• Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Major Major
    • cucumber-perf-plugin
    • None
    • Windows 7
      master only
      jenkins 1.519
      java 1.6 update 37

      When parsing reports the plugin thros an exception

      [Cucumber Tests] Parsing results.
      [Cucumber Tests] parsing cucumber.json
      [CucumberPerfRecorder] Starting Cucumber Performance Report generation...
      [CucumberPerfRecorder] Reporting on performance for cucumber #87
      [CucumberPerfRecorder] detected master build 
      [CucumberPerfRecorder] running project report
      [CucumberPerfRecorder] build report generation complete - true
      [CucumberPerfRecorder] running project reports on 1 builds
      ERROR: Publisher com.castlemon.jenkins.performance.CucumberPerfRecorder aborted due to exception
      java.lang.NullPointerException
      	at com.castlemon.jenkins.performance.util.CucumberPerfUtils.getRelevantSummaries(CucumberPerfUtils.java:36)
      	at com.castlemon.jenkins.performance.reporting.ReportBuilder.generateScenarioReports(ReportBuilder.java:128)
      	at com.castlemon.jenkins.performance.reporting.ReportBuilder.generateProjectReports(ReportBuilder.java:74)
      	at com.castlemon.jenkins.performance.CucumberPerfRecorder.generateProjectReport(CucumberPerfRecorder.java:102)
      	at com.castlemon.jenkins.performance.CucumberPerfRecorder.perform(CucumberPerfRecorder.java:71)
      	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:776)
      	at hudson.model.Build$BuildExecution.post2(Build.java:183)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:726)
      	at hudson.model.Run.execute(Run.java:1618)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:242)
      [CucumberReportPublisher] Compiling Cucumber Html Reports ...
      [CucumberReportPublisher] detected this build is running on the master 
      [CucumberReportPublisher] copying json to reports directory: C:\workarea\source\jenkinsci\cucumber-testresult-plugin\.\work\jobs\cucumber\builds\2013-10-04_14-06-20\cucumber-html-reports
      [CucumberReportPublisher] Generating HTML reports
      Finished: FAILURE
      

      This renders the plugin unusable and causes the builds to be marked as failures.

          [JENKINS-19888] NPE when running plugin

          Jim Gallagher added a comment -

          Hi,

          Sorry that this causing you an issue. Could you please attach a copy of the cucumber.json file that this plugin fails on, and I'll try to examine the issue & solve it?

          Thanks

          Jim

          Jim Gallagher added a comment - Hi, Sorry that this causing you an issue. Could you please attach a copy of the cucumber.json file that this plugin fails on, and I'll try to examine the issue & solve it? Thanks Jim

          Jim Gallagher added a comment -

          Without the file in question I'm unfortunately unable to progress this issue. If you could assist by supplying it I'd be delighted to assess the error and fix it.

          Unless I can progress this in the next 2 weeks, I'll mark the issue as closed.

          Jim Gallagher added a comment - Without the file in question I'm unfortunately unable to progress this issue. If you could assist by supplying it I'd be delighted to assess the error and fix it. Unless I can progress this in the next 2 weeks, I'll mark the issue as closed.

          I stumbled upon the same issue. See the attached cucumber.json file. HTH

          Matthias Naber added a comment - I stumbled upon the same issue. See the attached cucumber.json file. HTH

          Jim Gallagher added a comment -

          I've analysed the file you've attached - many thanks for that.

          The error is caused by step definitions in the json output that do not contain an id element, causing a null-pointer exception when trying to generate the step reports for each scenario.

          That's the cause - the solution is more tricky. I need to investigate the structure of the json output and determine if an id element is mandatory - you'd think it is, but your file indicates otherwise.

          This is still under investigation.

          Jim Gallagher added a comment - I've analysed the file you've attached - many thanks for that. The error is caused by step definitions in the json output that do not contain an id element, causing a null-pointer exception when trying to generate the step reports for each scenario. That's the cause - the solution is more tricky. I need to investigate the structure of the json output and determine if an id element is mandatory - you'd think it is, but your file indicates otherwise. This is still under investigation.

          Jim Gallagher added a comment -

          Matthias, could you supply me the feature file you used for this test? I need to relate the json output back to the feature file to see what's missing and how I can code around it.

          Thanks

          Jim Gallagher added a comment - Matthias, could you supply me the feature file you used for this test? I need to relate the json output back to the feature file to see what's missing and how I can code around it. Thanks

          Jim Gallagher added a comment -

          Having done a lot of investigation on this, no simple solution presents itself, as the issue is basically caused by a malformed JSON file (not invalid JSON, just missing elements) - results are also missing for certain steps.

          I'm going to close the issue, despite the lack of resolution, but will re-open should it be reported as a problem for anyone else.

          Jim Gallagher added a comment - Having done a lot of investigation on this, no simple solution presents itself, as the issue is basically caused by a malformed JSON file (not invalid JSON, just missing elements) - results are also missing for certain steps. I'm going to close the issue, despite the lack of resolution, but will re-open should it be reported as a problem for anyone else.

          James Nord added a comment -

          What is broken in the json?

          We where seeing this on our tests yet neither the cucumber test result plugin (and the cucumber plugin) had any issues parsing the json files.

          James Nord added a comment - What is broken in the json? We where seeing this on our tests yet neither the cucumber test result plugin (and the cucumber plugin) had any issues parsing the json files.

          Jim Gallagher added a comment -

          Hi Teilo

          I don't know your JSON file, as you never supplied it. The one from Mathias did seem to contain malformed objects.

          If you'd care to supply the JSON output file and your input feature file, I'll re-open the issue and re-examine it. I did ask you to do this back in October.

          Jim

          Jim Gallagher added a comment - Hi Teilo I don't know your JSON file, as you never supplied it. The one from Mathias did seem to contain malformed objects. If you'd care to supply the JSON output file and your input feature file, I'll re-open the issue and re-examine it. I did ask you to do this back in October. Jim

          Hi!

          Same Problem here.
          Log says

          [Cucumber Tests] Parsing results.
          [Cucumber Tests] parsing cucumber.json
          ERROR: Publisher org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver aborted due to exception
          java.lang.NullPointerException
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.GherkinCallback.before(GherkinCallback.java:220)
          	at gherkin.JSONParser.before(JSONParser.java:82)
          	at gherkin.JSONParser.parse(JSONParser.java:48)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:94)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:44)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:161)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:107)
          	at hudson.FilePath.act(FilePath.java:920)
          	at hudson.FilePath.act(FilePath.java:893)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl.parse(DefaultTestResultParserImpl.java:101)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:116)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.perform(CucumberTestResultArchiver.java:93)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:740)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:710)
          	at hudson.model.Build$BuildExecution.post2(Build.java:182)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:659)
          	at hudson.model.Run.execute(Run.java:1728)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:231)
          

          My cucumber.json attached above.
          All features do have an ID though.

          Does it matter, that this now failing job has been run before, without the plugin?

          Adam Zielinski added a comment - Hi! Same Problem here. Log says [Cucumber Tests] Parsing results. [Cucumber Tests] parsing cucumber.json ERROR: Publisher org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver aborted due to exception java.lang.NullPointerException at org.jenkinsci.plugins.cucumber.jsontestsupport.GherkinCallback.before(GherkinCallback.java:220) at gherkin.JSONParser.before(JSONParser.java:82) at gherkin.JSONParser.parse(JSONParser.java:48) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:94) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:44) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:161) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:107) at hudson.FilePath.act(FilePath.java:920) at hudson.FilePath.act(FilePath.java:893) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl.parse(DefaultTestResultParserImpl.java:101) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:116) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.perform(CucumberTestResultArchiver.java:93) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:740) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:710) at hudson.model.Build$BuildExecution.post2(Build.java:182) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:659) at hudson.model.Run.execute(Run.java:1728) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) My cucumber.json attached above. All features do have an ID though. Does it matter, that this now failing job has been run before, without the plugin?

          Jim Gallagher added a comment -

          Hi Adam

          When I get a chance I'll run your cucumber.json through a test system, but one thing puzzles me: there's no reference to my plugin in that stacktrace. Looking at it, the error occurs in org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver

          What leads you to believe that the Cucumber performance plugin is responsible?

          Regards

          Jim

          Jim Gallagher added a comment - Hi Adam When I get a chance I'll run your cucumber.json through a test system, but one thing puzzles me: there's no reference to my plugin in that stacktrace. Looking at it, the error occurs in org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver What leads you to believe that the Cucumber performance plugin is responsible? Regards Jim

            truedub Jim Gallagher
            teilo James Nord
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: