Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42812

Test Results are not displayed after several executions

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Minor Minor
    • None
    • Jenkins 2.32.3 (latest LTS) with "Cucumber json test reporting." v0.9.7
      Rails + Cucumber system under test runs on RVM-2.1.7+MacOS-10.11

      My problem is simply that the "Test Result" option is not shown in the Job page.

      I have installed the plugin, and in Job Configuration I have added the Post-build Action "Publish Cucumber test result report" with options: Cucumber JSON report files: 'features/reports/results.json'

      My cucumber exec build step looks like
      bundle exec cucumber -f pretty -f json -o ${WORKSPACE}/features/reports/results.json

      Trying to visit /job/ft/job/dev_pp/32 (the latest build) I do not find "Test Result" in the job side-panel.  I also have the "Cucumber reports" plugin enabled and "Cucumber reports" does show up there.  I saw in your screenshots that you are using Jenkins 1.x, maybe this is a regression in 2.x

      If there is some step I skipped, then I would make this a documentation bug, I think I have read the instructions carefully.  I have not tried building and installing your latest/unpublished release, so I can't say whether this is already fixed in HEAD.

      I would try building this and I'll be glad to contribute a fix if possible, but if you've already heard of this issue and know where the problem might be coming from, or if you can at least reproduce and confirm the bug, it would be great to know that it's not a user error before I go to that extent.

       

          [JENKINS-42812] Test Results are not displayed after several executions

          I should probably mention that, my project does not use Pipelines so I have not done anything with this step:

          To record the results in Pipeline use the following syntax:

          step($class: 'CucumberTestResultArchiver', testResults: 'glob')

           

          I just gave it a path from my workspace instead.  If I need to convert my project to a pipeline to use this project, I did not understand that from reading the documentation or Jenkins server logs.

          Kingdon Barrett added a comment - I should probably mention that, my project does not use Pipelines so I have not done anything with this step: To record the results in Pipeline use the following syntax: step($class: 'CucumberTestResultArchiver' , testResults: 'glob' )   I just gave it a path from my workspace instead.  If I need to convert my project to a pipeline to use this project, I did not understand that from reading the documentation or Jenkins server logs.

          James Nord added a comment -

          you may want to ask on the jenkins users mailing list or IRC (https://jenkins.io/participate/#help).

          Can you attach the build log.

          James Nord added a comment - you may want to ask on the jenkins users mailing list or IRC ( https://jenkins.io/participate/#help ). Can you attach the build log.

          Oh, thank you, there are the errors:

          46 scenarios (31 failed, 15 passed)
          532 steps (14 failed, 111 skipped, 407 passed)
          8m13.345s
          Build step 'Execute shell' marked build as failure
          $ ssh-agent -k
          unset SSH_AUTH_SOCK;
          unset SSH_AGENT_PID;
          echo Agent pid 18593 killed;
          [ssh-agent] Stopped.
          [Cucumber Tests] Parsing results.
          [Cucumber Tests] parsing results.json
          ERROR: Build step failed with exception
          java.lang.NullPointerException
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.GherkinCallback.before(GherkinCallback.java:221)
          	at gherkin.JSONParser.before(JSONParser.java:82)
          	at gherkin.JSONParser.parse(JSONParser.java:48)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:78)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:44)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:161)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:107)
          	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2731)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:153)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
          	at hudson.remoting.Request$2.run(Request.java:336)
          	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          	at hudson.remoting.Engine$1$1.run(Engine.java:94)
          	at java.lang.Thread.run(Thread.java:745)
          	at ......remote call to Channel to /10.32.0.1(Native Method)
          	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537)
          	at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
          	at hudson.remoting.Channel.call(Channel.java:822)
          	at hudson.FilePath.act(FilePath.java:985)
          	at hudson.FilePath.act(FilePath.java:974)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl.parseResult(DefaultTestResultParserImpl.java:101)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parseResult(CucumberJSONParser.java:101)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.publishReport(CucumberTestResultArchiver.java:128)
          	at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.perform(CucumberTestResultArchiver.java:107)
          	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
          	at hudson.model.Build$BuildExecution.post2(Build.java:185)
          	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
          	at hudson.model.Run.execute(Run.java:1753)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          	at hudson.model.ResourceController.execute(ResourceController.java:98)
          	at hudson.model.Executor.run(Executor.java:404)
          Build step 'Publish Cucumber test result report' marked build as failure
          [CucumberReport] Preparing Cucumber Reports
          [CucumberReport] Copied 1 json files from workspace "/Users/kingdonb/jenkins-slave/workspace/ft/dev_pp/features/reports" to reports directory "/var/jenkins_home/jobs/ft/jobs/dev_pp/builds/32/cucumber-html-reports/.cache"
          [CucumberReport] Filtered out 1 json files:
          [CucumberReport] /var/jenkins_home/jobs/ft/jobs/dev_pp/builds/32/cucumber-html-reports/.cache/results.json
          [CucumberReport] Found 14 failed steps, while expected not more than 0
          [CucumberReport] Build status is left unchanged
          Finished: FAILURE
          

          Kingdon Barrett added a comment - Oh, thank you, there are the errors: 46 scenarios (31 failed, 15 passed) 532 steps (14 failed, 111 skipped, 407 passed) 8m13.345s Build step 'Execute shell' marked build as failure $ ssh-agent -k unset SSH_AUTH_SOCK; unset SSH_AGENT_PID; echo Agent pid 18593 killed; [ssh-agent] Stopped. [Cucumber Tests] Parsing results. [Cucumber Tests] parsing results.json ERROR: Build step failed with exception java.lang.NullPointerException at org.jenkinsci.plugins.cucumber.jsontestsupport.GherkinCallback.before(GherkinCallback.java:221) at gherkin.JSONParser.before(JSONParser.java:82) at gherkin.JSONParser.parse(JSONParser.java:48) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:78) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:44) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:161) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:107) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2731) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:336) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at hudson.remoting.Engine$1$1.run(Engine.java:94) at java.lang. Thread .run( Thread .java:745) at ......remote call to Channel to /10.32.0.1(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:822) at hudson.FilePath.act(FilePath.java:985) at hudson.FilePath.act(FilePath.java:974) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl.parseResult(DefaultTestResultParserImpl.java:101) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parseResult(CucumberJSONParser.java:101) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.publishReport(CucumberTestResultArchiver.java:128) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.perform(CucumberTestResultArchiver.java:107) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720) at hudson.model.Build$BuildExecution.post2(Build.java:185) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665) at hudson.model.Run.execute(Run.java:1753) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Build step 'Publish Cucumber test result report' marked build as failure [CucumberReport] Preparing Cucumber Reports [CucumberReport] Copied 1 json files from workspace "/Users/kingdonb/jenkins-slave/workspace/ft/dev_pp/features/reports" to reports directory "/ var /jenkins_home/jobs/ft/jobs/dev_pp/builds/32/cucumber-html-reports/.cache" [CucumberReport] Filtered out 1 json files: [CucumberReport] / var /jenkins_home/jobs/ft/jobs/dev_pp/builds/32/cucumber-html-reports/.cache/results.json [CucumberReport] Found 14 failed steps, while expected not more than 0 [CucumberReport] Build status is left unchanged Finished: FAILURE

          Would like to attach the results.json that evidently failed to parse, but apparently attachments are disabled.

          Unless this message means it hasn't been found at all for some other reason

           

          Kingdon Barrett added a comment - Would like to attach the results.json that evidently failed to parse, but apparently attachments are disabled. Unless this message means it hasn't been found at all for some other reason  

          James Nord added a comment -

          the failure to parse is why you have no results.

          Seems like a dupe of JENKINS-29328

          James Nord added a comment - the failure to parse is why you have no results. Seems like a dupe of JENKINS-29328

          Thank you! Closing in favor of JENKINS-29328 .  My cucumber ruby gem version is 2.4.0.

          Kingdon Barrett added a comment - Thank you! Closing in favor of JENKINS-29328 .  My cucumber ruby gem version is 2.4.0.

          It's a dupe.

          Kingdon Barrett added a comment - It's a dupe.

            Unassigned Unassigned
            kingdonb Kingdon Barrett
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: