-
Bug
-
Resolution: Fixed
-
Major
-
CentOS 6
jenkins 1.546
cucumber-testresult 0.4
Ruby 2.0
What we did:
Ran minimal cucumber feature test with Jenkins:
Feature
Feature: Sample Test
Scenario Outline: Parsing scenarios with multiple examples
Given I navigate to the root page
Then I see the text 'home page'
Examples:
| a | b |
| 1 | 2 |
By a shell build step:
shell build
#!/bin/bash
# Set up RVM (Ruby 2.0)
. ~/.bashrc
set -e
CUCUMBER_OPTS='-f json -o cucumber.json features/sample.feature' rake cucumber
What we got:
[::1::]
What we expected:
No error, parsing of cucumber.json containing results from scenario outlines.
[::1::]
Output
/var/lib/jenkins/.rvm/rubies/ruby-2.0.0-p353/bin/ruby -S bundle exec cucumber -f json -o cucumber.json features/sample.feature Using the default profile... Feature: Sample Test Scenario Outline: Parsing scenarios with multiple examples # features/sample.feature:2 Given I navigate to the root page # features/step_definitions/navigation_steps.rb:15 Then I see the text 'home page' # features/step_definitions/study_admin.rb:23 Examples: | a | b | | 1 | 2 | 1 scenario (1 passed) 2 steps (2 passed) 0m1.080s Xvfb stopping Archiving artifacts [Cucumber Tests] Parsing results. [Cucumber Tests] parsing cucumber.json ERROR: Failed to archive cucumber reports java.io.IOException: Failed to parse Cucumber JSON at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:99) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:44) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:161) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:107) at hudson.FilePath.act(FilePath.java:914) at hudson.FilePath.act(FilePath.java:887) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl.parse(DefaultTestResultParserImpl.java:101) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:116) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.perform(CucumberTestResultArchiver.java:93) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:785) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:757) at hudson.model.Build$BuildExecution.post2(Build.java:183) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:706) at hudson.model.Run.execute(Run.java:1703) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Caused by: org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberModelException: Step received before previous step handled! at org.jenkinsci.plugins.cucumber.jsontestsupport.GherkinCallback.step(GherkinCallback.java:160) at gherkin.formatter.model.Step.replay(Step.java:68) at gherkin.JSONParser.step(JSONParser.java:106) at gherkin.JSONParser.parse(JSONParser.java:51) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:94) ... 17 more Build step 'Publish Cucumber test result report' changed build result to FAILURE Finished: FAILURE
[JENKINS-21835] Cucumber Test Result Plugin crashes on malformed output in Scenario Outline results produced by some tools
Description |
Original:
What we did: Ran minimal cucumber feature test with Jenkins Feature: Sample Test Scenario Outline: Parsing scenarios with multiple examples Given I navigate to the root page Then I see the text 'home page' Examples: | a | b | | 1 | 2 | By a shell build step: #!/bin/bash # Set up RVM (Ruby 2.0) . ~/.bashrc set -e CUCUMBER_OPTS='-f json -o cucumber.json features/sample.feature' rake cucumber What we got: [::1::] What we expected: No error, parsing of cucumber.json containing results from scenario outlines. [::1::] /var/lib/jenkins/.rvm/rubies/ruby-2.0.0-p353/bin/ruby -S bundle exec cucumber -f json -o cucumber.json features/sample.feature Using the default profile... Feature: Sample Test Scenario Outline: Parsing scenarios with multiple examples # features/sample.feature:2 Given I navigate to the root page # features/step_definitions/navigation_steps.rb:15 Then I see the text 'home page' # features/step_definitions/study_admin.rb:23 Examples: | a | b | | 1 | 2 | 1 scenario (1 passed) 2 steps (2 passed) 0m1.080s Xvfb stopping Archiving artifacts [Cucumber Tests] Parsing results. [Cucumber Tests] parsing cucumber.json ERROR: Failed to archive cucumber reports java.io.IOException: Failed to parse Cucumber JSON at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:99) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:44) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:161) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:107) at hudson.FilePath.act(FilePath.java:914) at hudson.FilePath.act(FilePath.java:887) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl.parse(DefaultTestResultParserImpl.java:101) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:116) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.perform(CucumberTestResultArchiver.java:93) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:785) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:757) at hudson.model.Build$BuildExecution.post2(Build.java:183) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:706) at hudson.model.Run.execute(Run.java:1703) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Caused by: org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberModelException: Step received before previous step handled! at org.jenkinsci.plugins.cucumber.jsontestsupport.GherkinCallback.step(GherkinCallback.java:160) at gherkin.formatter.model.Step.replay(Step.java:68) at gherkin.JSONParser.step(JSONParser.java:106) at gherkin.JSONParser.parse(JSONParser.java:51) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:94) ... 17 more Build step 'Publish Cucumber test result report' changed build result to FAILURE Finished: FAILURE |
New:
What we did: Ran minimal cucumber feature test with Jenkins: {code:title=Feature} Feature: Sample Test Scenario Outline: Parsing scenarios with multiple examples Given I navigate to the root page Then I see the text 'home page' Examples: | a | b | | 1 | 2 | {code} By a shell build step: {code:title=shell build} #!/bin/bash # Set up RVM (Ruby 2.0) . ~/.bashrc set -e CUCUMBER_OPTS='-f json -o cucumber.json features/sample.feature' rake cucumber {code} What we got: [::1::] What we expected: No error, parsing of cucumber.json containing results from scenario outlines. [::1::] {code:title=Output} /var/lib/jenkins/.rvm/rubies/ruby-2.0.0-p353/bin/ruby -S bundle exec cucumber -f json -o cucumber.json features/sample.feature Using the default profile... Feature: Sample Test Scenario Outline: Parsing scenarios with multiple examples # features/sample.feature:2 Given I navigate to the root page # features/step_definitions/navigation_steps.rb:15 Then I see the text 'home page' # features/step_definitions/study_admin.rb:23 Examples: | a | b | | 1 | 2 | 1 scenario (1 passed) 2 steps (2 passed) 0m1.080s Xvfb stopping Archiving artifacts [Cucumber Tests] Parsing results. [Cucumber Tests] parsing cucumber.json ERROR: Failed to archive cucumber reports java.io.IOException: Failed to parse Cucumber JSON at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:99) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:44) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:161) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl$ParseResultCallable.invoke(DefaultTestResultParserImpl.java:107) at hudson.FilePath.act(FilePath.java:914) at hudson.FilePath.act(FilePath.java:887) at org.jenkinsci.plugins.cucumber.jsontestsupport.DefaultTestResultParserImpl.parse(DefaultTestResultParserImpl.java:101) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:116) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver.perform(CucumberTestResultArchiver.java:93) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:785) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:757) at hudson.model.Build$BuildExecution.post2(Build.java:183) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:706) at hudson.model.Run.execute(Run.java:1703) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:231) Caused by: org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberModelException: Step received before previous step handled! at org.jenkinsci.plugins.cucumber.jsontestsupport.GherkinCallback.step(GherkinCallback.java:160) at gherkin.formatter.model.Step.replay(Step.java:68) at gherkin.JSONParser.step(JSONParser.java:106) at gherkin.JSONParser.parse(JSONParser.java:51) at org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberJSONParser.parse(CucumberJSONParser.java:94) ... 17 more Build step 'Publish Cucumber test result report' changed build result to FAILURE Finished: FAILURE {code} |
Attachment | New: cucumber_with_outline_and_basic_scenario.json [ 25509 ] |
Assignee | New: James Nord [ teilo ] |
Resolution | New: Fixed [ 1 ] | |
Status | Original: Open [ 1 ] | New: Resolved [ 5 ] |
Summary | Original: Cucumber Test Result Plugin does not correctly parse Scenario Outline results | New: Cucumber Test Result Plugin crashes on malformed output in Scenario Outline results produced by some tools |
Workflow | Original: JNJira [ 153836 ] | New: JNJira + In-Review [ 194716 ] |
Assignee | Original: James Nord [ teilo ] |