Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47194

xUnit should not consider report generated by other xUnit step

      node('master') {
          stage('1') {
              step([$class: 'XUnitBuilder',
                            thresholds: [[$class: 'FailedThreshold', failureThreshold: "1"]],
                             tools: [[$class: 'JUnitType', pattern: "tmp.xml"]]])
          }
          stage('2') {
              step([$class: 'XUnitBuilder',
                            thresholds: [[$class: 'FailedThreshold', failureThreshold: "0"]],
                             tools: [[$class: 'JUnitType', pattern: "tmp2.xml"]]])       
          }
      }
      

      tmp.xml contains one failure and tmp2.xml contains no failures.

       

      The result is as followed:

      [test_pipeline] Running shell script
      [Pipeline] step
      [xUnit] [INFO] - Starting to record.
      [xUnit] [INFO] - Processing JUnit
      [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern 'tmp.xml' relative to '/var/lib/jenkins/workspace/test_pipeline' for the testing framework 'JUnit'.
      [xUnit] [INFO] - Check 'Failed Tests' threshold.
      [xUnit] [INFO] - Setting the build status to SUCCESS
      [xUnit] [INFO] - Stopping recording.
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (2)
      [Pipeline] sh
      [test_pipeline] Running shell script
      [Pipeline] step
      [xUnit] [INFO] - Starting to record.
      [xUnit] [INFO] - Processing JUnit
      [xUnit] [INFO] - [JUnit] - 1 test report file(s) were found with the pattern 'tmp2.xml' relative to '/var/lib/jenkins/workspace/test_pipeline' for the testing framework 'JUnit'.
      [xUnit] [INFO] - Check 'Failed Tests' threshold.
      [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value.
      [xUnit] [INFO] - Setting the build status to FAILURE
      [xUnit] [INFO] - Stopping recording.
      

      Xunit keeps track of previous recorded failures, while it, in my opinion, should not

          [JENKINS-47194] xUnit should not consider report generated by other xUnit step

          roel postelmans created issue -
          Andrew Bayer made changes -
          Component/s Original: pipeline-model-definition-plugin [ 21706 ]
          Andrew Bayer made changes -
          Assignee Original: Andrew Bayer [ abayer ]

          Andrew Bayer added a comment -

          JENKINS-27395 may be relevant here.

          Andrew Bayer added a comment - JENKINS-27395 may be relevant here.
          Nikolas Falco made changes -
          Assignee New: Nikolas Falco [ nfalco ]
          Nikolas Falco made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          Nikolas Falco made changes -
          Summary Original: Can't have a different failure threshold with xunit per stage New: xUnit should not consider report generated by other xUnit step
          Nikolas Falco made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: In Progress [ 3 ] New: Resolved [ 5 ]

          Code changed in jenkins
          User: Nikolas Falco
          Path:
          src/main/java/org/jenkinsci/plugins/xunit/XUnitProcessor.java
          src/main/java/org/jenkinsci/plugins/xunit/service/XUnitTransformer.java
          src/test/java/org/jenkinsci/plugins/xunit/XUnitPublisherTest.java
          src/test/java/org/jenkinsci/plugins/xunit/XUnitWorkflowTest.java
          src/test/java/org/jenkinsci/plugins/xunit/service/TempWorkspace.java
          src/test/java/org/jenkinsci/plugins/xunit/service/XUnitTransformerTest.java
          src/test/java/org/jenkinsci/plugins/xunit/service/XUnitValidationServiceTest.java
          src/test/resources/org/jenkinsci/plugins/xunit/XUnitPublisherTest/workspace/JENKINS-47194/input.xml
          src/test/resources/org/jenkinsci/plugins/xunit/config.xml
          http://jenkins-ci.org/commit/xunit-plugin/10d52804cbf03c855fc0b9208ff6d8a62cf0efa0
          Log:
          JENKINS-47194 xUnit should not consider report generated by other xUnit step

          XUnit build step or publisher use an instance of processor than generates output report to a shared common folder. This cause on latest step thresholds consider also reports produced by previous steps. Processor now generate an identifier that permits to generate reports into an unique output folder.

          *NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

          Functionality will be removed from GitHub.com on January 31st, 2019.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nikolas Falco Path: src/main/java/org/jenkinsci/plugins/xunit/XUnitProcessor.java src/main/java/org/jenkinsci/plugins/xunit/service/XUnitTransformer.java src/test/java/org/jenkinsci/plugins/xunit/XUnitPublisherTest.java src/test/java/org/jenkinsci/plugins/xunit/XUnitWorkflowTest.java src/test/java/org/jenkinsci/plugins/xunit/service/TempWorkspace.java src/test/java/org/jenkinsci/plugins/xunit/service/XUnitTransformerTest.java src/test/java/org/jenkinsci/plugins/xunit/service/XUnitValidationServiceTest.java src/test/resources/org/jenkinsci/plugins/xunit/XUnitPublisherTest/workspace/ JENKINS-47194 /input.xml src/test/resources/org/jenkinsci/plugins/xunit/config.xml http://jenkins-ci.org/commit/xunit-plugin/10d52804cbf03c855fc0b9208ff6d8a62cf0efa0 Log: JENKINS-47194 xUnit should not consider report generated by other xUnit step XUnit build step or publisher use an instance of processor than generates output report to a shared common folder. This cause on latest step thresholds consider also reports produced by previous steps. Processor now generate an identifier that permits to generate reports into an unique output folder. * NOTE: * This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.
          Nikolas Falco made changes -
          Link New: This issue is duplicated by JENKINS-38530 [ JENKINS-38530 ]

            nfalco Nikolas Falco
            roel0 roel postelmans
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: