Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68061

On parallel execution stages are not marked according to threshold values

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • xunit-plugin
    • None
    • 3.0.6

      We use xunit in a parallel pipeline to analyze test report files.

      When there are multiple test failures, xunit only marks the first calling stage as unstable. All other stages are marked as passed.

      Here is an stage overview of a MWE illustrating the problem:

      All three stages perform the xunit analysis on the same test report file which contains one failing test. The xunit call is like so:

      xunit thresholds: [failed(unstableThreshold: '0')],
                                tools: [GoogleTest(deleteOutputFiles: true,
                                    failIfNotNew: false,
                                    pattern: "*.xml",
                                    skipNoTestFiles: false,
                                    stopProcessingIfError: true)]

      The only difference between the stages is a delay before calling xunit which has been swapped between the three stages for each run.

      I would expect all three stages to be marked unstable, like so:

       

       

      We use xunit version 3.0.5,  and Jenkins Core 2.319.3

        1. image-2022-03-17-11-38-41-869.png
          image-2022-03-17-11-38-41-869.png
          47 kB
        2. image-2022-03-17-11-45-06-156.png
          image-2022-03-17-11-45-06-156.png
          20 kB
        3. unstable_test.groovy
          2 kB
        4. unstable_test.xml
          0.7 kB

          [JENKINS-68061] On parallel execution stages are not marked according to threshold values

          Nikolas Falco added a comment -

          Could you provide me the piece of pipeline job are you using to run the parallel block. I mean the code that runs the stages no1, no2, no3.

          Could you also attach the report file you are using?

          Nikolas Falco added a comment - Could you provide me the piece of pipeline job are you using to run the parallel block. I mean the code that runs the stages no1, no2, no3. Could you also attach the report file you are using?

          O B added a comment -

          Sure thing, I attached the groovy script and test.xml

          O B added a comment - Sure thing, I attached the groovy script and test.xml

          Nikolas Falco added a comment -

          Could you look me if global build test result is correct? I mean in that case it should report 3 failures.

          Nikolas Falco added a comment - Could you look me if global build test result is correct? I mean in that case it should report 3 failures.

          Nikolas Falco added a comment -

          I got the issue. If the first stage mark build UNSTABLE the second one since the build is not worst than UNSTABLE do not do anything so it appear green.

          Nikolas Falco added a comment - I got the issue. If the first stage mark build UNSTABLE the second one since the build is not worst than UNSTABLE do not do anything so it appear green.

          O B added a comment -

          Regarding your question about global build result:

          I copy-pasted the test xml from some other source in order to create the MWE and might have made a mistake there. In the original file, the numbers were correct.

           

          I'm glad this got resolved so quickly

          O B added a comment - Regarding your question about global build result: I copy-pasted the test xml from some other source in order to create the MWE and might have made a mistake there. In the original file, the numbers were correct.   I'm glad this got resolved so quickly

            nfalco Nikolas Falco
            keksmaster O B
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: