Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59659

XUnit plugin result displays confused by aggregation of parallel builds

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Minor Minor
    • xunit-plugin
    • None

      So we use pipeline to build several platforms of our codebase, and prod/nonprod etc.  They all produce a unit test report, and the paths to the result files all contain the specific parallel branch they were built in.

      I was hoping this would be enough for it to properly identify the tests, but it seems that unless the tests have the build specific info in the test name, the graphing and result display of the tests gets very confused and will combine some tests based on race conditions which leads to a very spiky looking graph.

      I confirmed this by using the jest config in a node project we build the same way.  Jest lets you set the pattern used to build the suite name and test name. Once I set this to include the build info, the graph flattened out and remained consistent.

      In other projects we use other frameworks that you support googletest, etc.  I don't know if each of them support the option to configure the test suite/name patterns...     Could you please look at how the plugin could properly handle parallel builds for test display and graphing?

          [JENKINS-59659] XUnit plugin result displays confused by aggregation of parallel builds

          Nikolas Falco added a comment -

          I have some questions:

          • Do you have a jenkins job per branch (multibranch pipeline) or one job that build a specific branch?
          • Is the workspace folder shred among all build? Is the workspace wipe out?
          • What do you mean with parallel? Do you have a single job that where you have parallelized the test stage (which one that produce reports) or you have one job but parallel executor of the same job in the same workspace?

          Nikolas Falco added a comment - I have some questions: Do you have a jenkins job per branch (multibranch pipeline) or one job that build a specific branch? Is the workspace folder shred among all build? Is the workspace wipe out? What do you mean with parallel? Do you have a single job that where you have parallelized the test stage (which one that produce reports) or you have one job but parallel executor of the same job in the same workspace?

          Nikolas Falco added a comment - - edited

          Any news? If not I will close this issue.
          In pipeline there is the association with the stage. This means you can run parallel step and process reports. Each report is associated with a specific stage. Path of the reports are not taken into account.

          In non pipeline environment we can not support stage association. This means that tests name have to be unique across parallel execution.

          Nikolas Falco added a comment - - edited Any news? If not I will close this issue. In pipeline there is the association with the stage. This means you can run parallel step and process reports. Each report is associated with a specific stage. Path of the reports are not taken into account. In non pipeline environment we can not support stage association. This means that tests name have to be unique across parallel execution.

          Nikolas Falco added a comment -

          Path of report file could not be used to distinguish test. Perfom a xunit step in a different atep to differentiate them as explained in the improvement of linked issue.

          Nikolas Falco added a comment - Path of report file could not be used to distinguish test. Perfom a xunit step in a different atep to differentiate them as explained in the improvement of linked issue.

            nfalco Nikolas Falco
            ddumont Dan Dumont
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: