Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51702

Summarize history of same test case executed in different parallel splits

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: junit-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      I am using the Parallel Test Executor plugin, so my unit tests are splitted into splits executed in parallel on different nodes. This is a simplified part of my Jenkinsfile:

       

      #!/usr/bin/env groovy
      
      stage('Test') {
        def testSteps = [:]
        def splits = splitTests parallelism: count(env.NUMBER_EXECUTORS as int)
        for (int i = 0; i < splits.size(); i++) {
          def num = i
          def split = splits[num]
          testSteps["Test split${num}"] = {
            node('!master') {
              checkout scm
              writeFile file: ("test_exclusions.txt"), text: split.join("\n")
              sh './gradlew test'
              junit '**/build/test-results/**/TEST-*.xml'
            }
          }
        }
        parallel testSteps
      }
      
      

       

      We make heavy use of the test result history to see when the build first broke and who is responsible for that.

      The issue I am facing is the fact that the ID of the test result contains the name of the parallel execution step as the prefix, in this case "Test split2" for example. As a result, a broken test is shown in the history as "Test / Test split2 / my.package.MyTest.testStuff()".

      This is nice in theory, but there is one issue: The history only shows me the history of a test case executed in this specific split. For example, it may be that  "Test / Test split2 / my.package.MyTest.testStuff()" is shown as broken with an age of 1, although it may be broken since multiple builds, just that this specific test case happened to be put into different splits before.

      To summarize: "Test / Test split2 / my.package.MyTest.testStuff()" and "Test / Test split3 / my.package.MyTest.testStuff()" have separate histories, but I need to track their histories together.

       

        Attachments

          Activity

          Hide
          nardew A Ch added a comment -

          I would like to vote for this request. Stage names/node names were made part of the test result key without any functional backward compatibility, anyone who is running tests on several nodes randomly is getting completely unusable history of test results since one time test is running on node A, second time on node B and plugin treats it as separate test records (so you cannot track evolution of test results between builds and so on).

          Show
          nardew A Ch added a comment - I would like to vote for this request. Stage names/node names were made part of the test result key without any functional backward compatibility, anyone who is running tests on several nodes randomly is getting completely unusable history of test results since one time test is running on node A, second time on node B and plugin treats it as separate test records (so you cannot track evolution of test results between builds and so on).

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            christianciach Christian Ciach
            Votes:
            3 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: