Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39353

Include filtering by class name when splitting tests

    XMLWordPrintable

Details

    Description

      Starting with an scenario such as N test classes run:

      com.parallelizable.one (duration 10 mins)
      com.parallelizable.two (duration 10 mins)
      com.notparallelizable.one (duration 1 mins)
      com.notparallelizable.two (duration 1 mins)

      I am interested in parallelizing classes with packaging com.parallelizable, ending with an pseudo-pipeline such as:

      node {
      test (com.notparallelizable)
      splits = splitTests parallelism: 2
      parallel ( "one": test(splits[0].list.join('\n')),
      "two": test(splits[1].list.join('\n')))
      }

      My current workaround is filtering with grep:

      parallel ( "one": test(splits[0].list.grep(~/com.parallelizable.*/).join('\n')),
      "two": test(splits[1].list.grep(~/com.parallelizable.*/).join('\n')))

      but obviously, the resulting tests wont be ballanced in number of elements. I was thinking in filtering at the plugin level, so the returned groups would be properly ballanced.

      Im not sure if this improvement would be interesting for freestyle projects (i dont use them anymore), but for sure it will for pipelines.

      Attachments

        Issue Links

          Activity

            witokondoria Javier Delgado created issue -
            dcendents Daniel Beland added a comment -

             

            I'd like to have the same functionality but for a different reason.

            I have a maven project with multiple fast unit tests run by the surefire plugin and a few longer integration tests run by the failsafe plugin.

            When I split my test I always en up with one split that contains only unit tests or only integration tests so effectively neither the execution of surefire or failsafe is optimal.

            It would be nice if I could specify test includes/excludes patterns and split the tests twice for my 2 test execution context.

             

            dcendents Daniel Beland added a comment -   I'd like to have the same functionality but for a different reason. I have a maven project with multiple fast unit tests run by the surefire plugin and a few longer integration tests run by the failsafe plugin. When I split my test I always en up with one split that contains only unit tests or only integration tests so effectively neither the execution of surefire or failsafe is optimal. It would be nice if I could specify test includes/excludes patterns and split the tests twice for my 2 test execution context.  
            witokondoria Javier Delgado made changes -
            Field Original Value New Value
            Assignee Javier Delgado [ witokondoria ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 24 (Web Link)" [ 17198 ]
            jglick Jesse Glick added a comment -

            To the extent I understand the reporter’s original use case, it sounds like it would be better addressed by JENKINS-27395, under study.

            jglick Jesse Glick added a comment - To the extent I understand the reporter’s original use case, it sounds like it would be better addressed by  JENKINS-27395 , under study.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-27395 [ JENKINS-27395 ]

            From what I read, for JENKINS-27395 to help with this use case, several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering. Couldnt both functionalities coexist (filtering via regexp and via archiveId)? The mentioned ticket doesnt seem to be receiving work recently

            witokondoria Javier Delgado added a comment - From what I read, for JENKINS-27395 to help with this use case, several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering. Couldnt both functionalities coexist (filtering via regexp and via archiveId)? The mentioned ticket doesnt seem to be receiving work recently
            jglick Jesse Glick added a comment -

            several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering

            Yes, which seems the natural way to extend parallel-test-executor. Integration in this plugin is IMO a critical use case for JENKINS-27395. I cannot speak to the status of that issue currently but I am not keen on piling on features to work around lack of proper design.

            W.r.t. to the maven-project use case, best to just avoid this plugin (for many reasons). If it happens to work with parallel-test-executor currently, that is more by luck than anything else. The better integration is via Pipeline.

            jglick Jesse Glick added a comment - several invocations to junit would be needed to setup each archiveId and later reuse that ids for the filtering Yes, which seems the natural way to extend parallel-test-executor . Integration in this plugin is IMO a critical use case for  JENKINS-27395 . I cannot speak to the status of that issue currently but I am not keen on piling on features to work around lack of proper design. W.r.t. to the maven-project use case, best to just avoid this plugin (for many reasons). If it happens to work with parallel-test-executor currently, that is more by luck than anything else. The better integration is via Pipeline.

            Wont do as the linked ticket will eventually allow accessing test reports archives selectively (and parallelizing over them)

            witokondoria Javier Delgado added a comment - Wont do as the linked ticket will eventually allow accessing test reports archives selectively (and parallelizing over them)
            witokondoria Javier Delgado made changes -
            Resolution Won't Do [ 10001 ]
            Status Open [ 1 ] Closed [ 6 ]

            People

              witokondoria Javier Delgado
              witokondoria Javier Delgado
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: