Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40285

global libraries test harness

    XMLWordPrintable

Details

    Description

      There aren't any standards and guidelines regarding the testing of shared pipeline libraries.
      The expected folder structure (https://issues.jenkins-ci.org/browse/JENKINS-33925) doesn't play well with Maven or Gradle and it isn't clear whether committing and pushing anything else but the expected folder tree would break the implementation.

      I couldn't find any examples apart from https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/src/test/java/org/jenkinsci/plugins/workflow/cps/global/WorkflowLibRepositoryTest.java

      Tried to integrate a similar approach into an example standard shared library but it's of limited use, see https://github.com/zhelyan/jenkins-pipeline-test-harness

      Any hints?

      Attachments

        Issue Links

          Activity

            mkobit Mike Kobit added a comment -

            I'd like to see some guidance on this, too. We use Bitbucket Branch Source Plugin almost exclusively, so anything geared towards testing libraries this way would be great. I opened a discussion on the forum about this previously, but didn't really get a hugely satisfactory answer. Being able to test these libraries would be an awesome addition.

            mkobit Mike Kobit added a comment - I'd like to see some guidance on this, too. We use Bitbucket Branch Source Plugin almost exclusively, so anything geared towards testing libraries this way would be great. I opened a discussion on the forum about this previously, but didn't really get a hugely satisfactory answer. Being able to test these libraries would be an awesome addition.
            0x89 Martin Sander added a comment -

            jglick: are you sure that this is actually a duplicate?

            JENKINS-33925 is about unit-testing Jenkinsfiles without interacting with a "real" Jenkins at all.

            The requirement testing a global library is IMVHO partly a different usecase, where you might want to "integration test" the methods from your global library actually interacting with the Jenkins you are running this on. While this is not nicely isolated, it might give you useful feedback if your methods are actually correctly interacting with the Jenkins, especially when it comes to CPS transformed code.

            While you can just write assertions using if statements and failing the build using error if something is not what you expect it to be, this is a lot more cumbersome than e.g. using JUnit, and does not give you the same feedback.

            • your tests will abort if you call error
            • if you use assert instead, your tests will just as well abort (unless you write some kind of harness yourself)
              • assert in CPS-transformed code is much less useful than "normal" Groovy assert - it does not give you the values of the whole expression
            • you don't get Junit-style test reports (again unless you report those yourself)
            • the JenkinsRule-approach mentioned in JENKINS-33925 looked promising to me, however you only get to check the logs and the result of the build, so it is rather a black-box (or gray-box) test, as you cannot query the exact values your methods return (or you would have to parse them out of the log or using some other hacks)

            It would be really nice to have some kind of support here..

            0x89 Martin Sander added a comment - jglick : are you sure that this is actually a duplicate? JENKINS-33925 is about unit-testing Jenkinsfiles without interacting with a "real" Jenkins at all. The requirement testing a global library is IMVHO partly a different usecase, where you might want to "integration test" the methods from your global library actually interacting with the Jenkins you are running this on. While this is not nicely isolated, it might give you useful feedback if your methods are actually correctly interacting with the Jenkins, especially when it comes to CPS transformed code. While you can just write assertions using if statements and failing the build using error if something is not what you expect it to be, this is a lot more cumbersome than e.g. using JUnit, and does not give you the same feedback. your tests will abort if you call error if you use assert instead, your tests will just as well abort (unless you write some kind of harness yourself) assert in CPS-transformed code is much less useful than "normal" Groovy assert - it does not give you the values of the whole expression you don't get Junit-style test reports (again unless you report those yourself) the JenkinsRule-approach mentioned in JENKINS-33925 looked promising to me, however you only get to check the logs and the result of the build, so it is rather a black-box (or gray-box) test, as you cannot query the exact values your methods return (or you would have to parse them out of the log or using some other hacks) It would be really nice to have some kind of support here..
            jglick Jesse Glick added a comment -

            JENKINS-33925 is about unit-testing Jenkinsfiles without interacting with a "real" Jenkins at all.

            That is one approach along a spectrum of options discussed in that issue. All of your concerns are well known.

            jglick Jesse Glick added a comment - JENKINS-33925  is about unit-testing Jenkinsfiles without interacting with a "real" Jenkins at all. That is one approach along a spectrum of options discussed in that issue. All of your concerns are well known.

            People

              Unassigned Unassigned
              zhelyan Zhelyan Panchev
              Votes:
              5 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: