Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-69731

Want @Library annotation to always load same version (feature branch name) of library as pipeline script from SCM

      Currently the @Library annotation allows to specify only a fixed-string version of a library to be loaded. For pipelines that deal with branching (e.g. dev/qa/stage/prod or randomly-named feature branches as the pipelines and their libraries evolve) this causes either more configuration to separate jobs per folders with preloaded default library versions (and forfeiting benefits of e.g. Multi-Branch Pipeline and Organization Folders automagic), or needlessly deviating Git sources to reference a particular library branch.

      This situation has certain historic reasons, including security (not allowing arbitrary code outside the sandbox, which the library step apparently imposes among other inconveniences like lame access to library src/* classes - being a post-compilation step in a pipeline) and Groovy->Java CPS (annotated @Library is loaded and compiled before general-purpose groovy scripting of the pipeline and so its variables can be used).

      This issue states the desired ability to load at least same "version" (branch name) of the trusted globally configured library as that of the pipeline being executed, with little hassle to scale it on the pipeline coding and Jenkins configuration side.

      PR posted as https://github.com/jenkinsci/pipeline-groovy-lib-plugin/pull/19 to allow literally requesting this (optionally, site-configured):

      @Library('libname@${BRANCH_NAME}') _
      import my.lib.name.class;
      ... 

      If the "version" requested in pipeline script is a literal string ${BRANCH_NAME}, the implementation would try to learn the "BRANCH_NAME" environment variable set for the current Run object. If that is present, it would validateVersion() as implemented by this or that LibraryRetriever backend (SCM here, HTTP in workflow-cps-global-lib-http-plugin, etc.) and if the string is not rejected, it would be substituted as the version to use. Otherwise the configured default version would be used as fallback.

          [JENKINS-69731] Want @Library annotation to always load same version (feature branch name) of library as pipeline script from SCM

          Viktor added a comment -

          The templating syntax inside the Decorator maybe should not be ${...} as that can lead to mistakes when users use the wrong string literal quotes?

          e.g.:

          @Library("libname@${BRANCH_NAME}") _
          

          Might be a runtime/compiletime error?

          Maybe the syntax should be e.g. %{...} instead? - Something that doesn't collide with GString interpolation.
          Thoughts?

          Viktor added a comment - The templating syntax inside the Decorator maybe should not be ${... } as that can lead to mistakes when users use the wrong string literal quotes? e.g.: @Library("libname@${BRANCH_NAME}") _ Might be a runtime/compiletime error? Maybe the syntax should be e.g. %{... } instead? - Something that doesn't collide with GString interpolation. Thoughts?

          Jim Klimov added a comment - - edited

          It would not collide with GStrings - these annotations (and library "name@version" verbatim strings) in pipeline source text are interpreted by the plugin before Groovy gets to them. The pipeline text is interpreted, static libraries attached, and then it gets to compilation etc. (honoring `import` in particular).

          It is actually the "core of the problem" for those library consumers who aim to use a variable version in static-library context and can not: concept of pipeline/groovy variables appears at a later stage in processing.

          But thanks for the reminder - I will stress in the online help for the options that while this markup looks intentionally similar to script expansion for ease of use (I suppose people trying to naively use ${BRANCH_NAME} would just succeed without knowing of this feature or that any other dollar-expansion would not work), the values substituted here are not influenced by run-time interpretation of the pipeline script.

          I'll also change some tests to request with both quote types to make sure it is as I expect

          UPDATE: actually src/main/java/org/jenkinsci/plugins/workflow/libs/LibraryDecorator.java rejects use of double quotes, so that point is laid to rest (but for the future I'll still add the test):

          org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
          WorkflowScript: @Library value ?branchylib@$BRANCH_NAME? was not a constant; did you mean to use the ?library? step instead? 

          Jim Klimov added a comment - - edited It would not collide with GStrings - these annotations (and library "name@version" verbatim strings) in pipeline source text are interpreted by the plugin before Groovy gets to them. The pipeline text is interpreted, static libraries attached, and then it gets to compilation etc. (honoring `import` in particular). It is actually the "core of the problem" for those library consumers who aim to use a variable version in static-library context and can not: concept of pipeline/groovy variables appears at a later stage in processing. But thanks for the reminder - I will stress in the online help for the options that while this markup looks intentionally similar to script expansion for ease of use (I suppose people trying to naively use ${BRANCH_NAME } would just succeed without knowing of this feature or that any other dollar-expansion would not work), the values substituted here are not influenced by run-time interpretation of the pipeline script. I'll also change some tests to request with both quote types to make sure it is as I expect UPDATE: actually src/main/java/org/jenkinsci/plugins/workflow/libs/LibraryDecorator.java rejects use of double quotes, so that point is laid to rest (but for the future I'll still add the test): org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: WorkflowScript: @Library value ?branchylib@$BRANCH_NAME? was not a constant; did you mean to use the ?library? step instead?

          Jim Klimov added a comment -

          For now I'm done with the PR, using it in a project. Some lessons learned, tips and tricks summarized in https://github.com/jenkinsci/pipeline-groovy-lib-plugin/pull/19#issuecomment-1268047231

          Jim Klimov added a comment - For now I'm done with the PR, using it in a project. Some lessons learned, tips and tricks summarized in https://github.com/jenkinsci/pipeline-groovy-lib-plugin/pull/19#issuecomment-1268047231

          Jim Klimov added a comment -

          After some discussions and trial by fire (in practical server and varied pipeline script sources), PR was updated and functionally "done for now" again (looking at some generalization somewhat separately)

          Jim Klimov added a comment - After some discussions and trial by fire (in practical server and varied pipeline script sources), PR was updated and functionally "done for now" again (looking at some generalization somewhat separately)

          Mahdi added a comment - - edited

          I would love to have this functionality for integration testing of shared libraries. Is there something blocking the merging of this enhancement?

          Mahdi added a comment - - edited I would love to have this functionality for integration testing of shared libraries. Is there something blocking the merging of this enhancement?

          I'm using this syntax and that works for me:

           

          final String sharedLibRef = 'v1.0.0'
          library("libname@${sharedLibRef}") 

          Source

          Cyprien Quilici added a comment - I'm using this syntax and that works for me:   final String sharedLibRef = 'v1.0.0' library( "libname@${sharedLibRef}" ) Source

            jimklimov Jim Klimov
            jimklimov Jim Klimov
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: