• Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None

      Hi there,

      I was asked to open a Ticket here, because it seems like the feature is interesting for many peoples.

      It would be nice to have the possibility to set the dynamically set the values of the "axis".

      Use-Case:
      I want to compile multiple plugins with the same Jenkins-pipeline. To do so I wrote a "JenkinsLibrary" with an interface the plugins can use for parameters. One of this parameters is the axis-value for compiler.

      • e.g. I want to build Plugin "X" for gcc7 on armv8 and Plugin Y for vc15 x64.
        So I have a Jenkins-Library with the logic of the build parameters.
      • And each plugin calls the library with the build-parameters. (e.g. compiler and arch )

      Example:
      a "JenkinsLibrary" with a file my_library.groovy

      def call(Map i_options)
      {
          def axis_1_values = "axis_1_value_A"
          // later: def axis_1_values = i_options.get("axis_1_values")
          pipeline
          {
              agent any
              stages { stage("stage A") { matrix {
                  axes {
                      axis {
                          name "axis_1"
                          values "${axis_1_values}" 
                      }
                      axis {
                          name "axis_2"
                          values "axis_2_value_A", "axis_2_value_B"
                      }
                  }
                  stages {
                      stage("another stage") { steps {
                          echo "hello world from ${axis_1} && ${axis_2}"
                      } }
                  }
              } } }
          }
      }
      

      and each plugin contains a Jenkinsfile like that:

      @Library("JenkinsLibrary") _
      
      def options = [
          axis_1_values: "a_axis_value"
      ]
      
      my_library(options)
      

      We want to keep the Jenkinsfiles of the plugins as short as possible to keep the maintenance as low as possible.

       

      If you run the pipeline above you get the error:

      [...] Expected string literal but got "${axis_1_values}"
      

      Complete log:

          19: Expected string literal but got "${axis_1_values}" @ line 19, column 28.
                             values "${axis_1_values}" 
                                    ^
      
      1 error
      
          at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
          at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
          at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
          at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
          at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
          at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
          at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
          at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
          at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:761)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
          at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
          at org.jenkinsci.plugins.workflow.cps.global.UserDefinedGlobalVariable.getValue(UserDefinedGlobalVariable.java:57)
          at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
          at sun.reflect.GeneratedMethodAccessor729.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
          at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
          at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
          at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
          at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
          at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:160)
          at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
          at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
          at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
          at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
          at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
          at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
          at WorkflowScript.run(WorkflowScript:21)
          at ___cps.transform___(Native Method)
          at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
          at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
          at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
          at sun.reflect.GeneratedMethodAccessor500.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
          at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39)
          at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
          at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28)
          at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)
          at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)
          at com.cloudbees.groovy.cps.Next.step(Next.java:83)
          at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
          at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
          at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
          at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
          at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
          at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
          at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
          at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
          at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
          at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
          at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Finished: FAILURE
      

      I tried many ways for of setting the variables in the pipeline:

      // already tried the following alternatives ...
      //    def axis_1_values = 'axis_1_value_A'
      //    def axis_1_values = '''axis_1_value_A'''
      //    def axis_1_values = ["axis_1_value_A", "axis_2_value_A"]
          pipeline
          {
      [...]
      
      [...]
      axis {
          name "axis_1"
          // values "axis_1_value_A" // <- of course this works ...
          // but I want to read it from a variable
          values "${axis_1_values}" 
          // I already tried more variants
          // values ${axis_1_values}
          // values axis_1_values
      [...]
      }
      

      It would be nice if this will be possible.

          [JENKINS-62127] Dynamic Axis-Values for declarative Pipeline

          Rob McGee added a comment - - edited

          This would be very handy for us too. If an experienced jenkins dev could point us in the right direction we might be able to figure this out. We'd like to pull an axis' values, or better yet an entire matrix, from a shared library so we don't have to duplicate it everywhere.

          Just for one of our Linux Intel builds, we currently have up to 70 different pipelines that need updating anytime we make changes to our test matrix. 

          Rob McGee added a comment - - edited This would be very handy for us too. If an experienced jenkins dev could point us in the right direction we might be able to figure this out. We'd like to pull an axis' values, or better yet an entire matrix, from a shared library so we don't have to duplicate it everywhere. Just for one of our Linux Intel builds, we currently have up to 70 different pipelines that need updating anytime we make changes to our test matrix. 

          avi shaaya added a comment -

          This feature request is very important! In my use case i want to run the same pipeline all all the agents with a specific label. It cant be done with declarative pipeline 

          avi shaaya added a comment - This feature request is very important! In my use case i want to run the same pipeline all all the agents with a specific label. It cant be done with declarative pipeline 

          tharilya added a comment - - edited

          Hi bitwiseman,

          I was just looking into this and was suprised that it's still open / not even started.
          I am a software-developer for about 10 years now and just took a look into it.

          I am little confused, because I interpreted your answer on stackoverflow and here that it's kina tricky and is much effort to get this done.
          I assume the the code-lines which needs to be adapted are the overloads of the Axis-parameters which defines the allowed types.
          These are defined in the "matrix-plugin" (https://github.com/jenkinsci/matrix-project-plugin) in the "Axis.java":

          I've never really got in tough with java, but "isn't it just adding some overloads there" ?

          please correct me if I am wrong.

          kind regards

          EDIT:
          It seems linke the "other way around" could be to change the "AxisDirective" in the 'pipeline-model-definition-plugin' ( https://github.com/jenkinsci/pipeline-model-definition-plugin )

          tharilya added a comment - - edited Hi bitwiseman , I was just looking into this and was suprised that it's still open / not even started. I am a software-developer for about 10 years now and just took a look into it. I am little confused, because I interpreted your answer on stackoverflow and here that it's kina tricky and is much effort to get this done. I assume the the code-lines which needs to be adapted are the overloads of the Axis-parameters which defines the allowed types. These are defined in the "matrix-plugin" ( https://github.com/jenkinsci/matrix-project-plugin ) in the "Axis.java": https://github.com/jenkinsci/matrix-project-plugin/blob/master/src/main/java/hudson/matrix/Axis.java#L79-L89 I've never really got in tough with java, but "isn't it just adding some overloads there" ? please correct me if I am wrong. kind regards EDIT: It seems linke the "other way around" could be to change the "AxisDirective" in the 'pipeline-model-definition-plugin' ( https://github.com/jenkinsci/pipeline-model-definition-plugin ) https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/generator/AxisDirective.java

          avi shaaya added a comment -

          Does anyone has a workaround/ hack ? (I dont want to use scripted syntax to solve this)

          avi shaaya added a comment - Does anyone has a workaround/ hack ? (I dont want to use scripted syntax to solve this)

          Timo Sand added a comment -

          Timo Sand added a comment - aviss7 This seems like an okay workaround https://www.jenkins.io/blog/2019/11/22/welcome-to-the-matrix/#controlling-cell-behavior-at-runtime

          avi shaaya added a comment - - edited

          Thanks Timo,

          Its some sort of a workaround that can work for some users but still we have these limitations:

          • axis values are hard coded (no way to change them from parameter only from code)
          • This workaround allow to select only one item from the axis (no solution for running 2 items or more)

          avi shaaya added a comment - - edited Thanks Timo, Its some sort of a workaround that can work for some users but still we have these limitations: axis values are hard coded (no way to change them from parameter only from code) This workaround allow to select only one item from the axis (no solution for running 2 items or more)

          chandan added a comment -

          Hi bitwiseman ,

          I am happy to pick this up. Please guide me on how it can be done.

          chandan added a comment - Hi bitwiseman , I am happy to pick this up. Please guide me on how it can be done.

          Kyle Cronin added a comment -

          Located the chunk of code that validates the values are string literals:

          https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/4cfd8916915c861ad118c01443b0a7d5f6a74041/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/validator/ModelValidatorImpl.groovy#L823-L828

          Commenting it out allows GStrings in the value list. I don't know the full implications beyond that though. There is validation to check for duplicate values which could be violated assuming it's evaluated at a later point. Could be one issue.

          Here is where the axes are expanded to create the Matrix stages.

          https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/4cfd8916915c861ad118c01443b0a7d5f6a74041/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/RuntimeASTTransformer.groovy#L915-L932

          Would be nice to have an extension point for this directive which could open the possibility of different axes or axis value providers similar to matrix-project-plugin.

          Kyle Cronin added a comment - Located the chunk of code that validates the values are string literals: https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/4cfd8916915c861ad118c01443b0a7d5f6a74041/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/validator/ModelValidatorImpl.groovy#L823-L828 Commenting it out allows GStrings in the value list. I don't know the full implications beyond that though. There is validation to check for duplicate values which could be violated assuming it's evaluated at a later point. Could be one issue. Here is where the axes are expanded to create the Matrix stages. https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/4cfd8916915c861ad118c01443b0a7d5f6a74041/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/RuntimeASTTransformer.groovy#L915-L932 Would be nice to have an extension point for this directive which could open the possibility of different axes or axis value providers similar to matrix-project-plugin.

          David Navrkal added a comment - - edited

          Hi folks it will be soon 4 years since created this issue, do you have please any update or any estimate when this super useful feature might be implemented?

          David Navrkal added a comment - - edited Hi folks it will be soon 4 years since created this issue, do you have please any update or any estimate when this super useful feature might be implemented?

          Strangely, the "scripted" pipeline do seems to support such functionalities, as such, I fail to understand why the declarative version isn't.  It could be just the way the axis values are read, even their syntax specification seems strange:

             values 'a', 'b', 'c'

           

          Normally we should use a form that is more common in Jenkins

           

             values = { value(name: 'a'), value(name: 'b'), value(name:'c') }

          or 

             values = 'a,b,c'

          It is the first time I see a "parameters" accepting multiple value where those are not encoded within the string but within the Jenkins pipeline declarative syntax itself (the commas are not within the string but outside of it).

          Yannick Koehler added a comment - Strangely, the "scripted" pipeline do seems to support such functionalities, as such, I fail to understand why the declarative version isn't.  It could be just the way the axis values are read, even their syntax specification seems strange:    values 'a', 'b', 'c'   Normally we should use a form that is more common in Jenkins      values = { value(name: 'a'), value(name: 'b'), value(name:'c') } or     values = 'a,b,c' It is the first time I see a "parameters" accepting multiple value where those are not encoded within the string but within the Jenkins pipeline declarative syntax itself (the commas are not within the string but outside of it).

            Unassigned Unassigned
            tharilya tharilya
            Votes:
            59 Vote for this issue
            Watchers:
            59 Start watching this issue

              Created:
              Updated: