[JENKINS-62127] Dynamic Axis-Values for declarative Pipeline - Jenkins Jira

XML

Word

Printable

Details

Type: New Feature
Status: Open (View Workflow)
Priority: Critical
Resolution: Unresolved
Component/s: matrix-combinations-parameter-plugin, matrix-project-plugin, pipeline, pipeline-model-definition-plugin
Labels:
None
Environment:

Hide
System-Setup:
My System: Jenkins-Version: Jenkins ver. 2.190.1
Declarative-Plugin-Version 1.5.0
Declarative Agent API: 1.1.1
Declarative Extension Points API: 1.5.0
(many other plugins, I think this are the important ones)

The answer of the stackoverflow post say that it is currently not possible, even with the current version of the plugins.

Show
System-Setup: My System: Jenkins-Version: Jenkins ver. 2.190.1 Declarative-Plugin-Version 1.5.0 Declarative Agent API: 1.1.1 Declarative Extension Points API: 1.5.0 (many other plugins, I think this are the important ones) The answer of the stackoverflow post say that it is currently not possible, even with the current version of the plugins.

Similar Issues:

Show

Description

Hi there,

I was asked to open a Ticket here, because it seems like the feature is interesting for many peoples.

Stack Overflow - Declarative Pipeline with dynamic matrix axis values

It would be nice to have the possibility to set the dynamically set the values of the "axis".

Use-Case:
I want to compile multiple plugins with the same Jenkins-pipeline. To do so I wrote a "JenkinsLibrary" with an interface the plugins can use for parameters. One of this parameters is the axis-value for compiler.

e.g. I want to build Plugin "X" for gcc7 on armv8 and Plugin Y for vc15 x64.
So I have a Jenkins-Library with the logic of the build parameters.
And each plugin calls the library with the build-parameters. (e.g. compiler and arch )

Example:
a "JenkinsLibrary" with a file my_library.groovy

def call(Map i_options)
{
    def axis_1_values = "axis_1_value_A"
    // later: def axis_1_values = i_options.get("axis_1_values")
    pipeline
    {
        agent any
        stages { stage("stage A") { matrix {
            axes {
                axis {
                    name "axis_1"
                    values "${axis_1_values}" 
                }
                axis {
                    name "axis_2"
                    values "axis_2_value_A", "axis_2_value_B"
                }
            }
            stages {
                stage("another stage") { steps {
                    echo "hello world from ${axis_1} && ${axis_2}"
                } }
            }
        } } }
    }
}

and each plugin contains a Jenkinsfile like that:

@Library("JenkinsLibrary") _

def options = [
    axis_1_values: "a_axis_value"
]

my_library(options)

We want to keep the Jenkinsfiles of the plugins as short as possible to keep the maintenance as low as possible.

If you run the pipeline above you get the error:

[...] Expected string literal but got "${axis_1_values}"

Complete log:

    19: Expected string literal but got "${axis_1_values}" @ line 19, column 28.
                       values "${axis_1_values}" 
                              ^

1 error

    at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
    at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
    at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
    at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
    at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
    at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
    at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:761)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
    at org.jenkinsci.plugins.workflow.cps.global.UserDefinedGlobalVariable.getValue(UserDefinedGlobalVariable.java:57)
    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
    at sun.reflect.GeneratedMethodAccessor729.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:160)
    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
    at WorkflowScript.run(WorkflowScript:21)
    at ___cps.transform___(Native Method)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
    at sun.reflect.GeneratedMethodAccessor500.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39)
    at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
    at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28)
    at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)
    at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

I tried many ways for of setting the variables in the pipeline:

// already tried the following alternatives ...
//    def axis_1_values = 'axis_1_value_A'
//    def axis_1_values = '''axis_1_value_A'''
//    def axis_1_values = ["axis_1_value_A", "axis_2_value_A"]
    pipeline
    {
[...]

[...]
axis {
    name "axis_1"
    // values "axis_1_value_A" // <- of course this works ...
    // but I want to read it from a variable
    values "${axis_1_values}" 
    // I already tried more variants
    // values ${axis_1_values}
    // values axis_1_values
[...]
}

It would be nice if this will be possible.

Attachments

Activity

Ascending order - Click to sort in descending order

View 8 older comments

avi shaaya added a comment - 2022-11-17 19:24

This feature request is very important! In my use case i want to run the same pipeline all all the agents with a specific label. It cant be done with declarative pipeline

avi shaaya added a comment - 2022-11-17 19:24 This feature request is very important! In my use case i want to run the same pipeline all all the agents with a specific label. It cant be done with declarative pipeline

tharilya added a comment - 2022-12-07 10:21 - edited

Hi bitwiseman,

I was just looking into this and was suprised that it's still open / not even started.
I am a software-developer for about 10 years now and just took a look into it.

I am little confused, because I interpreted your answer on stackoverflow and here that it's kina tricky and is much effort to get this done.
I assume the the code-lines which needs to be adapted are the overloads of the Axis-parameters which defines the allowed types.
These are defined in the "matrix-plugin" (https://github.com/jenkinsci/matrix-project-plugin) in the "Axis.java":

https://github.com/jenkinsci/matrix-project-plugin/blob/master/src/main/java/hudson/matrix/Axis.java#L79-L89

I've never really got in tough with java, but "isn't it just adding some overloads there" ?

please correct me if I am wrong.

kind regards

EDIT:
It seems linke the "other way around" could be to change the "AxisDirective" in the 'pipeline-model-definition-plugin' ( https://github.com/jenkinsci/pipeline-model-definition-plugin )

https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/generator/AxisDirective.java

tharilya added a comment - 2022-12-07 10:21 - edited Hi bitwiseman , I was just looking into this and was suprised that it's still open / not even started. I am a software-developer for about 10 years now and just took a look into it. I am little confused, because I interpreted your answer on stackoverflow and here that it's kina tricky and is much effort to get this done. I assume the the code-lines which needs to be adapted are the overloads of the Axis-parameters which defines the allowed types. These are defined in the "matrix-plugin" ( https://github.com/jenkinsci/matrix-project-plugin ) in the "Axis.java": https://github.com/jenkinsci/matrix-project-plugin/blob/master/src/main/java/hudson/matrix/Axis.java#L79-L89 I've never really got in tough with java, but "isn't it just adding some overloads there" ? please correct me if I am wrong. kind regards EDIT: It seems linke the "other way around" could be to change the "AxisDirective" in the 'pipeline-model-definition-plugin' ( https://github.com/jenkinsci/pipeline-model-definition-plugin ) https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/master/pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/generator/AxisDirective.java

avi shaaya added a comment - 2022-12-07 12:21

Does anyone has a workaround/ hack ? (I dont want to use scripted syntax to solve this)

avi shaaya added a comment - 2022-12-07 12:21 Does anyone has a workaround/ hack ? (I dont want to use scripted syntax to solve this)

Timo Sand added a comment - 2023-04-13 18:37

aviss7 This seems like an okay workaround https://www.jenkins.io/blog/2019/11/22/welcome-to-the-matrix/#controlling-cell-behavior-at-runtime

Timo Sand added a comment - 2023-04-13 18:37 aviss7 This seems like an okay workaround https://www.jenkins.io/blog/2019/11/22/welcome-to-the-matrix/#controlling-cell-behavior-at-runtime

avi shaaya added a comment - 2023-04-14 08:25 - edited

Thanks Timo,

Its some sort of a workaround that can work for some users but still we have these limitations:

axis values are hard coded (no way to change them from parameter only from code)
This workaround allow to select only one item from the axis (no solution for running 2 items or more)

avi shaaya added a comment - 2023-04-14 08:25 - edited Thanks Timo, Its some sort of a workaround that can work for some users but still we have these limitations: axis values are hard coded (no way to change them from parameter only from code) This workaround allow to select only one item from the axis (no solution for running 2 items or more)

People

Assignee:: Liam Newman

Reporter:: tharilya

Votes:: 43 Vote for this issue

Watchers:: 43 Start watching this issue

Dates

Created:: 2020-04-30 16:06

Updated:: 2023-05-16 10:20