Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-11577

Matrix axis are not recognixed as parameter

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We want to want to create a matrix job which triggers the current another job with the current matrix parameters.

      Concrete example: Our setup creation consists of two steps. The first job (prepare-setup) is to collect all the files and potentially encrypt them, the second step (setup) is to create a setup.exe. prepare-setup is a matix build with the ecryption option as its values (encrypted, unencrypted). The second job should be triggered with the encryption option as its parameter, so that we have an encryted setup (for the customers) and an unencrypted (for internal tests and debugging)

      Currenty the setup job is triggered twice, both with the value "$encryption" for the encryption parameter. (see screenshot)

        Attachments

        1. jobs.zip
          41 kB
        2. nonparam.png
          nonparam.png
          46 kB

          Activity

          jotbepunkt Jens Baitinger created issue -
          Hide
          jotbepunkt Jens Baitinger added a comment -

          As a workaround you can write the matrix configuration to a property file (via batch or shell script) and trigger the other job using that property file.

          Show
          jotbepunkt Jens Baitinger added a comment - As a workaround you can write the matrix configuration to a property file (via batch or shell script) and trigger the other job using that property file.
          Hide
          vladichko Vlad Aginsky added a comment -

          Hi,
          Is there any progress on this?

          Show
          vladichko Vlad Aginsky added a comment - Hi, Is there any progress on this?
          Hide
          cjo9900 cjo9900 added a comment -

          When using the predefined Parameters option the variables are expanded using the following

          EnvVars env = build.getEnvironment(listener);
          Properties p = new Properties();
          p.load(new StringInputStream(properties));

          List<ParameterValue> values = new ArrayList<ParameterValue>();
          for (Map.Entry<Object, Object> entry : p.entrySet()) {
          values.add(new StringParameterValue(entry.getKey().toString(), env.expand(entry.getValue().toString())));
          }

          So the issues looks to be that the axis are not added to the EnvVars returned by build.getEnvironment(),
          as they are added in the MatrixRun class in the getBuildVariables() function.

          public Map<String,String> getBuildVariables() {
          Map<String,String> r = super.getBuildVariables();
          // pick up user axes
          AxisList axes = getParent().getParent().getAxes();
          for (Map.Entry<String,String> e : getParent().getCombination().entrySet())

          { Axis a = axes.find(e.getKey()); if (a!=null) a.addBuildVariable(e.getValue(),r); else r.put(e.getKey(), e.getValue()); }

          return r;
          }

          so the predefined parameters also needs to Expand the build variables.

          Show
          cjo9900 cjo9900 added a comment - When using the predefined Parameters option the variables are expanded using the following EnvVars env = build.getEnvironment(listener); Properties p = new Properties(); p.load(new StringInputStream(properties)); List<ParameterValue> values = new ArrayList<ParameterValue>(); for (Map.Entry<Object, Object> entry : p.entrySet()) { values.add(new StringParameterValue(entry.getKey().toString(), env.expand(entry.getValue().toString()))); } So the issues looks to be that the axis are not added to the EnvVars returned by build.getEnvironment(), as they are added in the MatrixRun class in the getBuildVariables() function. public Map<String,String> getBuildVariables() { Map<String,String> r = super.getBuildVariables(); // pick up user axes AxisList axes = getParent().getParent().getAxes(); for (Map.Entry<String,String> e : getParent().getCombination().entrySet()) { Axis a = axes.find(e.getKey()); if (a!=null) a.addBuildVariable(e.getValue(),r); else r.put(e.getKey(), e.getValue()); } return r; } so the predefined parameters also needs to Expand the build variables.
          Hide
          cjo9900 cjo9900 added a comment -

          Actually the issue is that the Dependancy is connected between the Matrix project and the and the downstream project, so when the Dependancy is called to contribute the parameters this variable is not present.

          Show
          cjo9900 cjo9900 added a comment - Actually the issue is that the Dependancy is connected between the Matrix project and the and the downstream project, so when the Dependancy is called to contribute the parameters this variable is not present.
          Hide
          cjo9900 cjo9900 added a comment -

          Axis are not added as Environment variables, just as buildVariables and therefore are not present when calling getEnviroment from the build trigger.

          Code to add these is at https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/matrix/MatrixRun.java#L114
          but does not add the to the Environment.

          Show
          cjo9900 cjo9900 added a comment - Axis are not added as Environment variables, just as buildVariables and therefore are not present when calling getEnviroment from the build trigger. Code to add these is at https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/matrix/MatrixRun.java#L114 but does not add the to the Environment.
          Hide
          cjo9900 cjo9900 added a comment -

          Currently the AXIS parameters are not added the environment that the parameterized trigger uses in the trigger build step.

          They are missing when the plugin does build.getEnvironment()[0] as MatrixRun does not add them to this, so defaults to the ones added in AbstractBuild [1]

          But they are added to the build.getBuildVariables(), as the are added in the MatrixRun [2]

          So the solution might be to add these via a EnvironmentContributor [3] in the MatrixRun class.

          [0] https://github.com/jenkinsci/parameterized-trigger-plugin/blob/c04e84770dfab3459f01047be95888ecb353cbbd/src/main/java/hudson/plugins/parameterizedtrigger/BuildTrigger.java#L59
          [1] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/AbstractBuild.java#L935
          [2] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/matrix/MatrixRun.java#L114
          [3] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/EnvironmentContributor.java

          Show
          cjo9900 cjo9900 added a comment - Currently the AXIS parameters are not added the environment that the parameterized trigger uses in the trigger build step. They are missing when the plugin does build.getEnvironment() [0] as MatrixRun does not add them to this, so defaults to the ones added in AbstractBuild [1] But they are added to the build.getBuildVariables(), as the are added in the MatrixRun [2] So the solution might be to add these via a EnvironmentContributor [3] in the MatrixRun class. [0] https://github.com/jenkinsci/parameterized-trigger-plugin/blob/c04e84770dfab3459f01047be95888ecb353cbbd/src/main/java/hudson/plugins/parameterizedtrigger/BuildTrigger.java#L59 [1] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/AbstractBuild.java#L935 [2] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/matrix/MatrixRun.java#L114 [3] https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/EnvironmentContributor.java
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/matrix/MatrixConfiguration.java
          http://jenkins-ci.org/commit/jenkins/a6dddf2917c1d9ae819f86947a7ccffa2a04d728
          Log:
          [FIXED JENKINS-11577]

          Traditionally, matrix axis values are only exposed as "build variables", and it was up to individual build steps and others to treat them equally like environment variables.

          However, in practice, this distinction between environment variables vs build variables didn't serve any useful purposes, and the down side (of some plugins only expanding env vars in the ${VAR} notation and not build variables) was probably bigger than whatever benefit this distinction was supposed to bring.

          In this fix, we are exposing matrix axis values also as environment variables. This will make them recognizable from plugins that only expand environment variables, such as parameterized trigger plugins as indicated in the original bug report.

          This was motivated by pull request #701 but fix was made differently.


          You received this message because you are subscribed to the Google Groups "Jenkins Commits" group.
          To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@googlegroups.com.
          For more options, visit https://groups.google.com/groups/opt_out.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/matrix/MatrixConfiguration.java http://jenkins-ci.org/commit/jenkins/a6dddf2917c1d9ae819f86947a7ccffa2a04d728 Log: [FIXED JENKINS-11577] Traditionally, matrix axis values are only exposed as "build variables", and it was up to individual build steps and others to treat them equally like environment variables. However, in practice, this distinction between environment variables vs build variables didn't serve any useful purposes, and the down side (of some plugins only expanding env vars in the ${VAR} notation and not build variables) was probably bigger than whatever benefit this distinction was supposed to bring. In this fix, we are exposing matrix axis values also as environment variables. This will make them recognizable from plugins that only expand environment variables, such as parameterized trigger plugins as indicated in the original bug report. This was motivated by pull request #701 but fix was made differently. – You received this message because you are subscribed to the Google Groups "Jenkins Commits" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out .
          scm_issue_link SCM/JIRA link daemon made changes -
          Field Original Value New Value
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          Hide
          dogfood dogfood added a comment -

          Integrated in jenkins_main_trunk #2282
          [FIXED JENKINS-11577] (Revision a6dddf2917c1d9ae819f86947a7ccffa2a04d728)

          Result = SUCCESS
          kohsuke : a6dddf2917c1d9ae819f86947a7ccffa2a04d728
          Files :

          • core/src/main/java/hudson/matrix/MatrixConfiguration.java
          • changelog.html
          Show
          dogfood dogfood added a comment - Integrated in jenkins_main_trunk #2282 [FIXED JENKINS-11577] (Revision a6dddf2917c1d9ae819f86947a7ccffa2a04d728) Result = SUCCESS kohsuke : a6dddf2917c1d9ae819f86947a7ccffa2a04d728 Files : core/src/main/java/hudson/matrix/MatrixConfiguration.java changelog.html
          Hide
          tommigun Tommi Kiviniemi added a comment - - edited

          I can not access axes before SCM sync. I have a multi-configuration job where I want to define a custom workspace for the master's checkout and reuse that for the child (child workspace "."). Setting custom workspace to $BRANCH_NAME_$PLATFORM_$BUILD_TYPE results in a workspace called "$BRANCH_NAME_$PLATFORM_$BUILD_TYPE". Trying to access the axes in "Prepare an environment for this run" can not access them yet either, nor via properties or a Groovy script. Right now there is no way for me to construct the workspace name due to this. Could you please make the axes available already at that point?

          Show
          tommigun Tommi Kiviniemi added a comment - - edited I can not access axes before SCM sync. I have a multi-configuration job where I want to define a custom workspace for the master's checkout and reuse that for the child (child workspace "."). Setting custom workspace to $BRANCH_NAME_$PLATFORM_$BUILD_TYPE results in a workspace called "$BRANCH_NAME_$PLATFORM_$BUILD_TYPE". Trying to access the axes in "Prepare an environment for this run" can not access them yet either, nor via properties or a Groovy script. Right now there is no way for me to construct the workspace name due to this. Could you please make the axes available already at that point?
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 141820 ] JNJira + In-Review [ 189832 ]

            People

            Assignee:
            huybrechts huybrechts
            Reporter:
            jotbepunkt Jens Baitinger
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: