Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43911

Environment variables can't be used in agent configuration

      If you try something like

      agent {
        label "${SOME_ENV_VAR}"
      }
      environment {
        SOME_ENV_VAR = "some-label"
      }
      

      You'll get an error due to SOME_ENV_VAR not existing in the binding. Not ideal, obviously, but I'm not sure if we can actually solve this without significant rewrites.

          [JENKINS-43911] Environment variables can't be used in agent configuration

          Robby Pocase added a comment -

          kshultz It looks like your example works as long as the environment variable existed before pipeline is entered. My previous example was apparently valid, but I had a slightly different initial test case. Modifying it with your additions:

          env.AGENT_NAME = ''
          pipeline {
            agent none
            stages {
                stage('setenv') {
                    steps { script { env.AGENT_NAME = 'label' } }
                }
                stage('echo') {
                    agent { label "${env.AGENT_NAME}" }
                    steps { echo NODE_NAME }
                }
            }
          }

          From my tests, this appears to be valid. This likely isn't a good long term workaround since the goal is to be able to support most use cases through the pipeline editor. As far as I can tell, the below wouldn't be valid.

          pipeline {
            agent none
            environment {
               AGENT_NAME = ''
            }
            stages {
                stage('setenv') {
                    steps { script { env.AGENT_NAME = 'label' } }
                }
                stage('echo') {
                    agent { label "${env.AGENT_NAME}" }
                    steps { echo NODE_NAME }
                }
            }
          }

           

          Robby Pocase added a comment - kshultz It looks like your example works as long as the environment variable existed before pipeline is entered. My previous example was apparently valid, but I had a slightly different initial test case. Modifying it with your additions: env.AGENT_NAME = '' pipeline {   agent none   stages {       stage( 'setenv' ) {           steps { script { env.AGENT_NAME = 'label' } }       }       stage( 'echo' ) {           agent { label "${env.AGENT_NAME}" }           steps { echo NODE_NAME }       }   } } From my tests, this appears to be valid. This likely isn't a good long term workaround since the goal is to be able to support most use cases through the pipeline editor. As far as I can tell, the below wouldn't be valid. pipeline {   agent none environment { AGENT_NAME = '' }   stages {       stage( 'setenv' ) {           steps { script { env.AGENT_NAME = 'label' } }       }       stage( 'echo' ) {           agent { label "${env.AGENT_NAME}" }           steps { echo NODE_NAME }       }   } }  

          Robby Pocase added a comment -

          After getting some runtime on my previous test, it looks like it does not work as expected. As far as I can tell, the env.AGENT_NAME is loaded at pipeline parse time and isn't taking the update into account. So the end result is more along the lines of:

          agent { label '' }
          

          This appears to be equivalent to agent any. Using a regular groovy variable in place of env also doesn't work.

          Robby Pocase added a comment - After getting some runtime on my previous test, it looks like it does not work as expected. As far as I can tell, the env.AGENT_NAME is loaded at pipeline parse time and isn't taking the update into account. So the end result is more along the lines of: agent { label '' } This appears to be equivalent to agent any. Using a regular groovy variable in place of env also doesn't work.

          Robert Hales added a comment -

          I looked at this issue recently as well. For completeness, I wanted to add my findings here for others who stumble on this JIRA later.

          As Robby said, the agent expression is evaluated when the pipeline is parsed. I used a GString to run a println in my agent name to see in the console output when it was running: 

          {{}}

          agentName = "Windows" 
          agentLabel = "${println 'Right Now the Agent Name is ' + agentName; return agentName}" 
          
          pipeline { 
              agent none 
              stages { 
                  stage('Prep') { 
                      steps { 
                          script { 
                              agentName = "Linux" 
                          } 
                      } 
                  } 
                  stage('Final') { 
                      agent { label agentLabel } 
             
                      steps { 
                          script { 
                              println agentLabel 
                              println agentName 
                          } 
                      } 
                  } 
              }
          }

           

          The console output will show the agent label being evaluated before the first stage runs. 

          The only way I know to workaround this is to use a node{} in the stage. This seems like an acceptable way to handle this. I think if you the pipeline is getting to the level of complexity that requires dynamically determining agent, you may be beyond the a declarative pipeline and should consider scripted. 

          Robert Hales added a comment - I looked at this issue recently as well. For completeness, I wanted to add my findings here for others who stumble on this JIRA later. As Robby said, the agent expression is evaluated when the pipeline is parsed. I used a GString to run a println in my agent name to see in the console output when it was running:  {{}} agentName = "Windows" agentLabel = "${println 'Right Now the Agent Name is ' + agentName; return agentName}" pipeline { agent none stages { stage( 'Prep' ) { steps { script { agentName = "Linux" } } } stage( 'Final' ) { agent { label agentLabel } steps { script { println agentLabel println agentName } } } } }   The console output will show the agent label being evaluated before the first stage runs.  The only way I know to workaround this is to use a node{} in the stage. This seems like an acceptable way to handle this. I think if you the pipeline is getting to the level of complexity that requires dynamically determining agent, you may be beyond the a declarative pipeline and should consider scripted. 

          Caleb Mayeux added a comment -

          My use case is that I have a pipeline, and part of the way through it I check out a git repo, and set the latest git commit hash to a variable which is used to tag and push a docker image. Later on I have a stage that uses that image, and thus I need the hash to run the stage using an agent based on that image. The only workaround to do this is to take the stage that checks out the code out of the declarative pipeline and do it before. If there were other steps that had to happen before that, I'd have to take them out of the pipeline too, so at that point I'd be basically just not using declarative pipeline because of this issue.

           

          I think being able to dynamically assign agents based on the pipeline is an important and powerful ability, and is something that exists in both scripted pipeline and freestyle jobs (you can assign node/label as a parameter for a freestyle job, and a pipeline of freestyle jobs effectively makes each job the equivalent of a "stage" in pipeline terminology).

          Caleb Mayeux added a comment - My use case is that I have a pipeline, and part of the way through it I check out a git repo, and set the latest git commit hash to a variable which is used to tag and push a docker image. Later on I have a stage that uses that image, and thus I need the hash to run the stage using an agent based on that image. The only workaround to do this is to take the stage that checks out the code out of the declarative pipeline and do it before. If there were other steps that had to happen before that, I'd have to take them out of the pipeline too, so at that point I'd be basically just not using declarative pipeline because of this issue.   I think being able to dynamically assign agents based on the pipeline is an important and powerful ability, and is something that exists in both scripted pipeline and freestyle jobs (you can assign node/label as a parameter for a freestyle job, and a pipeline of freestyle jobs effectively makes each job the equivalent of a "stage" in pipeline terminology).

          Allan Lewis added a comment -

          I have a very similar use case to buzzard44.
          This has been in progress for quite a while, can we please have a status update?

          Allan Lewis added a comment - I have a very similar use case to buzzard44 . This has been in progress for quite a while, can we please have a status update?

          Allan Lewis added a comment -

          I finally found a workaround for this:

          registry = 'my-registry'
          imageRepo = 'my-repo-name'
          
          def getImageName(def doCheckout = false) {
            def treeHash = node {
              if (doCheckout) checkout scm
              sh(script: 'git rev-parse HEAD:', returnStdout: true).trim()
            }
            "$registry/$imageRepo:$treeHash"
          }
          
          pipeline {
            agent {
              label 'docker-builder'
            }
          
            stages {
              stage('Build & Push Docker Image') {
                steps {
                  script {
                    docker.build(getImageName()).push()
                  }
                }
              }
              stage('Do Stuff') {
                agent {
                  docker {
                    image getImageName(true)
                  }
                }
                steps {
                  sh 'echo hello'
                }
              }
            }
          }
          

          Allan Lewis added a comment - I finally found a workaround for this: registry = 'my-registry' imageRepo = 'my-repo-name' def getImageName( def doCheckout = false ) { def treeHash = node { if (doCheckout) checkout scm sh(script: 'git rev-parse HEAD:' , returnStdout: true ).trim() } "$registry/$imageRepo:$treeHash" } pipeline { agent { label 'docker-builder' } stages { stage( 'Build & Push Docker Image' ) { steps { script { docker.build(getImageName()).push() } } } stage( 'Do Stuff' ) { agent { docker { image getImageName( true ) } } steps { sh 'echo hello' } } } }

          efo plo added a comment -

          Would also like to have this implemented.

           

          We need to dynamically assign a label based on branch, so that branches of Team A run on the resources of that team, etc. We contact Consul for the mapping of branch to label. Currently this is not possible as the label expression is evaluated before the pipeline is run, not at the beginning of the relevant step, so any changes to variables or environment do not affect the node where the step is run.

          efo plo added a comment - Would also like to have this implemented.   We need to dynamically assign a label based on branch, so that branches of Team A run on the resources of that team, etc. We contact Consul for the mapping of branch to label. Currently this is not possible as the label expression is evaluated before the pipeline is run, not at the beginning of the relevant step, so any changes to variables or environment do not affect the node where the step is run.

          Jack P added a comment -

           I would like to bump interest in this as well recently got diverted here from posting a [similar question on stack overflow|https://stackoverflow.com/questions/55267427/how-can-i-use-different-private-docker-agents-based-on-parameter-in-jenkins-decl.]

           

          The solution seems to mean restarting the same environment between stages which feels a bit off and results in me having to merge stages that would be nice to keep separate.

          Jack P added a comment -  I would like to bump interest in this as well recently got diverted here from posting a [similar question on stack overflow| https://stackoverflow.com/questions/55267427/how-can-i-use-different-private-docker-agents-based-on-parameter-in-jenkins-decl .]   The solution seems to mean restarting the same environment between stages which feels a bit off and results in me having to merge stages that would be nice to keep separate.

          efo plo added a comment -

          datajack We worked around this limitation by implementing a scripted pipeline before declarative:

          node('master') {
              stage('Choose Label') {
                  LABEL = 'my_desired_label' // script it anyway you want
              }
          }
          
          
          pipeline {
              agent {
                  node {
                      label "${LABEL}"
                  }
              }
          // etc.
          }
          

          efo plo added a comment - datajack We worked around this limitation by implementing a scripted pipeline before declarative: node( 'master' ) { stage( 'Choose Label' ) { LABEL = 'my_desired_label' // script it anyway you want } } pipeline { agent { node { label "${LABEL}" } } // etc. }

          Jack P added a comment - - edited

          eplodn1 Thank you! I am eternally greatful. Totally works with docker and azure storage plugin, here's my stackoverflow example fixed as per yours.

           
           

          node('master') { 
           stage('Choose Label') { 
            URL_VAR = "${env.registrySelection == "PROD" ? "urlProd.azure.io" : "urlTest.azure.io"}" 
            CREDS_VAR = "${env.registrySelection == "PROD" ? "credsProd" : "credsTest"}" 
           } 
          } 
          pipeline { 
            parameters { 
              choice( 
              name: 'registrySelection', 
              choices: ['TEST', 'PROD'], 
              description: 'Is this a deployment to STAGING or PRODUCTION environment?' ) 
            } 
            agent { 
              docker { 
                image "${URL_VAR}/image:tag" 
                registryUrl "https://${URL_VAR}" registryCredentialsId "${CREDS_VAR}" 
              } 
            } 
            stages{ 
              stage('test'){ 
                steps{ 
                  echo "${URL_VAR}" 
                  echo "${CREDS_VAR}" 
                } 
              } 
            } 
          }

           

          Jack P added a comment - - edited eplodn1 Thank you! I am eternally greatful. Totally works with docker and azure storage plugin, here's my stackoverflow example fixed as per yours.     node( 'master' ) { stage( 'Choose Label' ) { URL_VAR = "${env.registrySelection == " PROD " ? " urlProd.azure.io " : " urlTest.azure.io "}" CREDS_VAR = "${env.registrySelection == " PROD " ? " credsProd " : " credsTest "}" } } pipeline { parameters { choice( name: 'registrySelection' , choices: [ 'TEST' , 'PROD' ], description: 'Is this a deployment to STAGING or PRODUCTION environment?' ) } agent { docker { image "${URL_VAR}/image:tag" registryUrl "https: //${URL_VAR}" registryCredentialsId "${CREDS_VAR}" } } stages{ stage( 'test' ){ steps{ echo "${URL_VAR}" echo "${CREDS_VAR}" } } } }  

            Unassigned Unassigned
            abayer Andrew Bayer
            Votes:
            22 Vote for this issue
            Watchers:
            32 Start watching this issue

              Created:
              Updated: