Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45916

Path is not getting set correctly in pipeline when there is a variable present

    XMLWordPrintable

Details

    Description

      After upgrading to latest Pipeline version and it's dependencies (we also upgraded the blue ocean plugins but don't think that caused an issue here), we noticed that the PATH was not getting set in our Jenkins Pipeline Jobs. After debugging found that if I set PATH to a string inside the "environment" section, as such:

       

          environment

      {         PATH="/a/new/bin:$PATH"    }

       

      The PATH was updated successfully without any issues. However if I introduce a variable, the path does not get updated, and defaults to what is on the agent that runs the job. Example:

      pipeline

      {       agent

      {  label 'docker'    }

          environment {
              PATH="/a/new/bin:${env.SCM_HOME}:$PATH"
              SCM_HOME="${env.SCM_HOME}"
         }
        stages {
          stage ('build') {
            steps

      {         echo "PATH=$PATH"         echo "SCM_HOME=$SCM_HOME"       }

          }
        }
      }

       

      OUTPUT:

       
      [Pipeline] {

      [Pipeline] withEnv

      [Pipeline] {

      [Pipeline] stage

      [Pipeline] { (build)

      [Pipeline] echoPATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin

      [Pipeline] echoSCM_HOME=/usr/local/bin/scmtools/

      [Pipeline] }

      [Pipeline] // stage

      [Pipeline] }

      [Pipeline] // withEnv

      [Pipeline] }

      [Pipeline] // node

      [Pipeline] End of PipelineFinished: SUCCESS
       

       

      Notice that SCM_HOME resolves fine outside of PATH and that /a/new/bin was not added to the path. However if I drop the variable it will set PATH fine.

      Attachments

        Issue Links

          Activity

            What does that mean for the mere mortals?  Is "steps" or "sh" the thing to avoid?

            ilatypov Ilguiz Latypov added a comment - What does that mean for the mere mortals?  Is "steps" or "sh" the thing to avoid?

            Never mind, I see the magic recipe.  I still don't know the exact difference between "declarative" and "scripting", but I am mortal as well....

            the PATH+EXTRA system works for Scripted Pipeline. If it does not work for Declarative, please file a separate RFE in pipeline-model-definition-plugin. Workaround would I guess be something like (untested)

            environment {
              STUFF = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin"
            }
            // …
            steps {
              withEnv(["PATH+EXTRA=$STUFF"]) {
                sh 'whatever'
              }
            }
            

            https://issues.jenkins-ci.org/browse/JENKINS-41339?focusedCommentId=357645&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-357645

             

            ilatypov Ilguiz Latypov added a comment - Never mind, I see the magic recipe.  I still don't know the exact difference between "declarative" and "scripting", but I am mortal as well.... the PATH+EXTRA system works for Scripted Pipeline. If it does not work for Declarative, please file a separate RFE in pipeline-model-definition-plugin . Workaround would I guess be something like (untested) environment { STUFF = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin" } // … steps { withEnv([ "PATH+EXTRA=$STUFF" ]) { sh 'whatever' } } https://issues.jenkins-ci.org/browse/JENKINS-41339?focusedCommentId=357645&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-357645  
            ilatypov Ilguiz Latypov added a comment - - edited

            I figured the original issue appears fixed, and I am just confirming this for Jenkins 2.226 and current plugins. Sorry I just noticed Andrew Bayer's fix in this ticket.

            pipeline {
                agent {
                    label 'linux'
                }
            
                environment {
                    PATH="/a/new/bin:$PATH"
               }
               stages {
                    stage ('build') {
                        steps {
                            echo "PATH=$PATH"
                            echo "env.PATH=${env.PATH}"
                            sh "export -p | grep PATH"
                        }
                    }
                }
            }
            
             Running in Durability level: MAX_SURVIVABILITY
             [Pipeline] Start of Pipeline
             [Pipeline] node
             Running on Jenkins in /var/jenkins_home/workspace/System/Test_JENKINS-45916
             [Pipeline] {
             [Pipeline] withEnv
             [Pipeline] {
             [Pipeline] stage
             [Pipeline] { (build)
             [Pipeline] echo
             PATH=/a/new/bin:/usr/local/nvm/versions/node/v10.15.0/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
             [Pipeline] echo
             env.PATH=/a/new/bin:/usr/local/nvm/versions/node/v10.15.0/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
             [Pipeline] sh
             + export -p
             + grep PATH
             declare -x GEM_PATH="file:/var/jenkins_home/plugins/ruby-runtime/WEB-INF/lib/stapler-jruby-1.209.jar!/gem/"
             declare -x PATH="/a/new/bin:/usr/local/nvm/versions/node/v10.15.0/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
             [Pipeline] }
             [Pipeline] // stage
             [Pipeline] }
             [Pipeline] // withEnv
             [Pipeline] }
             [Pipeline] // node
             [Pipeline] End of Pipeline
             Finished: SUCCESS
            
            ilatypov Ilguiz Latypov added a comment - - edited I figured the original issue appears fixed, and I am just confirming this for Jenkins 2.226 and current plugins. Sorry I just noticed Andrew Bayer's fix in this ticket. pipeline { agent { label 'linux' } environment { PATH= "/a/ new /bin:$PATH" } stages { stage ( 'build' ) { steps { echo "PATH=$PATH" echo "env.PATH=${env.PATH}" sh "export -p | grep PATH" } } } } Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in /var/jenkins_home/workspace/System/Test_JENKINS-45916 [Pipeline] { [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (build) [Pipeline] echo PATH=/a/new/bin:/usr/local/nvm/versions/node/v10.15.0/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin [Pipeline] echo env.PATH=/a/new/bin:/usr/local/nvm/versions/node/v10.15.0/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin [Pipeline] sh + export -p + grep PATH declare -x GEM_PATH="file:/var/jenkins_home/plugins/ruby-runtime/WEB-INF/lib/stapler-jruby-1.209.jar!/gem/" declare -x PATH="/a/new/bin:/usr/local/nvm/versions/node/v10.15.0/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS
            ilatypov Ilguiz Latypov added a comment - - edited

            The environment clause appears to be ignored by plugins executing code such as the following,

            EnvVars vars = build.getEnvironment(listener); 
            

            My current work-around is to define hard-coded default values for the pipeline "parameters", even for the agent "master" which runs on the same machine with the server. (If Jenkins server needs to run its own checkout of, say, a pipeline library, then it will fail to use Git on the server using the agent-targeting PATH).

                agent {
                    label 'master'
                }
                
                parameters {
                    string(name: 'FORTIFY_HOME', defaultValue: "/MYTOOLDIR/fortify", 
                        description: 'A work-around to the plugin using FORTIFY_HOME of the Jenkins server instead of that of the agent')
                    string(name: 'PATH', defaultValue: 
                        "/MYTOOLDIR/fortify/bin:/usr/local/FOO/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
                        description: 'A work-around to the plugin using PATH of the Jenkins server instead of that of the agent')
                }
            

            I wish Jenkins in general its plugins in particular gave precendence to the agent's environment. I did not try to find the exact API to fetch that in the context of the plugin running on the server but aiming the given "agent".

            ilatypov Ilguiz Latypov added a comment - - edited The environment clause appears to be ignored by plugins executing code such as the following, EnvVars vars = build.getEnvironment(listener); My current work-around is to define hard-coded default values for the pipeline "parameters", even for the agent "master" which runs on the same machine with the server. (If Jenkins server needs to run its own checkout of, say, a pipeline library, then it will fail to use Git on the server using the agent-targeting PATH). agent { label 'master' } parameters { string(name: 'FORTIFY_HOME' , defaultValue: "/MYTOOLDIR/fortify" , description: 'A work-around to the plugin using FORTIFY_HOME of the Jenkins server instead of that of the agent' ) string(name: 'PATH' , defaultValue: "/MYTOOLDIR/fortify/bin:/usr/local/FOO/bin:/usr/local/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" , description: 'A work-around to the plugin using PATH of the Jenkins server instead of that of the agent' ) } I wish Jenkins in general its plugins in particular gave precendence to the agent's environment. I did not try to find the exact API to fetch that in the context of the plugin running on the server but aiming the given "agent".
            citizenkahn citizenkahn added a comment -

            Folks, I didn't notice this workaround but it does seem viable.  I use script vs declarative so you might need to tweak this a bit.

             

            Yes, withEnv will not work within the inside closure.  Normally, the path variable will not be passed via a docker ENV method as that would usually be a bad idea given host OS and image OS are likely not the same.

             

            So, if we know what the images path is, then we can augment the path via the docker run args field `.inside(...)` to achieve the end of path modification.   It does mean that I must know or obtain the original path from the image.  I own the image and the pipeline code so I can easily do that.  If not, then I'd run an ephemeral docker container, and get the path from that via shell + returnStdout

             

            // hardcoded because I own the image or have determined what the image's path normally is
            def knownOriginalPath="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            
            // my change to the path - in this case, a custom tool location used because I don't want to have to modify a bunch of existing pipeline code.
            def pathModification="/ansible2.10/bin"
            def desiredPath="${pathModification}:${knownOriginalPath}"
            
            
                docker.withRegistry(...) {
                   // for now, the plugin seems to respect the integrity of the inside option string.
                    docker.image(..)inside("-e PATH=${desiredPath}") {
                        sh 'echo PATH = $PATH'        
                    }
                } 
            citizenkahn citizenkahn added a comment - Folks, I didn't notice this workaround but it does seem viable.  I use script vs declarative so you might need to tweak this a bit.   Yes, withEnv will not work within the inside closure.  Normally, the path variable will not be passed via a docker ENV method as that would usually be a bad idea given host OS and image OS are likely not the same.   So, if we know what the images path is, then we can augment the path via the docker run args field `.inside(...)` to achieve the end of path modification.   It does mean that I must know or obtain the original path from the image.  I own the image and the pipeline code so I can easily do that.  If not, then I'd run an ephemeral docker container, and get the path from that via shell + returnStdout   // hardcoded because I own the image or have determined what the image's path normally is def knownOriginalPath= "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" // my change to the path - in this case , a custom tool location used because I don't want to have to modify a bunch of existing pipeline code. def pathModification= "/ansible2.10/bin" def desiredPath= "${pathModification}:${knownOriginalPath}" docker.withRegistry(...) { // for now, the plugin seems to respect the integrity of the inside option string. docker.image(..)inside( "-e PATH=${desiredPath}" ) { sh 'echo PATH = $PATH' } }

            People

              abayer Andrew Bayer
              jammurp James Murphy
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: