Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41339

Environment variables referencing other variables broken

      This seems to have broken fairly recently. I have a global PATH environment variable defined in Jenkins as follows:

      PATH: /path/to/toolchain/bin:$PATH

      Freestyle jobs work with this. An older version of the durable task plugin also worked. After updating to the latest, this pipeline job:

      node('master', {
          echo 'env.PATH=' + env.PATH
          sh('env')
         })
      

      results in this output:

      [Pipeline] node
      Running on master in /var/lib/jenkins/workspace/pipeline bug
      [Pipeline] {
      [Pipeline] echo
      env.PATH=/path/to/toolchain/bin:$PATH
      [Pipeline] sh
      [pipeline bug] Running shell script
      nohup: failed to run command ‘sh’: No such file or directory
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: script returned exit code -2
      Finished: FAILURE
      

          [JENKINS-41339] Environment variables referencing other variables broken

          Denys Digtiar added a comment -

          As oleg_nenashev pointed out, this doesn't only affect the PATH but other Global environment variables as well.

          If I were to define for example Global environment variable NODE_TMP=/tmp/${NODE_NAME}

          Then Create Freestyle project and Pipeline one with only one step that does env

          Then run both projects on master:

          In Console Output of the Freestyle build, I would see NODE_TMP=/tmp/master

          while in Console Output of the Pipeline it would be NODE_TMP=/tmp/${NODE_NAME}

          Denys Digtiar added a comment - As oleg_nenashev pointed out, this doesn't only affect the PATH but other Global environment variables as well. If I were to define for example Global environment variable NODE_TMP=/tmp/${NODE_NAME } Then Create Freestyle project and Pipeline one with only one step that does env Then run both projects on master: In Console Output of the Freestyle build, I would see NODE_TMP=/tmp/master while in Console Output of the Pipeline it would be NODE_TMP=/tmp/${NODE_NAME }

          Andrew Gray added a comment -

          What's the status of this bug?  It is being fixed?

          Andrew Gray added a comment - What's the status of this bug?  It is being fixed?

          Oleg Nenashev added a comment -

          AFAIK no

          Oleg Nenashev added a comment - AFAIK no

          eric gisse added a comment - - edited

          FWIW I'm seeing this one too, and this was exceptionally confusing since I have two different build pipelines which are largely the same and both built from blue ocean.

          The first one didn't have this issue, with an environment like this:

          environment {
              GOPATH = "${WORKSPACE}/go"
              PATH = "${PATH}:${WORKSPACE}/go:${WORKSPACE}/go/bin"
              BUILD_ROOT = "${WORKSPACE}/go/src/github.com/<things>l"
            }

          The second one, which used the same PATH variable, did.

          I can accept broken but pls consistency.

           

           

          eric gisse added a comment - - edited FWIW I'm seeing this one too, and this was exceptionally confusing since I have two different build pipelines which are largely the same and both built from blue ocean. The first one didn't have this issue, with an environment like this: environment {     GOPATH = "${WORKSPACE}/go"     PATH = "${PATH}:${WORKSPACE}/go:${WORKSPACE}/go/bin"     BUILD_ROOT = "${WORKSPACE}/go/src/github.com/<things>l"   } The second one, which used the same PATH variable, did. I can accept broken but pls consistency.    

          So really there is just this hack used by Jenkins available ? It's not even bash syntax and provides only prepend option.

          The old truth I learned working with Jenkins for 4 years is up to date - keep Jenkins installation and configuraion minimal and do everything you can by yourself in scripts (not by plugins). Jenkins and pipeline should only be glue, not the build system itself.

          Dawid Gosławski added a comment - So really there is just this hack used by Jenkins available ? It's not even bash syntax and provides only prepend option. The old truth I learned working with Jenkins for 4 years is up to date - keep Jenkins installation and configuraion minimal and do everything you can by yourself in scripts (not by plugins). Jenkins and pipeline should only be glue, not the build system itself.

          Andrew Gray added a comment -

          Disagree Dawid.  Having used Jenkins for 11 years.  Use the plugins as much as possible.

          Andrew Gray added a comment - Disagree Dawid.  Having used Jenkins for 11 years.  Use the plugins as much as possible.

          Kevin Lyda added a comment -

          This is crazy. Is there no sane way to do this in a declarative pipeline?

          I would like to do this rather basic thing in a declarative pipeline in a Jenkins file:

           

          environment {
            GOPATH = "$WORKSPACE/gopath/bin"
            PATH = "$GOPATH/bin:$PATH"
          }
          

           

          But for some inexplicable reason we can't modify environment variables like one does in every single other utility in the world? Fine, I'll write a script. But clearly looking at buildbot keeps being shoved forcibly up my priority list.

          Kevin Lyda added a comment - This is crazy. Is there no sane way to do this in a declarative pipeline? I would like to do this rather basic thing in a declarative pipeline in a Jenkins file:   environment {   GOPATH = "$WORKSPACE/gopath/bin"   PATH = "$GOPATH/bin:$PATH" }   But for some inexplicable reason we can't modify environment variables like one does in every single other utility in the world? Fine, I'll write a script. But clearly looking at buildbot keeps being shoved forcibly up my priority list.

          Oleg Nenashev added a comment -

          lyda EnvInject plugin does not support Pipeline, and there is no short-term plans to do so. Your comment is not related to this ticket IMHO, Declarative Pipeline has its own implementation. AFAIK there is a ticket for the case you raise (CC abayer svanoort)

          Oleg Nenashev added a comment - lyda EnvInject plugin does not support Pipeline, and there is no short-term plans to do so. Your comment is not related to this ticket IMHO, Declarative Pipeline has its own implementation. AFAIK there is a ticket for the case you raise (CC abayer svanoort )

          Tobin Davis added a comment - - edited

          I am really confused by all of this.

          First, I'm running 2.138.2 LTS (I know, I need to update), with all of the latests plugin version for pipeline (and no EnvInject plugin).  I have tried the following, with no success:

          1st:

          environment {
            DCP_VERSION = "1.2-alpha"
            DCP_LOC = "/opt/dcp/${DCP_VERSION}"
            QUARTUS_HOME = "/opt/inteldevstack/intelFPGA_pro/quartus" 
            MTI_HOME = "/opt/altera/17.1/modelsim_ase"
            TBB_HOME = "/opt/intel/tbb"
            PATH = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin:${PATH}"
            LD_LIBRARY_PATH = '${LD_LIBRARY_PATH}:${TBB_HOME}/lib/intel64_lin/gcc4.7'
          }

          Produced this error:

          nohup: failed to run command 'sh': No such file or directory

          2nd (based on googling the sh command error)

          environment {
            DCP_VERSION = "1.2-alpha"
            DCP_LOC = "/opt/dcp/${DCP_VERSION}"
            QUARTUS_HOME = "/opt/inteldevstack/intelFPGA_pro/quartus" 
            MTI_HOME = "/opt/altera/17.1/modelsim_ase"
            TBB_HOME = "/opt/intel/tbb"
            PATH+EXTRA = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin"
            LD_LIBRARY_PATH = '${LD_LIBRARY_PATH}:${TBB_HOME}/lib/intel64_lin/gcc4.7'
          }

          Now I get this error:
          (PATH + EXTRA) is a binary expression, but it should be a variable expression at line: 11 column: 19. File: WorkflowScript @ line 11, column 19.
          PATH+EXTRA="${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin"

          Note that neither of these solutions are actually documented in the pipeline guide (https://jenkins.io/doc/book/pipeline/syntax/), and BOTH options are fully acceptable and copied from the built-in syntax generator.

          So, what is the right way to do this? I can't set global paths as a lot of these paths change depending on the job (for example, MTI_HOME will change depending on simulator used - future script development task)

           

          Update:  I got around the first issue by setting Jenkins global shell to /bin/bash.  Not cross platform, but works for now.

          Tobin Davis added a comment - - edited I am really confused by all of this. First, I'm running 2.138.2 LTS (I know, I need to update), with all of the latests plugin version for pipeline (and no EnvInject plugin).  I have tried the following, with no success: 1st: environment {   DCP_VERSION = "1.2-alpha"   DCP_LOC = "/opt/dcp/${DCP_VERSION}"   QUARTUS_HOME = "/opt/inteldevstack/intelFPGA_pro/quartus"    MTI_HOME = "/opt/altera/17.1/modelsim_ase"   TBB_HOME = "/opt/intel/tbb"   PATH = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin:${PATH}"   LD_LIBRARY_PATH = '${LD_LIBRARY_PATH}:${TBB_HOME}/lib/intel64_lin/gcc4.7' } Produced this error: nohup: failed to run command 'sh': No such file or directory 2nd (based on googling the sh command error) environment {   DCP_VERSION = "1.2-alpha"   DCP_LOC = "/opt/dcp/${DCP_VERSION}"   QUARTUS_HOME = "/opt/inteldevstack/intelFPGA_pro/quartus"    MTI_HOME = "/opt/altera/17.1/modelsim_ase"   TBB_HOME = "/opt/intel/tbb"   PATH+EXTRA = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin"   LD_LIBRARY_PATH = '${LD_LIBRARY_PATH}:${TBB_HOME}/lib/intel64_lin/gcc4.7' } Now I get this error: (PATH + EXTRA) is a binary expression, but it should be a variable expression at line: 11 column: 19. File: WorkflowScript @ line 11, column 19. PATH+EXTRA="${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin" Note that neither of these solutions are actually documented in the pipeline guide ( https://jenkins.io/doc/book/pipeline/syntax/ ), and BOTH options are fully acceptable and copied from the built-in syntax generator. So, what is the right way to do this? I can't set global paths as a lot of these paths change depending on the job (for example, MTI_HOME will change depending on simulator used - future script development task)   Update:  I got around the first issue by setting Jenkins global shell to /bin/bash.  Not cross platform, but works for now.

          Jesse Glick added a comment -

          gruemaster the PATH+EXTRA system works for Scripted Pipeline. If it does not work for Declarative, please file a separate RFE in pipeline-model-definition-plugin. Workaround would I guess be something like (untested)

          environment {
            STUFF = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin"
          }
          // …
          steps {
            withEnv(["PATH+EXTRA=$STUFF"]) {
              sh 'whatever'
            }
          }
          

          Jesse Glick added a comment - gruemaster the PATH+EXTRA system works for Scripted Pipeline. If it does not work for Declarative, please file a separate RFE in pipeline-model-definition-plugin . Workaround would I guess be something like (untested) environment { STUFF = "${MTI_HOME}/linux:${MTI_HOME}/bin:${QUARTUS_HOME}/bin:${DCP_LOC}/bin" } // … steps { withEnv([ "PATH+EXTRA=$STUFF" ]) { sh 'whatever' } }

            Unassigned Unassigned
            jtatum James Tatum
            Votes:
            27 Vote for this issue
            Watchers:
            52 Start watching this issue

              Created:
              Updated: