Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52059

checkout scm step does not populate environment with GIT_*** variables in non-multibranch pipelines

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Critical Critical
    • git-plugin, scm-api-plugin
    • None

      Pipeline from SCM does not populate environment with variables as claimed by https://wiki.jenkins.io/display/JENKINS/Git+Plugin when pipeline is non-multibranch.

      Contrived example:

      // a library with convenience functions
// unused in this particular build, but relevant
// as it's used in all 'real' builds
@Library('jenkins-libraries') _ 

// ideally, we would love to know at least the remote branch name here

node ('linux_staging') {
  stage('checkout') {
    checkout scm
    // the next best thing would be to know the branch name here

    sh "echo ${env.GIT_BRANCH}"  // empty
    echo sh(script: "env | sort", returnStdout: true)  // no sign of them here
    echo env.GIT_BRANCH  // null
  }
}

      Build log

      Started by an SCM change
Checking out git git@git.tobii.intra:/project/oem/platform/tools/gitolite_testing.git into /jenkinsstorage/jenkins/workspace/infra/Tests/gitolite_hook_tests/test_trigger_feature@script to read Jenkinsfile
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git@git.tobii.intra:/project/oem/platform/tools/gitolite_testing.git # timeout=10
Fetching upstream changes from git@git.tobii.intra:/project/oem/platform/tools/gitolite_testing.git
 > git --version # timeout=10
using GIT_SSH to set credentials These credentials are to be used by jenkins/nodes to connect with git using user tts-tools
 > git fetch --tags --progress git@git.tobii.intra:/project/oem/platform/tools/gitolite_testing.git +refs/heads/f/*:refs/remotes/origin/f/*
Seen branch in repository origin/f/PTOOL-112233
Seen branch in repository origin/f/PTOOL-123456
Seen branch in repository origin/f/PTOOL-543
Seen branch in repository origin/f/PTOOL-666
Seen branch in repository origin/f/PTOOL-do-stuff
Seen 5 remote branches
 > git show-ref --tags -d # timeout=10
Checking out Revision 54d55d769c0ea5e320f8706b2d016c7bea22fe55 (origin/f/PTOOL-do-stuff)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 54d55d769c0ea5e320f8706b2d016c7bea22fe55
 > git branch -a -v --no-abbrev # timeout=10
 > git checkout -b f/PTOOL-do-stuff 54d55d769c0ea5e320f8706b2d016c7bea22fe55
Commit message: "Jenkinsfile"
First time build. Skipping changelog.
Running in Durability level: MAX_SURVIVABILITY
Loading library jenkins-libraries@master
Attempting to resolve master from remote references...
 > git --version # timeout=10
using GIT_SSH to set credentials These credentials are to be used by jenkins/nodes to connect with git using user tts-tools
 > git ls-remote -h -t git@git.tobii.intra:prereqs/jenkins_libraries.git # timeout=10
Found match: refs/heads/master revision 1dd4f56b1444ffa2b65de0e6d689c195d23632ff
Cloning the remote Git repository
Cloning repository git@git.tobii.intra:prereqs/jenkins_libraries.git
 > git init /jenkinsstorage/jenkins/workspace/infra/Tests/gitolite_hook_tests/test_trigger_feature@libs/jenkins-libraries # timeout=10
Fetching upstream changes from git@git.tobii.intra:prereqs/jenkins_libraries.git
 > git --version # timeout=10
using GIT_SSH to set credentials These credentials are to be used by jenkins/nodes to connect with git using user tts-tools
 > git fetch --tags --progress git@git.tobii.intra:prereqs/jenkins_libraries.git +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url git@git.tobii.intra:prereqs/jenkins_libraries.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git@git.tobii.intra:prereqs/jenkins_libraries.git # timeout=10
Fetching upstream changes from git@git.tobii.intra:prereqs/jenkins_libraries.git
using GIT_SSH to set credentials These credentials are to be used by jenkins/nodes to connect with git using user tts-tools
 > git fetch --tags --progress git@git.tobii.intra:prereqs/jenkins_libraries.git +refs/heads/*:refs/remotes/origin/*
Checking out Revision 1dd4f56b1444ffa2b65de0e6d689c195d23632ff (master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1dd4f56b1444ffa2b65de0e6d689c195d23632ff
Commit message: "PTOOL-1520_doublelock_remove_printouts"
[Pipeline] node
Running on linux_staging in /datadisk/jenkins/workspace/infra/Tests/gitolite_hook_tests/test_trigger_feature
[Pipeline] {
[Pipeline] stage
[Pipeline] { (checkout)
[Pipeline] checkout
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git@git.tobii.intra:/project/oem/platform/tools/gitolite_testing.git # timeout=10
Fetching upstream changes from git@git.tobii.intra:/project/oem/platform/tools/gitolite_testing.git
 > git --version # timeout=10
using GIT_SSH to set credentials These credentials are to be used by jenkins/nodes to connect with git using user tts-tools
 > git fetch --tags --progress git@git.tobii.intra:/project/oem/platform/tools/gitolite_testing.git +refs/heads/f/*:refs/remotes/origin/f/*
Seen branch in repository origin/f/PTOOL-112233
Seen branch in repository origin/f/PTOOL-123456
Seen branch in repository origin/f/PTOOL-543
Seen branch in repository origin/f/PTOOL-666
Seen branch in repository origin/f/PTOOL-do-stuff
Seen 5 remote branches
 > git show-ref --tags -d # timeout=10
Checking out Revision 54d55d769c0ea5e320f8706b2d016c7bea22fe55 (origin/f/PTOOL-do-stuff)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 54d55d769c0ea5e320f8706b2d016c7bea22fe55
 > git branch -a -v --no-abbrev # timeout=10
 > git checkout -b f/PTOOL-do-stuff 54d55d769c0ea5e320f8706b2d016c7bea22fe55
Commit message: "Jenkinsfile"
[Pipeline] sh
[test_trigger_feature] Running shell script
+ echo null
null
[Pipeline] sh
[test_trigger_feature] Running shell script
+ sort
+ env
[Pipeline] echo
BUILD_DISPLAY_NAME=#9
BUILD_ID=9
BUILD_NUMBER=9
BUILD_TAG=jenkins-infra-Tests-gitolite_hook_tests-test_trigger_feature-9
BUILD_URL=https://tobiitechjenkins.tobii.intra/job/infra/job/Tests/job/gitolite_hook_tests/job/test_trigger_feature/9/
CI=true
EXECUTOR_NUMBER=0
HOME=/home/tt-tools
HUDSON_COOKIE=4e1bdaa3-3ca4-4286-b9ee-6b8e57eb1a1a
HUDSON_HOME=/jenkinsstorage/jenkins
HUDSON_SERVER_COOKIE=8faf0bbbf409f2f0
HUDSON_URL=https://tobiitechjenkins.tobii.intra/
JENKINS_HOME=/jenkinsstorage/jenkins
JENKINS_NODE_COOKIE=fdfcfdf6-6441-4673-acf3-382798ae6c69
JENKINS_SERVER_COOKIE=durable-1f3503ac3f6209410f79edc5a86ce920
JENKINS_URL=https://tobiitechjenkins.tobii.intra/
JOB_BASE_NAME=test_trigger_feature
JOB_DISPLAY_URL=https://tobiitechjenkins.tobii.intra/job/infra/job/Tests/job/gitolite_hook_tests/job/test_trigger_feature/display/redirect
JOB_NAME=infra/Tests/gitolite_hook_tests/test_trigger_feature
JOB_URL=https://tobiitechjenkins.tobii.intra/job/infra/job/Tests/job/gitolite_hook_tests/job/test_trigger_feature/
LANG=en_US.UTF-8
LOGNAME=tt-tools
MAIL=/var/mail/tt-tools
NODE_LABELS=linux_staging
NODE_NAME=linux_staging
OLDPWD=/home/tt-tools
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PWD=/datadisk/jenkins/workspace/infra/Tests/gitolite_hook_tests/test_trigger_feature
RUN_CHANGES_DISPLAY_URL=https://tobiitechjenkins.tobii.intra/job/infra/job/Tests/job/gitolite_hook_tests/job/test_trigger_feature/9/display/redirect?page=changes
RUN_DISPLAY_URL=https://tobiitechjenkins.tobii.intra/job/infra/job/Tests/job/gitolite_hook_tests/job/test_trigger_feature/9/display/redirect
SHELL=/bin/bash
SHLVL=1
SignPassword=*
SSH_CLIENT=*
SSH_CONNECTION=*
STAGE_NAME=checkout
USER=tt-tools
_=/usr/bin/java
WORKSPACE=/datadisk/jenkins/workspace/infra/Tests/gitolite_hook_tests/test_trigger_feature
XDG_RUNTIME_DIR=/run/user/1000
XDG_SESSION_ID=2388

[Pipeline] echo
null
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

      The more real use case is that we have builds that use the parallel-command to execute builds on multiple OS's using various compilers, each (well, most) separate parallel stages do a checkout scm before compilation starts. Finally, after parallel returns, a final stage is supposed to use information about what was actually built to perform common things.

      Expected behavior:

      Environment populated with GIT_XYZ variables after checkout scm, or even better when the build starts.

      Actual behavior:

      checkout scm does not lead to the environment being populated with GIT_XYZ, unless in a multibranch configuration. Technically we could use the map returned from checkout scm to populate the environment ourselves, but it feels like this functionality should be provided by the plugin itself. Furthermore, in this particular instance the branch name is only necessary in a final stage, after parallel building has completed, so if using the workaround that stage would have to checkout scm as well, or rely on one of the parallel stages to set a variable in the outer scope.

        1. pipeline-syntax-snippet-generator-help.png
          pipeline-syntax-snippet-generator-help.png
          215 kB
        2. pipeline-syntax-snippet-generator-help-cropped.png
          pipeline-syntax-snippet-generator-help-cropped.png
          133 kB

          [JENKINS-52059] checkout scm step does not populate environment with GIT_*** variables in non-multibranch pipelines

          Amaretto Slim added a comment -

          Bump

          Amaretto Slim added a comment - Bump

          Mark Waite added a comment - - edited

          I've confirmed this bug with scripted pipeline when using Pipeline from SCM in a single Pipeline job (not multibranch).

          My test case is in the JENKINS-52059 scripted pipeline verification git job, the JENKINS-52059 scripted pipeline verification jgit job, the JENKINS-52059 declarative pipeline verification job, and the JENKINS-52059 verification branch.

          It would be consistent with declarative pipeline if the env variables were set. Refer to the Jenkins Minute video for the declarative use of env vars.

          What I observed:

          Pipeline Type Return from checkout env values assigned
          scripted pipeline from scm No No
          multibranch scripted pipeline Yes No
          declarative pipeline from scm Not checked Yes
          multibranch declarative pipeline Not checked Yes
          multibranch declarative pipeline without default checkout Yes (scripted inside declarative) No

          Mark Waite added a comment - - edited I've confirmed this bug with scripted pipeline when using Pipeline from SCM in a single Pipeline job (not multibranch). My test case is in the JENKINS-52059 scripted pipeline verification git job , the JENKINS-52059 scripted pipeline verification jgit job , the JENKINS-52059 declarative pipeline verification job , and the JENKINS-52059 verification branch . It would be consistent with declarative pipeline if the env variables were set. Refer to the Jenkins Minute video for the declarative use of env vars. What I observed: Pipeline Type Return from checkout env values assigned scripted pipeline from scm No No multibranch scripted pipeline Yes No declarative pipeline from scm Not checked Yes multibranch declarative pipeline Not checked Yes multibranch declarative pipeline without default checkout Yes (scripted inside declarative) No

          Simon added a comment -

          I have this exact issue when working with the github organization folder.

          Meaning that there is only access to the $BRANCH_NAME environment variable. All the other GIT_ Are not available.

          Would be nice to get this fixed.

          Simon added a comment - I have this exact issue when working with the github organization folder. Meaning that there is only access to the $BRANCH_NAME environment variable. All the other GIT_ Are not available. Would be nice to get this fixed.

          Mark Waite added a comment - - edited

          Would be nice to get this fixed.

          I don't plan to work on this. Using the return value from checkout scm is the best approach for multibranch scripted Pipeline and for multibranch declarative Pipeline that disables the default checkout.

          Mark Waite added a comment - - edited Would be nice to get this fixed. I don't plan to work on this. Using the return value from checkout scm is the best approach for multibranch scripted Pipeline and for multibranch declarative Pipeline that disables the default checkout.

          Simon added a comment -

          markewaite Do you by chance have a link to the documentation for returned object from 'checkout scm'.

          I didn't know it returned anything to work with as it is not described in:

          https://www.jenkins.io/doc/pipeline/steps/workflow-scm-step/#checkout-check-out-from-version-control

          https://plugins.jenkins.io/workflow-scm-step/

          Simon added a comment - markewaite Do you by chance have a link to the documentation for returned object from 'checkout scm'. I didn't know it returned anything to work with as it is not described in: https://www.jenkins.io/doc/pipeline/steps/workflow-scm-step/#checkout-check-out-from-version-control https://plugins.jenkins.io/workflow-scm-step/

          Mark Waite added a comment - - edited

          The Pipeline snippet syntax generator shows the online help for your local installation. That help is based on the exact plugins that are installed on your system. On my system, it shows the following image:

          The git plugin documentation includes a video that recommends the use of the Pipeline syntax snippet generator.

          Mark Waite added a comment - - edited The Pipeline snippet syntax generator shows the online help for your local installation. That help is based on the exact plugins that are installed on your system. On my system, it shows the following image: The git plugin documentation includes a video that recommends the use of the Pipeline syntax snippet generator.

          Simon added a comment -

          Thank you so much.

          Sad the information is not in an accessable variable or environment. But this is a fine workaround.

          Simon added a comment - Thank you so much. Sad the information is not in an accessable variable or environment. But this is a fine workaround.

          Mark Waite added a comment -

          No plan to implement this because the results are assigned to the map that is returned by 

          def checkoutSettings = checkout scm

          Mark Waite added a comment - No plan to implement this because the results are assigned to the map that is returned by  def checkoutSettings = checkout scm

            Unassigned Unassigned
            emanuel_tobiitech Emanuel Birge
            Votes:
            8 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: