Details
-
Type:
Improvement
-
Status: Closed (View Workflow)
-
Priority:
Major
-
Resolution: Fixed
-
Component/s: git-plugin
-
Labels:
-
Environment:Jenkins 2.89.3, workflow-job 2.18-SNAPSHOT
-
Similar Issues:
Description
Environment variables are not resolved in 'Advanced clone behaviours' -> 'Path of the reference repo to use during clone' when configuring git reference repository in the Pipeline SCM section.
Steps to reproduce:
- Create pipeline project
- In configure: fill in repository URL, Branch Specifier and Script Path (Jenkinsfile).
- In 'Additional Behaviours' select 'Advanced Clone Behaviours'. Set 'Path of the reference repo to use during clone' to some value depending on an environment variable, e.g. ${HOME}/jenkins/reference/big-git-repo.git
- At URL, create a git repository with Jenkinsfile. Put 'node('master') { checkout scm }' in the Jenkinsfile.
- Run the job. It will report:
Cloning repository ...URL... > git init ... # timeout=10 ERROR: Reference path does not exist: ${HOME}/jenkins/reference/big-git-repo.git
Would be very handy to have this feature. In my case, I want to use the 'checkout scm' shortcut in a pipeline script that runs several builds in parallel on different platforms. So I want to specify where a reference repository exists on a specific node. For instance, on a Windows node I want to put it in D:\jenkins\reference, on a Mac – to /Users/jenkins/reference and, finally, on linux – /home/jenkins/reference. I could set an environment variable ${REFERENCE} to the above values in node configuration and than use it in 'Path of the reference repo to use during clone' as ${REFERENCE}.
Attachments
Issue Links
- relates to
-
-
- Open
-
-
JENKINS-65315 Environment variables not resolved in Pipeline SCM -> Advanced sub-modules behaviours -> Path of the reference repo
-
- Open
-
- links to
Activity
Any updates on releasing this? I verified it does not work on my v4.0 plugin (can resolve $JENKINS_HOME but not $HOME so is useless) and the changelog does not list it for that or v4.1 beta
We have pipelines across many nodes, including different platforms, so we can't take full advantage of reference repos (the workaround of manually re-defining the checkout for every stage would be maintenance disaster) — we need a way of dynamically building the reference repo path per node per pipeline.
D Pasto
added a comment - Any updates on releasing this? I verified it does not work on my v4.0 plugin (can resolve $JENKINS_HOME but not $HOME so is useless) and the changelog does not list it for that or v4.1 beta
We have pipelines across many nodes, including different platforms, so we can't take full advantage of reference repos (the workaround of manually re-defining the checkout for every stage would be maintenance disaster) — we need a way of dynamically building the reference repo path per node per pipeline. I've proposed automatic repository caching as a Google Summer of Code idea and have offered the pull request as a starting point for consideration.
Have you tried the alternative that is described by Jesse Glick? It seems like it would work in your case if you perform the environment variable expansion in the Jenkins checkout statement.
D Pasto did you actually specify HOME as an explicit variable on the node or are you just depending on an existing environment variable. If I remember correctly we're using this with some 4.* version for a while, but always with explicitly specified node variables.
Patrick Ruckstuhl
added a comment - D Pasto did you actually specify HOME as an explicit variable on the node or are you just depending on an existing environment variable. If I remember correctly we're using this with some 4.* version for a while, but always with explicitly specified node variables. Good call Patrick Ruckstuhl It works! I assumed I could just use environment variables that I believe existed, but whether it doesn't really exist, or the plugin only sees a subset of environment variables, it correctly found the reference repo using an env defined in the node! It's not as clean as an automatically defined variable would be, but it looks like it works! We have a good sized repo so even though we're on-prem, I was seeing 2+ minute fetches for those stages where the hard-coded path was wrong, but this brings it down to 15 seconds in my first few runs
Edit: forgot the most important part - this lets me define a single path (using forward slashes) that works on both Linux and Windows nodes.
D Pasto
added a comment - - edited Good call Patrick Ruckstuhl It works! I assumed I could just use environment variables that I believe existed, but whether it doesn't really exist, or the plugin only sees a subset of environment variables, it correctly found the reference repo using an env defined in the node! It's not as clean as an automatically defined variable would be, but it looks like it works! We have a good sized repo so even though we're on-prem, I was seeing 2+ minute fetches for those stages where the hard-coded path was wrong, but this brings it down to 15 seconds in my first few runs
Edit: forgot the most important part - this lets me define a single path (using forward slashes) that works on both Linux and Windows nodes. 
These are of course better handled via Docker containers or the like if you can manage it.
I did mean to imply that the Jenkinsfile must list actual filesystem paths on each possible node, merely that the program has some way of determining this information on the fly, preferably not involving Jenkins global configuration. That could be an environment variable set on the agent’s account, etc. The point is that the selection and configuration of a reference repo is something under the control of the job maintainer or Jenkins admin, rather than being baked into a plugin.
I do not have to imagine that, it is a daily reality.
In the case of the Jenkins project, in fact we do, but in case you do not, there are various alternatives.