• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • workflow-cps-plugin
    • Jenins 1.613, Workflow plugin 1.6
      Also: Jenkins 1.651.1, workflow-job 2.3

      When selecting the "Groovy CPS DSL from SCM" option for a worflow job, the SCM plugins do not appear to resolve build parameters or environment variables. I am using the git plugin and when I use it from other jobs I can specify a build parameter, like "BuildBranch", and use that when specifying what branch should be built:

      Branches to build: */${BuildBranch}

      This does not work when I use the Groovy CPS DSL from SCM.

          [JENKINS-28447] CpsScmFlowDefinition does not resolve variables

          I don't understand why it's marked as resolved. I can still reproduce it with the latest workflow-cps (2.30) and workflow-job (2.10) plugins. Reproducible as it is in the description.

          • Create pipeline project
          • In configure: check 'This project is parametrized' and add a String Parameter "branch" with some default value. Select 'Pipeline script from SCM' in definition. Fill in repository URL, Branch Specifier as ${branch} and Script Path (Jenkinsfile)
          • At URL, create a git repository with Jenkinsfile. Put 'node('master') { checkout scm }' in the Jenkinsfile.
          • Run the job. It will fail with: 
            hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/${branch}:refs/remotes/origin/${branch} --prune" returned status code 128:
            stdout: 
            stderr: fatal: Couldn't find remote ref refs/heads/${branch}
            
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1833)
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1552)
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:66)
            	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:343)
            	at jenkins.plugins.git.GitSCMFileSystem$BuilderImpl.build(GitSCMFileSystem.java:306)
            	at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:196)
            	at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:172)
            	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:99)
            	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:59)
            	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:232)
            	at hudson.model.ResourceController.execute(ResourceController.java:98)
            	at hudson.model.Executor.run(Executor.java:405)
            

          Anna Tikhonova added a comment - I don't understand why it's marked as resolved. I can still reproduce it with the latest workflow-cps (2.30) and workflow-job (2.10) plugins. Reproducible as it is in the description. Create pipeline project In configure: check 'This project is parametrized' and add a String Parameter "branch" with some default value. Select 'Pipeline script from SCM' in definition. Fill in repository URL, Branch Specifier as ${branch} and Script Path (Jenkinsfile) At URL, create a git repository with Jenkinsfile. Put 'node('master') { checkout scm }' in the Jenkinsfile. Run the job. It will fail with:  hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/${branch}:refs/remotes/origin/${branch} --prune" returned status code 128: stdout: stderr: fatal: Couldn't find remote ref refs/heads/${branch} at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1833) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1552) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:66) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:343) at jenkins.plugins.git.GitSCMFileSystem$BuilderImpl.build(GitSCMFileSystem.java:306) at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:196) at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:172) at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:99) at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:59) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:232) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:405)

          Also, I'm facing the issue when configuring git reference repository in the Pipeline SCM section.

          • All the same configuration as in my comment above, but replace ${branch} with an existing branch, e.g master
          • In 'Additional Behaviours' select 'Advanced Clone Behaviours'
          • Set 'Path of the reference repo to use during clone' to some value depending on an environment variable, e.g. ${HOME}/jenkins/reference/big-git-repo.git. The job will report: 
            Cloning repository ...URL...
             > git init ... # timeout=10
            ERROR: Reference path does not exist: ${HOME}/jenkins/reference/big-git-repo.git

          Would be very handy to have this feature. In my case, I want to use the 'checkout scm' shortcut in my pipeline script that runs several builds in parallel on different platforms. So I want to specify where a reference repository exists on a specific node. For instance, on a Windows node I want to put it in D:\jenkins\reference, on a Mac – to /Users/jenkins/reference and, finally, on linux – /home/jenkins/reference. I could set an environment variable ${REFERENCE} to the above values in node configuration and than use it in 'Path of the reference repo to use during clone' as ${REFERENCE}.

          Anna Tikhonova added a comment - Also, I'm facing the issue when configuring git reference repository in the Pipeline SCM section. All the same configuration as in my comment above, but replace ${branch} with an existing branch, e.g master In 'Additional Behaviours' select 'Advanced Clone Behaviours' Set 'Path of the reference repo to use during clone' to some value depending on an environment variable, e.g. ${HOME}/jenkins/reference/big-git-repo.git. The job will report:  Cloning repository ...URL... > git init ... # timeout=10 ERROR: Reference path does not exist: ${HOME}/jenkins/reference/big-git-repo.git Would be very handy to have this feature. In my case, I want to use the 'checkout scm' shortcut in my pipeline script that runs several builds in parallel on different platforms. So I want to specify where a reference repository exists on a specific node. For instance, on a Windows node I want to put it in D:\jenkins\reference, on a Mac – to /Users/jenkins/reference and, finally, on linux – /home/jenkins/reference. I could set an environment variable ${REFERENCE} to the above values in node configuration and than use it in 'Path of the reference repo to use during clone' as ${REFERENCE}.

          Jesse Glick added a comment -

          atikhono you must disable lightweight checkout.

          Jesse Glick added a comment - atikhono you must disable lightweight checkout.

          Jesse Glick added a comment -

          Jesse Glick added a comment - jwillemsen see  JENKINS-42836 .

          jglick Thanks, with lightweight checkout off ${branch} is resolved. Still, git reference repository path is not. Should I file another issue for that?

          Anna Tikhonova added a comment - jglick Thanks, with lightweight checkout off ${branch} is resolved. Still, git reference repository path is not. Should I file another issue for that?

          Jesse Glick added a comment -

          atikhono that sounds like an RFE for git-plugin not specific to Pipeline.

          Jesse Glick added a comment - atikhono that sounds like an RFE for git-plugin not specific to Pipeline.

          jglick I came across this issue as well, and unchecking "lightweight checkout" also helped. However I cannot figure out why exactly, can you give a bit more context into why the env/parameter interpolation is disabled when LC is on ?

          Pavel Savshenko added a comment - jglick I came across this issue as well, and unchecking "lightweight checkout" also helped. However I cannot figure out why exactly, can you give a bit more context into why the env/parameter interpolation is disabled when LC is on ?

          Jesse Glick added a comment -

          Lightweight checkout is going through a completely unrelated code path, and currently SCMFileSystem implementations do not honor variables (or even have an API allowing them to honor variables IIRC). Would have to be a separate feature.

          Jesse Glick added a comment - Lightweight checkout is going through a completely unrelated code path, and currently SCMFileSystem implementations do not honor variables (or even have an API allowing them to honor variables IIRC). Would have to be a separate feature.

          Has this "separate feature" been added yet to the "lightweight checkout" functionality? In my case, I have a pipeline job that is parameterized. One of the parameters is the branch name which I intend to use later on in the job when selecting the definition "Pipeline script from SCM". When selecting that you have the option to select your Git repository and Branches to build, but it doesn't recognize the parameter when "Lightweight checkout" is selected.

          Matthew Mallard added a comment - Has this "separate feature" been added yet to the "lightweight checkout" functionality? In my case, I have a pipeline job that is parameterized. One of the parameters is the branch name which I intend to use later on in the job when selecting the definition "Pipeline script from SCM". When selecting that you have the option to select your Git repository and Branches to build, but it doesn't recognize the parameter when "Lightweight checkout" is selected.

          I have a jenkins installation which was set up in 2019. A job for building releases has a build parameter TAG_NAME. This variable is used as branch specifier

          refs/tags/${TAG_NAME}

          in the "Pipelinescript from SCM" git section and i have "lightweight checkout" activated. (it didn't work btw. telling me that it is falling back to full checkout)

           

          Until yesterday, it worked flawlessly, the variable was resolved correctly and the corresponding tag was checked out.

          Then i updated a bunch of plugins and restarted jenkins. Then the variable wasn't resolved any more. Only unchecking the "lighweight checkout" option helped.

          How could this have worked for me before if this problem is already so old?

          Started by upstream project "XXXXXXX" build number 200
          originally caused by:
           GitHub tag trigger
          hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune -- origin +refs/tags/${TAG_NAME}:refs/remotes/origin/${TAG_NAME}" returned status code 128:
          stdout: 
          stderr: fatal: Couldn't find remote ref refs/tags/${TAG_NAME}	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2436)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2050)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:572)
          	at jenkins.plugins.git.GitSCMFileSystem$BuilderImpl.build(GitSCMFileSystem.java:364)
          	at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:197)
          	at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:173)
          	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:115)
          	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:69)
          	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)
          	at hudson.model.ResourceController.execute(ResourceController.java:97)
          	at hudson.model.Executor.run(Executor.java:428)
          Finished: FAILURE
          

          Could it be that now with the updated plugins i would actually get a "lightweight" checkout and therefore the variable resolution doesn't work?

          Jürgen Wakunda added a comment - I have a jenkins installation which was set up in 2019. A job for building releases has a build parameter TAG_NAME. This variable is used as branch specifier refs/tags/${TAG_NAME} in the "Pipelinescript from SCM" git section and i have "lightweight checkout" activated. (it didn't work btw. telling me that it is falling back to full checkout)   Until yesterday, it worked flawlessly, the variable was resolved correctly and the corresponding tag was checked out. Then i updated a bunch of plugins and restarted jenkins. Then the variable wasn't resolved any more. Only unchecking the "lighweight checkout" option helped. How could this have worked for me before if this problem is already so old? Started by upstream project "XXXXXXX" build number 200 originally caused by: GitHub tag trigger hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune -- origin +refs/tags/${TAG_NAME}:refs/remotes/origin/${TAG_NAME}" returned status code 128: stdout: stderr: fatal: Couldn't find remote ref refs/tags/${TAG_NAME} at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2436) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2050) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:572) at jenkins.plugins.git.GitSCMFileSystem$BuilderImpl.build(GitSCMFileSystem.java:364) at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:197) at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:173) at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:115) at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:69) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) Finished: FAILURE Could it be that now with the updated plugins i would actually get a "lightweight" checkout and therefore the variable resolution doesn't work?

            jglick Jesse Glick
            nsnewland Nicholas Newland
            Votes:
            50 Vote for this issue
            Watchers:
            68 Start watching this issue

              Created:
              Updated:
              Resolved: