• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • workflow-cps-plugin
    • None

      Steps to reproduce

      1. Create a pipeline job
      2. Add a build parameter to specify the URL of a git repository, ${gitUrl}
      3. Add a build parameter to specify the build script to execute, ${script}
      4. Within the Pipeline section
        1. Select "Pipeline script from SCM"
        2. Select "Git" for the SCM
        3. Enter ${gitUrl} for the Repository URL
        4. Enter the branch to build
        5. Enter ${script} for the "Script Path"

      Perform a "Build with Parameters" and enter values for the gitUrl and script values and select Build.  The following error is reported

      Mar 16, 2017 2:43:53 PM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
      INFO: Worker #4 completed: FAILURE
      Mar 16, 2017 2:43:53 PM org.jenkinsci.plugins.workflow.flow.FlowExecutionList unregister
      WARNING: Owner[Worker/4:Worker #4] was not in the list to begin with: []
      Mar 16, 2017 2:43:53 PM io.jenkins.blueocean.events.PipelineEventListener$1 run
      SEVERE: Unexpected error publishing pipeline FlowNode event.
      java.util.concurrent.ExecutionException: hudson.AbortException: /var/jenkins_home/workspace/Worker@script/${script} not found
      at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:289)
      at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:276)
      at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
      at io.jenkins.blueocean.events.PipelineEventListener$1.run(PipelineEventListener.java:226)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: hudson.AbortException: /var/jenkins_home/workspace/Worker@script/${script} not found
      at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:136)
      at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:59)
      at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:232)
      at hudson.model.ResourceController.execute(ResourceController.java:98)
      at hudson.model.Executor.run(Executor.java:404)
      

      If ${script} is set to the location of a script, the job starts successfully.

          [JENKINS-42836] Build parameter not expanded in script path

          I use SVN and I'm able to checkout the Jenkinsfile from a path with a parameter and define the local directory, but I can't get the jenkinsfile as there is no expansion of the parameter

          Tiago Magalhães added a comment - I use SVN and I'm able to checkout the Jenkinsfile from a path with a parameter and define the local directory, but I can't get the jenkinsfile as there is no expansion of the parameter

          Jesse Glick added a comment -

          Not a duplicate.

          Jesse Glick added a comment - Not a duplicate.

          Jesse Glick added a comment -

          This is about CpsScmFlowDefinition.script expansion, which is unrelated to expansion of variables in SCM plugin configuration.

          Jesse Glick added a comment - This is about CpsScmFlowDefinition.script expansion, which is unrelated to expansion of variables in SCM plugin configuration.

          Alon Bar-Lev added a comment -

          Hi,

          This is very important without a generic pipeline job cannot be implemented.

          Example: selection of what pipeline to build within source tree, invoke a generic script out of trigger etc...

          I was sure that the $XXX substitution is done automagically not per usage.

          Thanks!

          Alon Bar-Lev added a comment - Hi, This is very important without a generic pipeline job cannot be implemented. Example: selection of what pipeline to build within source tree, invoke a generic script out of trigger etc... I was sure that the $XXX substitution is done automagically not per usage. Thanks!

          Jesse Glick added a comment -

          Workaround: you can simply have an inline script (CpsFlowDefinition) which looks something like

          evaluate(readTrusted THE_PARAMETER_NAME)
          

          to get the same effect I think. Assuming you were using lightweight checkout mode.

          Jesse Glick added a comment - Workaround: you can simply have an inline script ( CpsFlowDefinition ) which looks something like evaluate(readTrusted THE_PARAMETER_NAME) to get the same effect I think. Assuming you were using lightweight checkout mode.

          Alon Bar-Lev added a comment -

          Thanks!

          However, I cannot use the lightweight checkout as it does not accept parameters as well... I wasted so much time on that, but I did not try this notation... Will try now.

          Alon Bar-Lev added a comment - Thanks! However, I cannot use the lightweight checkout as it does not accept parameters as well... I wasted so much time on that, but I did not try this notation... Will try now.

          Jesse Glick added a comment -

          For heavyweight checkout the idiom would be something like

          def script
          node {
            checkout …
            script = loadFile THE_PARAMETER_NAME
          }
          evaluate(script)

          Jesse Glick added a comment - For heavyweight checkout the idiom would be something like def script node { checkout … script = loadFile THE_PARAMETER_NAME } evaluate(script)

          Alon Bar-Lev added a comment -

          Hi jglick,

          I tried your suggestion and indeed it works!! However... it breaks the checkout(scm) within the pipeline, see below.

          So either I use the regular git in which all works but I cannot have control over script name or I have control but cannot checkout the sources properly.

          Any other suggestion?

          Thanks!
          [Pipeline] { (Declarative: Checkout SCM)[Pipeline] checkoutCloning the remote Git repository
          Using shallow clone
          Avoid fetching tags
          Cloning repository evaluate(readTrusted('GERRIT_SCHEME'))://evaluate(readTrusted('GERRIT_HOST')):evaluate(readTrusted('GERRIT_PORT'))/evaluate(readTrusted('GERRIT_PROJECT')).git
          > git init /home/msnext-ci/workspace/test1 # timeout=10
          Fetching upstream changes from evaluate(readTrusted('GERRIT_SCHEME'))://evaluate(readTrusted('GERRIT_HOST')):evaluate(readTrusted('GERRIT_PORT'))/evaluate(readTrusted('GERRIT_PROJECT')).git
          > git --version # timeout=10
          using GIT_SSH to set credentials msnext-ci (Authenticate using SSH)
          > git fetch --no-tags --progress evaluate(readTrusted('GERRIT_SCHEME'))://evaluate(readTrusted('GERRIT_HOST')):evaluate(readTrusted('GERRIT_PORT'))/evaluate(readTrusted('GERRIT_PROJECT')).git +refs/heads/:refs/remotes/origin/ --depth=1
          ERROR: Error cloning remote repo 'origin'

          Alon Bar-Lev added a comment - Hi jglick , I tried your suggestion and indeed it works!! However... it breaks the checkout(scm) within the pipeline, see below. So either I use the regular git in which all works but I cannot have control over script name or I have control but cannot checkout the sources properly. Any other suggestion? Thanks! [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout Cloning the remote Git repository Using shallow clone Avoid fetching tags Cloning repository evaluate(readTrusted('GERRIT_SCHEME'))://evaluate(readTrusted('GERRIT_HOST')):evaluate(readTrusted('GERRIT_PORT'))/evaluate(readTrusted('GERRIT_PROJECT')).git > git init /home/msnext-ci/workspace/test1 # timeout=10 Fetching upstream changes from evaluate(readTrusted('GERRIT_SCHEME'))://evaluate(readTrusted('GERRIT_HOST')):evaluate(readTrusted('GERRIT_PORT'))/evaluate(readTrusted('GERRIT_PROJECT')).git > git --version # timeout=10 using GIT_SSH to set credentials msnext-ci (Authenticate using SSH) > git fetch --no-tags --progress evaluate(readTrusted('GERRIT_SCHEME'))://evaluate(readTrusted('GERRIT_HOST')):evaluate(readTrusted('GERRIT_PORT'))/evaluate(readTrusted('GERRIT_PROJECT')).git +refs/heads/ :refs/remotes/origin/ --depth=1 ERROR: Error cloning remote repo 'origin'

          Jesse Glick added a comment -

          Yes checkout scm will check out whatever the top-level script definition used, so you would instead checkout some GitSCM configuration (see Pipeline Syntax for suggestions).

          Jesse Glick added a comment - Yes checkout scm will check out whatever the top-level script definition used, so you would instead checkout some GitSCM configuration (see Pipeline Syntax for suggestions).

          I have the same problem (duplicated here: https://issues.jenkins-ci.org/browse/JENKINS-44073)
          But we use PlasticSCM and this scm NEEDS unique workspace names, so the script paths are always dynamic and contain the ${NODE_NAME} environment variable. This issue is really blocking us from using build scripts stored in repositories...

          Benjamin Buchfink added a comment - I have the same problem (duplicated here: https://issues.jenkins-ci.org/browse/JENKINS-44073 ) But we use PlasticSCM and this scm NEEDS unique workspace names, so the script paths are always dynamic and contain the ${NODE_NAME} environment variable. This issue is really blocking us from using build scripts stored in repositories...

          Andrew Bayer added a comment -

          PR up at https://github.com/jenkinsci/workflow-cps-plugin/pull/197 to make sure that script path gets expanded from the environment. Note that this doesn't do anything about using parameters in the repo URL/refspec/branch etc - that works unless you're using a lightweight checkout. See JENKINS-42971 for more on that.

          Andrew Bayer added a comment - PR up at https://github.com/jenkinsci/workflow-cps-plugin/pull/197 to make sure that script path gets expanded from the environment. Note that this doesn't do anything about using parameters in the repo URL/refspec/branch etc - that works unless you're using a lightweight checkout. See JENKINS-42971 for more on that.

          Joseph Lim added a comment -

          Hi, May I know when this patch will be released ?  This fixed is very critical for our build.  We need the script path to be expanded in order for us to pass in git branch so that the checkout will not be overwritten by other instance that calls the same job

           

          Joseph Lim added a comment - Hi, May I know when this patch will be released ?  This fixed is very critical for our build.  We need the script path to be expanded in order for us to pass in git branch so that the checkout will not be overwritten by other instance that calls the same job  

          I'm very much interested in this patch, too. Though I've found a workaround that works fine for me.

          I use Workspacenames like: Jenkins-${JOB_NAME}-${NODE_NAME}

          And on Windows you can then use the shortname, like: JENKIN~1\build.jenkins

          That way you don't need the environment variables expanded

          Benjamin Buchfink added a comment - I'm very much interested in this patch, too. Though I've found a workaround that works fine for me. I use Workspacenames like: Jenkins-${JOB_NAME}-${NODE_NAME} And on Windows you can then use the shortname, like: JENKIN~1\build.jenkins That way you don't need the environment variables expanded

          Joseph Lim added a comment -

          @Benjamin - Thanks for your suggestion.  I'm not sure how the workspacename can be modified.  If you are referring to Jenkins wide configuration, then I don't think this is doable for my environment.  If this can be done as part of the "Pipeline script from SCM", I don't see this option, perhaps you can shed me some light.  For now, I will try to checkout the specific code from github and compile it myself and test it in my own environment until the official binaries is being released.

          Joseph Lim added a comment - @Benjamin - Thanks for your suggestion.  I'm not sure how the workspacename can be modified.  If you are referring to Jenkins wide configuration, then I don't think this is doable for my environment.  If this can be done as part of the "Pipeline script from SCM", I don't see this option, perhaps you can shed me some light.  For now, I will try to checkout the specific code from github and compile it myself and test it in my own environment until the official binaries is being released.

          Joseph Lim added a comment - - edited

          @Benjamin - I have fork the workflow-cps source master branch and merge with the change done by Andrew Bayer.  I have committed the changes to my fork github  https://github.com/josephlim75/workflow-cps-plugin

          I've compiled successfully and produces the plugin hpi.  Tried it and it works

           

           

          Joseph Lim added a comment - - edited @Benjamin - I have fork the workflow-cps source master branch and merge with the change done by Andrew Bayer.  I have committed the changes to my fork github  https://github.com/josephlim75/workflow-cps-plugin I've compiled successfully and produces the plugin hpi.  Tried it and it works    

          Chris Broll added a comment -

          Any update on using variables from the environment in the Script file path?

          Chris Broll added a comment - Any update on using variables from the environment in the Script file path?

          Code changed in jenkins
          User: Andrew Bayer
          Path:
          src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java
          src/test/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinitionTest.java
          http://jenkins-ci.org/commit/workflow-cps-plugin/2dc4bf82a1e095b09dde4638a50c893320432f93
          Log:
          [FIXED JENKINS-42836] env expand the script path from SCM.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java src/test/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinitionTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/2dc4bf82a1e095b09dde4638a50c893320432f93 Log: [FIXED JENKINS-42836] env expand the script path from SCM.

          Andrew Bayer added a comment -

          Whoops, this was merged and released a while back, as of workflow-cps 2.49.

          Andrew Bayer added a comment - Whoops, this was merged and released a while back, as of workflow-cps 2.49.

          Fabian Holler added a comment -

          This does not work for me.

          I have the "Pipeline: Groovy" Plugin version 2.62 installed.
          Similiar to described in the ticket I do the following:

          • Configure my Pipeline job to have a string parameter called branch
          • In the "Pipeline script from SCM" section, I enter as branch specifier ${branch}
          • I start the job and as value for the branch parameter develop.

          The job fails during checkout with
          stderr: fatal: Couldn't find remote ref refs/heads/${branch}

          The variable is not dereferenced.

          Fabian Holler added a comment - This does not work for me. I have the "Pipeline: Groovy" Plugin version 2.62 installed. Similiar to described in the ticket I do the following: Configure my Pipeline job to have a string parameter called branch In the "Pipeline script from SCM" section, I enter as branch specifier ${branch } I start the job and as value for the branch parameter develop . The job fails during checkout with stderr: fatal: Couldn't find remote ref refs/heads/${branch } The variable is not dereferenced.

          Zishan Ahmad added a comment -

          fho abayer

          also a continued issue for me with "Pipeline: Groovy" plugin version 2.65

          can we get this reopened and fixed?

          Zishan Ahmad added a comment - fho abayer also a continued issue for me with "Pipeline: Groovy" plugin version 2.65 can we get this reopened and fixed?

          Jesse Glick added a comment -

          zishan a particular issue was identified and fixed, with test coverage. Please do not reopen. If you continue to have some issue which you believe is related, file a distinct issue report with complete, minimal steps to reproduce from scratch and link to this issue.

          Jesse Glick added a comment - zishan a particular issue was identified and fixed, with test coverage. Please do not reopen. If you continue to have some issue which you believe is related, file a distinct issue report with complete, minimal steps to reproduce from scratch and link to this issue.

          Fabian Holler added a comment - - edited

          It works only when 'lightweight checkout' is disabled.
          There is the ticket https://issues.jenkins-ci.org/browse/JENKINS-42971 for the issue.

          Fabian Holler added a comment - - edited It works only when 'lightweight checkout' is disabled. There is the ticket https://issues.jenkins-ci.org/browse/JENKINS-42971 for the issue.

          fho thanks a bunch. Was on a wild goose chase here trying to figure this out and was just about to file a ticket here when I saw your comment. Thanks! Once I disabled the lightweight checkout, all worked fine.

          Oliver Schneider added a comment - fho thanks a bunch. Was on a wild goose chase here trying to figure this out and was just about to file a ticket here when I saw your comment. Thanks! Once I disabled the lightweight checkout, all worked fine.

          ipleten added a comment - - edited

          It doesn't work anymore with or without 'lightweight checkout'.
          https://issues.jenkins-ci.org/browse/JENKINS-60250
          UPDATE: it might not work in some cases: see https://issues.jenkins-ci.org/browse/JENKINS-42971?focusedCommentId=370658&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-370658
          UPDATE2: it fails if readTrusted is used.

          ipleten added a comment - - edited It doesn't work anymore with or without 'lightweight checkout'. https://issues.jenkins-ci.org/browse/JENKINS-60250 UPDATE: it might not work in some cases: see https://issues.jenkins-ci.org/browse/JENKINS-42971?focusedCommentId=370658&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-370658 UPDATE2: it fails if readTrusted is used.

            abayer Andrew Bayer
            bkeyser_dgi Brian Keyser
            Votes:
            13 Vote for this issue
            Watchers:
            26 Start watching this issue

              Created:
              Updated:
              Resolved: