Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49466

job directory empty when using declarative pipeline

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Minor Minor
    • pipeline, pipeline-model-definition-plugin
    • None

      I'm creating a job with a declarative pipeline; I'm coming across an issue similar to:

      https://issues.jenkins-ci.org/browse/JENKINS-42440

      only with a declarative pipeline. Jenkins does automagically check out the code, but it ends up in a

      $WORKSPACE/foo_pipeline-XXXXX@script

      directory instead of

      $WORKSPACE/foo_pipeline-XXXXX
      giving an error like:

      Running on Jenkins in /var/lib/jenkins/workspace/foo_pipeline-XXXXX
[...]
java.io.FileNotFoundException: /var/lib/jenkins/workspace/foo_pipeline-XXXXX/package.json does not exist.
	at org.jenkinsci.plugins.pipeline.utility.steps.json.ReadJSONStepExecution.doRun(ReadJSONStepExecution.java:69)
	at org.jenkinsci.plugins.pipeline.utility.steps.json.ReadJSONStepExecution.doRun(ReadJSONStepExecution.java:45)
	at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepExecution.run(AbstractFileOrTextStepExecution.java:23)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
	at hudson.security.ACL.impersonate(ACL.java:290)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

      pipeline {
  agent any
  parameters {
    choice(choices: 'foo\nbar',
           description: 'Which package to build',
           name: 'packageType'
    )
  }
  stages {
    stage('Build and push image') {
      steps {
        script {
          // requires "Pipeline Utility Steps" plugin
          def packageJson = readJSON file:'package.json'

          echo "${packageJson.version}"
          // more stuff here
        }
      }
    }

  }
  post {
    always {
      // Uses Workspace Cleanup plugin
      cleanWs()
    }
  }
}

      This was easy to miss when I wasn't clearing out the job workspace each time, but I think that would actually have been worse, since I'm guessing Jenkins was looking in one place, but running / changing things in the other.

          [JENKINS-49466] job directory empty when using declarative pipeline

          Will Yardley created issue -
          Will Yardley made changes -
          Description Original: I'm creating a job with a declarative pipeline; I'm coming across an issue similar to:

          https://issues.jenkins-ci.org/browse/JENKINS-42440

          only with a declarative pipeline. Jenkins does automagically check out the code, but it ends up in a

          {{$WORKSPACE/foo_pipeline-XXXXX@script}}

          directory instead of

          {{$WORKSPACE/foo_pipeline-XXXXX}}
          giving an error like:

          {noformat}
          Running on Jenkins in /var/lib/jenkins/workspace/foo_pipeline-XXXXX
          [...]
          java.io.FileNotFoundException: /var/lib/jenkins/workspace/foo_pipeline-XXXXX/package.json does not exist.
          at org.jenkinsci.plugins.pipeline.utility.steps.json.ReadJSONStepExecution.doRun(ReadJSONStepExecution.java:69)
          at org.jenkinsci.plugins.pipeline.utility.steps.json.ReadJSONStepExecution.doRun(ReadJSONStepExecution.java:45)
          at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepExecution.run(AbstractFileOrTextStepExecution.java:23)
          at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
          at hudson.security.ACL.impersonate(ACL.java:290)
          at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
          Finished: FAILURE
          {noformat}

          {code}
          pipeline {
            agent any
            parameters {
              choice(choices: 'foo\nbar',
                     description: 'Which package to build',
                     name: 'packageType'
              )
            }
            stages {
              stage('Build and push image') {
                steps {
                  script {
                    // requires "Pipeline Utility Steps" plugin
                    def packageJson = readJSON file:'package.json'

                    echo "${packageJson.version}"
                    // more stuff here
                  }
                }
              }

            }
            post {
              always {
                // Uses Workspace Cleanup plugin
                cleanWs()
              }
            }
          }
          {code}           		New: I'm creating a job with a declarative pipeline; I'm coming across an issue similar to:

          https://issues.jenkins-ci.org/browse/JENKINS-42440

          only with a declarative pipeline. Jenkins does automagically check out the code, but it ends up in a

          {{$WORKSPACE/foo_pipeline-XXXXX@script}}

          directory instead of

          {{$WORKSPACE/foo_pipeline-XXXXX}}
          giving an error like:

          {noformat}
          Running on Jenkins in /var/lib/jenkins/workspace/foo_pipeline-XXXXX
          [...]
          java.io.FileNotFoundException: /var/lib/jenkins/workspace/foo_pipeline-XXXXX/package.json does not exist.
          at org.jenkinsci.plugins.pipeline.utility.steps.json.ReadJSONStepExecution.doRun(ReadJSONStepExecution.java:69)
          at org.jenkinsci.plugins.pipeline.utility.steps.json.ReadJSONStepExecution.doRun(ReadJSONStepExecution.java:45)
          at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepExecution.run(AbstractFileOrTextStepExecution.java:23)
          at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
          at hudson.security.ACL.impersonate(ACL.java:290)
          at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
          Finished: FAILURE
          {noformat}

          {code}
          pipeline {
            agent any
            parameters {
              choice(choices: 'foo\nbar',
                     description: 'Which package to build',
                     name: 'packageType'
              )
            }
            stages {
              stage('Build and push image') {
                steps {
                  script {
                    // requires "Pipeline Utility Steps" plugin
                    def packageJson = readJSON file:'package.json'

                    echo "${packageJson.version}"
                    // more stuff here
                  }
                }
              }

            }
            post {
              always {
                // Uses Workspace Cleanup plugin
                cleanWs()
              }
            }
          }
          {code}

          This was easy to miss when I wasn't clearing out the job workspace each time, but I think that would actually have been worse, since I'm guessing Jenkins was looking in one place, but running / changing things in the other.

          Will Yardley added a comment - - edited

          If I print out env.WORKSPACE either inside or outside of the script block, I get the same value as Running on Jenkins in /var/lib/jenkins/workspace/foo_pipeline-XXXXX, but that directory is empty if I don't clean it out after builds, whereas the one with @script has the checked out copy.

          I tried adding a checkout SCM stage, then I get:
          ERROR: Could not determine exact tip revision of pipeline

          Will Yardley added a comment - - edited If I print out env.WORKSPACE either inside or outside of the script block, I get the same value as Running on Jenkins in /var/lib/jenkins/workspace/foo_pipeline-XXXXX , but that directory is empty if I don't clean it out after builds, whereas the one with @script has the checked out copy. I tried adding a checkout SCM stage, then I get: ERROR: Could not determine exact tip revision of pipeline
          Oleg Nenashev made changes -
          Component/s New: pipeline-model-definition-plugin [ 21706 ]
          Andrew Bayer made changes -
          Link New: This issue duplicates JENKINS-40862 [ JENKINS-40862 ]
          Andrew Bayer made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]
          Will Yardley made changes -
          Resolution Original: Duplicate [ 3 ]
          Status Original: Resolved [ 5 ] New: Reopened [ 4 ]

          Will Yardley added a comment -

          I don't think this is a dupe of JENKINS-40862 – the error in the comment only shows up if I try to work around the problem by adding "checkout scm" in the Jenkinsfile.

          Will Yardley added a comment - I don't think this is a dupe of JENKINS-40862 – the error in the comment only shows up if I try to work around the problem by adding "checkout scm" in the Jenkinsfile.

          Andrew Bayer added a comment -

          wyardley - so I'm 99% sure that this is the same thing. We do an automatic checkout scm whenever you go to a new agent, but only if the skipDefaultCheckout option isn't specified and the scm object exists/is valid. The Could not determine exact tip revision of pipeline error message is, basically, what happens when you try to do a checkout scm on a null or otherwise broken scm object - you don't see that show up without the explicit checkout scm because Declarative doesn't even try doing it when the scm object is null/invalid.

          Andrew Bayer added a comment - wyardley - so I'm 99% sure that this is the same thing. We do an automatic checkout scm whenever you go to a new agent, but only if the skipDefaultCheckout option isn't specified and the scm object exists/is valid. The Could not determine exact tip revision of pipeline error message is, basically, what happens when you try to do a checkout scm on a null or otherwise broken scm object - you don't see that show up without the explicit checkout scm because Declarative doesn't even try doing it when the scm object is null/invalid.

          Will Yardley added a comment -

          abayer In the case where it's not explicit, the code does get checked out, though. Just that it ends up in foo@script instead of foo. So the code gets checked out, just not to the place where Jenkins expects it to be.

          Will Yardley added a comment - abayer In the case where it's not explicit, the code does get checked out, though. Just that it ends up in foo@script instead of foo . So the code gets checked out, just not to the place where Jenkins expects it to be.

            Unassigned Unassigned
            wyardley Will Yardley
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: