Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41894

deleteDir() for cwd removes workspace when run in docker.image('builder').inside

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • docker-workflow-plugin
    • None
    • Pipeline basic steps: 2.3
      Docker pipeline: 1.9.1
      Jenkins: 2.19.3

      This snippet (with docker.image.inside) fails:

      node('default') {
        docker.image('builder').inside {
          deleteDir()
          checkout scm
          sh 'git rev-parse HEAD > git-revision.tmp'
          env.REVISION = readFile('git-revision.tmp').trim()
        }
      }
      

      While this snippet doesn't:

      node('default') {
        deleteDir()
        checkout scm
        sh 'git rev-parse HEAD > git-revision.tmp'
        env.REVISION = readFile('git-revision.tmp').trim()
      }
      

      The failure begins with the attempt to use the checked out files, with this error:

      [Pipeline] sh
      [FLE3OSMVFPS5YXYDBJO2SDKESHOZUWIWXDMRIPXDFZ27TAUUNB6Q] Running shell script
      shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
      job-working-directory: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
      shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
      + git rev-parse HEAD
      job-working-directory: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
      /home/jenkins/workspace/-FLE3OSMVFPS5YXYDBJO2SDKESHOZUWIWXDMRIPXDFZ27TAUUNB6Q@tmp/durable-f4c5cca7/script.sh: line 2: git-revision.tmp: No such file or directory
      Raw
      

      deleteDir() calls hudson.FilePath.deleteRecursive() which "Deletes this directory, including all its contents recursively."

      Somehow when not working with docker, the pipeline recovers from this situation. But when doing that in a docker image, it fails.

          [JENKINS-41894] deleteDir() for cwd removes workspace when run in docker.image('builder').inside

          Jesse Glick added a comment -

          Put the deleteDir outside the Image.inside.

          Probably WithContainerStep is failing to mkdirs its workspace.

          Jesse Glick added a comment - Put the deleteDir outside the Image.inside . Probably WithContainerStep is failing to mkdirs its workspace.

          David Resnick added a comment -

          Sure, though this is easier said than done when logic is in an external library (like here) and calling git requires the container image.

          David Resnick added a comment - Sure, though this is easier said than done when logic is in an external library (like here ) and calling git requires the container image.

          Robby Pocase added a comment -

          Is it safe to assume this also affects post stages for a declarative docker agent? Seem to be hitting issues with something along the following.

          pipeline {
             agent { docker { image 'image' } }
             ...
             post { always { deleteDir() } }
          }
          

          Robby Pocase added a comment - Is it safe to assume this also affects post stages for a declarative docker agent? Seem to be hitting issues with something along the following. pipeline { agent { docker { image 'image' } } ... post { always { deleteDir() } } }

          I confirm Robby assumption my workaround was to use WipeWorkspace plugin with checkout

          checkout([
          $class: 'GitSCM',
          branches: scm.branches,
          extensions: scm.extensions + [[$class: 'WipeWorkspace']],
          userRemoteConfigs: scm.userRemoteConfigs
          ])

          Frederic Rousseau added a comment - I confirm Robby assumption my workaround was to use WipeWorkspace plugin with checkout checkout([ $class: 'GitSCM', branches: scm.branches, extensions: scm.extensions + [ [$class: 'WipeWorkspace'] ], userRemoteConfigs: scm.userRemoteConfigs ])

          Jon B added a comment -

          I found my way here after I noticed that declarative pipeline sh steps fail if you try to ensure the workspace is clear when using either deleteDir() or cleanWs()

          I have to do some really hacky workaround to make this work as desired.

          A reasonable person would assume that you can call deleteDir() anywhere to help ensure the WORKSPACE is empty. Unfortunately, deleteDir() physically blows away the WORKSPACE which screws up docker declarative agents.

          Jon B added a comment - I found my way here after I noticed that declarative pipeline sh steps fail if you try to ensure the workspace is clear when using either deleteDir() or cleanWs() I have to do some really hacky workaround to make this work as desired. A reasonable person would assume that you can call deleteDir() anywhere to help ensure the WORKSPACE is empty. Unfortunately, deleteDir() physically blows away the WORKSPACE which screws up docker declarative agents.

          As above, echo Jon B.

          I had a scripted pipline, converted it to a declarative (is there a good reason to still?) and found that deleteDir() hoses the working folder. I semantically wanted a cleanWorkspace() method, for newbies to groovy ; a better docs description of each method?

          Zaphod Beeblebrox added a comment - As above, echo Jon B. I had a scripted pipline, converted it to a declarative (is there a good reason to still?) and found that deleteDir() hoses the working folder. I semantically wanted a cleanWorkspace() method, for newbies to groovy ; a better docs description of each method?

          Jesse Glick added a comment -

          I was able to reproduce this. I do not think there is any fix possible—you just should not use deleteDir at this point. You may use it before running the withDockerContainer step. The reason is that the container will mount the working directory as a volume, and if you delete that directory, it has nothing to work with.

          Jesse Glick added a comment - I was able to reproduce this. I do not think there is any fix possible—you just should not use deleteDir at this point. You may use it before running the withDockerContainer step. The reason is that the container will mount the working directory as a volume, and if you delete that directory, it has nothing to work with.

          Joan Touzet added a comment - - edited

          Just ran into this when using declarative pipeline + docker agent steps ... how very unfortunate.

          A workaround for now is to replace deleteDir calls with sh 'rm -rf ${WORKSPACE}/' (possibly including ${WORKSPACE}/.[a-zA-Z]) but it's ugly.

          This should be documented, at the very least.

          Joan Touzet added a comment - - edited Just ran into this when using declarative pipeline + docker agent steps ... how very unfortunate. A workaround for now is to replace deleteDir calls with sh 'rm -rf ${WORKSPACE}/ ' (possibly including ${WORKSPACE}/. [a-zA-Z] ) but it's ugly. This should be documented, at the very least.

          jglick is it reasonable to throw some kind of exception or user-friendlier error when this is attempted, since it's a limitation of Docker and not going to be fixed? "You cannot use that command in this context" or similar?

          Aaron D. Marasco added a comment - jglick is it reasonable to throw some kind of exception or user-friendlier error when this is attempted, since it's a limitation of Docker and not going to be fixed? "You cannot use that command in this context" or similar?

          Jesse Glick added a comment -

          wohali if Declarative does not provide an option to start the build with a clean workspace then perhaps it should. I do not recall offhand. It is just sugar for some underlying Scripted definition.

          aarondmarasco_vsi Not straightforward to have the deleteDir warn anything, but the sh in this context could perhaps abort with a message to the effect that the expected mount point disappeared unexpectedly between the start of the withDockerContainer block and now. Technically, that would be an enhancement in this plugin’s DecoratedLauncher.

          Jesse Glick added a comment - wohali if Declarative does not provide an option to start the build with a clean workspace then perhaps it should. I do not recall offhand. It is just sugar for some underlying Scripted definition. aarondmarasco_vsi Not straightforward to have the deleteDir warn anything, but the sh in this context could perhaps abort with a message to the effect that the expected mount point disappeared unexpectedly between the start of the withDockerContainer block and now. Technically, that would be an enhancement in this plugin’s DecoratedLauncher .

            Unassigned Unassigned
            david_resnick David Resnick
            Votes:
            11 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated: