Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28821

Docker Workflow demos: sh step on Docker image hangs if workspace is not writable

    XMLWordPrintable

Details

    Description

      A sample demo workflow script hangs infinitely on the shell step.
      Logs are available below, stackdump is attached.

      Workflow:

      docker.image('maven:3.3.3-jdk-8').inside {
        git url: "https://github.com/oleg-nenashev/ircbot.git"
        sh 'echo hello'
        //Same thing with mvn -B clean install'
      }
      

      Build log

      Started by user anonymous
      Running: Allocate node : Start
      Running on master in /Users/nenashev/Documents/cloudbees/demo/docker-traceability/work/jobs/test-example-1/workspace
      Running: Allocate node : Body : Start
      Running: Shell Script
      [workspace] Running shell script
      + docker inspect -f . maven:3.3.3-jdk-8
      .
      Running: Run build steps inside a Docker container : Start
      $ docker run -t -d -u 502:20 -w /Users/nenashev/Documents/cloudbees/demo/docker-traceability/work/jobs/test-example-1/workspace -v /Users/nenashev/Documents/cloudbees/demo/docker-traceability/work/jobs/test-example-1/workspace:/Users/nenashev/Documents/cloudbees/demo/docker-traceability/work/jobs/test-example-1/workspace:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** maven:3.3.3-jdk-8 cat
      Running: Run build steps inside a Docker container : Body : Start
      Running: Git
       > git rev-parse --is-inside-work-tree # timeout=10
      Fetching changes from the remote Git repository
       > git config remote.origin.url https://github.com/oleg-nenashev/ircbot.git # timeout=10
      Fetching upstream changes from https://github.com/oleg-nenashev/ircbot.git
       > git --version # timeout=10
       > git -c core.askpass=true fetch --tags --progress https://github.com/oleg-nenashev/ircbot.git +refs/heads/*:refs/remotes/origin/*
       > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
       > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
      Checking out Revision eaa8c349f12288276e5c27fcef78c245bb41517a (refs/remotes/origin/master)
       > git config core.sparsecheckout # timeout=10
       > git checkout -f eaa8c349f12288276e5c27fcef78c245bb41517a
       > git rev-list eaa8c349f12288276e5c27fcef78c245bb41517a # timeout=10
      Running: Shell Script
      [workspace] Running shell script
      <<< hangs here
      

      Attachments

        Issue Links

          Activity

            kmusard Kris Musard added a comment -

            kmadel In my setup I had a Docker server that was also running jenkins. Once I setup a jenkins user inside the container with matching uid:gid to the server running jenkins, the sh commands no longer hung.

            kmusard Kris Musard added a comment - kmadel In my setup I had a Docker server that was also running jenkins. Once I setup a jenkins user inside the container with matching uid:gid to the server running jenkins, the sh commands no longer hung.
            oleg_nenashev Oleg Nenashev added a comment -

            Stopped the progress since jglick didn't like https://github.com/jenkinsci/docker-workflow-plugin/pull/7 . I have not implemented anything else

            oleg_nenashev Oleg Nenashev added a comment - Stopped the progress since jglick didn't like https://github.com/jenkinsci/docker-workflow-plugin/pull/7 . I have not implemented anything else
            kzantow Keith Zantow added a comment -

            I was able to work around this without modifying the container by explicitly specifying the uid to match the mounted filesystem (uid: 1000, gid: 50).

            Like this:

            docker.image('maven:3.3.3-jdk-8').inside('-u 1000:50') {
              git '...'
              sh 'mvn -B clean install'
            }
            

            Less than ideal, as the uid is specified twice in the command line...

            kzantow Keith Zantow added a comment - I was able to work around this without modifying the container by explicitly specifying the uid to match the mounted filesystem (uid: 1000, gid: 50). Like this: docker.image( 'maven:3.3.3-jdk-8' ).inside( '-u 1000:50' ) { git '...' sh 'mvn -B clean install' } Less than ideal, as the uid is specified twice in the command line...
            jglick Jesse Glick added a comment -

            Seems there are various things being mixed together here.

            First is the robustness issue: if the control directory cannot be created, we had better report that properly, and fail. JENKINS-28400 covers this.

            Then there is an issue reported with DinD, which is perhaps best avoided by using bind-mounts of the Docker socket; PR 31 purports to address this.

            Then there was a comment about containers with an entry point. If reproducible, clearly a distinct bug.

            Then there is some discussion of UID mismatches, which PR 25 might address, though there is no information given there about how to reproduce.

            jglick Jesse Glick added a comment - Seems there are various things being mixed together here. First is the robustness issue: if the control directory cannot be created, we had better report that properly, and fail. JENKINS-28400 covers this. Then there is an issue reported with DinD, which is perhaps best avoided by using bind-mounts of the Docker socket; PR 31 purports to address this. Then there was a comment about containers with an entry point. If reproducible, clearly a distinct bug. Then there is some discussion of UID mismatches, which PR 25 might address, though there is no information given there about how to reproduce.
            jglick Jesse Glick added a comment -

            Closing given the confused set of issues here. For any particular bug which causes the workspace to not be writable (or not mounted, etc.), please file a distinct report with steps to reproduce.

            jglick Jesse Glick added a comment - Closing given the confused set of issues here. For any particular bug which causes the workspace to not be writable (or not mounted, etc.), please file a distinct report with steps to reproduce.

            People

              Unassigned Unassigned
              oleg_nenashev Oleg Nenashev
              Votes:
              3 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: