Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37069

Permission denied on durable task directory when using docker.image.inside step on fresh install of jenkins

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • docker-workflow-plugin

    Description

      [Maybe related to issues JENKINS-28821, JENKINS-33632 and JENKINS-36842]

      Hello,

      I am trying to stablish a new CI environment with jobs running on docker but I am running into permission issues. I tried to create a minimal reproducible scenario without slave machines.

      Looking at some of the related issues, I wonder whether I was supposed to configure permissions or group membership for the jenkins user on the container (instead of just using the image as is), but I assume that is not the case.

      Starting on a clean Centos 7 vm, I installed jenkins 2.7.1 and docker, and then added the jenkins user to the docker group (ansible playbook follows). Then I only installed "Pipeline" and "CloudBees Docker Pipeline" plugins and its dependencies. Everything is updated as of today. Then I created a single pipeline job:

      node {
         docker.image('centos:7').inside {
            sh 'pwd'
         }
      }
      

      This job fails with permission issues:

      Started by user admin
      [Pipeline] node
      Running on master in /var/lib/jenkins/workspace/container-test
      [Pipeline] {
      [Pipeline] sh
      [container-test] Running shell script
      + docker inspect -f . centos:7
      .
      [Pipeline] withDockerContainer
      $ docker run -t -d -u 992:989 -w /var/lib/jenkins/workspace/container-test -v /var/lib/jenkins/workspace/container-test:/var/lib/jenkins/workspace/container-test:rw -v /var/lib/jenkins/workspace/container-test@tmp:/var/lib/jenkins/workspace/container-test@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** centos:7 cat
      [Pipeline] {
      [container-test] Running shell script
      [Pipeline] sh
      sh: /var/lib/jenkins/workspace/container-test@tmp/durable-890dccc6/pid: Permission denied
      sh: /var/lib/jenkins/workspace/container-test@tmp/durable-890dccc6/jenkins-log.txt: Permission denied
      sh: /var/lib/jenkins/workspace/container-test@tmp/durable-890dccc6/jenkins-result.txt: Permission denied
      [Pipeline] }
      $ docker stop c71c65555ff53a1bd87db33a9d240c6eb4ae14d9c61a0a0a348c7f72f82b7a50
      $ docker rm -f c71c65555ff53a1bd87db33a9d240c6eb4ae14d9c61a0a0a348c7f72f82b7a50
      [Pipeline] // withDockerContainer
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: script returned exit code -2
      Finished: FAILURE
      

      Tools were installed using the following ansible recipe:

      ---
      - hosts: jenkins-minimal
      
        tasks:
        - yum: name={{ item }} state=installed
          with_items:
            - libselinux-python
            - dejavu-sans-fonts
            - fontconfig
            - java-1.8.0-openjdk-headless
            - docker
      
        - yum_repository:
            name: jenkins
            description: 'Jenkins-stable'
            baseurl: http://pkg.jenkins.io/redhat-stable
            gpgkey: http://pkg.jenkins.io/redhat-stable/jenkins.io.key
      
        - yum: name=jenkins state=installed
      
        - group: name=docker
        - user: name=jenkins groups=docker
      
        - firewalld: port=8080/tcp state=enabled permanent=true immediate=yes
      
        - service: name={{ item }} state=started enabled=yes
          with_items:
            - jenkins
            - docker
      

      Attachments

        Activity

          rodrigc Craig Rodrigues added a comment - - edited

          jglick Ah! OK, that makes a lot of sense. I switched to an a completely Linux + Docker environment,
          and re-ran the pipeline and it worked.
          I don't know if it is possible, but it might be nice to put some error messages
          in the Docker plugin to indicate that this type of thing is not supported on platforms that are non-native Docker ports, like Mac.

          rodrigc Craig Rodrigues added a comment - - edited jglick Ah! OK, that makes a lot of sense. I switched to an a completely Linux + Docker environment, and re-ran the pipeline and it worked. I don't know if it is possible, but it might be nice to put some error messages in the Docker plugin to indicate that this type of thing is not supported on platforms that are non-native Docker ports, like Mac.
          jglick Jesse Glick added a comment -

          Well the documentation does mention this but if I can figure out a way for Jenkins to automatically detect this situation and report a nicer error I will do so.

          jglick Jesse Glick added a comment - Well the documentation does mention this but if I can figure out a way for Jenkins to automatically detect this situation and report a nicer error I will do so.

          rodrigc - If it is helpful I was able to get it to work using the xhyve driver with experimental NFS share

          brew install docker-machine-driver-xhyve
          sudo chown root:wheel /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
          sudo chmod u+s /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
          docker-machine create -d xhyve --xhyve-experimental-nfs-share default2
          eval $(docker-machine env default2)
          jenkins
          

          Then it worked for me.

          mmccaskill Michael McCaskill added a comment - rodrigc - If it is helpful I was able to get it to work using the xhyve driver with experimental NFS share brew install docker-machine-driver-xhyve sudo chown root:wheel /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve sudo chmod u+s /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve docker-machine create -d xhyve --xhyve-experimental-nfs-share default2 eval $(docker-machine env default2) jenkins Then it worked for me.

          rodrigc - Another option I've used successfully recently was using vagrant.

          • vagrant init ubuntu/xenial64
          • vagrant up
          • install JDK and Docker
          • Copy appropriate SSH public key to /home/ubuntu/.ssh/authorized_keys
          • Setup this vagrant machine as a SSH Slave

          For my purposes I did mount my /Users -> /Users via the Vagrantfile and it works nicely. You may want to label the node as 'docker' and have the Jenkinsfile use that node.

          mmccaskill Michael McCaskill added a comment - rodrigc - Another option I've used successfully recently was using vagrant. vagrant init ubuntu/xenial64 vagrant up install JDK and Docker Copy appropriate SSH public key to /home/ubuntu/.ssh/authorized_keys Setup this vagrant machine as a SSH Slave For my purposes I did mount my /Users -> /Users via the Vagrantfile and it works nicely. You may want to label the node as 'docker' and have the Jenkinsfile use that node.

          rodrigc - Another option that works that's much easier to continue using the virtualbox driver and

          https://github.com/adlogix/docker-machine-nfs

          brew install docker-machine-nfs
          docker-machine-nfs <name of docker-machine>
          
          mmccaskill Michael McCaskill added a comment - rodrigc - Another option that works that's much easier to continue using the virtualbox driver and https://github.com/adlogix/docker-machine-nfs brew install docker-machine-nfs docker-machine-nfs <name of docker-machine>

          People

            Unassigned Unassigned
            seuvitor Vitor Dantas
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated: