• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • docker-workflow-plugin
    • None
    • Jenkins Core 2.73

      When running a build inside a docker container, some commands don't work because they rely on the user being properly set. For example, ssh doesn't work with the following error:

       

      No user exists for uid 150.

       

      I think this could be solved by append to passwd on container startup, something like this (untested, for proof of concept):

      if [ "$(id -u)" != "0" ]; then

          echo "jenkins:x:$(id -u):$(id -g):Jenkins:${HOME}:/sbin/nologin" >> /etc/passwd

      fi

          [JENKINS-47026] User not completely set in docker containers

          Eric Dahlseng created issue -

          Nicola Worthington added a comment - - edited

          Manually modifying /etc/passwd  like this feels quite wrong to me. I understand that this is intended to be a lightweight fix that doesn't rely on extraneous system maintenance packages, but who is to say that any given Docker container is even configured to perform a lookup using /etc/passwd?

          I think the root problem here is that Jenkins relies upon a shared volume model as the transport mechanism to get source into the container, and the build artefacts back out. While this is arguably reasonable for the input vector (permissions of the files being injected can be left fairly open), we clearly see the problems when trying to get files back out.

          Would a better approach be to decouple container workspace filesystem from the host, and implement an RPC mechanism to deliver the build artefacts back to Jenkins.

          I believe GitLab CI does something similar with its job caching and artefact management, so running tasks inside the Docker container as root is not a problem.

          Nicola Worthington added a comment - - edited Manually modifying /etc/passwd  like this feels quite wrong to me. I understand that this is intended to be a lightweight fix that doesn't rely on extraneous system maintenance packages, but who is to say that any given Docker container is even configured to perform a lookup using /etc/passwd? I think the root problem here is that Jenkins relies upon a shared volume model as the transport mechanism to get source into the container, and the build artefacts back out. While this is arguably reasonable for the input vector (permissions of the files being injected can be left fairly open), we clearly see the problems when trying to get files back out. Would a better approach be to decouple container workspace filesystem from the host, and implement an RPC mechanism to deliver the build artefacts back to Jenkins. I believe GitLab CI does something similar with its job caching and artefact management, so running tasks inside the Docker container as root is not a problem.
          Nicola Worthington made changes -
          Link New: This issue relates to JENKINS-31944 [ JENKINS-31944 ]

          Waldek M added a comment - - edited

          I've prepared an example of how using this fixed, non-root user breaks RPM builds, using:

          I'll attach the content of Jenkinsfile and output log (slightly redacted) shortly; example failure:

           

          [docker-build-weeps] Running shell script
          + . /tmp/venv/bin/activate
          [...]
          + python setup.py bdist_rpm
          running bdist_rpm
          running egg_info
          writing semver.egg-info/PKG-INFO
          writing top-level names to semver.egg-info/top_level.txt
          writing dependency_links to semver.egg-info/dependency_links.txt
          reading manifest file 'semver.egg-info/SOURCES.txt'
          reading manifest template 'MANIFEST.in'
          writing manifest file 'semver.egg-info/SOURCES.txt'
          creating build/bdist.linux-x86_64
          creating build/bdist.linux-x86_64/rpm
          creating build/bdist.linux-x86_64/rpm/SOURCES
          creating build/bdist.linux-x86_64/rpm/SPECS
          creating build/bdist.linux-x86_64/rpm/BUILD
          creating build/bdist.linux-x86_64/rpm/RPMS
          creating build/bdist.linux-x86_64/rpm/SRPMS
          writing 'build/bdist.linux-x86_64/rpm/SPECS/semver.spec'
          running sdist
          running check
          creating semver-2.7.9
          creating semver-2.7.9/semver.egg-info
          making hard links in semver-2.7.9...
          hard linking MANIFEST.in -> semver-2.7.9
          hard linking README.rst -> semver-2.7.9
          hard linking semver.py -> semver-2.7.9
          hard linking setup.py -> semver-2.7.9
          hard linking semver.egg-info/PKG-INFO -> semver-2.7.9/semver.egg-info
          hard linking semver.egg-info/SOURCES.txt -> semver-2.7.9/semver.egg-info
          hard linking semver.egg-info/dependency_links.txt -> semver-2.7.9/semver.egg-info
          hard linking semver.egg-info/top_level.txt -> semver-2.7.9/semver.egg-info
          Writing semver-2.7.9/setup.cfg
          creating dist
          Creating tar archive
          removing 'semver-2.7.9' (and everything under it)
          copying dist/semver-2.7.9.tar.gz -> build/bdist.linux-x86_64/rpm/SOURCES
          building RPMs
          rpmbuild -ba --define _topdir /var/spool/jenkins/workspace/docker-build-weeps/build/bdist.linux-x86_64/rpm --clean build/bdist.linux-x86_64/rpm/SPECS/semver.spec
          error: Bad owner/group: /var/spool/jenkins/workspace/docker-build-weeps/build/bdist.linux-x86_64/rpm/SOURCES/semver-2.7.9.tar.gz
          error: command 'rpmbuild' failed with exit status 1

           

           

          Pipeline to reproduce the behaviour is pretty simple:

           

          pipeline {
              agent {
                  docker {
                      image "weakcamel/centos-python2-build:3"
                      label 'cam1'
                  }
              }
              stages {
                  stage('build and unit test') {
                      steps {
                          git changelog: false, poll: false, url: 'https://github.com/k-bx/python-semver.git'
                          script {
                              sh('virtualenv /tmp/venv')
                              sh '. /tmp/venv/bin/activate && python setup.py build'
                              sh('. /tmp/venv/bin/activate && python setup.py bdist_rpm')
                          }
                      }
                  }
              }
          }

          Waldek M added a comment - - edited I've prepared an example of how using this fixed, non-root user breaks RPM builds, using: a sample Docker image based on Centos ( https://hub.docker.com/r/weakcamel/centos-python2-build/) a (picked at almost random) simple PyPI package: https://github.com/k-bx/python-semver.git to use as source code base I'll attach the content of Jenkinsfile and output log (slightly redacted) shortly; example failure:   [docker-build-weeps] Running shell script + . /tmp/venv/bin/activate [...] + python setup.py bdist_rpm running bdist_rpm running egg_info writing semver.egg-info/PKG-INFO writing top-level names to semver.egg-info/top_level.txt writing dependency_links to semver.egg-info/dependency_links.txt reading manifest file 'semver.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'semver.egg-info/SOURCES.txt' creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/rpm creating build/bdist.linux-x86_64/rpm/SOURCES creating build/bdist.linux-x86_64/rpm/SPECS creating build/bdist.linux-x86_64/rpm/BUILD creating build/bdist.linux-x86_64/rpm/RPMS creating build/bdist.linux-x86_64/rpm/SRPMS writing 'build/bdist.linux-x86_64/rpm/SPECS/semver.spec' running sdist running check creating semver-2.7.9 creating semver-2.7.9/semver.egg-info making hard links in semver-2.7.9... hard linking MANIFEST.in -> semver-2.7.9 hard linking README.rst -> semver-2.7.9 hard linking semver.py -> semver-2.7.9 hard linking setup.py -> semver-2.7.9 hard linking semver.egg-info/PKG-INFO -> semver-2.7.9/semver.egg-info hard linking semver.egg-info/SOURCES.txt -> semver-2.7.9/semver.egg-info hard linking semver.egg-info/dependency_links.txt -> semver-2.7.9/semver.egg-info hard linking semver.egg-info/top_level.txt -> semver-2.7.9/semver.egg-info Writing semver-2.7.9/setup.cfg creating dist Creating tar archive removing 'semver-2.7.9' (and everything under it) copying dist/semver-2.7.9.tar.gz -> build/bdist.linux-x86_64/rpm/SOURCES building RPMs rpmbuild -ba --define _topdir /var/spool/jenkins/workspace/docker-build-weeps/build/bdist.linux-x86_64/rpm --clean build/bdist.linux-x86_64/rpm/SPECS/semver.spec error: Bad owner/group: /var/spool/jenkins/workspace/docker-build-weeps/build/bdist.linux-x86_64/rpm/SOURCES/semver-2.7.9.tar.gz error: command 'rpmbuild' failed with exit status 1     Pipeline to reproduce the behaviour is pretty simple:   pipeline {     agent {         docker {             image "weakcamel/centos-python2-build:3"             label 'cam1'         }     }     stages {         stage('build and unit test') {             steps {                 git changelog: false, poll: false, url: 'https://github.com/k-bx/python-semver.git'                 script {                     sh('virtualenv /tmp/venv')                     sh '. /tmp/venv/bin/activate && python setup.py build'                     sh('. /tmp/venv/bin/activate && python setup.py bdist_rpm')                 }             }         }     } }
          Waldek M made changes -
          Attachment New: build_log.txt [ 41383 ]

          Waldek M added a comment - - edited

          Note also that in case you try and use Python's `pip` command, there's a warning as well related to not fully set up user:

           
          {{ + pip install virtualenv}}
          {{ The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.}}
          {{ The directory '/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.}}
          {{ Requirement already satisfied: virtualenv in /usr/lib/python2.7/site-packages}}

          Waldek M added a comment - - edited Note also that in case you try and use Python's `pip` command, there's a warning as well related to not fully set up user:   {{ + pip install virtualenv}} {{ The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.}} {{ The directory '/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.}} {{ Requirement already satisfied: virtualenv in /usr/lib/python2.7/site-packages}}
          Waldek M made changes -
          Attachment New: Jenkinsfile [ 41384 ]
          Nicolas De Loof made changes -
          Component/s New: declarative-pipeline-when-conditions-plugin [ 23168 ]
          Component/s Original: docker [ 20834 ]
          Nicolas De Loof made changes -
          Component/s New: docker-workflow-plugin [ 20625 ]
          Component/s Original: declarative-pipeline-when-conditions-plugin [ 23168 ]

          Waldek M added a comment -

          Related to what the Original Poster mentioned about SSH problems in this setup, there are general problems with using plain Git commands within the container (as initiated by Jenkins).

           

          a) Over SSH:

          $ docker run -u 1223:1223 -it weakcamel/centos-python2-build:3
          bash-4.2$ cd /tmp
          bash-4.2$ git clone git@github.com:k-bx/python-semver.git
          Cloning into 'python-semver'...
          No user exists for uid 1223
          fatal: Could not read from remote repository.
          
          Please make sure you have the correct access rights
          and the repository exists.

           

          b) over HTTPS as well:

           

          bash-4.2$ git clone https://github.com/k-bx/python-semver.git
          Cloning into 'python-semver'...
          remote: Counting objects: 534, done.
          remote: Total 534 (delta 0), reused 0 (delta 0), pack-reused 534
          Receiving objects: 100% (534/534), 89.52 KiB | 0 bytes/s, done.
          Resolving deltas: 100% (285/285), done.
          fatal: unable to look up current user in the passwd file: no such user
          Unexpected end of command stream
          bash-4.2$ echo $?
          128
          bash-4.2$ ls -al python-semver
          ls: cannot access python-semver: No such file or directory

          Waldek M added a comment - Related to what the Original Poster mentioned about SSH problems in this setup, there are general problems with using plain Git commands within the container (as initiated by Jenkins).   a) Over SSH: $ docker run -u 1223:1223 -it weakcamel/centos-python2-build:3 bash-4.2$ cd /tmp bash-4.2$ git clone git@github.com:k-bx/python-semver.git Cloning into 'python-semver' ... No user exists for uid 1223 fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.   b) over HTTPS as well:   bash-4.2$ git clone https: //github.com/k-bx/python-semver.git Cloning into 'python-semver' ... remote: Counting objects: 534, done. remote: Total 534 (delta 0), reused 0 (delta 0), pack-reused 534 Receiving objects: 100% (534/534), 89.52 KiB | 0 bytes/s, done. Resolving deltas: 100% (285/285), done. fatal: unable to look up current user in the passwd file: no such user Unexpected end of command stream bash-4.2$ echo $? 128 bash-4.2$ ls -al python-semver ls: cannot access python-semver: No such file or directory

            Unassigned Unassigned
            edahlseng Eric Dahlseng
            Votes:
            6 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: