Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-29760

Docker in Docker support - use volumes-from for handling host filesystem

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Right now the plugin will launch a single Docker container and mount the job folder from host machine. The parameters used will start the container with the same user and group as the host so that the Docker container can write these files.

      This is a problem when you need to run Docker as root, for example when running Docker-in-Docker.

      To address this, the plugin could create a separate, storage only container, which mounts the host's folder with the right credentials. After that, the job Docker container would start with whatever user we need and mount the data container with 'volumes-from'.

      This way we have best of both worlds - ability to use any user in the job container, and write to host's filesystem.

      It might be worth exploring which option would work better in terms of performance and concurrency - having a single storage container per host or create a storage container for each job that is being requested.

        Attachments

          Issue Links

            Activity

            Hide
            ndeloof Nicolas De Loof added a comment - - edited

            Proposed approach won't fix the permission issue : as long as a process do run with arbitrary user ID, it will create files in workspace with unexpected permission that jenkins won't be able to handle later.

            About DinD, it's possible to make the lanch command configurable, so you can run `wrapdocker /bin/cat`, then have all subsequent build steps ran with `docker exec` with the user option set. But you won't then be able to run nested containers until you have prepared your docker image so Jenkins build user belongs the docker group :-\

            Show
            ndeloof Nicolas De Loof added a comment - - edited Proposed approach won't fix the permission issue : as long as a process do run with arbitrary user ID, it will create files in workspace with unexpected permission that jenkins won't be able to handle later. About DinD, it's possible to make the lanch command configurable, so you can run `wrapdocker /bin/cat`, then have all subsequent build steps ran with `docker exec` with the user option set. But you won't then be able to run nested containers until you have prepared your docker image so Jenkins build user belongs the docker group :-\
            Hide
            ndeloof Nicolas De Loof added a comment -

            I guess we will get stuck with this user issue until docker do use user-namespace

            Show
            ndeloof Nicolas De Loof added a comment - I guess we will get stuck with this user issue until docker do use user-namespace
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Nicolas De Loof
            Path:
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/BuiltInContainer.java
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/Docker.java
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper.java
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerDecoratedLauncher.java
            src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/config.jelly
            src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-command.html
            src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-group.html
            http://jenkins-ci.org/commit/docker-custom-build-environment-plugin/aaebd9fc5449ca9b4e68c57342e7ce06c733b5f1
            Log:
            JENKINS-29760 Docker in Docker support (and other similar advanced use cases)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/BuiltInContainer.java src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/Docker.java src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper.java src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerDecoratedLauncher.java src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/config.jelly src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-command.html src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-group.html http://jenkins-ci.org/commit/docker-custom-build-environment-plugin/aaebd9fc5449ca9b4e68c57342e7ce06c733b5f1 Log: JENKINS-29760 Docker in Docker support (and other similar advanced use cases)
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Nicolas De Loof
            Path:
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/BuiltInContainer.java
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/Docker.java
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper.java
            src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerDecoratedLauncher.java
            src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/config.jelly
            src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-command.html
            src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-group.html
            src/test/java/com/cloudbees/jenkins/plugins/docker_build_env/FunctionalTests.java
            http://jenkins-ci.org/commit/docker-custom-build-environment-plugin/8bcb202a7b83a131894cad802ea98748f94e13fb
            Log:
            JENKINS-29760 Docker in Docker support (and other similar advanced use cases)

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/BuiltInContainer.java src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/Docker.java src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper.java src/main/java/com/cloudbees/jenkins/plugins/docker_build_env/DockerDecoratedLauncher.java src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/config.jelly src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-command.html src/main/resources/com/cloudbees/jenkins/plugins/docker_build_env/DockerBuildWrapper/help-group.html src/test/java/com/cloudbees/jenkins/plugins/docker_build_env/FunctionalTests.java http://jenkins-ci.org/commit/docker-custom-build-environment-plugin/8bcb202a7b83a131894cad802ea98748f94e13fb Log: JENKINS-29760 Docker in Docker support (and other similar advanced use cases)

              People

              Assignee:
              ndeloof Nicolas De Loof
              Reporter:
              jsniecikowski Jacek Sniecikowski
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: