Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64954

Docker-in-docker connection misuse



    • Bug
    • Resolution: Unresolved
    • Minor
    • docker-workflow-plugin
    • None
    • Jenkins: 2.190.3
      docker-workflow-plugin: docker-workflow-1.26-1-gb174d46

      (Running via `mvn hpi:run` from docker-workflow-plugin revision gb174d46)


      We are trying to start a docker daemon inside a docker container from a Jenkinsfile and selectivelly use that daemon for certain operations.
      However it seems that Jenkins gets confused about which docker daemon to contact, failing the build.

      Reproduction steps

      Tested with docker-workflow-plugin docker-workflow-1.26-1-gb174d46 (current git version).


      node {
          docker.image("docker:dind").withRun("--privileged", "dockerd --tls=false --host") { dockerContainer ->
              docker.image("docker:dind").inside("--link ${dockerContainer.id}:docker --entrypoint=''") {
                  docker.withServer("tcp://docker") {
                      sh "echo with nested docker"

      Job output

      Started by user unknown or anonymous
      Running in Durability level: MAX_SURVIVABILITY
      [Pipeline] Start of Pipeline
      [Pipeline] node
      Running on Jenkins in ...
      [Pipeline] {
      [Pipeline] isUnix
      [Pipeline] sh
      + docker run -d --privileged docker:dind dockerd --tls=false --host
      [Pipeline] isUnix
      [Pipeline] sh
      + docker inspect -f . docker:dind
      [Pipeline] withDockerContainer
      Jenkins does not seem to be running inside a container
      $ docker run -t -d -u 62407:100 --link 983fa1038732d92d6490e70a13d02b81287bf737eecce485efd48d356c2ac593:docker --entrypoint= -w ... -v ...:rw,z -v ...:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** docker:dind cat
      $ docker top 42a5f9bbb303b7c37736f712fe6d24ca6f7d1b247aa0c89f5fb2cfc8087a3e56 -eo pid,comm
      [Pipeline] {
      [Pipeline] withDockerServer
      [Pipeline] {
      [Pipeline] sh
      process apparently never started in .../workspace/dind@tmp/durable-1c014c22
      (running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
      [Pipeline] }
      [Pipeline] // withDockerServer
      [Pipeline] }
      $ docker stop --time=1 42a5f9bbb303b7c37736f712fe6d24ca6f7d1b247aa0c89f5fb2cfc8087a3e56
      $ docker rm -f 42a5f9bbb303b7c37736f712fe6d24ca6f7d1b247aa0c89f5fb2cfc8087a3e56
      [Pipeline] // withDockerContainer
      [Pipeline] isUnix
      [Pipeline] sh
      + docker stop 983fa1038732d92d6490e70a13d02b81287bf737eecce485efd48d356c2ac593
      + docker rm -f 983fa1038732d92d6490e70a13d02b81287bf737eecce485efd48d356c2ac593
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: script returned exit code -2
      Finished: FAILURE

      Started containers

      CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
      42a5f9bbb303        docker:dind         "cat"                    5 seconds ago       Up 4 seconds        2375-2376/tcp       epic_cori
      983fa1038732        docker:dind         "dockerd-entrypoint.…"   6 seconds ago       Up 5 seconds        2375-2376/tcp       fervent_heyrovsky

      Command executed by DurableTask:

      # heavily trimmed down
      DOCKER_HOST=tcp://docker docker exec --env DOCKER_HOST=tcp://docker 42a5f9bbb303b7c $COMMAND


      The execution of commands is intercepted by WithContainerStep through org.jenkinsci.plugins.docker.workflow.WithContainerStep.Decorator.
      While this LauncherDecorator explicitly keeps track of the container id to launch the command in, it uses the DOCKER_HOST variable from the environment variables that are active at the time of the execution of the step instead of the one that was used during the creation of the container.

      In this example the "docker exec" invocation itself should not have the environment variable `DOCKER_HOST` set to `tcp://docker`, while it should be passed to the command started inside the container.

      I propose (and offer to implement) to keep explicitly track of the docker connection used to create a container and use it every time this container is to be interacted with.




            Unassigned Unassigned
            t8ch Thomas Weißschuh
            1 Vote for this issue
            2 Start watching this issue