Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39243

image.inside() does not respect docker.withServer() settings

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • docker-workflow-plugin
    • None
    • CloudBees Docker Pipeline 1.9
      Jenkins is running inside a docker container
      Docker Server is accessed via tcp (TLS)
      Jenkins home directory is mounted via volume

      Using image.inside() inside docker.withServer() does fails with the following error:

      [Pipeline] withDockerContainer
      $ docker run -t -d -u 1000:1000 -w "/var/jenkins_home/workspace/Utility Jobs/copy-schema" -v "/var/jenkins_home/workspace/Utility Jobs/copy-schema:/var/jenkins_home/workspace/Utility Jobs/copy-schema:rw" -v "/var/jenkins_home/workspace/Utility Jobs/copy-schema@tmp:/var/jenkins_home/workspace/Utility Jobs/copy-schema@tmp:rw" -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** --entrypoint cat dockerimage
      [Pipeline] {
      [Pipeline] sh
      [copy-schema] Running shell script
      sh: /var/jenkins_home/workspace/Utility Jobs/copy-schema@tmp/durable-f8d02fd0/pid: No such file or directory
      sh: /var/jenkins_home/workspace/Utility Jobs/copy-schema@tmp/durable-f8d02fd0/jenkins-log.txt: No such file or directory
      sh: /var/jenkins_home/workspace/Utility Jobs/copy-schema@tmp/durable-f8d02fd0/jenkins-result.txt: No such file or directory
      [Pipeline] }
      $ docker stop --time=1 d1f1d8173fcc89762825aeee7019aa24e88e4f9c87c08036083118cd4c708f26
      $ docker rm -f d1f1d8173fcc89762825aeee7019aa24e88e4f9c87c08036083118cd4c708f26
      [Pipeline] // withDockerContainer
      

      I've debugged my Jenkins instance and it turns out that the docker inspect calls inside org.jenkinsci.plugins.docker.workflow.WithContainerStep.Execution.start() fail silently because the docker client can't connect to the docker host. The reason seems to be that the call to org.jenkinsci.plugins.docker.workflow.client.DockerClient.getVolumes(EnvVars, String) uses envHost which does not contain the environment variables set by withServer(). If I pass in all environment variables ( env) the call succeeds and the inside() step succeeds as well (the volume from the master is reused using --volumes-from).

      I'm not sure about the other implications of passing in env instead of envHost.

          [JENKINS-39243] image.inside() does not respect docker.withServer() settings

          Christian Fraenkel created issue -
          Jesse Glick made changes -
          Link New: This issue is duplicated by JENKINS-32709 [ JENKINS-32709 ]

          Jesse Glick added a comment -

          Sounds plausible. Would need to first strengthen the demo to actually run (and rely on) withServer, which perhaps means ceasing to directly bind-mount docker.sock, and switching to creating some kind of TCP proxy (ideally behind TLS) that would in turn connect to the host daemon.

          Jesse Glick added a comment - Sounds plausible. Would need to first strengthen the demo to actually run (and rely on) withServer , which perhaps means ceasing to directly bind-mount docker.sock , and switching to creating some kind of TCP proxy (ideally behind TLS) that would in turn connect to the host daemon.
          Jesse Glick made changes -
          Link New: This issue relates to JENKINS-36348 [ JENKINS-36348 ]

          cfraenkel can you post an example of how you passed the value of DOCKER_HOST down so that image.inside() works with the settings specified in docker.withServer() ?

           

          I'm trying to do the same thing you are doing, where my Docker server is on a separate host from where my Jenkins job is running.

          Craig Rodrigues added a comment - cfraenkel can you post an example of how you passed the value of DOCKER_HOST down so that image.inside() works with the settings specified in docker.withServer() ?   I'm trying to do the same thing you are doing, where my Docker server is on a separate host from where my Jenkins job is running.

          Christian Fraenkel added a comment - - edited

          rodrigc: I've recompiled the plugin using the attached patch (JENKINS-39243_env.patch). As stated above, this may expose (far) too many environment variables though.

          Christian Fraenkel added a comment - - edited rodrigc : I've recompiled the plugin using the attached patch ( JENKINS-39243_env.patch ). As stated above, this may expose (far) too many environment variables though.
          Christian Fraenkel made changes -
          Attachment New: JENKINS-39243_env.patch [ 36572 ]

          Mark Austin added a comment - - edited

          I'm experiencing a very similar problem and using the changes in the attached patch don't seem to fix my issue. I used the latest commit to master when I tested against docker-workflow-plugin (eb4193c33d0a1a3c1f09b4197b5052729f97dcbb).

          My setup is very simple...

          • 1 docker host
          • 1 jenkins master

          On the Jenkins master, I have an env var set to:
          DOCKER_HOST=tcp://10.x.x.x:4243

          On the docker host, I have it listening over the same tcp port:
          OPTIONS="-H tcp://0.0.0.0:4243"

          No firewall rules are in the way and I can reach the other host with docker -H tcp://10.x.x.x:4243 ...

          When I trigger a build, docker is invoked and will pull down a recent image on the external docker host, but once the docker.image().inside() function is called, I get an identical error to the OP.

          When I install docker directly onto the Jenkins master, add jenkins to the docker group and unset the DOCKER_HOST env var, it works as expected but I obviously don't want the docker daemon running on the master.

          Jenkins Core: 2.32.3
          Pipeline: 2.5
          Durable Task Plugin: 1.13
          Docker Pipeline: 1.11-SNAPSHOT

           

          Mark Austin added a comment - - edited I'm experiencing a very similar problem and using the changes in the attached patch don't seem to fix my issue. I used the latest commit to master when I tested against docker-workflow-plugin (eb4193c33d0a1a3c1f09b4197b5052729f97dcbb). My setup is very simple... 1 docker host 1 jenkins master On the Jenkins master, I have an env var set to: DOCKER_HOST=tcp://10.x.x.x:4243 On the docker host, I have it listening over the same tcp port: OPTIONS="-H tcp://0.0.0.0:4243" No firewall rules are in the way and I can reach the other host with docker -H tcp://10.x.x.x:4243 ... When I trigger a build, docker is invoked and will pull down a recent image on the external docker host, but once the docker.image().inside() function is called, I get an identical error to the OP. When I install docker directly onto the Jenkins master, add jenkins to the docker group and unset the DOCKER_HOST env var, it works as expected but I obviously don't want the docker daemon running on the master. Jenkins Core: 2.32.3 Pipeline: 2.5 Durable Task Plugin: 1.13 Docker Pipeline: 1.11-SNAPSHOT  
          Jesse Glick made changes -
          Link New: This issue duplicates JENKINS-43222 [ JENKINS-43222 ]
          Jesse Glick made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

            Unassigned Unassigned
            cfraenkel Christian Fraenkel
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: