Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30200

Nullpointer exception while connecting to a docker container

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • docker-plugin
    • None
    • Linux, Docker plugin, docker 1.8.1

      Jenkins ver. 1.626
      Docker plugin version: 0.12.1

      Container settings:
      Docker command: /sbin/init 3 (without custom command also crashes)
      Hostname: debian8_32bit_32bit_INIT3_CONTAINER
      DNS: <Some ip address>
      Volumes: /sys/fs/cgroup:/sys/fs/cgroup:ro
      Environment: container=docker
      Privileged : true
      Remote filing root: /home/jenkins
      labels: debian8_32bit_32bit_INIT3_CONTAINER

      Port: 22

      Problem:
      Crash when running a job. The container is started within docker server but jenkins cannot see it. Jenkins logs state:

      Aug 28, 2015 2:29:20 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Asked to provision 1 slave(s) for: debian8_32bit_32bit_INIT3_CONTAINER
      Aug 28, 2015 2:29:20 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Will provision 'debian8_32bit_32bit', for label: 'debian8_32bit_32bit_INIT3_CONTAINER', in cloud: 'docker-main'
      Aug 28, 2015 2:29:20 PM com.nirima.jenkins.plugins.docker.DockerCloud addProvisionedSlave
      INFORMATION: Provisioning 'debian8_32bit_32bit' number '0' on 'docker-main'; Total containers: '0'
      Aug 28, 2015 2:29:20 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
      INFORMATION: Started provisioning Image of debian8_32bit_32bit from docker-main with 1 executors. Remaining excess workload: 0
      Aug 28, 2015 2:29:20 PM com.nirima.jenkins.plugins.docker.DockerCloud provisionWithWait
      INFORMATION: Trying to run container for debian8_32bit_32bit
      Aug 28, 2015 2:29:21 PM com.nirima.jenkins.plugins.docker.DockerCloud$1 call
      SCHWERWIEGEND: Error in provisioning; template='DockerTemplate{configVersion=2, labelString='debian8_32bit_32bit_INIT3_CONTAINER', launcher=com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher@1cc6aff2, remoteFsMapping='', remoteFs='/home/jenkins', instanceCap=1, mode=NORMAL, retentionStrategy=com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy@52fd763e, numExecutors=1, dockerTemplateBase=DockerTemplateBase

      {image=debian8_32bit_32bit}

      , removeVolumes=false, pullStrategy=PULL_LATEST}' for cloud='docker-main'
      java.lang.NullPointerException
      at com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher.getPortUtils(DockerComputerSSHLauncher.java:118)
      at com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher.waitUp(DockerComputerSSHLauncher.java:77)
      at com.nirima.jenkins.plugins.docker.DockerCloud.provisionWithWait(DockerCloud.java:364)
      at com.nirima.jenkins.plugins.docker.DockerCloud.access$000(DockerCloud.java:55)
      at com.nirima.jenkins.plugins.docker.DockerCloud$1.call(DockerCloud.java:219)
      at com.nirima.jenkins.plugins.docker.DockerCloud$1.call(DockerCloud.java:216)
      at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)

      Aug 28, 2015 2:29:30 PM hudson.slaves.NodeProvisioner$2 run
      WARNUNG: Provisioned slave Image of debian8_32bit_32bit failed to launch
      java.lang.NullPointerException
      at com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher.getPortUtils(DockerComputerSSHLauncher.java:118)
      at com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher.waitUp(DockerComputerSSHLauncher.java:77)
      at com.nirima.jenkins.plugins.docker.DockerCloud.provisionWithWait(DockerCloud.java:364)
      at com.nirima.jenkins.plugins.docker.DockerCloud.access$000(DockerCloud.java:55)
      at com.nirima.jenkins.plugins.docker.DockerCloud$1.call(DockerCloud.java:219)
      at com.nirima.jenkins.plugins.docker.DockerCloud$1.call(DockerCloud.java:216)
      at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)

      Aug 28, 2015 2:29:30 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Asked to provision 1 slave(s) for: debian8_32bit_32bit_INIT3_CONTAINER
      Aug 28, 2015 2:29:30 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Will provision 'debian8_32bit_32bit', for label: 'debian8_32bit_32bit_INIT3_CONTAINER', in cloud: 'docker-main'
      Aug 28, 2015 2:29:30 PM com.nirima.jenkins.plugins.docker.DockerCloud addProvisionedSlave
      INFORMATION: Not Provisioning 'debian8_32bit_32bit'. Instance limit of '1' reached on server 'docker-main'
      Aug 28, 2015 2:29:40 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Asked to provision 1 slave(s) for: debian8_32bit_32bit_INIT3_CONTAINER
      Aug 28, 2015 2:29:40 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Will provision 'debian8_32bit_32bit', for label: 'debian8_32bit_32bit_INIT3_CONTAINER', in cloud: 'docker-main'
      Aug 28, 2015 2:29:40 PM com.nirima.jenkins.plugins.docker.DockerCloud addProvisionedSlave
      INFORMATION: Not Provisioning 'debian8_32bit_32bit'. Instance limit of '1' reached on server 'docker-main'
      Aug 28, 2015 2:29:50 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Asked to provision 1 slave(s) for: debian8_32bit_32bit_INIT3_CONTAINER
      Aug 28, 2015 2:29:50 PM com.nirima.jenkins.plugins.docker.DockerCloud provision
      INFORMATION: Will provision 'debian8_32bit_32bit', for label: 'debian8_32bit_32bit_INIT3_CONTAINER', in cloud: 'docker-main'
      Aug 28, 2015 2:29:50 PM com.nirima.jenkins.plugins.docker.DockerCloud addProvisionedSlave
      INFORMATION: Not Provisioning 'debian8_32bit_32bit'. Instance limit of '1' reached on server 'docker-main'

          [JENKINS-30200] Nullpointer exception while connecting to a docker container

          Jaroslav Barton added a comment - - edited

          I've played with it and:

          Docker API v1.19:

              "PortSpecs": [
                  "22/tcp"
              ]
          

          Is not part of Docker API v1.20. In both v1.19 and v1.20 API documentation I see:

              "ExposedPorts": {
                  "22/tcp": {}
               },
          

          When I update API request sent by jenkins (removed PortSpec, added ExposedPorts), I'm able to connect running container.

          Jaroslav Barton added a comment - - edited I've played with it and: Docker API v1.19 : "PortSpecs" : [ "22/tcp" ] Is not part of Docker API v1.20 . In both v1.19 and v1.20 API documentation I see: "ExposedPorts" : { "22/tcp" : {} }, When I update API request sent by jenkins (removed PortSpec , added ExposedPorts ), I'm able to connect running container.

          Thanks for verbose output. So is it an newer API issue or docker-plugin bug?

          Kanstantsin Shautsou added a comment - Thanks for verbose output. So is it an newer API issue or docker-plugin bug?

          Also please increase DockerOnceRetentionStrategy timeout to 10 as it may kill Slave randomly.

          Kanstantsin Shautsou added a comment - Also please increase DockerOnceRetentionStrategy timeout to 10 as it may kill Slave randomly.

          Kanstantsin Shautsou added a comment - - edited

          jabarton i copied your cloud config section and it works fine with Docker version 1.8.1, build d12ea79 (but set evarga/jenkins-slave as image)

          Kanstantsin Shautsou added a comment - - edited jabarton i copied your cloud config section and it works fine with Docker version 1.8.1, build d12ea79 (but set evarga/jenkins-slave as image)

          I've played with it in my environment and following change works for me on both Docker 1.7.1 and 1.8.2:
          https://github.com/jenkinsci/docker-plugin/pull/320

          Jaroslav Barton added a comment - I've played with it in my environment and following change works for me on both Docker 1.7.1 and 1.8.2 : https://github.com/jenkinsci/docker-plugin/pull/320

          Tom Canova added a comment -

          we are also running into an issue with jenkins connecting to a remote docker which is on docker version 1.8.1, api 1.20
          {{Will provision 'evarga/jenkins-slave', for label: 'evarga', in cloud: 'dockerac3'

          Sep 15, 2015 4:56:13 PM WARNING com.nirima.jenkins.plugins.docker.DockerCloud provision

          Bad template 'evarga/jenkins-slave' in cloud 'dockerac3': 'shaded.org.apache.http.conn.ConnectTimeoutException: Connect to }}

          Tom Canova added a comment - we are also running into an issue with jenkins connecting to a remote docker which is on docker version 1.8.1, api 1.20 {{Will provision 'evarga/jenkins-slave', for label: 'evarga', in cloud: 'dockerac3' Sep 15, 2015 4:56:13 PM WARNING com.nirima.jenkins.plugins.docker.DockerCloud provision Bad template 'evarga/jenkins-slave' in cloud 'dockerac3': 'shaded.org.apache.http.conn.ConnectTimeoutException: Connect to }}

          canova please provide full log, also to what ip and port is it trying connect and what is the right ip/port of running container (login into docker host and docker inspect appeared containerId)?

          Kanstantsin Shautsou added a comment - canova please provide full log, also to what ip and port is it trying connect and what is the right ip/port of running container (login into docker host and docker inspect appeared containerId)?

          Ashish Behl added a comment - - edited

          So, I tried with the latest docker plugin, version 0.14.0, and I can connect to the docker service.
          The problem that I now have is that I cannot use docker version > 1.8.0 with jenkins, but as I read above, this might be due to the custom containers that I create.
          So, Please let me know what I should I change while creating containers to make them work with docker > 1.8.0 and latest docker plugin?

          [Edit]
          Also, any way to start systemd in docker containers and connect to them through the docker plugin?

          Thanks,
          Ashish.

          Ashish Behl added a comment - - edited So, I tried with the latest docker plugin, version 0.14.0, and I can connect to the docker service. The problem that I now have is that I cannot use docker version > 1.8.0 with jenkins, but as I read above, this might be due to the custom containers that I create. So, Please let me know what I should I change while creating containers to make them work with docker > 1.8.0 and latest docker plugin? [Edit] Also, any way to start systemd in docker containers and connect to them through the docker plugin? Thanks, Ashish.

          Code changed in jenkins
          User: Jaroslav Barton
          Path:
          docker-plugin/src/main/java/com/nirima/jenkins/plugins/docker/launcher/DockerComputerSSHLauncher.java
          http://jenkins-ci.org/commit/docker-plugin/5b06e3b9d94a177b7ca98e972054ca7018ba6557
          Log:
          JENKINS-30200: Nullpointer exception while connecting to a docker
          container

          Looks like due to Docker API changes between `v1.19` and `v1.20`
          create container call should contain `ExposedPorts` instead of
          `PortSpecs`. Change was tested with Docker `1.8.2` and `1.7.1`.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jaroslav Barton Path: docker-plugin/src/main/java/com/nirima/jenkins/plugins/docker/launcher/DockerComputerSSHLauncher.java http://jenkins-ci.org/commit/docker-plugin/5b06e3b9d94a177b7ca98e972054ca7018ba6557 Log: JENKINS-30200 : Nullpointer exception while connecting to a docker container Looks like due to Docker API changes between `v1.19` and `v1.20` create container call should contain `ExposedPorts` instead of `PortSpecs`. Change was tested with Docker `1.8.2` and `1.7.1`.

          Code changed in jenkins
          User: magnayn
          Path:
          docker-plugin/src/main/java/com/nirima/jenkins/plugins/docker/launcher/DockerComputerSSHLauncher.java
          http://jenkins-ci.org/commit/docker-plugin/498c14371b10a03765f1761fbcfd125c55067125
          Log:
          Merge pull request #320 from djaara/master

          JENKINS-30200: Nullpointer exception while connecting to a docker

          Compare: https://github.com/jenkinsci/docker-plugin/compare/1ccd2e7ce449...498c14371b10

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: magnayn Path: docker-plugin/src/main/java/com/nirima/jenkins/plugins/docker/launcher/DockerComputerSSHLauncher.java http://jenkins-ci.org/commit/docker-plugin/498c14371b10a03765f1761fbcfd125c55067125 Log: Merge pull request #320 from djaara/master JENKINS-30200 : Nullpointer exception while connecting to a docker Compare: https://github.com/jenkinsci/docker-plugin/compare/1ccd2e7ce449...498c14371b10

            magnayn magnayn
            ashish_behl Ashish Behl
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: