Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55577

docker agent using a custom Dockerfile is broken with docker buildkit

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • docker-workflow-plugin
    • ubuntu 16.04
      docker 18.09.0
      jenkins 2.150.1
      docker-workflow-plugin 1.17

      When enabling buildkit in docker (either via `/etc/docker/daemon.json` or a global env var in jenkins: DOCKER_BUILDKIT=1), the job fails just after building the docker image used for the agent:

       https://jenkins.io/doc/book/pipeline/docker/#dockerfile

      [Pipeline] // stage
      [Pipeline] withEnv
      [Pipeline] {
      [Pipeline] stage
      [Pipeline] { (Declarative: Agent Setup)
      [Pipeline] isUnix
      [Pipeline] readFile
      [Pipeline] sh
      [workspace] Running shell script
      + docker build -t xxxx -f Dockerfile .
      
      ... buildkit logs ...
      
      [Pipeline] dockerFingerprintFrom
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] End of Pipeline
      java.io.IOException: Cannot retrieve .Id from 'docker inspect ubuntu:18.04'
      	at org.jenkinsci.plugins.docker.workflow.client.DockerClient.inspectRequiredField(DockerClient.java:220)
      	at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:133)
      	at org.jenkinsci.plugins.docker.workflow.FromFingerprintStep$Execution.run(FromFingerprintStep.java:85)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:290)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Finished: FAILURE
      

      Analysis:

      jenkins tries to get information about the base image used in the Dockerfile: `dockerFingerprintFrom`. It does so by assuming the base image is available in `docker images`, as it was just used to build the agent image.

      (This has a small timing issue if a system-wide `docker image prune` happens between the build and the call to `dockerFingerprintFrom`.)

      This worked great with the old docker builder, which effectively did a `docker pull` of the base image, but buildkit does not behave like that: it keeps things completely internally.

      $ cat Dockerfile
      FROM ubuntu:18.04
      $ DOCKER_BUILDKIT=1 docker build .
      [+] Building 1.7s (5/5) FINISHED                                                                                                                                                                                                             
       => [internal] load .dockerignore                                                                                                                                                                                                       0.0s
       => => transferring context: 2B                                                                                                                                                                                                         0.0s
       => [internal] load build definition from Dockerfile                                                                                                                                                                                    0.0s
       => => transferring dockerfile: 37B                                                                                                                                                                                                     0.0s
       => [internal] load metadata for docker.io/library/ubuntu:18.04                                                                                                                                                                         1.6s
       => [1/1] FROM docker.io/library/ubuntu:18.04@sha256:868fd30a0e47b8d8ac485df174795b5e2fe8a6c8f056cc707b232d65b8a1ab68                                                                                                                   0.0s
       => => resolve docker.io/library/ubuntu:18.04@sha256:868fd30a0e47b8d8ac485df174795b5e2fe8a6c8f056cc707b232d65b8a1ab68                                                                                                                   0.0s
       => exporting to image                                                                                                                                                                                                                  0.0s
       => => exporting layers                                                                                                                                                                                                                 0.0s
       => => writing image sha256:645e081eb3fdb8c828216921411fc0c08335e4fd098c151b44aba797fa334839                                                                                                                                            0.0s
      $ docker inspect ubuntu:18.04
      []
      Error: No such object: ubuntu:18.04
      

            Unassigned Unassigned
            thomas_deepomatic Thomas Riccardi
            Votes:
            8 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: