Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49567

Automatically support docker networks in pipelines with multiple containers

XMLWordPrintable

      As is, it's not possible to create a Jenkinsfile with docker containers that need to communicate with each other (and not just a basic connection from container A to container B). For example, when running Behat tests for Drupal:

      • The webapp container (with PHP etc) çontains Behat, which needs to be able to link to the Selenium container.
      • The Selenium container needs to be able to link back to the webapp container to load pages.

      This leads to a chicken-and-the-egg problem, since you can't know the ID of a container before it starts. I imagine there's a way to hack it with container names, but that seems collision prone.

      Under other tools like Circle CI, a docker network is transparently created and all containers ports from their Dockerfiles are automatically exposed to each other. For example, even though there are two containers, from both perspectives all ports are available on localhost.

      On top of this, the recommended --link parameter has been deprecated by docker, so moving to bridge networks will keep the plugin working with future Docker releases.

          [JENKINS-49567] Automatically support docker networks in pipelines with multiple containers

          Andrew Berry created issue -

          Ryan Desmond added a comment - - edited

          This would be great, particularly with the deprecation of the --link option in docker.

          An alternative that would be a more purposeful with a sidecar pattern would be to have a function that surrounds any docker calls and switches each from the default network to a local one:

          docker.withNetwork {
   docker.image('sidecar').withRun() { c->
      docker.image('main').inside() {
         // do something with host "${c.name}"
      }
   }
}

           

          Currently, I have a function that works around this:

           

          def withDockerNetwork(Closure inner) {
    try {
        networkId = UUID.randomUUID().toString()
        sh "docker network create ${networkId}"
        inner.call(networkId)
    } finally {
        sh "docker network rm ${networkId}"
    }
}

          withDockerNetwork{ n ->
   docker.image('sidecar').withRun("--network ${n} --name sidecar") { c->
      docker.image('main').inside("--network ${n}") {
         // do something with host "sidecar"
      }
   }
}

           (note that the names have to be globally unique, so they have to be generated)

           

          Ryan Desmond added a comment - - edited This would be great, particularly with the deprecation of the --link option in docker. An alternative that would be a more purposeful with a sidecar pattern would be to have a function that surrounds any docker calls and switches each from the default network to a local one: docker.withNetwork {    docker.image( 'sidecar' ).withRun() { c->       docker.image( 'main' ).inside() { // do something with host "${c.name}" }    } }   Currently, I have a function that works around this:   def withDockerNetwork(Closure inner ) { try { networkId = UUID.randomUUID().toString() sh "docker network create ${networkId}" inner .call(networkId) } finally { sh "docker network rm ${networkId}" } } withDockerNetwork{ n ->   docker.image( 'sidecar' ).withRun( "--network ${n} --name sidecar" ) { c-> docker.image( 'main' ).inside( "--network ${n}" ) { // do something with host "sidecar" }   } }  (note that the names have to be globally unique, so they have to be generated)  
          Vivek Anand made changes -
          Comment [ I was looking for the exact same thing, but then we could use the docker.withRun to start a container(pause container) and then join all the other containers to the pause container network.

          This gives us the benefit of not having to remember to delete the network and is similar to how AWS fargate or Kubernetes does the networking to provide inter container communciation. ]

            Unassigned Unassigned
            deviantintegral Andrew Berry
            Votes:
            4 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: