Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48050

Replace Declarative Docker agent directive with new implementation

    • Declarative - 1.2, Pipeline - December, Pipeline - April 2018

      Goals

      • We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
      • We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
      • We want customers to try this new implementation without having to use patched plugins or special builds from a branch

      Scope

      • Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
      • The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
      • Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
      • Honour the entry point of the container
      • Sharing the workspace from the top level agent to a per stage Docker agent (and back again!)

      Out of scope

      • Declarative Docker agent directive that specifies a Dockerfile
      • Scripted docker.inside
      • Scripted docker.build
      • Building containers
      • Linked containers
      • Caching directories

      Acceptance criteria
      The following steps should work using the new implementation (even if modifications are needed to these steps):

      • withMaven()
      • dir()
      • withCredentials

      The following functionality should work:

      • Tool installers
      • Config File Provider

      Examples
      Per stage

      pipeline {
        options {
          dockerPipelineVersion(2)
        }
        agent none // Will not reuse workspace
        stages {
           stage (‘build’) {
              agent { docker ‘maven3’ } 
              steps { 
      	   sh ‘mvn clean test’
              } 
           }
        }
      }
      

      Whole Pipeline

      pipeline {
        options {
          dockerPipelineVersion(2)
        }
        agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
        stages {
           stage (‘build’) {
              steps { 
      	   sh ‘mvn clean test’
              } 
           }
        }
      }
      

          [JENKINS-48050] Replace Declarative Docker agent directive with new implementation

          James Dumay created issue -
          James Dumay made changes -
          Epic Link New: JENKINS-47962 [ 186491 ]
          James Dumay made changes -
          Description Original: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author.

          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins.

          *Out of scope*
          * Declarative Docker agent directive that specifies a Dockerfile
          * Scripted {{docker.inside}}
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * withMaven()
          * dir()
          New: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author.

          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a Dockerfile
          * Scripted {{docker.inside}}
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * withMaven()
          * dir()
          James Dumay made changes -
          Description Original: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author.

          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a Dockerfile
          * Scripted {{docker.inside}}
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * withMaven()
          * dir()
          New: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a Dockerfile
          * Scripted {{docker.inside}}
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * withMaven()
          * dir()

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          James Dumay made changes -
          Description Original: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a Dockerfile
          * Scripted {{docker.inside}}
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * withMaven()
          * dir()

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          New: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a {{Dockerfile}}
          * Scripted {{docker.inside}}
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * {{withMaven()}}
          * {{dir()}}

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          James Dumay made changes -
          Description Original: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a {{Dockerfile}}
          * Scripted {{docker.inside}}
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * {{withMaven()}}
          * {{dir()}}

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          New: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a {{Dockerfile}}
          * Scripted {{docker.inside}}
          * Scripted {{docker.build}}
          * Building containers
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * {{withMaven()}}
          * {{dir()}}

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          James Dumay made changes -
          Description Original: *Goal*
          We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a {{Dockerfile}}
          * Scripted {{docker.inside}}
          * Scripted {{docker.build}}
          * Building containers
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * {{withMaven()}}
          * {{dir()}}

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          New: *Goals*
          * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
          * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
          * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a {{Dockerfile}}
          * Scripted {{docker.inside}}
          * Scripted {{docker.build}}
          * Building containers
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * {{withMaven()}}
          * {{dir()}}

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          Andrew Bayer made changes -
          Description Original: *Goals*
          * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
          * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
          * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a {{Dockerfile}}
          * Scripted {{docker.inside}}
          * Scripted {{docker.build}}
          * Building containers
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * {{withMaven()}}
          * {{dir()}}

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion: 2
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}
          New: *Goals*
          * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
          * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
          * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


          *Scope*
          * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
          * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
          * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
          * Honour the entry point of the container

          *Out of scope*
          * Declarative Docker agent directive that specifies a {{Dockerfile}}
          * Scripted {{docker.inside}}
          * Scripted {{docker.build}}
          * Building containers
          * Linked containers

          *Acceptance criteria*
          The following steps should work using the new implementation (even if modifications are needed to these steps):
          * {{withMaven()}}
          * {{dir()}}

          *Examples*
          _Per stage_
          {code}
          pipeline {
            options {
              dockerPipelineVersion(2)
            }
            agent none // Will not reuse workspace
            stages {
               stage (‘build’) {
                  agent { docker ‘maven3’ }
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          _Whole Pipeline_
          {code}
          pipeline {
            options {
              dockerPipelineVersion(2)
            }
            agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
            stages {
               stage (‘build’) {
                  steps {
          sh ‘mvn clean test’
                  }
               }
            }
          }
          {code}

          Andrew Bayer added a comment -

          A couple more criteria I’d like to suggest:

          • Doesn’t require any additional configuration on the master. This is a key aspect of the existing behavior, so I’d say it really needs to be retained.
          • JENKINS-46831 is fixed - that is, it’s possible to have one docker agent at the top level and a different one on a stage. This’ll probably come for free with dir() working and the like, but it’s still worth mentioning.

          The scope here also doesn’t address the question of sharing the workspace from the top level agent to a per stage Docker agent (and back again!). I think that’s important and should be retained, but not as important as the lack of additional master configuration.

          Andrew Bayer added a comment - A couple more criteria I’d like to suggest: Doesn’t require any additional configuration on the master. This is a key aspect of the existing behavior, so I’d say it really needs to be retained. JENKINS-46831 is fixed - that is, it’s possible to have one docker agent at the top level and a different one on a stage. This’ll probably come for free with dir() working and the like, but it’s still worth mentioning. The scope here also doesn’t address the question of sharing the workspace from the top level agent to a per stage Docker agent (and back again!). I think that’s important and should be retained, but not as important as the lack of additional master configuration.
          Michael Neale made changes -
          Component/s New: pipeline-model-definition-plugin [ 21706 ]

            Unassigned Unassigned
            jamesdumay James Dumay
            Votes:
            14 Vote for this issue
            Watchers:
            41 Start watching this issue

              Created:
              Updated: