Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48050

Replace Declarative Docker agent directive with new implementation

    XMLWordPrintable

Details

    • Declarative - 1.2, Pipeline - December, Pipeline - April 2018

    Description

      Goals

      • We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
      • We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
      • We want customers to try this new implementation without having to use patched plugins or special builds from a branch

      Scope

      • Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
      • The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
      • Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
      • Honour the entry point of the container
      • Sharing the workspace from the top level agent to a per stage Docker agent (and back again!)

      Out of scope

      • Declarative Docker agent directive that specifies a Dockerfile
      • Scripted docker.inside
      • Scripted docker.build
      • Building containers
      • Linked containers
      • Caching directories

      Acceptance criteria
      The following steps should work using the new implementation (even if modifications are needed to these steps):

      • withMaven()
      • dir()
      • withCredentials

      The following functionality should work:

      • Tool installers
      • Config File Provider

      Examples
      Per stage

      pipeline {
        options {
          dockerPipelineVersion(2)
        }
        agent none // Will not reuse workspace
        stages {
           stage (‘build’) {
              agent { docker ‘maven3’ } 
              steps { 
      	   sh ‘mvn clean test’
              } 
           }
        }
      }
      

      Whole Pipeline

      pipeline {
        options {
          dockerPipelineVersion(2)
        }
        agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
        stages {
           stage (‘build’) {
              steps { 
      	   sh ‘mvn clean test’
              } 
           }
        }
      }
      

      Attachments

        Issue Links

          Activity

            jamesdumay James Dumay created issue -
            jamesdumay James Dumay made changes -
            Field Original Value New Value
            Epic Link JENKINS-47962 [ 186491 ]
            jamesdumay James Dumay made changes -
            Description *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author.

            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins.

            *Out of scope*
            * Declarative Docker agent directive that specifies a Dockerfile
            * Scripted {{docker.inside}}
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * withMaven()
            * dir()
            *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author.

            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a Dockerfile
            * Scripted {{docker.inside}}
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * withMaven()
            * dir()
            jamesdumay James Dumay made changes -
            Description *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author.

            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a Dockerfile
            * Scripted {{docker.inside}}
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * withMaven()
            * dir()
            *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a Dockerfile
            * Scripted {{docker.inside}}
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * withMaven()
            * dir()

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            jamesdumay James Dumay made changes -
            Description *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a Dockerfile
            * Scripted {{docker.inside}}
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * withMaven()
            * dir()

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            jamesdumay James Dumay made changes -
            Description *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            jamesdumay James Dumay made changes -
            Description *Goal*
            We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author. We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)

            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            abayer Andrew Bayer made changes -
            Description *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion: 2
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            abayer Andrew Bayer added a comment -

            A couple more criteria I’d like to suggest:

            • Doesn’t require any additional configuration on the master. This is a key aspect of the existing behavior, so I’d say it really needs to be retained.
            • JENKINS-46831 is fixed - that is, it’s possible to have one docker agent at the top level and a different one on a stage. This’ll probably come for free with dir() working and the like, but it’s still worth mentioning.

            The scope here also doesn’t address the question of sharing the workspace from the top level agent to a per stage Docker agent (and back again!). I think that’s important and should be retained, but not as important as the lack of additional master configuration.

            abayer Andrew Bayer added a comment - A couple more criteria I’d like to suggest: Doesn’t require any additional configuration on the master. This is a key aspect of the existing behavior, so I’d say it really needs to be retained. JENKINS-46831 is fixed - that is, it’s possible to have one docker agent at the top level and a different one on a stage. This’ll probably come for free with dir() working and the like, but it’s still worth mentioning. The scope here also doesn’t address the question of sharing the workspace from the top level agent to a per stage Docker agent (and back again!). I think that’s important and should be retained, but not as important as the lack of additional master configuration.
            michaelneale Michael Neale made changes -
            Component/s pipeline-model-definition-plugin [ 21706 ]
            jamesdumay James Dumay added a comment - - edited

            abayer

            Doesn’t require any additional configuration on the master

            I suppose this means everything falls back to using the docker socket if Jenkins doesn't provide an implementation?

            The scope here also doesn’t address the question of sharing the workspace from the top level agent to a per stage Docker agent

            Yes, it should. Ill add this to the description.

            jamesdumay James Dumay added a comment - - edited abayer Doesn’t require any additional configuration on the master I suppose this means everything falls back to using the docker socket if Jenkins doesn't provide an implementation? The scope here also doesn’t address the question of sharing the workspace from the top level agent to a per stage Docker agent Yes, it should. Ill add this to the description.
            jamesdumay James Dumay made changes -
            Description *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container
            * Sharing the workspace from the top level agent to a per stage Docker agent (and back again!)

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            teilo James Nord added a comment -

            I would suggest adding withCredentials to the scope as without credentials many things are dead in the water.

            I would also suggest that any new implementation should be /aware/ of windows containers (ie. .its fine if it doesn't work with them on day zero, but if it needs architectural changes to work with them in the future it should be a showstopper. (this may involve things like flagging the desired agent type so that you can choose the correct agent on which to launch the container).

            Doesn’t require any additional configuration on the master. This is a key aspect of the existing behavior, so I’d say it really needs to be retained.

            well today you 100% have to do this, because not all agents (even if they are cattle) are the same and some herds will not be docker enabled and if you do not also specify a label then you will eventually go bang, when the farmer introduces a new breed of cattle (or perhaps branches out into pig farming)

            teilo James Nord added a comment - I would suggest adding withCredentials to the scope as without credentials many things are dead in the water. I would also suggest that any new implementation should be /aware/ of windows containers (ie. .its fine if it doesn't work with them on day zero, but if it needs architectural changes to work with them in the future it should be a showstopper. (this may involve things like flagging the desired agent type so that you can choose the correct agent on which to launch the container). Doesn’t require any additional configuration on the master. This is a key aspect of the existing behavior, so I’d say it really needs to be retained. well today you 100% have to do this, because not all agents (even if they are cattle) are the same and some herds will not be docker enabled and if you do not also specify a label then you will eventually go bang, when the farmer introduces a new breed of cattle (or perhaps branches out into pig farming)
            cleclerc Cyrille Le Clerc added a comment - - edited

            Can we add some criterion of feature parity of dockerized build agents with "permanent jenkins agents":

            • Tool installers
            • Environment variable contribution by plugins, including the PATH variable
            • Config file provider plugin
            • Caching of some directories: maven/gradle repo, npm... in a mode that can be extended for other programming languages

            Note that some of these criterion are implied by the criterion to support withMaven

            cleclerc Cyrille Le Clerc added a comment - - edited Can we add some criterion of feature parity of dockerized build agents with "permanent jenkins agents": Tool installers Environment variable contribution by plugins, including the PATH variable Config file provider plugin Caching of some directories: maven/gradle repo, npm... in a mode that can be extended for other programming languages Note that some of these criterion are implied by the criterion to support withMaven
            abayer Andrew Bayer added a comment -

            withCredentials, tool installers, and variables on the agent like PATH should definitely work - I think in practice, all of those will work without any additional effort so long as we're interacting with the container as a true agent behind the scenes, as discussed already, but it'll definitely be worth having tests for those.

            re: directory caching - I'd say that comes later.

            abayer Andrew Bayer added a comment - withCredentials , tool installers, and variables on the agent like PATH should definitely work - I think in practice, all of those will work without any additional effort so long as we're interacting with the container as a true agent behind the scenes, as discussed already, but it'll definitely be worth having tests for those. re: directory caching - I'd say that comes later.
            michaelneale Michael Neale made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jamesdumay James Dumay made changes -
            Description *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container
            * Sharing the workspace from the top level agent to a per stage Docker agent (and back again!)

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container
            * Sharing the workspace from the top level agent to a per stage Docker agent (and back again!)

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}
            * {{withCredentials}}

            The following functionality should work:
            * Tool installers

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            michaelneale Michael Neale made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            jamesdumay James Dumay made changes -
            Description *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container
            * Sharing the workspace from the top level agent to a per stage Docker agent (and back again!)

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}
            * {{withCredentials}}

            The following functionality should work:
            * Tool installers

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            *Goals*
            * We want to have a new implementation for running steps inside a Docker container that is transparent to the Pipeline author
            * We want to ensure that as this developed that we start making changes that allow us to swap out the implementation in the future (e.g. docker, k8, etc)
            * We want customers to try this new implementation without having to use patched plugins or special builds from a branch


            *Scope*
            * Declarative Docker agent directive can be used to run steps for a stage or for the whole pipeline
            * The new implementation can be enabled by specifying Pipeline->Options->dockerPipelineVersion = 2
            * Must be able to use an off the shelf docker container without modifying it so it works with Jenkins
            * Honour the entry point of the container
            * Sharing the workspace from the top level agent to a per stage Docker agent (and back again!)

            *Out of scope*
            * Declarative Docker agent directive that specifies a {{Dockerfile}}
            * Scripted {{docker.inside}}
            * Scripted {{docker.build}}
            * Building containers
            * Linked containers
            * Caching directories

            *Acceptance criteria*
            The following steps should work using the new implementation (even if modifications are needed to these steps):
            * {{withMaven()}}
            * {{dir()}}
            * {{withCredentials}}

            The following functionality should work:
            * Tool installers
            * Config File Provider

            *Examples*
            _Per stage_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent none // Will not reuse workspace
              stages {
                 stage (‘build’) {
                    agent { docker ‘maven3’ }
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}

            _Whole Pipeline_
            {code}
            pipeline {
              options {
                dockerPipelineVersion(2)
              }
              agent { docker ‘maven3’ } // will reuse workspace for all stages unless specified
              stages {
                 stage (‘build’) {
                    steps {
            sh ‘mvn clean test’
                    }
                 }
              }
            }
            {code}
            michaelneale Michael Neale made changes -
            Labels cloudbees-internal-pipeline
            jamesdumay James Dumay made changes -
            Sprint Declarative - 1.2 [ 366 ]
            Labels cloudbees-internal-pipeline
            jamesdumay James Dumay made changes -
            Link This issue relates to JENKINS-40484 [ JENKINS-40484 ]
            cleclerc Cyrille Le Clerc made changes -
            Summary Replace Declarative Docker agent directive with new impelementation Replace Declarative Docker agent directive with new implementation
            cleclerc Cyrille Le Clerc added a comment - - edited

            jamesdumay as discussed with abayer, I feel that we should restrict the list of supported credentials types for withCredentials because it's the responsibility of each credentials plugin to implement the "not very well documented" contract that files created by plugins for builds should be created in hudson.slaves.WorkspaceList#tempDir. I propose to restrict to

            • secret file: should be ok because it's implemented by the credentials binding plugin directly
            • secret zip file: should be ok because it's implemented by the credentials binding plugin directly
            • docker client certificate: provided by the docker commons plugin
            • AWS access key: provided by the AWS Credentials Plugin, nothing to test it's environment variables, not files
            • username and password conjoined and separated: provided by the Credentials Binding Plugin, nothing to test it's environment variables, not files
            • SSH private key: I didn't find how it's handled by withCredentials
            • GPG certs: I didn't find how it's handled by withCredentials

            Moreover, can we clarify/document the behavior of the declarative docker agent regarding:

            • The "tmp" dir of the JVM of the agent. docker-pipeline decided to not mount it, do we confirm this choice? Not mounting the temp folder of the build agent JVM has the benefit of providing more isolation of the builds but has the drawback of breaking a bunch of plugin (was breaking the config file provider plugin and some credentials files)
            • The mount / or not mount of the tools folder. docker-pipeline-plugin decided to not mounting it (breaking tool installers), custom-build-environment-plugin decided to mount this folder, preserving feature parity on tool installers.

            Last but not least, we could check that the following things works

            • "git" and "checkout" using an ssh private key
            • "sshAgent": I don't know if it relies on a file on disk

            Docker things

            • Can we confirm that we fix JENKINS-41316 "docker.image('my-image').inside{...} no longer honors Dockerfile "entrypoint" since version 1.8"

            Pending questions:

            • OpenShift credentials by the kubernetes plugin: see OpenShiftTokenCredentialImpl.java,
              • Doesn't seem to be integrated with withCredentials
            • Kubernetes plugin seem to rely on disk on
            • perforce credentials by the P4 plugin
            cleclerc Cyrille Le Clerc added a comment - - edited jamesdumay as discussed with abayer , I feel that we should restrict the list of supported credentials types for withCredentials because it's the responsibility of each credentials plugin to implement the "not very well documented" contract that files created by plugins for builds should be created in hudson.slaves.WorkspaceList#tempDir . I propose to restrict to secret file: should be ok because it's implemented by the credentials binding plugin directly secret zip file: should be ok because it's implemented by the credentials binding plugin directly docker client certificate: provided by the docker commons plugin AWS access key: provided by the AWS Credentials Plugin, nothing to test it's environment variables, not files username and password conjoined and separated: provided by the Credentials Binding Plugin, nothing to test it's environment variables, not files SSH private key: I didn't find how it's handled by withCredentials GPG certs: I didn't find how it's handled by withCredentials Moreover, can we clarify/document the behavior of the declarative docker agent regarding: The "tmp" dir of the JVM of the agent. docker-pipeline decided to not mount it, do we confirm this choice? Not mounting the temp folder of the build agent JVM has the benefit of providing more isolation of the builds but has the drawback of breaking a bunch of plugin (was breaking the config file provider plugin and some credentials files) The mount / or not mount of the tools folder. docker-pipeline-plugin decided to not mounting it (breaking tool installers), custom-build-environment-plugin decided to mount this folder, preserving feature parity on tool installers. Last but not least, we could check that the following things works " git " and " checkout " using an ssh private key "sshAgent": I don't know if it relies on a file on disk Docker things Can we confirm that we fix JENKINS-41316 "docker.image('my-image').inside{...} no longer honors Dockerfile "entrypoint" since version 1.8" Pending questions: OpenShift credentials by the kubernetes plugin: see OpenShiftTokenCredentialImpl.java , Doesn't seem to be integrated with withCredentials Kubernetes plugin seem to rely on disk on See ServiceAccountCredential.java and SERVICEACCOUNT_TOKEN_PATH = "/var/run/secrets/kubernetes.io/serviceaccount/token" perforce credentials by the P4 plugin
            csanchez Carlos Sanchez made changes -
            Link This issue blocks JENKINS-48135 [ JENKINS-48135 ]
            jglick Jesse Glick added a comment -

            Huh? Any kind of credentials with a Binding implementation is supported by withCredentials.

            jglick Jesse Glick added a comment - Huh? Any kind of credentials with a Binding implementation is supported by withCredentials .
            jglick Jesse Glick added a comment -

            And of course docker-workflow mounts the workspace temp dir. Tons of stuff would break otherwise.

            Mounting a tools directory is unsafe because it is shared across builds.

            I think you need me on design review.

            jglick Jesse Glick added a comment - And of course docker-workflow mounts the workspace temp dir. Tons of stuff would break otherwise. Mounting a tools directory is unsafe because it is shared across builds. I think you need me on design review.
            jglick Jesse Glick added a comment -

            checkout with GitSCM ignores any decorated Launcher due to a well-known design flaw in the git-client plugin, so this is irrelevant. Test with a properly implemented SCM, like mercurial.

            jglick Jesse Glick added a comment - checkout with GitSCM ignores any decorated Launcher due to a well-known design flaw in the git-client plugin, so this is irrelevant. Test with a properly implemented SCM, like mercurial .
            abayer Andrew Bayer added a comment -

            The plan is that there will be a true Docker agent - i.e., it'll mount in the workspace but nothing else from the underlying host, so anything that works on, say, a container agent (via the k8s plugin, or Docker plugin's cloud, or Azure Container Service, or...you get the point), will work the same on this new agent as well.

            So let's not worry about complexity of edge cases that don't work in docker-workflow due to it not being an actual agent - that's why dir, tools, etc don't work, because they expect to be acting on the agent, not inside the container within the agent. When we're interacting with the container as a true agent, running remoting.jar and all, those problems should just vanish. And to be sure, we'll be adding tests for a wide array of behaviors, both ones that don't work on docker-workflow and ones that do, of course.

            abayer Andrew Bayer added a comment - The plan is that there will be a true Docker agent - i.e., it'll mount in the workspace but nothing else from the underlying host, so anything that works on, say, a container agent (via the k8s plugin, or Docker plugin's cloud, or Azure Container Service, or...you get the point), will work the same on this new agent as well. So let's not worry about complexity of edge cases that don't work in docker-workflow due to it not being an actual agent - that's why dir , tools, etc don't work, because they expect to be acting on the agent, not inside the container within the agent. When we're interacting with the container as a true agent, running remoting.jar and all, those problems should just vanish. And to be sure, we'll be adding tests for a wide array of behaviors, both ones that don't work on docker-workflow and ones that do, of course.
            jamesdumay James Dumay made changes -
            Sprint Declarative - 1.2 [ 366 ] Declarative - 1.2, Pipeline - December [ 366, 446 ]
            jamesdumay James Dumay made changes -
            Rank Ranked lower
            jamesdumay James Dumay added a comment - - edited

            ndeloof you mentioned some changes in a recent docker release that could be beneficial to this work. Would you be able to find out roughly when this change gets into a Docker Enterprise release?

            Random thought: if Docker is too old, then use the current Docker Pipeline implementation or fit its new enough use the new Docker implementation?

            jamesdumay James Dumay added a comment - - edited ndeloof you mentioned some changes in a recent docker release that could be beneficial to this work. Would you be able to find out roughly when this change gets into a Docker Enterprise release? Random thought: if Docker is too old, then use the current Docker Pipeline implementation or fit its new enough use the new Docker implementation?
            rsandell rsandell added a comment -

            So why should we do this?

            I see goals and acceptance criteria, but no motivation on why this is needed?

            rsandell rsandell added a comment - So why should we do this? I see goals and acceptance criteria, but no motivation on why this is needed?
            jamesdumay James Dumay added a comment -

            The current implementation is broken by design. Steps like dir and withMaven do not work

            jamesdumay James Dumay added a comment - The current implementation is broken by design. Steps like dir and withMaven do not work
            jamesdumay James Dumay added a comment -

            Also tool installers

            jamesdumay James Dumay added a comment - Also tool installers
            abayer Andrew Bayer added a comment -

            fwiw, here are some JIRAs that are basically unfixable due to the architecture of Docker Pipeline (a few of them may be fixed by https://github.com/jenkinsci/docker-workflow-plugin/pull/123, but that requires Docker 17.12 and none of them would be an issue if we were actually running the agent process on the container itself anyway):

            Then there's the issue of Docker Pipeline's maintainability - there are a number of things about its design and implementation that are now considered mistakes, it mashes up a lot of different functionality into one codebase that can be hard to unravel, the docker.image(...).inside syntax is problematic, etc...

            abayer Andrew Bayer added a comment - fwiw, here are some JIRAs that are basically unfixable due to the architecture of Docker Pipeline (a few of them may be fixed by https://github.com/jenkinsci/docker-workflow-plugin/pull/123 , but that requires Docker 17.12 and none of them would be an issue if we were actually running the agent process on the container itself anyway): JENKINS-46831 JENKINS-33510 JENKINS-41894 JENKINS-47890 JENKINS-47805 JENKINS-43590 JENKINS-41316 JENKINS-36913 JENKINS-48082 JENKINS-48319 Then there's the issue of Docker Pipeline's maintainability - there are a number of things about its design and implementation that are now considered mistakes, it mashes up a lot of different functionality into one codebase that can be hard to unravel, the docker.image(...).inside syntax is problematic, etc...
            cleclerc Cyrille Le Clerc made changes -
            Link This issue is related to JENKINS-47823 [ JENKINS-47823 ]
            cleclerc Cyrille Le Clerc made changes -
            Link This issue is related to JENKINS-48284 [ JENKINS-48284 ]
            schukai Volker Schukai made changes -
            Link This issue relates to JENKINS-48460 [ JENKINS-48460 ]
            stodorov Steve Todorov made changes -
            Link This issue relates to JENKINS-47805 [ JENKINS-47805 ]
            stodorov Steve Todorov added a comment -

            This seems like it will fix a lot of the issues we're having with running maven builds in docker containers.
            What is the approximate timeline for having this feature released - I'm guessing a few months? 

            stodorov Steve Todorov added a comment - This seems like it will fix a lot of the issues we're having with running maven builds in docker containers. What is the approximate timeline for having this feature released - I'm guessing a few months? 

            Nothing planned yet, not even agreement on implementation decisions. 

            ndeloof Nicolas De Loof added a comment - Nothing planned yet, not even agreement on implementation decisions. 
            stodorov Steve Todorov added a comment -

            Thanks for your reply. Please keep us in the loop in that case.  

            stodorov Steve Todorov added a comment - Thanks for your reply. Please keep us in the loop in that case.  
            weakcamel Waldek M added a comment -

            Hello,

            I'm wondering  - would the new implementation be also solving problems like https://issues.jenkins-ci.org/browse/JENKINS-47026 ?

            I.e. would it also honor the isolation of Docker container from host (processes running within and outside of Docker container with different UID/GID)?

            weakcamel Waldek M added a comment - Hello, I'm wondering  - would the new implementation be also solving problems like https://issues.jenkins-ci.org/browse/JENKINS-47026 ? I.e. would it also honor the isolation of Docker container from host (processes running within and outside of Docker container with different UID/GID)?

            weakcamel this is uncertain so far, as this issue is more about balancing what we'd like to get and what is feasible . Typically

            • we'd like to be able to run arbitrary docker image but jenkins require a JVM for it's agent
            • we'd like to be able to share workspace between steps running in containers, but then the uid/gid issue will come back 
            • etc ... always with but xx at some point

            IMHO we need to define a fresh new pipeline approach for docker usage that would not try to hack docker container. Working on a proposal on this purpose

             

            ndeloof Nicolas De Loof added a comment - weakcamel this is uncertain so far, as this issue is more about balancing what we'd like to get and what is feasible . Typically we'd like to be able to run arbitrary docker image but  jenkins require a JVM for it's agent we'd like to be able to share workspace between steps running in containers,  but then the uid/gid issue will come back  etc ... always with  but xx at some point IMHO we need to define a fresh new pipeline approach for docker usage that would not try to hack docker container. Working on a proposal on this purpose  
            weakcamel Waldek M added a comment -

            ndeloof sure, I hear what you're saying. There's a lot of features Jenkins users take for granted (which they wouldn't like to use) and general design of agents stand in the way too.

            I've had a couple of discussions with my colleagues on the subject of sharing files between a host and a container and the bottom line was: the alternatives may be inconvenient (or complicated, or come  with performance trade-off), but using a simple volume mount just isn't clean and versatile enough and

            IMO the only clean way to transfer files back and forth is some sort of RPC (as nicolaw has suggested in JENKINS-47026), be it directly via SSH/SFTP/what have you or e.g. https://github.com/vieux/docker-volume-sshfs..] otherwise the design is always going to make assumptions on the image or Docker host setup. Just my £0.02.

            weakcamel Waldek M added a comment - ndeloof sure, I hear what you're saying. There's a lot of features Jenkins users take for granted (which they wouldn't like to use) and general design of agents stand in the way too. I've had a couple of discussions with my colleagues on the subject of sharing files between a host and a container and the bottom line was: the alternatives may be inconvenient (or complicated, or come  with performance trade-off), but using a simple volume mount just isn't clean and versatile enough and IMO the only clean way to transfer files back and forth is some sort of RPC (as nicolaw has suggested in JENKINS-47026 ), be it directly via SSH/SFTP/what have you or e.g. https://github.com/vieux/docker-volume-sshfs. .] otherwise the design is  always going to make assumptions on the image or Docker host setup. Just my £0.02.

            weakcamel yes indeed, I'd prefer we enforce use of pipeline `squash` for file sharing, and get rid of the uid/gid ownership constraint

            ndeloof Nicolas De Loof added a comment - weakcamel yes indeed, I'd prefer we enforce use of pipeline `squash` for file sharing, and get rid of the uid/gid ownership constraint

            Some update on this issue:

            we'd like to introduce some "version:2" in declarative pipeline so changing behaviour to this new agent design would be opt-in only and we wouldn't break existing pipelines.

            docker-plugin do support `dockerNode` pipeline DSL keyword to provision a full jenkins node within a pipeline. This sounds like a nice candidate for docker agent implementation in declarative, as we would then get a full agent to fully honour Jenkins APIs and expectations (wrappers, env, path, etc).

            A major issue to remain is the way docker.inside to let one access workspace from higher level agent. This introduce filesystem permission issues, so forced user to run in container. To get rid of this, safer option seems imho to not keep this behaviour for nested agents, and only support use of `stash` to share files between agents.

            ndeloof Nicolas De Loof added a comment - Some update on this issue: we'd like to introduce some "version:2" in declarative pipeline so changing behaviour to this new agent design would be opt-in only and we wouldn't break existing pipelines. docker-plugin do support `dockerNode` pipeline DSL keyword to provision a full jenkins node within a pipeline. This sounds like a nice candidate for docker agent implementation in declarative, as we would then get a full agent to fully honour Jenkins APIs and expectations (wrappers, env, path, etc). A major issue to remain is the way docker.inside to let one access workspace from higher level agent. This introduce filesystem permission issues, so forced user to run in container. To get rid of this, safer option seems imho to not keep this behaviour for nested agents, and only support use of `stash` to share files between agents.

            Code changed in jenkins
            User: Nicolas De Loof
            Path:
            pom.xml
            src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy
            src/main/java/io/jenkins/docker/pipeline/DockerAgent.java
            src/main/java/io/jenkins/docker/pipeline/DockerAgentScript.groovy
            http://jenkins-ci.org/commit/docker-plugin/a8240b9a147153a9c2c772bcc8df4acdd8798d19
            Log:
            JENKINS-48050 declarative « container » agent

            Compare: https://github.com/jenkinsci/docker-plugin/compare/6acd9e848f61^...a8240b9a1471

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: pom.xml src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy src/main/java/io/jenkins/docker/pipeline/DockerAgent.java src/main/java/io/jenkins/docker/pipeline/DockerAgentScript.groovy http://jenkins-ci.org/commit/docker-plugin/a8240b9a147153a9c2c772bcc8df4acdd8798d19 Log: JENKINS-48050 declarative « container » agent Compare: https://github.com/jenkinsci/docker-plugin/compare/6acd9e848f61 ^...a8240b9a1471

            Code changed in jenkins
            User: Nicolas De Loof
            Path:
            pom.xml
            src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy
            src/main/java/io/jenkins/docker/pipeline/DockerAgent.java
            http://jenkins-ci.org/commit/docker-plugin/65dce18da936fbeea34d89245595d085866dd725
            Log:
            JENKINS-48050 declarative « container » agent

            Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: pom.xml src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy src/main/java/io/jenkins/docker/pipeline/DockerAgent.java http://jenkins-ci.org/commit/docker-plugin/65dce18da936fbeea34d89245595d085866dd725 Log: JENKINS-48050 declarative « container » agent Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
            ndeloof Nicolas De Loof made changes -
            Issue Type Story [ 10002 ] Epic [ 10001 ]
            ndeloof Nicolas De Loof made changes -
            Epic Link JENKINS-47962 [ 186491 ]
            ndeloof Nicolas De Loof made changes -
            Labels docker
            ndeloof Nicolas De Loof made changes -
            Epic Child JENKINS-49748 [ 188691 ]

            Code changed in jenkins
            User: Nicolas De Loof
            Path:
            pom.xml
            src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy
            src/main/java/io/jenkins/docker/pipeline/DockerAgent.java
            http://jenkins-ci.org/commit/docker-plugin/2e2c79fc3aba672e6a19d812d47ac55b2c7a418b
            Log:
            JENKINS-48050 declarative « container » agent

            Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: pom.xml src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy src/main/java/io/jenkins/docker/pipeline/DockerAgent.java http://jenkins-ci.org/commit/docker-plugin/2e2c79fc3aba672e6a19d812d47ac55b2c7a418b Log: JENKINS-48050 declarative « container » agent Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
            vivek Vivek Pandey made changes -
            Sprint Declarative - 1.2, Pipeline - December [ 366, 446 ] Declarative - 1.2, Pipeline - December, Pipeline - March 2018 [ 366, 446, 506 ]

            Code changed in jenkins
            User: Nicolas De Loof
            Path:
            pom.xml
            src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy
            src/main/java/io/jenkins/docker/pipeline/DockerAgent.java
            http://jenkins-ci.org/commit/docker-plugin/3cea7e1a5c49d069be4aeea1b027210101ce4bac
            Log:
            JENKINS-48050 declarative « container » agent

            Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De Loof Path: pom.xml src/main/groovy/io/jenkins/docker/pipeline/DockerAgentScript.groovy src/main/java/io/jenkins/docker/pipeline/DockerAgent.java http://jenkins-ci.org/commit/docker-plugin/3cea7e1a5c49d069be4aeea1b027210101ce4bac Log: JENKINS-48050 declarative « container » agent Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
            ndeloof Nicolas De Loof made changes -
            Component/s docker-plugin [ 18724 ]
            ndeloof Nicolas De Loof made changes -
            Assignee Nicolas De Loof [ ndeloof ] Andrew Bayer [ abayer ]
            csanchez Carlos Sanchez made changes -
            Link This issue blocks JENKINS-48135 [ JENKINS-48135 ]
            csanchez Carlos Sanchez made changes -
            Link This issue is related to JENKINS-48135 [ JENKINS-48135 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "pipeline-model-definition PR 255 (Web Link)" [ 21348 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "Unfiled docker PR: ndeloof:JENKINS-48050 (Web Link)" [ 21350 ]
            jglick Jesse Glick added a comment -

            Alternate simpler approach: docker PR 681

            jglick Jesse Glick added a comment - Alternate simpler approach: docker PR 681
            jglick Jesse Glick made changes -
            Remote Link This issue links to "docker PR 681 (Web Link)" [ 21352 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-36776 [ JENKINS-36776 ]
            bitwiseman Liam Newman made changes -
            Remote Link This issue links to "PR-255 (Web Link)" [ 23025 ]
            bitwiseman Liam Newman added a comment -
            bitwiseman Liam Newman added a comment - I've added a link to some work done for this https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/255
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-58732 [ JENKINS-58732 ]
            abayer Andrew Bayer made changes -
            Assignee Andrew Bayer [ abayer ]
            jglick Jesse Glick made changes -
            Component/s docker-workflow-plugin [ 20625 ]

            People

              Unassigned Unassigned
              jamesdumay James Dumay
              Votes:
              14 Vote for this issue
              Watchers:
              41 Start watching this issue

              Dates

                Created:
                Updated: