-
Improvement
-
Resolution: Fixed
-
Minor
-
None
With the traditional pipeline we have been able to do a mixture of working with the Jenkins workspace and using docker.inside() to pull in docker containers which have additional build tools needed to do a build. Docker.inside() maps the jenkins workspace to docker volume, so that what happens inside the docker container can read from, and write to the jenkins workspace.
Part of this is our build environment. The builds all run from slaves which run as docker containers inside Kubernetes. Those pods run privileged, so that they can reach the docker socket on the host to spin up "peer" docker containers when the docker.inside() commands are run. This is needed in order to run docker commands and to build new docker containers.
So for instance a slave does a checkout scm. gathers information about the repository commit using git commands, then does docker.inside() to pull in a pre-configured docker image with all the necessary npm modules already installed. It runs the gulp tasks inside() the container which thanks to the way things are mapped, writes the gulp output back to the jenkins workpace.
Then the inside() block closes, and the next steps of the pipeline do a docker build, and docker push to our registry.
I initially tired doing this in declarative pipeline by using an agent statement at the top just inside the pipeline{
agent label:'swarm', docker:'our-registry/npm-build:latest'
This initially failed because while the slave has git on it, git does not exist inside that npm-build image, so I couldn't use the git commands to determine the url and git commit hash
I initially added git to that image, and that part worked, but then I realized I could go no further, since there was no way to run docker commands inside this image which was already being run inside a docker container on the slave. I had no way of making it privileged so that I could access /var/run/docker.sock on the host.
I tried setting
pipeline{
agent label:'swarm'
which worked for the first part, of pulling in the code, and running the git commands, but then in another stage I tried
stage('gulp build'){
agent label:'swarm', docker:'our-registry/npm-build:latest'
but was greeted with a completely blank workspace.
I know I can use stash/unstash, to run the git commands on the slave workspace, then stash, then unstash inside the docker container, run the gulp commands, stash again, then grab a blank workspace again on slave and unstash, but the inefficiencies in doing that bug me. It would become a nightmare in some of our more complex workflows that use docker.inside() multiple times, and even have multiple different containers, each configured for just the tool needed for that stage.
I assume I can probably do some of that if I use a script{} block, but was hoping to avoid that as much as possible, as I hope to hand these pipelines off to people not as familiar with Jenkins and pipeline in general.
I'm just looking for a way to do docker.inside() like commands where we pull in another docker image and do work in there, but still have access to the workspace.
[JENKINS-40866] docker.inside() like syntax that keeps the workspace
I've improved the logic for determining if we're on a node, so that's nice - https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/93/commits/a262c3e9fa90372fadb9d2b0a64b27658f3b2d81 is the commit with the bulk of it including the tests. tomlarrow - any thoughts on that syntax?
Changing this to be the default behavior will cause existing pipelines to have possible unexpected outcomes. It also sounds like maintaining the workspace across stages with different containers will likely break if the top-level agent is a docker directive itself.
In that case it is probably best to keep the default behavior that each agent call in a stage will allocate a new node and execute on that node. Users would then need to set reuseNode true inside docker for the agent to maintain the workspace context but in a new container.
In addition, what if we add a top-level options to enable workspace reuse as the default.
pipeline {
options {
reuseNode true
}
agent {
label "my-label"
}
stages {
stage("Build") {
steps {
echo "I'm not in a container"
}
}
stage("Docker") {
agent {
docker {
image "ubuntu"
}
steps {
echo "I'm in a container on same workspace"
}
}
}
}
I agree with hrmpw that if I had already written pipelines that expected a new workspace every time, I wouldn't want the default changed on me. I like the option of reuseNode being something I could set in the options, so I wouldn't need to specify it every time I used it.
Just to make sure I was reading the code correctly, It looked like the vars would still be available to use in the stage agent definition. We don't do true docker in docker, rather if we have one docker container building another, we pass in the arg "-v /var/run/docker.sock:/var/run/docker.sock" in order to let the docker daemon inside the container control the host's docker and build sibling containers.
Other than that, it looks good, and will save quite a bit of script{} blocks in my declarative pipeline code.
Code changed in jenkins
User: Andrew Bayer
Path:
pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/AbstractDockerAgent.java
pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipeline.java
pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineFromDockerfile.java
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineFromDockerfileScript.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineScript.groovy
pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/AgentTest.java
pipeline-model-definition/src/test/resources/agentDockerDontReuseNode.groovy
pipeline-model-definition/src/test/resources/agentDockerReuseNode.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/a262c3e9fa90372fadb9d2b0a64b27658f3b2d81
Log:
JENKINS-40866 First work on reusing nodes for stage agents
More tests still needed.
Code changed in jenkins
User: Andrew Bayer
Path:
pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/AbstractDockerAgent.java
pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipeline.java
pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineFromDockerfile.java
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/AbstractDockerPipelineScript.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineFromDockerfileScript.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineScript.groovy
pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/AgentTest.java
pipeline-model-definition/src/test/resources/agentDockerDontReuseNode.groovy
pipeline-model-definition/src/test/resources/agentDockerReuseNode.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/3222d5ababdf848fe7b81d78f40140330a9d6051
Log:
Merge pull request #93 from abayer/jenkins-40866
JENKINS-40866 Docker agent in stage can reuse global node
Compare: https://github.com/jenkinsci/pipeline-model-definition-plugin/compare/b4d556f90759...3222d5ababdf
Merged! This'll be in the 0.9 release I'll be doing in the next couple days. Only syntax change is the addition of the reuseNode boolean option on docker and dockerfile.
Hey, abayer, thank you for adding this, I noticed that its not in syntax.md or the wiki (under changelog), might want to add that so other people can find it!
Stepan Mazurov
added a comment - - edited Hey, abayer , thank you for adding this, I noticed that its not in syntax.md or the wiki (under changelog), might want to add that so other people can find it! smazurov - how's https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/Controlling-your-build-environment#reusing-nodeworkspace-with-per-stage-docker-agents for now? I'll update SYNTAX.md as well.
I've got a very work-in-progress PR for this up at https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/93 - I decided to keep it simple in that it doesn't error out if you use it with agent none at the global level; instead, it'll just say "Oh, no node context, I'll use a label". And I left it still able to do docker-in-docker because...well, I couldn't trivially figure out how to prevent that at validation time. =) Will think about it more further and sleep on this.