Details
-
Bug
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Fixed
-
None
-
pipeline-model-definition-plugin v1.2
Description
Version 1.2 of pipeline-model-definition-plugin changes the behavior of docker images. `docker pull` is now performed for docker images specified in `agent`. Previously this was not performed.
This change affects use cases where the docker image only exists locally. It will obviously fail if `docker pull` is called.
The change was introduced in https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/182
The simple workaround is to patch the plugin to revert this change, but it would be nice to avoid that.
Hopefully this could be reverted to old behavior or/and an option added for configuration.
Attachments
Issue Links
- links to
Activity
Denis Boulas
added a comment - Worth to add that there is no workaround currently as it seems.
I've tried to replace local image with remote, however Jenkins fails to authenticate with ECR prior to actual check for image update... So I would change priority or provide any working workaround for this for now.
Denis Boulas
added a comment - Worth to add that there is no workaround currently as it seems.
I've tried to replace local image with remote, however Jenkins fails to authenticate with ECR prior to actual check for image update... So I would change priority or provide any working workaround for this for now. Our build system has been hit by this problem too. All our pipelines now fail as they need to use locally built images!
Scott Hilleard
added a comment - - edited Our build system has been hit by this problem too. All our pipelines now fail as they need to use locally built images! 
Dan Ackroyd
added a comment - I think it would be appropriate for the priority to be updated to be 'major' or 'critical', and for there to be some announcement about this being a breaking bug.
Upgrading to this version of the plugin, when you have images that are stored locally, apparently breaks the build pipeline.
So anyone using the previous version and having local images in their pipeline need to be aware this bug will break their build.
(apologies in advance for cross-posting, but this caused quite a bit of headaches for us, which other people shouldn't have to suffer.)
Dan Ackroyd
added a comment - I think it would be appropriate for the priority to be updated to be 'major' or 'critical', and for there to be some announcement about this being a breaking bug.
Upgrading to this version of the plugin, when you have images that are stored locally, apparently breaks the build pipeline.
So anyone using the previous version and having local images in their pipeline need to be aware this bug will break their build.
(apologies in advance for cross-posting, but this caused quite a bit of headaches for us, which other people shouldn't have to suffer.) Any chance someone could give me a reproduction case from scratch so that I can write a unit test? I'd like to be able to verify the correct behavior in the fix, so that I can retain the JENKINS-46277 fix as well.
Also, it's be immensely helpful if someone could provide the logs/errors they get with the current behavior.
abayer: To reproduce I merely need to build the local image on the node so that it resides in the local docker repo. The image: argument of the docker agent just specifies that local image.
docker {
image 'build_linux:latest'
}
The pipeline error log thusly looks like:
12:09:13 + docker pull build_linux:latest 12:09:14 Error response from daemon: repository build_linux not found: does not exist or no pull access
Peter Wilcock
added a comment - - edited abayer : To reproduce I merely need to build the local image on the node so that it resides in the local docker repo. The image: argument of the docker agent just specifies that local image.
docker {
image 'build_linux:latest'
}
The pipeline error log thusly looks like:
12:09:13 + docker pull build_linux:latest
12:09:14 Error response from daemon: repository build_linux not found: does not exist or no pull access
PR up at https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/199 for changing to alwaysDoDockerPull in options, with the default being not to pull.
Code changed in jenkins
User: Andrew Bayer
Path:
content/doc/book/pipeline/syntax.adoc
http://jenkins-ci.org/commit/jenkins.io/2546503a7f816ed1cb9b764c2d01b19079bf1253
Log:
JENKINS-47106 Add docs for alwaysDoDockerPull option
Code changed in jenkins
User: R. Tyler Croy
Path:
content/doc/book/pipeline/syntax.adoc
http://jenkins-ci.org/commit/jenkins.io/10a1828a6afd68ac8341969bb7abb21937775b2f
Log:
Merge pull request #1152 from abayer/jenkins-47106
JENKINS-47106 Add docs for alwaysDoDockerPull option
Compare: https://github.com/jenkins-infra/jenkins.io/compare/9e529c025017...10a1828a6afd
Code changed in jenkins
User: Andrew Bayer
Path:
pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Agent.groovy
pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/options/impl/AlwaysDoDockerPull.java
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineScript.groovy
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/options/impl/AlwaysDoDockerPull/help.jelly
pipeline-model-definition/src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/AgentTest.java
pipeline-model-definition/src/test/resources/dockerPullLocalImage.groovy
pipeline-model-extensions/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/agent/DeclarativeAgent.java
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/1c275cd3958c595fb32decdc80177efd46af5b19
Log:
[FIXED JENKINS-47106] Only do explicit docker pull if specified
Automatically doing docker pull was breaking usage of local images. It
may make sense to re-enable the automatic pull for the top-level
agent, though.
Also added a way to store arbitrary DeclarativeOptions on the
DeclarativeAgent, so that we don't have to hardcode any new options in
the future.
Scott Hilleard
added a comment - abayer Thanks for getting a fix implemented so quickly! 
Ryan Fitzsimon
added a comment - AFAICT, a regression was introduced by a367db7c2a727e5ea87381777b72b4c84b52f1a2.
It seems this commit removes the check of alwaysDoDockerPull for agents that aren't within a stage, making it impossible to use a local image in top level scope.
I could easily be wrong, please let me know if I've misunderstood something.
Ryan Fitzsimon
added a comment - AFAICT, a regression was introduced by a367db7c2a727e5ea87381777b72b4c84b52f1a2.
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/a367db7c2a727e5ea87381777b72b4c84b52f1a2
It seems this commit removes the check of alwaysDoDockerPull for agents that aren't within a stage, making it impossible to use a local image in top level scope.
I could easily be wrong, please let me know if I've misunderstood something. 
Johnson Xu
added a comment - I met same issue as rmf.
When I set 'alwaysPull false' within stage block, it works fine as expect to use local image. However, if the same config is set within the top 'pipeline/agent' block, it still always do 'docker pull' and the local image does not work.
Johnson Xu
added a comment - I met same issue as rmf .
When I set 'alwaysPull false' within stage block, it works fine as expect to use local image. However, if the same config is set within the top 'pipeline/agent' block, it still always do 'docker pull' and the local image does not work. Well, that was dumb on my part. Will get a fix up today or tomorrow. Sorry!
Dan Cioiu
added a comment - Hi abayer ,
Any news on this regression? Code changed in jenkins
User: Segev Finer
Path:
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineScript.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/5c4946495236afe8c778b9662ffd754923a728e5
Log:
[FIXED JENKINS-47106] Support alwaysPull in the top level agent section
Code changed in jenkins
User: Andrew Bayer
Path:
pipeline-model-definition/src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineScript.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/fa8ed8bb0861220ff36b730656dc12701ba86ffd
Log:
Merge pull request #254 from segevfiner/jenkins-47106-top-level-agent-local-image
[FIXED JENKINS-47106] Support alwaysPull in the top level agent section
Compare: https://github.com/jenkinsci/pipeline-model-definition-plugin/compare/f5deef1255d0...fa8ed8bb0861
Also confirmed. Support for local images +1
The original bug requested that this behaviour be toggled with a flag. That seems to be a sensible option.