-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
None
-
Jenkins 1.651.1
CloudBees Docker Pipeline 1.4
Pipeline 2.0
Docker 1.11.0
RHEL 7.2
With a simple Jenkinsfile when building, at some point it'll fail for no obvious reason.
An example Jenkinsfile:
def img = 'centos:7'; node('docker') { stage "pulling"; sh "docker pull ${img}"; // workaround for JENKINS-34288 checkout scm; docker.image(img).inside { sh 'for i in $(seq 30); do sleep 1; echo $i; done'; sh 'ls -alh --color'; } }
Partial output:
[Pipeline] Run build steps inside a Docker container : Start
$ docker run -t -d -u 995:993 -w /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:rw -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** centos:7 cat
[Pipeline] withDockerContainer {
[Pipeline] sh
[master] Running shell script
++ seq 30
+ for i in '$(seq 30)'
+ sleep 1
[Pipeline] } //withDockerContainer
$ docker stop 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10
$ docker rm -f 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10
[Pipeline] Run build steps inside a Docker container : End
[Pipeline] } //node
[Pipeline] Allocate node : End
[Pipeline] End of Pipeline
- is related to
-
-
- Resolved
-
-
-
- Closed
-
- relates to
-
-
- Resolved
-
- links to
[JENKINS-34289] docker.image.inside fails unexpectedly with Jenkinsfile
| Description |
Original:
With a simple {{Jenkinsfile}} when building, at some point it'll fail for no obvious reason. An example {{Jenkinsfile}}: {{ def img = 'centos:7'; node('docker') { stage "pulling"; sh "docker pull ${img}"; // workaround for checkout scm; docker.image(img).inside { sh 'for i in $(seq 30); do sleep 1; echo $i; done'; sh 'ls -alh --color'; } } def img = 'centos:7'; node('docker') { stage "pulling"; sh "docker pull ${img}"; // workaround for checkout scm; docker.image(img).inside { sh 'for i in $(seq 30); do sleep 1; echo $i; done'; sh 'ls -alh --color'; } } }} Partial output: {{ [Pipeline] Run build steps inside a Docker container : Start $ docker run -t -d -u 995:993 -w /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:rw -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** centos:7 cat [Pipeline] withDockerContainer { [Pipeline] sh [master] Running shell script ++ seq 30 + for i in '$(seq 30)' + sleep 1 [Pipeline] } //withDockerContainer $ docker stop 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 $ docker rm -f 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 [Pipeline] Run build steps inside a Docker container : End [Pipeline] } //node [Pipeline] Allocate node : End [Pipeline] End of Pipeline }} |
New:
With a simple {{Jenkinsfile}} when building, at some point it'll fail for no obvious reason. An example {{Jenkinsfile}}: {code:groovy} def img = 'centos:7'; node('docker') { stage "pulling"; sh "docker pull ${img}"; // workaround for checkout scm; docker.image(img).inside { sh 'for i in $(seq 30); do sleep 1; echo $i; done'; sh 'ls -alh --color'; } } {code} Partial output: {noformat} [Pipeline] Run build steps inside a Docker container : Start $ docker run -t -d -u 995:993 -w /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:rw -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** centos:7 cat [Pipeline] withDockerContainer { [Pipeline] sh [master] Running shell script ++ seq 30 + for i in '$(seq 30)' + sleep 1 [Pipeline] } //withDockerContainer $ docker stop 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 $ docker rm -f 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 [Pipeline] Run build steps inside a Docker container : End [Pipeline] } //node [Pipeline] Allocate node : End [Pipeline] End of Pipeline {noformat} |
| Description |
Original:
With a simple {{Jenkinsfile}} when building, at some point it'll fail for no obvious reason. An example {{Jenkinsfile}}: {code:groovy} def img = 'centos:7'; node('docker') { stage "pulling"; sh "docker pull ${img}"; // workaround for checkout scm; docker.image(img).inside { sh 'for i in $(seq 30); do sleep 1; echo $i; done'; sh 'ls -alh --color'; } } {code} Partial output: {noformat} [Pipeline] Run build steps inside a Docker container : Start $ docker run -t -d -u 995:993 -w /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:rw -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** centos:7 cat [Pipeline] withDockerContainer { [Pipeline] sh [master] Running shell script ++ seq 30 + for i in '$(seq 30)' + sleep 1 [Pipeline] } //withDockerContainer $ docker stop 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 $ docker rm -f 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 [Pipeline] Run build steps inside a Docker container : End [Pipeline] } //node [Pipeline] Allocate node : End [Pipeline] End of Pipeline {noformat} |
New:
With a simple {{Jenkinsfile}} when building, at some point it'll fail for no obvious reason. An example {{Jenkinsfile}}: {code:java} def img = 'centos:7'; node('docker') { stage "pulling"; sh "docker pull ${img}"; // workaround for checkout scm; docker.image(img).inside { sh 'for i in $(seq 30); do sleep 1; echo $i; done'; sh 'ls -alh --color'; } } {code} Partial output: {noformat} [Pipeline] Run build steps inside a Docker container : Start $ docker run -t -d -u 995:993 -w /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master:rw -v /var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:/var/lib/jenkins/workspace/tron/docwhat-test-jenkinsfile/master@tmp:rw -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** centos:7 cat [Pipeline] withDockerContainer { [Pipeline] sh [master] Running shell script ++ seq 30 + for i in '$(seq 30)' + sleep 1 [Pipeline] } //withDockerContainer $ docker stop 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 $ docker rm -f 7fcbfd6ab39cf05257a43a774bd20b670bc39674a2047777fe603ee1a3162b10 [Pipeline] Run build steps inside a Docker container : End [Pipeline] } //node [Pipeline] Allocate node : End [Pipeline] End of Pipeline {noformat} |
| Environment |
Original:
Jenkins 1.651.1 CloudBees Docker Pipeline 1.4 Pipeline 2.0 |
New:
Jenkins 1.651.1 CloudBees Docker Pipeline 1.4 Pipeline 2.0 Docker 1.11.0 RHEL 7.2 |
Looking into the logs, I see this error when it dies:
Apr 19, 2016 11:42:51 AM FINE org.jenkinsci.plugins.docker.workflow.client.DockerClient
Executing docker command docker inspect -f {{.Image}} ab2e6f4420e8f74916096a64cc5dda59739f63d5585e6d0eee23ee4ed47fd411
Apr 19, 2016 11:42:51 AM FINE org.jenkinsci.plugins.docker.workflow.WithContainerStep
execution failure container=ab2e6f4420e8f74916096a64cc5dda59739f63d5585e6d0eee23ee4ed47fd411
hudson.AbortException: script returned exit code -1
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.check(DurableTaskStep.java:198)
at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.run(DurableTaskStep.java:150)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
From a quick glance this sounds like the (poorly stated) problem docker-workflow PR 25 purports to address. Is there a simple way to reproduce the problem from scratch?
| Remote Link | New: This issue links to "docker-workflow PR 25 (Web Link)" [ 14223 ] |
How about this?
node {
def img = docker.image('busybox');
img.pull();
img.inside {
sh 'for i in $(seq 30); do sleep 1; echo $i; done';
}
}
I'm pretty sure that the shell step isn't even executed inside the container and that AbortException is being returned from something else. If I change the step to run sleep 1 || true it still aborts with -1.
So part of the problem appears to be the -u 995:993 option passed to docker run. This user doesn't exist inside the container and certain commands cause the container to just "exit" with an unsuccessful exit code.
I made a simple Dockerfile that creates the user and group and it helps with some cases (e.g. ps -ef works now) but my for loop still dies on the first sleep 1.
New Jenkinsfile :
node('docker') { checkout scm; docker.build('fish', '.').inside { sh 'for i in $(seq 30); do sleep 1; echo $i; done'; sh 'ls -alh --color'; } }Dockerfile:
Edits: