[JENKINS-60517] Kubernetes plugin crashes with docker plugin .inside command

Type: Bug
Resolution: Fixed
Priority: Critical
Component/s: docker-workflow-plugin, kubernetes-plugin
Labels:
Environment:
jenkins v2.190.3
kubernetes plugin v1.22.1
docker workflow plugin v1.21

Similar Issues:
Powered by SuggestiMate

Show
Released As:
kubernetes-1.22.4

I have the following jenkinsfile:

        stage('build') {
            container('docker') {
                def tag = "${env.JOB_NAME}:${env.BUILD_ID}".toLowerCase()
                devTools = docker.build(tag, "--pull -f .docker/dev-tools/Dockerfile .")
                devTools.inside() {
                    sh 'make -f Makefile.test -j$(nproc) build-test'
                }
            }
        }

It works as expected with kubernetes plugin v1.22.0 (previous stable) and docker pipeline plugin v1.21 (current stable).

Today I installed a newer kubernetes plugin v1.22.1 and my jobs started failing with:

00:03:31.157  + docker inspect -f . <redacted>
00:03:31.157  .
00:03:31.185  [Pipeline] withDockerContainer
00:03:31.198  [Pipeline] // withDockerContainer
00:03:31.208  [Pipeline] }
00:03:31.234  [Pipeline] // container
00:03:31.251  [Pipeline] }
00:03:31.282  [Pipeline] // stage
00:03:31.303  [Pipeline] }
00:03:31.347  [Pipeline] // node
00:03:31.372  [Pipeline] }
00:03:31.398  [Pipeline] // podTemplate
00:03:31.430  [Pipeline] }
00:03:31.457  [Pipeline] // timeout
00:03:31.496  [Pipeline] End of Pipeline
00:03:31.526  java.lang.NullPointerException
00:03:31.526  	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:267)
00:03:31.526  	at hudson.Launcher$ProcStarter.start(Launcher.java:455)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:301)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:282)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:279)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.client.DockerClient.version(DockerClient.java:251)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:148)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:286)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
00:03:31.526  	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
00:03:31.526  	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
00:03:31.526  	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
00:03:31.526  	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:126)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66)
00:03:31.526  	at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:114)
00:03:31.526  	at WorkflowScript.run(WorkflowScript:26)
00:03:31.526  	at ___cps.transform___(Native Method)
00:03:31.526  	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
00:03:31.526  	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
00:03:31.526  	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
00:03:31.526  	at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
00:03:31.526  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
00:03:31.526  	at java.lang.reflect.Method.invoke(Method.java:498)
00:03:31.526  	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
00:03:31.526  	at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
00:03:31.526  	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
00:03:31.526  	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
00:03:31.526  	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
00:03:31.526  	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
00:03:31.526  	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
00:03:31.526  	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
00:03:31.526  	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
00:03:31.526  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:03:31.526  	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
00:03:31.526  	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
00:03:31.526  	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
00:03:31.526  	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
00:03:31.526  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:03:31.526  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:03:31.526  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:03:31.526  	at java.lang.Thread.run(Thread.java:748)

is caused by

JENKINS-58975 Executing shell in a container with a custom workingDir times out

Resolved

links to

PR #671

Ivan Kurnosov created issue - 2019-12-17 20:42

Ivan Kurnosov made changes - 2019-12-17 20:43

Environment

Original: 2.190.3

New: jenkins v2.190.3
kubernetes plugin v1.22.0
docker workflow plugin v1.21

Ivan Kurnosov made changes - 2019-12-19 22:56

Description

Original: I have the following jenkinsfile:

{code:groovy}
        stage('build') {
            container('docker') {
                def tag = "${env.JOB_NAME}:${env.BUILD_ID}".toLowerCase()
                devTools = docker.build(tag, "--pull -f .docker/dev-tools/Dockerfile .")
                devTools.inside() {
                    repository.code(this)
                    sh 'make -f Makefile.test -j$(nproc) build-test'
                }
            }
        }

{code}

It works as expected with kubernetes plugin v1.22.0 (previous stable) and docker pipeline plugin v1.21 (current stable).

Today I installed a newer kubernetes plugin v1.22.1 and my jobs started failing with:

{code}
00:03:31.157 + docker inspect -f . <redacted>
00:03:31.157 .
00:03:31.185 [Pipeline] withDockerContainer
00:03:31.198 [Pipeline] // withDockerContainer
00:03:31.208 [Pipeline] }
00:03:31.234 [Pipeline] // container
00:03:31.251 [Pipeline] }
00:03:31.282 [Pipeline] // stage
00:03:31.303 [Pipeline] }
00:03:31.347 [Pipeline] // node
00:03:31.372 [Pipeline] }
00:03:31.398 [Pipeline] // podTemplate
00:03:31.430 [Pipeline] }
00:03:31.457 [Pipeline] // timeout
00:03:31.496 [Pipeline] End of Pipeline
00:03:31.526 java.lang.NullPointerException
00:03:31.526 at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:267)
00:03:31.526 at hudson.Launcher$ProcStarter.start(Launcher.java:455)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:301)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:282)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:279)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.version(DockerClient.java:251)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:148)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:286)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
00:03:31.526 at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
00:03:31.526 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
00:03:31.526 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
00:03:31.526 at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:126)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:114)
00:03:31.526 at WorkflowScript.run(WorkflowScript:26)
00:03:31.526 at ___cps.transform___(Native Method)
00:03:31.526 at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
00:03:31.526 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
00:03:31.526 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
00:03:31.526 at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
00:03:31.526 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
00:03:31.526 at java.lang.reflect.Method.invoke(Method.java:498)
00:03:31.526 at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
00:03:31.526 at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
00:03:31.526 at com.cloudbees.groovy.cps.Next.step(Next.java:83)
00:03:31.526 at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
00:03:31.526 at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
00:03:31.526 at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
00:03:31.526 at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
00:03:31.526 at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
00:03:31.526 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:03:31.526 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
00:03:31.526 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
00:03:31.526 at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
00:03:31.526 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
00:03:31.526 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:03:31.526 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:03:31.526 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:03:31.526 at java.lang.Thread.run(Thread.java:748)
{code}

New: I have the following jenkinsfile:

{code:groovy}
        stage('build') {
            container('docker') {
                def tag = "${env.JOB_NAME}:${env.BUILD_ID}".toLowerCase()
                devTools = docker.build(tag, "--pull -f .docker/dev-tools/Dockerfile .")
                devTools.inside() {
                    sh 'make -f Makefile.test -j$(nproc) build-test'
                }
            }
        }

{code}

It works as expected with kubernetes plugin v1.22.0 (previous stable) and docker pipeline plugin v1.21 (current stable).

Today I installed a newer kubernetes plugin v1.22.1 and my jobs started failing with:

{code}
00:03:31.157 + docker inspect -f . <redacted>
00:03:31.157 .
00:03:31.185 [Pipeline] withDockerContainer
00:03:31.198 [Pipeline] // withDockerContainer
00:03:31.208 [Pipeline] }
00:03:31.234 [Pipeline] // container
00:03:31.251 [Pipeline] }
00:03:31.282 [Pipeline] // stage
00:03:31.303 [Pipeline] }
00:03:31.347 [Pipeline] // node
00:03:31.372 [Pipeline] }
00:03:31.398 [Pipeline] // podTemplate
00:03:31.430 [Pipeline] }
00:03:31.457 [Pipeline] // timeout
00:03:31.496 [Pipeline] End of Pipeline
00:03:31.526 java.lang.NullPointerException
00:03:31.526 at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:267)
00:03:31.526 at hudson.Launcher$ProcStarter.start(Launcher.java:455)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:301)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:282)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:279)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.client.DockerClient.version(DockerClient.java:251)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:148)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:286)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
00:03:31.526 at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
00:03:31.526 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
00:03:31.526 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
00:03:31.526 at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:126)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66)
00:03:31.526 at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(Docker.groovy:114)
00:03:31.526 at WorkflowScript.run(WorkflowScript:26)
00:03:31.526 at ___cps.transform___(Native Method)
00:03:31.526 at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
00:03:31.526 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
00:03:31.526 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
00:03:31.526 at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
00:03:31.526 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
00:03:31.526 at java.lang.reflect.Method.invoke(Method.java:498)
00:03:31.526 at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
00:03:31.526 at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
00:03:31.526 at com.cloudbees.groovy.cps.Next.step(Next.java:83)
00:03:31.526 at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
00:03:31.526 at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
00:03:31.526 at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
00:03:31.526 at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
00:03:31.526 at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
00:03:31.526 at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
00:03:31.526 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:03:31.526 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
00:03:31.526 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
00:03:31.526 at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
00:03:31.526 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
00:03:31.526 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:03:31.526 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:03:31.526 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:03:31.526 at java.lang.Thread.run(Thread.java:748)
{code}

Ivan Kurnosov made changes - 2019-12-19 23:11

Environment

Original: jenkins v2.190.3
kubernetes plugin v1.22.0
docker workflow plugin v1.21

New: jenkins v2.190.3
kubernetes plugin v1.22.1
docker workflow plugin v1.21

Vincent Latombe made changes - 2020-01-06 10:17

Remote Link

New: This issue links to "PR #671 (Web Link)" [ 24234 ]

Vincent Latombe made changes - 2020-01-06 10:18

Assignee

New: Narayanan Singaram [ narayanan ]

Vincent Latombe made changes - 2020-01-06 10:18

Resolution		New: Fixed [ 1 ]
Status	Original: Open [ 1 ]	New: Fixed but Unreleased [ 10203 ]

Vincent Latombe made changes - 2020-01-07 08:54

Released As		New: kubernetes-1.22.4
Status	Original: Fixed but Unreleased [ 10203 ]	New: Resolved [ 5 ]

Vincent Latombe made changes - 2021-01-08 09:30

Link

New: This issue is caused by ~~JENKINS-58975~~ [ ~~JENKINS-58975~~ ]

Assignee:: Narayanan Singaram

Reporter:: Ivan Kurnosov

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Created:: 2019-12-17 20:42

Updated:: 2021-01-08 09:30

Resolved:: 2020-01-06 10:18

Jenkins

Details

Description

Attachments

Issue Links

Activity

People

Dates