[JENKINS-53297] Pod name is detected as `localhost` in Bluemix IKS

Type: Bug
Resolution: Fixed
Priority: Blocker
Component/s: kubernetes-plugin
Labels:
- jenkins
- kubernetes-plugin
Environment:
Jenkins 2.121.3, 2.141
Kubernetes IKS 1.11 (Bluemix)
Kubernetes Plugin 1.12.4

Similar Issues:
Powered by SuggestiMate

Show

I've Jenkins running on Kubernetes Cluster (Bluemix IKS).

I am using the kubernetes plugin to create dynamic agents.

The jenkins connection with agent is made in the standard way:

Jenkins URL: jenkins:8080

Jenkins Tunnel: jenkins-agent:50000

When i start a job, the agent connect with Jenkins master successfully. But he keeps long time and not execute pipeline steps.

Agent logs:

Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorWarning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorAug 29, 2018 12:07:02 AM hudson.remoting.jnlp.Main createEngineINFO: Setting up agent: jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener <init>INFO: Jenkins agent is running in headless mode.Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineINFO: Using Remoting version: 3.23Aug 29, 2018 12:07:03 AM hudson.remoting.Engine startEngineWARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jarsAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Locating server among [http://jenkins:8080/]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]Aug 29, 2018 12:07:03 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability checkAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Agent discovery successful  Agent address: jenkins-agent  Agent port:    50000  Identity:      bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: HandshakingAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Connecting to jenkins-agent:50000Aug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Trying protocol: JNLP4-connectAug 29, 2018 12:07:03 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: Remote identity confirmed: bb:d3:5f:09:02:5c:56:70:04:7f:8f:72:f5:bd:23:79Aug 29, 2018 12:07:04 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: ConnectedAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave$SlaveDisconnector callINFO: Disabled slave engine reconnects.Aug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Error while determining if vetoers existhudson.remoting.RequestAbortedException: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at hudson.remoting.Request.abort(Request.java:340) at hudson.remoting.Channel.terminate(Channel.java:1038) at hudson.remoting.Channel.close(Channel.java:1439) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1270) at hudson.remoting.Channel$1.handle(Channel.java:565) at hudson.remoting.AbstractByteBufferCommandTransport.processCommand(AbstractByteBufferCommandTransport.java:203) at hudson.remoting.AbstractByteBufferCommandTransport.receive(AbstractByteBufferCommandTransport.java:189) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onRead(ChannelApplicationLayer.java:187) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecv(ApplicationLayer.java:207) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:369) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669) at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$2200(BIONetworkLayer.java:48) at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:283) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to jenkins-agent/172.21.160.215:50000 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.Request.call(Request.java:202) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286) at com.sun.proxy.$Proxy5.fetch3(Unknown Source) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:209) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.lang.ClassLoader.defineClass(ClassLoader.java:642) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:383) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at hudson.util.ProcessTree.get(ProcessTree.java:399) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 4 moreCaused by: hudson.remoting.ChannelClosedException: Channel "unknown": Protocol stack cannot write data anymore. ChannelApplicationLayer reports that the NIO Channel is closed at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:302) at hudson.remoting.AbstractByteBufferCommandTransport.write(AbstractByteBufferCommandTransport.java:304) at hudson.remoting.Channel.send(Channel.java:721) at hudson.remoting.Channel.close(Channel.java:1436) ... 18 moreCaused by: java.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doSend(ProtocolStack.java:694) at org.jenkinsci.remoting.protocol.ApplicationLayer.write(ApplicationLayer.java:157) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.write(ChannelApplicationLayer.java:299) ... 21 more
Aug 29, 2018 12:12:09 AM hudson.remoting.jnlp.Main$CuiListener statusINFO: TerminatedAug 29, 2018 12:12:09 AM hudson.util.ProcessTree getWARNING: Failed to load winp. Reverting to the defaultjava.lang.NoClassDefFoundError: hudson/util/ProcessTree$Linux$LinuxProcess at hudson.util.ProcessTree.get(ProcessTree.java:415) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.ClassNotFoundException: hudson.util.ProcessTree$Linux$LinuxProcess at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:171) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 12 more

Jenkins Master logs

Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionAug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Excess workload after pending Kubernetes agents: 1Aug 29, 2018 12:06:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provisionINFO: Template: Kubernetes Pod TemplateAug 29, 2018 12:06:51 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:06:51 AM hudson.slaves.NodeProvisioner$StandardStrategyImpl applyINFO: Started provisioning Kubernetes Pod Template from devops-k8s with 1 executors. Remaining excess workload: 0Aug 29, 2018 12:07:01 AM hudson.slaves.NodeProvisioner$2 runINFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)Aug 29, 2018 12:07:01 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Created Pod: jenkins-slave-n39mq-9zgzz in namespace jenkinsAug 29, 2018 12:07:01 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launchINFO: Waiting for Pod to be scheduled (0/100): jenkins-slave-n39mq-9zgzzAug 29, 2018 12:07:03 AM hudson.TcpSlaveAgentListener$ConnectionHandler runINFO: Accepted JNLP4-connect connection #3 from /172.30.89.194:44930Aug 29, 2018 12:07:08 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:08 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Terminating Kubernetes instance for agent jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM okhttp3.internal.platform.Platform logINFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?Aug 29, 2018 12:12:09 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosedWARNING: Computer.threadPoolForRemoting [#88] for jenkins-slave-n39mq-9zgzz terminatedjava.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314) at hudson.remoting.Channel.close(Channel.java:1450) at hudson.remoting.Channel.close(Channel.java:1403) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:799) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:103) at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:715) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave deleteSlavePodINFO: Terminated Kubernetes instance for agent jenkins/jenkins-slave-n39mq-9zgzzDisconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminateINFO: Disconnected computer jenkins-slave-n39mq-9zgzzAug 29, 2018 12:12:09 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finishedINFO: Removing pod template jenkins-slave-n39mq from cloud devops-k8sAug 29, 2018 12:12:10 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finishINFO: teste-deploy #45 completed: FAILURE

Console Output

Started by user Vinicius Xavier Running in Durability level: MAX_SURVIVABILITY [Pipeline] podTemplate [Pipeline] { [Pipeline] node Still waiting to schedule task There are no nodes with the label ‘jenkins-pipeline’ Agent jenkins-slave-n39mq-9zgzz is provisioned from template Kubernetes Pod Template Agent specification [Kubernetes Pod Template] (jenkins-pipeline): * [helm] lachlanevenson/k8s-helm Running on jenkins-slave-n39mq-9zgzz in /home/jenkins/workspace/teste-deploy [Pipeline] { [Pipeline] container [Pipeline] { [Pipeline] sh [teste-deploy] Running shell script [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) Caused: java.io.IOException: Failed to execute shell script inside container [helm] of pod [localhost]. Timed out waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:449) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810) at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:46) at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20) at WorkflowScript.run(WorkflowScript:10) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor375.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE

Jenkinsfile

podTemplate(label: 'jenkins-pipeline', cloud: 'devops-k8s', 
  containers: [
    containerTemplate(name: 'helm', image: 'lachlanevenson/k8s-helm', command: 'cat', ttyEnabled: true)], 
  volumes:[ 
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')]
){ 
node ('jenkins-pipeline') { 
  container('helm') { 
    sh 'helm init --client-only' sh 'helm ls --namespace default' 
  } 
 } 
}

Obs.: This happened after update Kubernetes 1.10 to 1.11. IKS changed RuntimeContainer from Docker to Containerd

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

image-2018-10-22-09-47-09-082.png
29 kB
2018-10-22 07:47
image-2018-10-22-09-40-49-930.png
26 kB
2018-10-22 07:40
jenkins-master.log
23 kB
2018-08-29 19:41

links to

PR-385

PR-388

Vinicius Xavier added a comment - 2018-08-29 18:39

I actived the ALL log level and caught this:

<-- 404 Not Found https://kubernetes.default/api/v1/namespaces/jenkins/pods/localhost (3ms)Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Type: application/jsonAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logDate: Wed, 29 Aug 2018 18:32:10 GMTAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Length: 186Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform log{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"localhost\" not found","reason":"NotFound","details":{"name":"localhost","kind":"pods"},"code":404}

Why jenkins using 'localhost' with name of the pod???

Vinicius Xavier added a comment - 2018-08-29 18:39 I actived the ALL log level and caught this: <-- 404 Not Found https: //kubernetes. default /api/v1/namespaces/jenkins/pods/localhost (3ms)Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Type: application/jsonAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logDate: Wed, 29 Aug 2018 18:32:10 GMTAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logContent-Length: 186Aug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform logAug 29, 2018 6:32:10 PM INFO okhttp3.internal.platform.Platform log{ "kind" : "Status" , "apiVersion" : "v1" , "metadata" :{}, "status" : "Failure" , "message" : "pods \" localhost\ " not found" , "reason" : "NotFound" , "details" :{ "name" : "localhost" , "kind" : "pods" }, "code" :404} Why jenkins using 'localhost' with name of the pod???

Carlos Sanchez added a comment - 2018-08-29 19:14

what are the log lines before that one

Carlos Sanchez added a comment - 2018-08-29 19:14 what are the log lines before that one

Vinicius Xavier added a comment - 2018-08-29 19:31

I noticed that when using for example: 'echo hi', the pipeline runs successfully, jenkins GET pods/<slave name> endpoint.
But, when using any command 'sh "cat /etc/hosts" ', jenkins try GET pods/localhost endpoint

--> GET https://kubernetes.default/api/v1/namespaces/jenkins/pods/localhost http/1.1
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJqZW5raW5zIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImplbmtpbnMtdG9rZW4tZnFsZmMiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiamVua2lucyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjM3NzU1YTY1LTk1YmEtMTFlOC04MzlkLTJhZGJhODJkZmMxMSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpqZW5raW5zOmplbmtpbnMifQ.mfHpadNaF6JhwS7YL171TBg1J_RxOQJwfcyAYUHDjPnc3VI_E6_LwXZ9CRJIIXxtcN7Ny8EWwL09EonaFq20LfZNK5TQaYbu6FIyipMNGaZ1JxA-vPZ-ZYghxOJcbK36JK_ap71g9Jiun7qUVbqGo39yuF6Ga-ELcDil30BOX3FMrZiAH3D6yTy0FUKMLt81KnCzwUEdyKSOO86Z6nNzzZTQ8qw_lVJOJbUPjicbMcOLz1yp19UW8sFYr2smY-BSv6oeHkoXCfbccKu7UsRFSxFKnFOK-5bv8TBwuZm8Wkm93RVUg9dNhrGnYUI5RtzJ1i2ntjqyBSBdYDeinYz5VPPTRhK7zG_k8bmIsw1OAiUdKEQZIsON1EaNYxFEwJ2yd3byQ5N-BUvkfEbOXwM6pWbKDM_RjEfZcFcSXr2XOC_1q6vbwaWBLKtEyD_xvz_ZLq7dbIU5N-FYm8NwAtSYcyxssnHleOqpUfUcDvVepkXJySMs__aarzMjdQYqLhpoCDQtiAZUq6LyFRtW3AezOClj1vweUoUKVx5KCyHm3meEKFc38TUUn9BkYY3lMlz4Q579TLzKzlTmQSFy9sdY1eSvQlikzLx-VSCb5aSc9D6xrINbky0tDmFH1uM7eT7oNRE9ZphctWADz3kZrP_cU-EWW4Ja798jpln93YgrFpY
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Host: kubernetes.default
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Connection: Keep-Alive
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Accept-Encoding: gzip
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
User-Agent: okhttp/3.9.1
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
--> END GET
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
<-- 404 Not Found https://kubernetes.default/api/v1/namespaces/jenkins/pods/localhost (3ms)
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Content-Type: application/json
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Date: Wed, 29 Aug 2018 19:28:34 GMT
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Content-Length: 186
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"localhost\" not found","reason":"NotFound","details":{"name":"localhost","kind":"pods"},"code":404}

Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log
<-- END HTTP (186-byte body)

Vinicius Xavier added a comment - 2018-08-29 19:31 I noticed that when using for example: 'echo hi', the pipeline runs successfully, jenkins GET pods/<slave name> endpoint. But, when using any command 'sh "cat /etc/hosts" ', jenkins try GET pods/localhost endpoint --> GET https: //kubernetes. default /api/v1/namespaces/jenkins/pods/localhost http/1.1 Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJqZW5raW5zIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImplbmtpbnMtdG9rZW4tZnFsZmMiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiamVua2lucyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjM3NzU1YTY1LTk1YmEtMTFlOC04MzlkLTJhZGJhODJkZmMxMSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpqZW5raW5zOmplbmtpbnMifQ.mfHpadNaF6JhwS7YL171TBg1J_RxOQJwfcyAYUHDjPnc3VI_E6_LwXZ9CRJIIXxtcN7Ny8EWwL09EonaFq20LfZNK5TQaYbu6FIyipMNGaZ1JxA-vPZ-ZYghxOJcbK36JK_ap71g9Jiun7qUVbqGo39yuF6Ga-ELcDil30BOX3FMrZiAH3D6yTy0FUKMLt81KnCzwUEdyKSOO86Z6nNzzZTQ8qw_lVJOJbUPjicbMcOLz1yp19UW8sFYr2smY-BSv6oeHkoXCfbccKu7UsRFSxFKnFOK-5bv8TBwuZm8Wkm93RVUg9dNhrGnYUI5RtzJ1i2ntjqyBSBdYDeinYz5VPPTRhK7zG_k8bmIsw1OAiUdKEQZIsON1EaNYxFEwJ2yd3byQ5N-BUvkfEbOXwM6pWbKDM_RjEfZcFcSXr2XOC_1q6vbwaWBLKtEyD_xvz_ZLq7dbIU5N-FYm8NwAtSYcyxssnHleOqpUfUcDvVepkXJySMs__aarzMjdQYqLhpoCDQtiAZUq6LyFRtW3AezOClj1vweUoUKVx5KCyHm3meEKFc38TUUn9BkYY3lMlz4Q579TLzKzlTmQSFy9sdY1eSvQlikzLx-VSCb5aSc9D6xrINbky0tDmFH1uM7eT7oNRE9ZphctWADz3kZrP_cU-EWW4Ja798jpln93YgrFpY Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Host: kubernetes. default Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Connection: Keep-Alive Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Accept-Encoding: gzip Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log User-Agent: okhttp/3.9.1 Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log --> END GET Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log <-- 404 Not Found https: //kubernetes. default /api/v1/namespaces/jenkins/pods/localhost (3ms) Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Content-Type: application/json Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Date: Wed, 29 Aug 2018 19:28:34 GMT Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Content-Length: 186 Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log { "kind" : "Status" , "apiVersion" : "v1" , "metadata" :{}, "status" : "Failure" , "message" : "pods \" localhost\ " not found" , "reason" : "NotFound" , "details" :{ "name" : "localhost" , "kind" : "pods" }, "code" :404} Aug 29, 2018 7:28:34 PM INFO okhttp3.internal.platform.Platform log <-- END HTTP (186- byte body)

Vinicius Xavier added a comment - 2018-08-29 19:42

I attached a log, from the beginning of the job to the first 404 in the endpoint pods / localhost.

Vinicius Xavier added a comment - 2018-08-29 19:42 I attached a log, from the beginning of the job to the first 404 in the endpoint pods / localhost.

leon gregori added a comment - 2018-09-10 15:52 - edited

Hello,

I have also been using the Jenkins Kubernetes Plugin 1.12.4 on IBM Cloud.
It also stopped working over the Weekend and im currently in investigation.

Job output is this, but i will provide more soon. As soon as I can identify the issue and see if it is another root cause, I will create a seperate issue.

java.io.IOException: container [java] does not exist in pod [jenkins-slave-sm0j8-666ng]
 at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:435)
 at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
 at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
 at hudson.Launcher$ProcStarter.start(Launcher.java:454)
 at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186)
 at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86)
 at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182)
 at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229)
 at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153)
 at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
 at sun.reflect.GeneratedMethodAccessor906.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
 at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
 at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
 at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
 at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
 at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
 at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:133)
 at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
 at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
 at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
 at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
 at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
 at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
 at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
 at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
 at WorkflowScript.run(WorkflowScript:44)
 at __cps.transform__(Native Method)
 at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
 at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:103)
 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
 at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
 at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:60)
 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
 at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
 at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
 at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
 at com.cloudbees.groovy.cps.Next.step(Next.java:83)
 at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
 at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
 at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
 at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
 at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
 at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
 at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
 at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
 at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
 at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
 at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
 at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
 Finished: FAILURE

leon gregori added a comment - 2018-09-10 15:52 - edited Hello, I have also been using the Jenkins Kubernetes Plugin 1.12.4 on IBM Cloud. It also stopped working over the Weekend and im currently in investigation. Job output is this, but i will provide more soon. As soon as I can identify the issue and see if it is another root cause, I will create a seperate issue. java.io.IOException: container [java] does not exist in pod [jenkins-slave-sm0j8-666ng] at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:435) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:186) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:86) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:182) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:229) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:153) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor906.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:133) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:44) at __cps.transform__(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:103) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:60) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE

Alok Rajiv added a comment - 2018-09-18 09:03 - edited

Hi ,

So, I think what everyone is facing here is an issue stemming from the runtime dependency change. Thanks to @rtheis from IBM Cloud Network Dev Team on Slack who reminded me of the change.

IKS on K8S v1.11 now uses containerd runtime instead of docker runtime. I fear this is why our container state cannot be ready by the master. I experimented on 2 new fresh clusters with 1.10 and 1.11 on IBM Cloud and do confirm that 1.10 works just fine and replicated the above issue on 1.11 (The docker mount itself is failing on 1.11)

IBM migration doc link: https://console.bluemix.net/docs/containers/cs_versions.html#containerd

Do correct me if anyone has another side to this story.

Alok Rajiv added a comment - 2018-09-18 09:03 - edited Hi , So, I think what everyone is facing here is an issue stemming from the runtime dependency change. Thanks to @rtheis from IBM Cloud Network Dev Team on Slack who reminded me of the change. IKS on K8S v1.11 now uses containerd runtime instead of docker runtime. I fear this is why our container state cannot be ready by the master. I experimented on 2 new fresh clusters with 1.10 and 1.11 on IBM Cloud and do confirm that 1.10 works just fine and replicated the above issue on 1.11 (The docker mount itself is failing on 1.11) IBM migration doc link: https://console.bluemix.net/docs/containers/cs_versions.html#containerd Do correct me if anyone has another side to this story.

leon gregori added a comment - 2018-09-18 14:19

alokrajiv yes that is the root cause if using IKS 1.11.

In Jenkins Pipeline it should be just fine to mount containerd socket instead of docker socket.
I could not yet test this as our cluster is not updated yet to containerd.

I will keep you updated here.

Currently (Working on old IBM IKS)

    volumes: [  hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
                hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')]

Possible fix (Working on new containerd IBM IKS)

    volumes: [  hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: 'Certificates'),
                hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: 'SOCKET FOR CONTAINERD')]

leon gregori added a comment - 2018-09-18 14:19 alokrajiv yes that is the root cause if using IKS 1.11. In Jenkins Pipeline it should be just fine to mount containerd socket instead of docker socket. I could not yet test this as our cluster is not updated yet to containerd. I will keep you updated here. Currently (Working on old IBM IKS) volumes: [ hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' )] Possible fix (Working on new containerd IBM IKS) volumes: [ hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: 'Certificates' ), hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: 'SOCKET FOR CONTAINERD' )]

leon gregori added a comment - 2018-10-10 11:48

Still no solution until now.
1.11 Kubernetes on IBM Cloud uses containerd runtime and we dont know how to use the correct socket. I think same is applicable for kubernetes plugin in jenkins.

See also:
https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd?noredirect=1#comment92152693_52179148

leon gregori added a comment - 2018-10-10 11:48 Still no solution until now. 1.11 Kubernetes on IBM Cloud uses containerd runtime and we dont know how to use the correct socket. I think same is applicable for kubernetes plugin in jenkins. See also: https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd?noredirect=1#comment92152693_52179148

David Currie added a comment - 2018-10-10 15:41

Is that Jenkinsfile just an example or what you're actually trying to achieve? You shouldn't need a Docker socket in order to use helm. That is not to say that this isn't a problem in other scenarios e.g. when trying to use the daemon on the Kubernetes node to perform Docker builds. In that case, you could look at using kaniko or fall back to Docker-in-Docker.

David Currie added a comment - 2018-10-10 15:41 Is that Jenkinsfile just an example or what you're actually trying to achieve? You shouldn't need a Docker socket in order to use helm . That is not to say that this isn't a problem in other scenarios e.g. when trying to use the daemon on the Kubernetes node to perform Docker builds. In that case, you could look at using kaniko or fall back to Docker-in-Docker.

Alok Rajiv added a comment - 2018-10-10 15:56

Hi David,

About the docker builds I was thinking on the same grounds. But, before we reach that step the issue is the builder containers spawned up for the job itself is the issue.

Currently, jenkins master doesn't see the slave builder container ready status. My error is `Timed out waiting for container to become ready!`

Is the Kubernetes Pod Template, using docker daemon underneath?

Alok Rajiv added a comment - 2018-10-10 15:56 Hi David, About the docker builds I was thinking on the same grounds. But, before we reach that step the issue is the builder containers spawned up for the job itself is the issue. Currently, jenkins master doesn't see the slave builder container ready status. My error is `Timed out waiting for container to become ready!` Is the Kubernetes Pod Template, using docker daemon underneath?

David Currie added a comment - 2018-10-10 16:24 - edited

Is that even if you remove the hostPathVolume from the podTemplate? My assumption was that the slave pod was failing to start because it couldn't satisfy the requested mount.

David Currie added a comment - 2018-10-10 16:24 - edited Is that even if you remove the hostPathVolume from the podTemplate ? My assumption was that the slave pod was failing to start because it couldn't satisfy the requested mount.

Alok Rajiv added a comment - 2018-10-10 16:26

I'll need to get back on this. Will need to spin up a new cluster in 1.11 to test again tomorrow.
Could just have been the bad mount. Thanks very much for the leads !

Alok Rajiv added a comment - 2018-10-10 16:26 I'll need to get back on this. Will need to spin up a new cluster in 1.11 to test again tomorrow. Could just have been the bad mount. Thanks very much for the leads !

leon gregori added a comment - 2018-10-11 11:19

docker-sock-volume:
Type: HostPath (bare host directory volume)
Path: /var/run/containerd/containerd.sock
HostPathType: File

Seems to work. So within a docker container from pipeline this can be mounted to execute eg. "docker build ."
But it gives new error

Error: failed to create containerd task: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"docker pull jenkins\": executable file not found in $PATH": unknown
Warning BackOff 0s (x2 over 2s) kubelet, 10.144.181.244 Back-off restarting failed container

leon gregori added a comment - 2018-10-11 11:19 docker-sock-volume: Type: HostPath (bare host directory volume) Path: /var/run/containerd/containerd.sock HostPathType: File Seems to work. So within a docker container from pipeline this can be mounted to execute eg. "docker build ." But it gives new error Error: failed to create containerd task: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"docker pull jenkins\": executable file not found in $PATH": unknown Warning BackOff 0s (x2 over 2s) kubelet, 10.144.181.244 Back-off restarting failed container

David Currie added a comment - 2018-10-11 11:29

You can't expect to be able to just swap out the Docker socket for a containerd socket and have that work. Even setting that aside, containerd doesn't even have the concept of building images. I've been out of the loop a bit with IKS but I suspect the path of least resistance here is to have Jenkins use the IBM Container Registry to build your Docker images i.e. via ibmcloud cr build.

David Currie added a comment - 2018-10-11 11:29 You can't expect to be able to just swap out the Docker socket for a containerd socket and have that work. Even setting that aside, containerd doesn't even have the concept of building images. I've been out of the loop a bit with IKS but I suspect the path of least resistance here is to have Jenkins use the IBM Container Registry to build your Docker images i.e. via ibmcloud cr build .

leon gregori added a comment - 2018-10-17 06:35 - edited

dcurrie IBM container registry "cr build" command uses local docker deamon. I just tried it and it build locally.

So i think there are only 2 workarounds:
1. Having docker build Slave on some VM outside of K8
2. Using some Java based building service like kaniko

See also

https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd/52871199#52871199

leon gregori added a comment - 2018-10-17 06:35 - edited dcurrie IBM container registry "cr build" command uses local docker deamon. I just tried it and it build locally. So i think there are only 2 workarounds: 1. Having docker build Slave on some VM outside of K8 2. Using some Java based building service like kaniko See also https://stackoverflow.com/questions/52179148/jenkins-kubernetes-plugin-with-containerd/52871199#52871199

leon gregori added a comment - 2018-10-18 10:47 - edited

So i can only say that the issue in this ticket was now reproduced by me:
(IKS 1.11, Plugin 1.12.4 or 1.12.9, happens on both)

[testingSimple] Running shell script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep(Native Method)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
	at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out waiting for container to become ready!
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
	at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
	at WorkflowScript.run(WorkflowScript:43)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
	at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

So in the end the plugin is not capable of running sh inside the slaves

leon gregori added a comment - 2018-10-18 10:47 - edited So i can only say that the issue in this ticket was now reproduced by me: (IKS 1.11, Plugin 1.12.4 or 1.12.9, happens on both) [testingSimple] Running shell script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang. Thread .sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:417) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:438) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:43) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE So in the end the plugin is not capable of running sh inside the slaves

Carlos Sanchez added a comment - 2018-10-18 17:01

Can you try building the ~~JENKINS-53297~~ branch in PR-385 and see what is the error now?

Carlos Sanchez added a comment - 2018-10-18 17:01 Can you try building the JENKINS-53297 branch in PR-385 and see what is the error now?

leon gregori added a comment - 2018-10-18 19:59 - edited

@Carlos Sanchez will try tomorrow. Appreciate your commits

leon gregori added a comment - 2018-10-18 19:59 - edited @Carlos Sanchez will try tomorrow. Appreciate your commits

leon gregori added a comment - 2018-10-19 07:00 - edited

csanchez I tested the ~~JENKINS-53297~~ plugin

Manage Plugins:
Kubernetes plugin
This plugin integrates Jenkins with Kubernetes
1.12.10-SNAPSHOT (private-d0dc3394-blwdev)

I tested your Commit with the following Jenkinsfile:

// Simple Jenkinsfile for testing Kubernetes Plugin
// Developers input is required here.
def appname = "test"
def jarname = "test.jar"// Jenkins Deployment Variables and Credentials
def cloud = env.CLOUD ?: "kubernetes"
def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id"
def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id"
def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id"
def serviceAccount = env.SERVICE_ACCOUNT ?: "default"
def sonarCredsID = "sonarID"
appname = appname.toLowerCase()

// Pod Environment Variables
def kubenamespace = "default"
def regnamespace = "blw-msa"
def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net"podTemplate(label: 'mypod', cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [
        envVar(key: 'NAMESPACE', value: kubenamespace),
        envVar(key: 'REGNAMESPACE', value: regnamespace),
	envVar(key: 'JARNAME', value: jarname),
        envVar(key: 'APPNAME', value: appname),
        envVar(key: 'REGISTRY', value: registry)
    ],
    volumes: [
    //hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
    //hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')
    //hostPathVolume(hostPath: '/tmp/gradle', mountPath: '/root/.gradle/caches/')
    ],
    containers: [
        containerTemplate(name: 'java', image: 'openjdk:8-jdk', ttyEnabled: true, command: 'cat')
  ]) {    node('mypod') {
        stage('Checkout') {
		echo "test"
		}
        container('java') {
		stage('Static Code Analysis') {
                    		sh "java -version"}
                }
}
}

This unfortunately has not helped, the issue is still the same (Providing Console Output of above Jenkinsfile) :

Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Still waiting to schedule task
Waiting for next available executor
Agent jenkins-slave-brxph-v8fff is provisioned from template Kubernetes Pod Template
Agent specification [Kubernetes Pod Template] (mypod): 
* [java] openjdk:8-jdkRunning on jenkins-slave-brxph-v8fff in /home/jenkins/workspace/testingSimple
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Checkout)
[Pipeline] echo
test
[Pipeline] }
[Pipeline] // stage
[Pipeline] container
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Static Code Analysis)[Pipeline] sh
[testingSimple] Running shell script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep(Native Method)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219)
	at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:420)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out (299 seconds) waiting for container to become ready!
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:447)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
	at WorkflowScript.run(WorkflowScript:44)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
	at sun.reflect.GeneratedMethodAccessor283.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

If you like, and can also provide you with credentials of my public jenkins testing instance, its running on a free ibm cluster for the next 20 days.

leon gregori added a comment - 2018-10-19 07:00 - edited csanchez I tested the JENKINS-53297 plugin Manage Plugins: Kubernetes plugin This plugin integrates Jenkins with Kubernetes 1.12.10-SNAPSHOT (private-d0dc3394-blwdev) I tested your Commit with the following Jenkinsfile: // Simple Jenkinsfile for testing Kubernetes Plugin // Developers input is required here. def appname = "test" def jarname = "test.jar" // Jenkins Deployment Variables and Credentials def cloud = env.CLOUD ?: "kubernetes" def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id" def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id" def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id" def serviceAccount = env.SERVICE_ACCOUNT ?: " default " def sonarCredsID = "sonarID" appname = appname.toLowerCase() // Pod Environment Variables def kubenamespace = " default " def regnamespace = "blw-msa" def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net" podTemplate(label: 'mypod' , cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [ envVar(key: 'NAMESPACE' , value: kubenamespace), envVar(key: 'REGNAMESPACE' , value: regnamespace), envVar(key: 'JARNAME' , value: jarname), envVar(key: 'APPNAME' , value: appname), envVar(key: 'REGISTRY' , value: registry) ], volumes: [ //hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), //hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' ) //hostPathVolume(hostPath: '/tmp/gradle' , mountPath: '/root/.gradle/caches/' ) ], containers: [ containerTemplate(name: 'java' , image: 'openjdk:8-jdk' , ttyEnabled: true , command: 'cat' ) ]) { node( 'mypod' ) { stage( 'Checkout' ) { echo "test" } container( 'java' ) { stage( 'Static Code Analysis' ) { sh "java -version" } } } } This unfortunately has not helped, the issue is still the same (Providing Console Output of above Jenkinsfile) : Started by user admin Running in Durability level: MAX_SURVIVABILITY [Pipeline] podTemplate [Pipeline] { [Pipeline] node Still waiting to schedule task Waiting for next available executor Agent jenkins-slave-brxph-v8fff is provisioned from template Kubernetes Pod Template Agent specification [Kubernetes Pod Template] (mypod): * [java] openjdk:8-jdkRunning on jenkins-slave-brxph-v8fff in /home/jenkins/workspace/testingSimple [Pipeline] { [Pipeline] stage [Pipeline] { (Checkout) [Pipeline] echo test [Pipeline] } [Pipeline] // stage [Pipeline] container [Pipeline] { [Pipeline] stage [Pipeline] { (Static Code Analysis)[Pipeline] sh [testingSimple] Running shell script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // container [Pipeline] } [Pipeline] // node [Pipeline] } [Pipeline] // podTemplate [Pipeline] End of Pipeline java.lang.InterruptedException: sleep interrupted at java.lang. Thread .sleep(Native Method) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilExists(BaseOperation.java:959) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:219) at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.waitUntilReady(HasMetadataOperation.java:37) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:420) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) Caused: java.io.IOException: Failed to execute shell script inside container [java] of pod [localhost]. Timed out (299 seconds) waiting for container to become ready! at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:447) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:44) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor283.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE If you like, and can also provide you with credentials of my public jenkins testing instance, its running on a free ibm cluster for the next 20 days.

Carlos Sanchez added a comment - 2018-10-19 09:48

The plugin gets the pod name from the /etc/hostname file and looks like IKS just sets hostname to localhost instead of the pod name

https://github.com/jenkinsci/kubernetes-plugin/blob/2e10cd29fa7bd1bca86e3716f04af61af818639e/src/main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesNodeContext.java#L42

Only solution I can think of is the use of the downward API

https://stackoverflow.com/questions/37253068/programmatically-get-the-name-of-the-pod-that-a-container-belongs-to-in-kubernet/37261036

Please try with the latest commit in the branch and adding this to your pod definition

      env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name

Carlos Sanchez added a comment - 2018-10-19 09:48 The plugin gets the pod name from the /etc/hostname file and looks like IKS just sets hostname to localhost instead of the pod name https://github.com/jenkinsci/kubernetes-plugin/blob/2e10cd29fa7bd1bca86e3716f04af61af818639e/src/main/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesNodeContext.java#L42 Only solution I can think of is the use of the downward API https://stackoverflow.com/questions/37253068/programmatically-get-the-name-of-the-pod-that-a-container-belongs-to-in-kubernet/37261036 Please try with the latest commit in the branch and adding this to your pod definition env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name

leon gregori added a comment - 2018-10-19 11:43 - edited

Im currently fighting with the pod template definition. The variable is available in the pod but empty / not defined, need some more try and error until i make it i guess.

// Simple Jenkinsfile for testing Kubernetes Plugin
// Developers input is required here.
def appname = "test"
def jarname = "test.jar"// Jenkins Deployment Variables and Credentials
def cloud = env.CLOUD ?: "kubernetes"
def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id"
def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id"
def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id"
def serviceAccount = env.SERVICE_ACCOUNT ?: "default"
def sonarCredsID = "sonarID"
appname = appname.toLowerCase()// Pod Environment Variables
def kubenamespace = "default"
def regnamespace = "test"
def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net"podTemplate(label: 'mypod', cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [
        envVar(key: 'NAMESPACE', value: kubenamespace),
        envVar(key: 'REGNAMESPACE', value: regnamespace),
	envVar(key: 'JARNAME', value: jarname),
        envVar(key: 'APPNAME', value: appname),
        envVar(key: 'POD_NAME', valuefrom: { fieldref: { fieldPath: metadata.name }}),
        envVar(key: 'REGISTRY', value: registry)
    ],
    volumes: [
    //hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
    //hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')
	//hostPathVolume(hostPath: '/tmp/gradle', mountPath: '/root/.gradle/caches/')
    ],
    containers: [
        // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle.
	// But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle
        containerTemplate(name: 'java', image: 'openjdk:8-jdk', ttyEnabled: true, command: 'cat')
  ]) {    node('mypod') {
        stage('Checkout') {
		echo "test"
		}
        container('java') {
		stage('Static Code Analysis') {
                    		sh "java -version"}
                }
}
}

$ kubectl get pod jenkins-slave-hcwsg-c9ntw -o yaml
apiVersion: v1
kind: Pod
metadata:
 annotations:
 kubernetes.io/psp: ibm-privileged-psp
 creationTimestamp: 2018-10-19T11:11:45Z
 deletionGracePeriodSeconds: 30
 deletionTimestamp: 2018-10-19T11:12:28Z
 labels:
 jenkins: slave
 jenkins/mypod: "true"
 name: jenkins-slave-hcwsg-c9ntw
 namespace: default
 resourceVersion: "20068"
 selfLink: /api/v1/namespaces/default/pods/jenkins-slave-hcwsg-c9ntw
 uid: c3ad40dc-d38f-11e8-a534-0623e2fdc71b
spec:
 containers:
 - command:
 - cat
 env:
 - name: NAMESPACE
 value: default
 - name: JENKINS_SECRET
 value: -
 - name: JENKINS_TUNNEL
 value: my-release-jenkins-agent:50000
 - name: JENKINS_AGENT_NAME
 value: jenkins-slave-hcwsg-c9ntw
 - name: REGNAMESPACE
 value: blw-msa
 - name: JARNAME
 value: AddressManagement.jar
 - name: POD_NAME
 - name: APPNAME
 value: addressmanagement
 - name: JENKINS_NAME
 value: jenkins-slave-hcwsg-c9ntw
 - name: JENKINS_URL
 value: http://my-release-jenkins:8080/
 - name: HOME
 value: /home/jenkins
 - name: REGISTRY
 value: registry.eu-de.bluemix.net
 image: openjdk:8-jdk
 imagePullPolicy: IfNotPresent
 name: java
 resources: {}
 securityContext:
 privileged: false
 terminationMessagePath: /dev/termination-log
 terminationMessagePolicy: File
 tty: true
 volumeMounts:
 - mountPath: /home/jenkins
 name: workspace-volume
 - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
 name: default-token-9hntr
 readOnly: true
 workingDir: /home/jenkins
 - env:
 - name: NAMESPACE
 value: default
 - name: JENKINS_SECRET
 value: -
 - name: JENKINS_TUNNEL
 value: my-release-jenkins-agent:50000
 - name: JENKINS_AGENT_NAME
 value: jenkins-slave-hcwsg-c9ntw
 - name: REGNAMESPACE
 value: blw-msa
 - name: JARNAME
 value: AddressManagement.jar
 - name: POD_NAME
 - name: APPNAME
 value: addressmanagement
 - name: JENKINS_NAME
 value: jenkins-slave-hcwsg-c9ntw
 - name: JENKINS_URL
 value: http://my-release-jenkins:8080/
 - name: HOME
 value: /home/jenkins
 - name: REGISTRY
 value: registry.eu-de.bluemix.net
 image: jenkins/jnlp-slave:alpine
 imagePullPolicy: IfNotPresent
 name: jnlp
 resources: {}
 terminationMessagePath: /dev/termination-log
 terminationMessagePolicy: File
 volumeMounts:
 - mountPath: /home/jenkins
 name: workspace-volume
 - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
 name: default-token-9hntr
 readOnly: true
 dnsPolicy: ClusterFirst
 imagePullSecrets:
 - name: bluemix-default-secret
 - name: bluemix-default-secret-regional
 - name: bluemix-default-secret-international
 nodeName: 10.144.180.238
 priority: 0
 restartPolicy: Never
 schedulerName: default-scheduler
 securityContext: {}
 serviceAccount: default
 serviceAccountName: default
 terminationGracePeriodSeconds: 30
 tolerations:
 - effect: NoExecute
 key: node.kubernetes.io/not-ready
 operator: Exists
 tolerationSeconds: 300
 - effect: NoExecute
 key: node.kubernetes.io/unreachable
 operator: Exists
 tolerationSeconds: 300
 volumes:
 - emptyDir: {}
 name: workspace-volume
 - name: default-token-9hntr
 secret:
 defaultMode: 420
 secretName: default-token-9hntr
status:
 conditions:
 - lastProbeTime: null
 lastTransitionTime: 2018-10-19T11:11:45Z
 status: "True"
 type: Initialized
 - lastProbeTime: null
 lastTransitionTime: 2018-10-19T11:11:47Z
 status: "True"
 type: Ready
 - lastProbeTime: null
 lastTransitionTime: null
 status: "True"
 type: ContainersReady
 - lastProbeTime: null
 lastTransitionTime: 2018-10-19T11:11:45Z
 status: "True"
 type: PodScheduled
 containerStatuses:
 - containerID: containerd://e8ce755b312e1d14201905eae9d09acd5b21e214f4df08a4959d6576a1f140ff
 image: docker.io/library/openjdk:8-jdk
 imageID: docker.io/library/openjdk@sha256:dfc720be416bdc7b5df39fb8b237a3bca88dcdf39ad00568f8c3cd3558ff1bc2
 lastState: {}
 name: java
 ready: true
 restartCount: 0
 state:
 running:
 startedAt: 2018-10-19T11:11:46Z
 - containerID: containerd://a5694b45c0a9bb88e54e24d66a37440a1fbe695cb8236187c1f37c92b9f3b416
 image: docker.io/jenkins/jnlp-slave:alpine
 imageID: docker.io/jenkins/jnlp-slave@sha256:7a826ce43cb7eeb59b0dc5000f89e00060ba31e51524d87ea811ba23453185ed
 lastState: {}
 name: jnlp
 ready: true
 restartCount: 0
 state:
 running:
 startedAt: 2018-10-19T11:11:46Z
 hostIP: 10.144.180.238
 phase: Running
 podIP: 172.30.202.87
 qosClass: BestEffort
 startTime: 2018-10-19T11:11:45Z

Job Console Output

java.lang.IllegalArgumentException: Name must be provided.
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:234)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:49)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:419)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255)
	at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236)
	at hudson.Launcher$ProcStarter.start(Launcher.java:454)
	at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185)
	at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98)
	at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
	at sun.reflect.GeneratedMethodAccessor262.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
	at WorkflowScript.run(WorkflowScript:45)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
	at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

leon gregori added a comment - 2018-10-19 11:43 - edited Im currently fighting with the pod template definition. The variable is available in the pod but empty / not defined, need some more try and error until i make it i guess. // Simple Jenkinsfile for testing Kubernetes Plugin // Developers input is required here. def appname = "test" def jarname = "test.jar" // Jenkins Deployment Variables and Credentials def cloud = env.CLOUD ?: "kubernetes" def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id" def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id" def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id" def serviceAccount = env.SERVICE_ACCOUNT ?: " default " def sonarCredsID = "sonarID" appname = appname.toLowerCase() // Pod Environment Variables def kubenamespace = " default " def regnamespace = "test" def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net" podTemplate(label: 'mypod' , cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [ envVar(key: 'NAMESPACE' , value: kubenamespace), envVar(key: 'REGNAMESPACE' , value: regnamespace), envVar(key: 'JARNAME' , value: jarname), envVar(key: 'APPNAME' , value: appname), envVar(key: 'POD_NAME' , valuefrom: { fieldref: { fieldPath: metadata.name }}), envVar(key: 'REGISTRY' , value: registry) ], volumes: [ //hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), //hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' ) //hostPathVolume(hostPath: '/tmp/gradle' , mountPath: '/root/.gradle/caches/' ) ], containers: [ // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle. // But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle containerTemplate(name: 'java' , image: 'openjdk:8-jdk' , ttyEnabled: true , command: 'cat' ) ]) { node( 'mypod' ) { stage( 'Checkout' ) { echo "test" } container( 'java' ) { stage( 'Static Code Analysis' ) { sh "java -version" } } } } $ kubectl get pod jenkins-slave-hcwsg-c9ntw -o yaml apiVersion: v1 kind: Pod metadata: annotations: kubernetes.io/psp: ibm-privileged-psp creationTimestamp: 2018-10-19T11:11:45Z deletionGracePeriodSeconds: 30 deletionTimestamp: 2018-10-19T11:12:28Z labels: jenkins: slave jenkins/mypod: " true " name: jenkins-slave-hcwsg-c9ntw namespace: default resourceVersion: "20068" selfLink: /api/v1/namespaces/ default /pods/jenkins-slave-hcwsg-c9ntw uid: c3ad40dc-d38f-11e8-a534-0623e2fdc71b spec: containers: - command: - cat env: - name: NAMESPACE value: default - name: JENKINS_SECRET value: - - name: JENKINS_TUNNEL value: my-release-jenkins-agent:50000 - name: JENKINS_AGENT_NAME value: jenkins-slave-hcwsg-c9ntw - name: REGNAMESPACE value: blw-msa - name: JARNAME value: AddressManagement.jar - name: POD_NAME - name: APPNAME value: addressmanagement - name: JENKINS_NAME value: jenkins-slave-hcwsg-c9ntw - name: JENKINS_URL value: http: //my-release-jenkins:8080/ - name: HOME value: /home/jenkins - name: REGISTRY value: registry.eu-de.bluemix.net image: openjdk:8-jdk imagePullPolicy: IfNotPresent name: java resources: {} securityContext: privileged: false terminationMessagePath: /dev/termination-log terminationMessagePolicy: File tty: true volumeMounts: - mountPath: /home/jenkins name: workspace-volume - mountPath: / var /run/secrets/kubernetes.io/serviceaccount name: default -token-9hntr readOnly: true workingDir: /home/jenkins - env: - name: NAMESPACE value: default - name: JENKINS_SECRET value: - - name: JENKINS_TUNNEL value: my-release-jenkins-agent:50000 - name: JENKINS_AGENT_NAME value: jenkins-slave-hcwsg-c9ntw - name: REGNAMESPACE value: blw-msa - name: JARNAME value: AddressManagement.jar - name: POD_NAME - name: APPNAME value: addressmanagement - name: JENKINS_NAME value: jenkins-slave-hcwsg-c9ntw - name: JENKINS_URL value: http: //my-release-jenkins:8080/ - name: HOME value: /home/jenkins - name: REGISTRY value: registry.eu-de.bluemix.net image: jenkins/jnlp-slave:alpine imagePullPolicy: IfNotPresent name: jnlp resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /home/jenkins name: workspace-volume - mountPath: / var /run/secrets/kubernetes.io/serviceaccount name: default -token-9hntr readOnly: true dnsPolicy: ClusterFirst imagePullSecrets: - name: bluemix- default -secret - name: bluemix- default -secret-regional - name: bluemix- default -secret-international nodeName: 10.144.180.238 priority: 0 restartPolicy: Never schedulerName: default -scheduler securityContext: {} serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations: - effect: NoExecute key: node.kubernetes.io/not-ready operator : Exists tolerationSeconds: 300 - effect: NoExecute key: node.kubernetes.io/unreachable operator : Exists tolerationSeconds: 300 volumes: - emptyDir: {} name: workspace-volume - name: default -token-9hntr secret: defaultMode: 420 secretName: default -token-9hntr status: conditions: - lastProbeTime: null lastTransitionTime: 2018-10-19T11:11:45Z status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: 2018-10-19T11:11:47Z status: "True" type: Ready - lastProbeTime: null lastTransitionTime: null status: "True" type: ContainersReady - lastProbeTime: null lastTransitionTime: 2018-10-19T11:11:45Z status: "True" type: PodScheduled containerStatuses: - containerID: containerd: //e8ce755b312e1d14201905eae9d09acd5b21e214f4df08a4959d6576a1f140ff image: docker.io/library/openjdk:8-jdk imageID: docker.io/library/openjdk@sha256:dfc720be416bdc7b5df39fb8b237a3bca88dcdf39ad00568f8c3cd3558ff1bc2 lastState: {} name: java ready: true restartCount: 0 state: running: startedAt: 2018-10-19T11:11:46Z - containerID: containerd: //a5694b45c0a9bb88e54e24d66a37440a1fbe695cb8236187c1f37c92b9f3b416 image: docker.io/jenkins/jnlp-slave:alpine imageID: docker.io/jenkins/jnlp-slave@sha256:7a826ce43cb7eeb59b0dc5000f89e00060ba31e51524d87ea811ba23453185ed lastState: {} name: jnlp ready: true restartCount: 0 state: running: startedAt: 2018-10-19T11:11:46Z hostIP: 10.144.180.238 phase: Running podIP: 172.30.202.87 qosClass: BestEffort startTime: 2018-10-19T11:11:45Z Job Console Output java.lang.IllegalArgumentException: Name must be provided. at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:234) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.withName(BaseOperation.java:49) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.waitUntilContainerIsReady(ContainerExecDecorator.java:419) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.doLaunch(ContainerExecDecorator.java:255) at org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1.launch(ContainerExecDecorator.java:236) at hudson.Launcher$ProcStarter.start(Launcher.java:454) at org.jenkinsci.plugins.durabletask.BourneShellScript.launchWithCookie(BourneShellScript.java:185) at org.jenkinsci.plugins.durabletask.FileMonitoringTask.launch(FileMonitoringTask.java:98) at org.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep$Execution.start(DurableTaskStep.java:264) at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:270) at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:178) at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122) at sun.reflect.GeneratedMethodAccessor262.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157) at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:155) at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17) at WorkflowScript.run(WorkflowScript:45) at ___cps.transform___(Native Method) at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109) at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82) at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72) at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21) at com.cloudbees.groovy.cps.Next.step(Next.java:83) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174) at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163) at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122) at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261) at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108) at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:182) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Finished: FAILURE

Carlos Sanchez added a comment - 2018-10-19 11:45

you would need to use the yaml syntax, I don't think valuefrom will work

Carlos Sanchez added a comment - 2018-10-19 11:45 you would need to use the yaml syntax, I don't think valuefrom will work

leon gregori added a comment - 2018-10-22 07:24 - edited

csanchez I used the yaml syntax now for jnlp container, but the yaml is not used properly:

leon gregori added a comment - 2018-10-22 07:24 - edited csanchez I used the yaml syntax now for jnlp container, but the yaml is not used properly:

Carlos Sanchez added a comment - 2018-10-22 07:34

don't name the container jenkins-slave, name it jnlp

Carlos Sanchez added a comment - 2018-10-22 07:34 don't name the container jenkins-slave , name it jnlp

leon gregori added a comment - 2018-10-22 07:47

Okay, i named it jnlp, only 1 container is created now. But im still missing the variable pod_name

$ kubectl describe pod test-2jrwj
Name:               test-2jrwj
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               10.144.180.238/10.144.180.238
Start Time:         Mon, 22 Oct 2018 09:44:34 +0200
Labels:             jenkins=slave
                    jenkins/test=true
Annotations:        kubernetes.io/psp=ibm-privileged-psp
Status:             Running
IP:                 172.30.202.66
Containers:
  jnlp:
    Container ID:  containerd://60f5b2ad8849852ac09e5afb018d6b625a25710e9618b4d20910387eb763d7c7
    Image:         jenkinsci/jnlp-slave
    Image ID:      docker.io/jenkinsci/jnlp-slave@sha256:1b30012786011b62417a9ee9b612280c62bb9deac0ffd939a8dd31b6633cc0aa
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c
    Args:
      cat
    State:          Running
      Started:      Mon, 22 Oct 2018 09:44:36 +0200
    Ready:          True
    Restart Count:  0
    Environment:
      JENKINS_SECRET:      46c5400d9757f360eeb132758bee5ea444954d558bb8543cd715e09c21a180c9
      JENKINS_TUNNEL:      my-release-jenkins-agent:50000
      JENKINS_AGENT_NAME:  test-2jrwj
      JENKINS_NAME:        test-2jrwj
      JENKINS_URL:         http://my-release-jenkins:8080/
      HOME:                /home/jenkins
    Mounts:
      /home/jenkins from workspace-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-9hntr (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  workspace-volume:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  default-token-9hntr:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-9hntr
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From                     Message
  ----    ------     ----  ----                     -------
  Normal  Scheduled  22s   default-scheduler        Successfully assigned default/test-2jrwj to 10.144.180.238
  Normal  Pulled     21s   kubelet, 10.144.180.238  Container image "jenkinsci/jnlp-slave" already present on machine
  Normal  Created    21s   kubelet, 10.144.180.238  Created container
  Normal  Started    20s   kubelet, 10.144.180.238  Started container

Definition:

leon gregori added a comment - 2018-10-22 07:47 Okay, i named it jnlp, only 1 container is created now. But im still missing the variable pod_name $ kubectl describe pod test-2jrwj Name: test-2jrwj Namespace: default Priority: 0 PriorityClassName: <none> Node: 10.144.180.238/10.144.180.238 Start Time: Mon, 22 Oct 2018 09:44:34 +0200 Labels: jenkins=slave jenkins/test= true Annotations: kubernetes.io/psp=ibm-privileged-psp Status: Running IP: 172.30.202.66 Containers: jnlp: Container ID: containerd: //60f5b2ad8849852ac09e5afb018d6b625a25710e9618b4d20910387eb763d7c7 Image: jenkinsci/jnlp-slave Image ID: docker.io/jenkinsci/jnlp-slave@sha256:1b30012786011b62417a9ee9b612280c62bb9deac0ffd939a8dd31b6633cc0aa Port: <none> Host Port: <none> Command: /bin/sh -c Args: cat State: Running Started: Mon, 22 Oct 2018 09:44:36 +0200 Ready: True Restart Count: 0 Environment: JENKINS_SECRET: 46c5400d9757f360eeb132758bee5ea444954d558bb8543cd715e09c21a180c9 JENKINS_TUNNEL: my-release-jenkins-agent:50000 JENKINS_AGENT_NAME: test-2jrwj JENKINS_NAME: test-2jrwj JENKINS_URL: http: //my-release-jenkins:8080/ HOME: /home/jenkins Mounts: /home/jenkins from workspace-volume (rw) / var /run/secrets/kubernetes.io/serviceaccount from default -token-9hntr (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: workspace-volume: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: default -token-9hntr: Type: Secret (a volume populated by a Secret) SecretName: default -token-9hntr Optional: false QoS Class : BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 22s default -scheduler Successfully assigned default /test-2jrwj to 10.144.180.238 Normal Pulled 21s kubelet, 10.144.180.238 Container image "jenkinsci/jnlp-slave" already present on machine Normal Created 21s kubelet, 10.144.180.238 Created container Normal Started 20s kubelet, 10.144.180.238 Started container Definition:

Carlos Sanchez added a comment - 2018-10-22 07:53

do you have more pod templates defined and is picking a different one ?

Carlos Sanchez added a comment - 2018-10-22 07:53 do you have more pod templates defined and is picking a different one ?

leon gregori added a comment - 2018-10-22 08:05 - edited

No its only this one. I will try to do some try and error testing to get the variable somehow into the pod. Will update as soon as i have news

leon gregori added a comment - 2018-10-22 08:05 - edited No its only this one. I will try to do some try and error testing to get the variable somehow into the pod. Will update as soon as i have news

Carlos Sanchez added a comment - 2018-10-23 07:56

see https://github.com/jenkinsci/kubernetes-plugin/pull/388 for a proper fix

Carlos Sanchez added a comment - 2018-10-23 07:56 see https://github.com/jenkinsci/kubernetes-plugin/pull/388 for a proper fix

leon gregori added a comment - 2018-10-23 09:03

csanchez Thanks for this information. I will try it as soon as the pull builds successfull.
So with this fix in place, pod template out of pipeline can still be used?

leon gregori added a comment - 2018-10-23 09:03 csanchez Thanks for this information. I will try it as soon as the pull builds successfull. So with this fix in place, pod template out of pipeline can still be used?

Carlos Sanchez added a comment - 2018-10-23 09:12

yes, that fix should work in all cases

Carlos Sanchez added a comment - 2018-10-23 09:12 yes, that fix should work in all cases

leon gregori added a comment - 2018-10-23 15:50 - edited

I have build the pull request, it works!
finally able to execute commands in a container with a simple jenkinsfile like:

// Developers input is required here.
def appname = "test"
def jarname = "test.jar"// Jenkins Deployment Variables and Credentials
def cloud = env.CLOUD ?: "kubernetes"
def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id"
def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id"
def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id"
def serviceAccount = env.SERVICE_ACCOUNT ?: "default"
def sonarCredsID = "sonarID"
appname = appname.toLowerCase()

// Pod Environment Variables
def kubenamespace = "default"
def regnamespace = "blw-msa"
def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net"podTemplate(label: 'mypod', cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [
        envVar(key: 'NAMESPACE', value: kubenamespace),
        envVar(key: 'REGNAMESPACE', value: regnamespace),
	envVar(key: 'JARNAME', value: jarname),
        envVar(key: 'APPNAME', value: appname),
        envVar(key: 'REGISTRY', value: registry)
    ],
    volumes: [
    //hostPathVolume(hostPath: '/etc/docker/certs.d', mountPath: '/etc/docker/certs.d'),
    //hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')
	//hostPathVolume(hostPath: '/tmp/gradle', mountPath: '/root/.gradle/caches/')
    ],
    containers: [
        // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle.
	// But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle
        containerTemplate(name: 'java', image: 'openjdk:8-jdk', ttyEnabled: true, command: 'cat')
  ]) {    node('mypod') {
        stage('Checkout') {
		echo "test"
		}
        container('java') {
		stage('Static Code Analysis') {
                    		sh "java -version"}
                }
}
}

leon gregori added a comment - 2018-10-23 15:50 - edited I have build the pull request, it works! finally able to execute commands in a container with a simple jenkinsfile like: // Developers input is required here. def appname = "test" def jarname = "test.jar" // Jenkins Deployment Variables and Credentials def cloud = env.CLOUD ?: "kubernetes" def registryCredsID = env.REGISTRY_CREDENTIALS ?: "registry-credentials-id" def camundaRepoID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-repo-id" def camundaLicenseID = env.CAMUNDA_REPO_CREDENTIALS ?: "camunda-lic-id" def serviceAccount = env.SERVICE_ACCOUNT ?: " default " def sonarCredsID = "sonarID" appname = appname.toLowerCase() // Pod Environment Variables def kubenamespace = " default " def regnamespace = "blw-msa" def registry = env.REGISTRY ?: "registry.eu-de.bluemix.net" podTemplate(label: 'mypod' , cloud: cloud, serviceAccount: serviceAccount, kubenamespace: kubenamespace, envVars: [ envVar(key: 'NAMESPACE' , value: kubenamespace), envVar(key: 'REGNAMESPACE' , value: regnamespace), envVar(key: 'JARNAME' , value: jarname), envVar(key: 'APPNAME' , value: appname), envVar(key: 'REGISTRY' , value: registry) ], volumes: [ //hostPathVolume(hostPath: '/etc/docker/certs.d' , mountPath: '/etc/docker/certs.d' ), //hostPathVolume(hostPath: '/ var /run/docker.sock' , mountPath: '/ var /run/docker.sock' ) //hostPathVolume(hostPath: '/tmp/gradle' , mountPath: '/root/.gradle/caches/' ) ], containers: [ // In Theory, java container is not needed as the normal jenkins jnlp slave would be capable of executing gradle. // But as we mount the cache from host, we need a root user running inside the build container, otherwise gradle wont be able to write to .gradle containerTemplate(name: 'java' , image: 'openjdk:8-jdk' , ttyEnabled: true , command: 'cat' ) ]) { node( 'mypod' ) { stage( 'Checkout' ) { echo "test" } container( 'java' ) { stage( 'Static Code Analysis' ) { sh "java -version" } } } }

leon gregori added a comment - 2018-10-24 08:12 - edited

I can confirm that this issue is fixed now with PR 388
Our Shared library fully able to run!

Special Thanks to csanchez vlatombe

leon gregori added a comment - 2018-10-24 08:12 - edited I can confirm that this issue is fixed now with PR 388 Our Shared library fully able to run! Special Thanks to csanchez vlatombe

Assignee:: Vincent Latombe

Reporter:: Vinicius Xavier

Votes:: 2 Vote for this issue

Watchers:: 7 Start watching this issue

Created:: 2018-08-29 00:54

Updated:: 2018-12-07 08:12

Resolved:: 2018-10-24 13:01

Jenkins

Details

Description

Attachments

Attachments

Issue Links

Activity

Collapse comment: Vinicius Xavier added a comment - 2018-08-29 18:39

Expand comment: Vinicius Xavier added a comment - 2018-08-29 18:39

Collapse comment: Carlos Sanchez added a comment - 2018-08-29 19:14

Expand comment: Carlos Sanchez added a comment - 2018-08-29 19:14

Collapse comment: Vinicius Xavier added a comment - 2018-08-29 19:31

Expand comment: Vinicius Xavier added a comment - 2018-08-29 19:31

Collapse comment: Vinicius Xavier added a comment - 2018-08-29 19:42

Expand comment: Vinicius Xavier added a comment - 2018-08-29 19:42

Collapse comment: leon gregori added a comment - 2018-09-10 15:52, Edited by leon gregori - 2018-09-10 15:53

Expand comment: leon gregori added a comment - 2018-09-10 15:52, Edited by leon gregori - 2018-09-10 15:53

Collapse comment: Alok Rajiv added a comment - 2018-09-18 09:03, Edited by Alok Rajiv - 2018-09-18 09:05

Expand comment: Alok Rajiv added a comment - 2018-09-18 09:03, Edited by Alok Rajiv - 2018-09-18 09:05

Collapse comment: leon gregori added a comment - 2018-09-18 14:19

Expand comment: leon gregori added a comment - 2018-09-18 14:19

Collapse comment: leon gregori added a comment - 2018-10-10 11:48

Expand comment: leon gregori added a comment - 2018-10-10 11:48

Collapse comment: David Currie added a comment - 2018-10-10 15:41

Expand comment: David Currie added a comment - 2018-10-10 15:41

Collapse comment: Alok Rajiv added a comment - 2018-10-10 15:56

Expand comment: Alok Rajiv added a comment - 2018-10-10 15:56

Collapse comment: David Currie added a comment - 2018-10-10 16:24, Edited by David Currie - 2018-10-10 16:24

Expand comment: David Currie added a comment - 2018-10-10 16:24, Edited by David Currie - 2018-10-10 16:24

Collapse comment: Alok Rajiv added a comment - 2018-10-10 16:26

Expand comment: Alok Rajiv added a comment - 2018-10-10 16:26

Collapse comment: leon gregori added a comment - 2018-10-11 11:19

Expand comment: leon gregori added a comment - 2018-10-11 11:19

Collapse comment: David Currie added a comment - 2018-10-11 11:29

Expand comment: David Currie added a comment - 2018-10-11 11:29

Collapse comment: leon gregori added a comment - 2018-10-17 06:35, Edited by leon gregori - 2018-10-18 15:32

Expand comment: leon gregori added a comment - 2018-10-17 06:35, Edited by leon gregori - 2018-10-18 15:32

Collapse comment: leon gregori added a comment - 2018-10-18 10:47, Edited by leon gregori - 2018-10-18 15:32

Expand comment: leon gregori added a comment - 2018-10-18 10:47, Edited by leon gregori - 2018-10-18 15:32

Collapse comment: Carlos Sanchez added a comment - 2018-10-18 17:01

Expand comment: Carlos Sanchez added a comment - 2018-10-18 17:01

Collapse comment: leon gregori added a comment - 2018-10-18 19:59, Edited by leon gregori - 2018-10-18 20:06

Expand comment: leon gregori added a comment - 2018-10-18 19:59, Edited by leon gregori - 2018-10-18 20:06

Collapse comment: leon gregori added a comment - 2018-10-19 07:00, Edited by leon gregori - 2018-12-07 08:12

Expand comment: leon gregori added a comment - 2018-10-19 07:00, Edited by leon gregori - 2018-12-07 08:12

Collapse comment: Carlos Sanchez added a comment - 2018-10-19 09:48

Expand comment: Carlos Sanchez added a comment - 2018-10-19 09:48

Collapse comment: leon gregori added a comment - 2018-10-19 11:43, Edited by leon gregori - 2018-12-07 08:12

Expand comment: leon gregori added a comment - 2018-10-19 11:43, Edited by leon gregori - 2018-12-07 08:12

Collapse comment: Carlos Sanchez added a comment - 2018-10-19 11:45

Expand comment: Carlos Sanchez added a comment - 2018-10-19 11:45

Collapse comment: leon gregori added a comment - 2018-10-22 07:24, Edited by leon gregori - 2018-10-22 07:40

Expand comment: leon gregori added a comment - 2018-10-22 07:24, Edited by leon gregori - 2018-10-22 07:40

Collapse comment: Carlos Sanchez added a comment - 2018-10-22 07:34

Expand comment: Carlos Sanchez added a comment - 2018-10-22 07:34

Collapse comment: leon gregori added a comment - 2018-10-22 07:47

Expand comment: leon gregori added a comment - 2018-10-22 07:47

Collapse comment: Carlos Sanchez added a comment - 2018-10-22 07:53

Expand comment: Carlos Sanchez added a comment - 2018-10-22 07:53

Collapse comment: leon gregori added a comment - 2018-10-22 08:05, Edited by leon gregori - 2018-10-22 08:05

Expand comment: leon gregori added a comment - 2018-10-22 08:05, Edited by leon gregori - 2018-10-22 08:05

Collapse comment: Carlos Sanchez added a comment - 2018-10-23 07:56

Expand comment: Carlos Sanchez added a comment - 2018-10-23 07:56

Collapse comment: leon gregori added a comment - 2018-10-23 09:03

Expand comment: leon gregori added a comment - 2018-10-23 09:03

Collapse comment: Carlos Sanchez added a comment - 2018-10-23 09:12

Expand comment: Carlos Sanchez added a comment - 2018-10-23 09:12

Collapse comment: leon gregori added a comment - 2018-10-23 15:50, Edited by leon gregori - 2018-12-07 08:10

Expand comment: leon gregori added a comment - 2018-10-23 15:50, Edited by leon gregori - 2018-12-07 08:10

Collapse comment: leon gregori added a comment - 2018-10-24 08:12, Edited by leon gregori - 2018-10-24 08:12

Expand comment: leon gregori added a comment - 2018-10-24 08:12, Edited by leon gregori - 2018-10-24 08:12

People

Dates