Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40745

Kubernetes plugin mkdirs error when configuring a persistent volume claim for /home/jenkins on slave

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Major Major
    • kubernetes-plugin
    • kubernetes 1.4.4
      jenkins 2.32.1
      kubernetes-plugin 0.10
      slave image jenkinsci/jnlp-slave

      I'm attempting to create a persistent volume claim at /home/jenkins for my build container so that I can get the working directory off of the kubernetes node. I'm running into kubernetes eviction errors due to the size of the container / source code that is being checked out on the node.

      However, I'm running into a permissions error it looks like when hitting the checkout step. Any thoughts on what to try here?

      java.io.IOException: Failed to mkdirs: /home/jenkins/workspace/vendasta_AA_develop-IEOQWSXH4FZN4YOLA4GNTUPEHK5V6WLUSFSBDOOPY2CKPWC5MHVQ
      	at hudson.FilePath.mkdirs(FilePath.java:1169)
      	at hudson.plugins.git.GitSCM.createClient(GitSCM.java:736)
      	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1088)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:83)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:73)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:221)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Finished: FAILURE
      

      kubernetes GCE persistent volume claim:

      apiVersion: "v1"
      kind: "List"
      items:
        - apiVersion: "v1"
          kind: "PersistentVolume"
          metadata:
            name: "jenkins-slaves"
          spec:
            accessModes:
              - ReadWriteMany
            capacity:
              storage: 200Gi
            gcePersistentDisk:
              pdName: jenkins-slaves
              fsType: ext4
      
        - apiVersion: "v1"
          kind: "PersistentVolumeClaim"
          metadata:
            name: "jenkins-slaves"
            namespace: "jenkins"
          spec:
            accessModes:
              - ReadWriteMany
            resources:
              requests:
                storage: 1Gi
      
      

      jenkinsfile

      #!groovy
      
      podTemplate(label: 'build', inheritFrom: 'default', containers: [
          containerTemplate(
              name: 'build-dependencies',
              image: 'myorg/build-dependencies',
              ttyEnabled: true,
              command: 'cat',
              alwaysPullImage: true,
              workingDir: '/home/jenkins',
              resourceRequestCpu: '4',
              resourceLimitCpu: '4',
              resourceRequestMemory: '8Gi',
              resourceLimitMemory: '8Gi'
          )],
          volumes: [
              persistentVolumeClaim(mountPath: '/home/jenkins', claimName: 'jenkins-slaves', readOnly: false)
          ]) {
          node('build') {
      
              stage("Checkout") {
                  checkout scm
              }
          }
      }
      

          [JENKINS-40745] Kubernetes plugin mkdirs error when configuring a persistent volume claim for /home/jenkins on slave

          Jesse Redl added a comment -

          Well...I was able to workaround this error but i'm not entirely sure why this fixed the issue. Rather than using inheritFrom I specified the containerTemplate for the jnlp slave and it worked just fine?

          #!groovy
          
          podTemplate(label: 'build', containers: [
              containerTemplate(
                  name: 'jnlp',
                  image: 'jenkinsci/jnlp-slave/',
                  args: '${computer.jnlpmac} ${computer.name}',
                  resourceRequestCpu: '.5',
                  resourceLimitCpu: '1',
                  resourceRequestMemory: '500Mi',
                  resourceLimitMemory: '1Gi',
              ),
              containerTemplate(
                  name: 'build-dependencies',
                  image: 'myorg/build-dependencies',
                  ttyEnabled: true,
                  command: 'cat',
                  alwaysPullImage: true,
                  resourceRequestCpu: '4',
                  resourceLimitCpu: '4',
                  resourceRequestMemory: '8Gi',
                  resourceLimitMemory: '8Gi'
              )],
              volumes: [
                  persistentVolumeClaim(mountPath: '/home/jenkins', claimName: 'jenkins-slaves', readOnly: false)
              ]) {
              node('build') {
          
                  stage("Checkout") {
                      checkout scm
                  }
              }
          }
          
          

          Jesse Redl added a comment - Well...I was able to workaround this error but i'm not entirely sure why this fixed the issue. Rather than using inheritFrom I specified the containerTemplate for the jnlp slave and it worked just fine? #!groovy podTemplate(label: 'build' , containers: [ containerTemplate( name: 'jnlp' , image: 'jenkinsci/jnlp-slave/' , args: '${computer.jnlpmac} ${computer.name}' , resourceRequestCpu: '.5' , resourceLimitCpu: '1' , resourceRequestMemory: '500Mi' , resourceLimitMemory: '1Gi' , ), containerTemplate( name: 'build-dependencies' , image: 'myorg/build-dependencies' , ttyEnabled: true , command: 'cat' , alwaysPullImage: true , resourceRequestCpu: '4' , resourceLimitCpu: '4' , resourceRequestMemory: '8Gi' , resourceLimitMemory: '8Gi' )], volumes: [ persistentVolumeClaim(mountPath: '/home/jenkins' , claimName: 'jenkins-slaves' , readOnly: false ) ]) { node( 'build' ) { stage( "Checkout" ) { checkout scm } } }

          I think that your code was running on jnlp slave before you explicitly added `custom jnlp` container. You can try to check this theory

          Sergey Kulanov added a comment - I think that your code was running on jnlp slave before you explicitly added `custom jnlp` container. You can try to check this theory

          Jesse Redl added a comment -

          I think you're right. Going to close this issue as we've been running fine since making these tweaks.

          Jesse Redl added a comment - I think you're right. Going to close this issue as we've been running fine since making these tweaks.

          Jesse Redl added a comment -

          Closing issue I logged as this seems to be user error.

          Jesse Redl added a comment - Closing issue I logged as this seems to be user error.

            csanchez Carlos Sanchez
            jredl Jesse Redl
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: