Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64250

k8s plugin schedule pod twice in each job

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • kubernetes-plugin
    • None
    • jenkins: 2.235.5
      kubernetes-plugin: 1.27.2 or above
      kubernetes-client-api: 4.11.1

      Hi,

      I encounter a strange issue from our Jenkins master. k8s client start to schedule two pods for

      each builds.

      builds console log:  you can see event log from k8s that it shows it creates pod twice

      [Pipeline] Start of Pipeline
      [Pipeline] podTemplate
      [Pipeline] {
      [Pipeline] node
      Created Pod: devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx][Scheduled] Successfully assigned devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx to eks-test-ebilm4u7zb7c-minion-0
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx][Pulled] Container image "docker.io/library/python:3.6-slim" already present on machine
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx][Created] Created container docker-io-library-python-3-6-slim
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx][Started] Started container docker-io-library-python-3-6-slim
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx][Pulled] Container image "hub.ecns.io/library/jnlp-slave:4.3-1" already present on machine[Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx][Created] Created container jnlp
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx][Started] Started container jnlpCreated Pod: devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z
      Agent 2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx is provisioned from template 2357fdb6-f93c-4f55-9b80-528366a75dff_807-sjl47-h94sx
      ---
      apiVersion: "v1"
      kind: "Pod"
      metadata:
        annotations:
          buildUrl: "http://devops.easystack.cn/job/2357fdb6-f93c-4f55-9b80-528366a75dff/807/"
          runUrl: "job/2357fdb6-f93c-4f55-9b80-528366a75dff/807/"
        labels:
          jenkins: "slave"
          jenkins/label-digest: "bae50ed141cfd133cdb349314a79a9c753d615eb"
          jenkins/label: "2357fdb6-f93c-4f55-9b80-528366a75dff_807-sjl47"
        name: "2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx"
      spec:
        containers:
        - command:
          - "cat"
          image: "docker.io/library/python:3.6-slim"
          imagePullPolicy: "IfNotPresent"
          name: "docker-io-library-python-3-6-slim"
          tty: true
          volumeMounts:
          - mountPath: "/home/jenkins/agent"
            name: "workspace-volume"
            readOnly: false
        - env:
          - name: "JENKINS_SECRET"
            value: "********"
          - name: "JENKINS_AGENT_NAME"
            value: "2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx"
          - name: "JENKINS_WEB_SOCKET"
            value: "true"
          - name: "JENKINS_NAME"
            value: "2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx"
          - name: "JENKINS_AGENT_WORKDIR"
            value: "/home/jenkins/agent"
          - name: "JENKINS_URL"
            value: "http://devops.easystack.cn/"
          image: "hub.ecns.io/library/jnlp-slave:4.3-1"
          name: "jnlp"
          resources:
            requests:
              cpu: "100m"
              memory: "256Mi"
          volumeMounts:
          - mountPath: "/home/jenkins/agent"
            name: "workspace-volume"
            readOnly: false
        dnsConfig:
          options:
          - name: "single-request-reopen"
        hostAliases:
        - hostnames:
          - "devops.easystack.cn"
          ip: "172.110.0.2"
        nodeSelector:
          kubernetes.io/os: "linux"
        restartPolicy: "Never"
        volumes:
        - emptyDir:
            medium: ""
          name: "workspace-volume"[Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z][Scheduled] Successfully assigned devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z to eks-test-ebilm4u7zb7c-minion-0[Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z][Pulled] Container image "docker.io/library/python:3.6-slim" already present on machine
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z][Created] Created container docker-io-library-python-3-6-slim
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z][Started] Started container docker-io-library-python-3-6-slim
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z][Pulled] Container image "hub.ecns.io/library/jnlp-slave:4.3-1" already present on machine
      Running on 2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-jsmfx in /home/jenkins/agent/workspace/2357fdb6-f93c-4f55-9b80-528366a75dff
      [Pipeline] {
      [Pipeline] stage
      [Pipeline] { (Scm_Clone_master)
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z][Created] Created container jnlp[Pipeline] checkout
      [Normal][devops/2357fdb6-f93c-4f55-9b80-528366a75dff-807-sjl47-h94sx-cg54z][Started] Started container jnlp
      The recommended git tool is: NONEusing credential d53155ef-0d13-4fb0-be3f-91331623fb2e
      Cloning the remote Git repository
      Cloning repository https://github.com/qaz-wsx-12/test-aa.git
       > git init /home/jenkins/agent/workspace/2357fdb6-f93c-4f55-9b80-528366a75dff # timeout=10
      Fetching upstream changes from https://github.com/qaz-wsx-12/test-aa.git
       > git --version # timeout=10 > git --version # 'git version 2.20.1'
      using GIT_ASKPASS to set credentials d53155ef-0d13-4fb0-be3f-91331623fb2e
       > git fetch --tags --force --progress -- https://github.com/qaz-wsx-12/test-aa.git +refs/heads/*:refs/remotes/origin/* # timeout=10Avoid second fetch
      Checking out Revision e6630c9aff0251940efb3548b412d37610bde78d (refs/remotes/origin/master) > git config remote.origin.url https://github.com/qaz-wsx-12/test-aa.git # timeout=10
       > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
       > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
       > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
       > git config core.sparsecheckout # timeout=10
       > git checkout -f e6630c9aff0251940efb3548b412d37610bde78d # timeout=10Commit message: "Initial commit"
       > git rev-list --no-walk e6630c9aff0251940efb3548b412d37610bde78d # timeout=10
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] stage
      [Pipeline] { (First Stage)
      [Pipeline] container
      [Pipeline] {[Pipeline] script
      [Pipeline] {
      [Pipeline] withEnv
      [Pipeline] {
      [Pipeline] sh (U3RlcC0x)
      + sleep 600Sending interrupt signal to process
      Killing processes
      kill finished with exit code 0Terminated
      script returned exit code 143
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }[Pipeline] // script
      [Pipeline] }
      [Pipeline] // container
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] }
      [Pipeline] // podTemplate
      [Pipeline] End of Pipeline
      Finished: ABORTED
      

      my pipeline is

      pipeline {
        agent {
          kubernetes {
            cloud 'single-cluster-default'
            yaml '''apiVersion: v1
      kind: Pod
      spec:
        containers:
        - image: hub.ecns.io/library/jnlp-slave:4.3-1
          name: jnlp
        - command:
          - cat
          image: docker.io/library/python:3.6-slim
          imagePullPolicy: IfNotPresent
          name: docker-io-library-python-3-6-slim
          tty: true
        dnsConfig:
          options:
          - name: single-request-reopen
        hostAliases:
        - hostnames:
          - devops.easystack.cn
          ip: 172.110.0.2
      '''
          }  }
        stages {
          stage('Scm_Clone_master') {
            steps {
              checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'd53155ef-0d13-4fb0-be3f-91331623fb2e', url: 'https://github.com/qaz-wsx-12/test-aa.git']]])
            }
          }    stage('First Stage') {
            steps {
              container('docker-io-library-python-3-6-slim') {
                script {
                  withEnv([]) {
                    sh script: ''' sleep 600 ''', label: 'U3RlcC0x'
                  }
                }        }      }
          }  }
      }
      

      Could you please have a check?

            Unassigned Unassigned
            bibline He Bihong
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: