Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51695

ssh-add fails randomly on Kubernetes

XMLWordPrintable

      The ssh-agent plugin was made to work in Kubernetes in issue JENKINS-42582

      However, as mentioned in comments of that issue, the plugin fails to run ssh-add randomly.  In my case it happens in almost 50% of the time, which makes the ssh-agent plugin unusable on Kubernetes (builds fail too often). 

      https://issues.jenkins-ci.org/browse/JENKINS-42582?focusedCommentId=332237&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-332237

      https://issues.jenkins-ci.org/browse/JENKINS-42582?focusedCommentId=332300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-332300

      I have done some investigation and I believe it actually is a race condition due to how the kubernetes-plugin parses output.  By comparing the output of a working build with a failing one we can see that there is output from the ssh-agent plugin AFTER the EXITCODE 0 output that the kubernetes plugin looks for.  The kubernetes plugin then looks for the exit code but finds the extra output and fails. I can reproduce very easily, so I could obtain more info if needed.

      FAILING:

      Executing command: "ssh-add" "/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key"
      printf "EXITCODE %3d" $?; exit
      *EXITCODE 0EXITCODE 0*Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key)
      Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key)

      WORKING:

      Executing command: "ssh-add" "/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key"
      printf "EXITCODE %3d" $?; exit
      Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key)
      Identity added: /home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key (/home/jenkins/workspace/bowie_feature_doc_upload-2NDL5PPAHMMAWBMRX6CFCZI7VDOLJYNMA6HK2HG3NCDKNGOA6YVA@tmp/private_key_8883554255349387919.key)

      EXITCODE 0EXITCODE 0

            csanchez Carlos Sanchez
            marckhouzam Marc Khouzam
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: