Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47561

Pipelines wait indefinitely for kubernetes slaves to come back online


    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • kubernetes-plugin
    • None

      Our situation

      We run Jenkins Master and ephemeral slaves in kubernetes clusters and love it. However we quite regularly run into stalled pipelines that wait indefinitely for a kubernetes ephemeral slave to come back online, which will never happen. I have no 100% reproducible steps because this does not happen every time. Instead it happens on average once out of 15 slaves. A slave will inexplicably disconnect/crash/be-decommissioned and the Jenkins pipeline using it thinks, "Ok, durable task means I wait for the slave that just went offline to come back online... forever." And so it does. :/

      Someone else has opened a ticket with Jenkins Master team requesting an option to flag a job/task as not durable, however they do make clear that this behavior is something that needs to be addressed by the plugin as well.

      The Story "Requirement"

      This may be a bug or a feature/improvement request, depending on how your team has intended the plugin to function. I have flagged this as a bug because I do not believe this behavior is intended.

      Kubernetes slaves should fail a pipeline step/task that is running on it whenever the slave goes offline.

            Unassigned Unassigned
            sbeckwithiii Sam Beckwith III
            0 Vote for this issue
            6 Start watching this issue