Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58878

withCredentials hangs

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      workflow-cps 2.76

      Description

      On Jenkins 2.176.2 and credentials-binding-plugin 1.19, the following pipeline will intermittently hang:

      pipeline {
          agent any
      
          stages {
              stage('bug') {
                  steps {
                      script {
                          for (int i = 0; i < 100; i++) {
                              withEnv(['A=b']) {
                                  withCredentials([]) {
                                      sh "echo hello ${i}"
                                  }
                              }
                          }
                      }
                  }
              }
          }
      }
      

      The build log looks like this:

      ...
      [Pipeline] withEnv
      [Pipeline] {
      [Pipeline] withCredentials
      

      until the build is aborted.

      I've seen two independent things that both seem to work around the problem:

      • flipping so `withCredentials([])` wraps `withEnv([...])` seems to avoid the hang.
      • ensuring there's always at least one item in the `withCredentials` list parameter seems to avoid the hang.

      We run into this in a pipeline where the parameter to withCredentials varies by projects, and sometimes is an empty list.

        Attachments

          Issue Links

            Activity

            Hide
            jons Jon Sten added a comment -

            You're welcome, and thank you Devin Nusbaum for fixing the problem! I started to try to fix the bug, but became very fast confused and disoriented by all the different threads and objects involved, so I gave up . In other words, I appreciate experts like you that can fix these problems!

            I've given the linked experimental release a spin on our staging environment and I'm no longer able to reproduce the issue

            Show
            jons Jon Sten added a comment - You're welcome, and thank you Devin Nusbaum for fixing the problem! I started to try to fix the bug, but became very fast confused and disoriented by all the different threads and objects involved, so I gave up . In other words, I appreciate experts like you that can fix these problems! I've given the linked experimental release a spin on our staging environment and I'm no longer able to reproduce the issue
            Hide
            dnusbaum Devin Nusbaum added a comment -

            A fix for this issue was just released in Pipeline: Groovy Plugin version 2.76.

            Show
            dnusbaum Devin Nusbaum added a comment - A fix for this issue was just released in Pipeline: Groovy Plugin version 2.76.
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            Devin Nusbaum By any chance could this fix also solve my sporadically experienced hanging pipelines in withDockerRegistry/withDockerServer context (with executor on Jenkins master *cough*)?

            Example thread dump:

                at DSL.withDockerRegistry(not currently scheduled, or running blocks)
                at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:40)
                at DSL.withEnv(Native Method)
                at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:39)
                at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66)
                at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:38)
                at com.acme.php.phpstorm.OfflineInspection.execute(OfflineInspection.groovy:59)
                at acme.executePhpStormInspections(acme.groovy:256)
                at WorkflowScript.run(WorkflowScript:101)
            ...
            

            Please note, however, that it appeared as if the single or first (?) step inside such withDockerRegistry/withDockerServer contexts may sometimes actually be still executed...

            Show
            reinholdfuereder Reinhold Füreder added a comment - Devin Nusbaum By any chance could this fix also solve my sporadically experienced hanging pipelines in withDockerRegistry / withDockerServer context (with executor on Jenkins master *cough*) ? Example thread dump: at DSL.withDockerRegistry(not currently scheduled, or running blocks) at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:40) at DSL.withEnv(Native Method) at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:39) at org.jenkinsci.plugins.docker.workflow.Docker.node(Docker.groovy:66) at org.jenkinsci.plugins.docker.workflow.Docker.withRegistry(Docker.groovy:38) at com.acme.php.phpstorm.OfflineInspection.execute(OfflineInspection.groovy:59) at acme.executePhpStormInspections(acme.groovy:256) at WorkflowScript.run(WorkflowScript:101) ... Please note, however, that it appeared as if the single or first (?) step inside such withDockerRegistry / withDockerServer contexts may sometimes actually be still executed...
            Hide
            dnusbaum Devin Nusbaum added a comment -

            Reinhold Füreder Yes, if you were running Docker Pipeline 1.19 or newer, in which case those steps use GeneralizedNonBlockingStepExecution (see docker-workflow PR 158), then they could have been affected by the bug and should be fixed now.

            Show
            dnusbaum Devin Nusbaum added a comment - Reinhold Füreder Yes, if you were running Docker Pipeline 1.19 or newer, in which case those steps use GeneralizedNonBlockingStepExecution (see docker-workflow PR 158 ), then they could have been affected by the bug and should be fixed now.
            Hide
            reinholdfuereder Reinhold Füreder added a comment -

            Thanks, this is very promising/good news!

            Show
            reinholdfuereder Reinhold Füreder added a comment - Thanks, this is very promising/good news!

              People

              Assignee:
              dnusbaum Devin Nusbaum
              Reporter:
              jonathanb1 Jonathan B
              Votes:
              4 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: