Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51421

limit the number of concurrent request to a workflow library repository

XMLWordPrintable

      We have a workflow-lib repository enabled that contains some groovy script utilities commons for all JenkinsFile. We have named it as "workflow-libs" an located on a git repository :

      ssh://bitbucket.server/prj/workflow-lib.git

      and we use it as usual :

      @Library ('jworkflow-libs') _
      
       node {
           stage ("Test") {
           builder = new workflow.BuildHelper()
      //  ....
           echo "Hello world"
         }
      }
      

      This can be emulated using :

      stage("Launch all") {
      parallel(
          test1 : {build job: 'test1', wait: true},
          test2 : {build job: 'test2', wait: true},
          test3 : {build job: 'test3', wait: true},
          test4 : {build job: 'test4', wait: true},
          test5 : {build job: 'test5', wait: true},
          test6 : {build job: 'test6', wait: true},
          test7 : {build job: 'test7', wait: true},
          test8 : {build job: 'test8', wait: true},
          test9 : {build job: 'test9', wait: true},
          )
      

      Where job 'testX' contains the previous one commented.

      But we are suffering a problem with the communication between Jenkins and Bitbucket server, when you launch >5 jobs simultaneously, the 6^th^ job fails with an error message .

      hudson.plugins.git.GitException: Failed to fetch from ssh://bitbucket.server/prj/workflow-lib.git
      	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:862)
      	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1129)
      	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1160)
      	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113)
      	at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.doRetrieve(SCMSourceRetriever.java:112)
      	at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrieve(SCMSourceRetriever.java:84)
      	at org.jenkinsci.plugins.workflow.libs.LibraryAdder.retrieve(LibraryAdder.java:153)
      	at org.jenkinsci.plugins.workflow.libs.LibraryAdder.add(LibraryAdder.java:134)
      	at org.jenkinsci.plugins.workflow.libs.LibraryDecorator$1.call(LibraryDecorator.java:125)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1065)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:129)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:123)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:517)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:480)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:269)
      	at hudson.model.ResourceController.execute(ResourceController.java:97)
      	at hudson.model.Executor.run(Executor.java:421)
      Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --progress ssh://bitbucket.server/prj/workflow-lib.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
      stdout: 
      stderr: ssh_exchange_identification: read: Connection reset by peer
      fatal: Could not read from remote repository.
      
      Please make sure you have the correct access rights
      and the repository exists.
      

      If you relaunch the job individually it works.

      We are trying to solve it ( it could be a security restriction between bitbucket - jenkins to avoid DoS ) but would be very useful a configuration parameter for worfklow libs that limit the number of simultaneous request to the same repository (like pooling configuration has).

            Unassigned Unassigned
            fernando_rosado Fernando Rosado Altamirano
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: