Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62479

Rate limit reached for Bitbucket server

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We have a repository with ~120+ opened PRs and >2000 branches at own Bitbucket server (not a cloud instance) which is scanned for Jenkins files.

      The plugin hits the rate limit for Bitbucket every time it scans the repository for Jenkinsfiles. For cloud instances a fix was implemented and merged in scope of JENKINS-42458

      We need the same fix for on-premise instances.

      java.io.IOException: Communication error for url: Jenkinsfile status code: 429
      	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:493)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1478)
      	at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99)
      	at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74)
      	at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrievePullRequests(BitbucketSCMSource.java:675)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:593)
      	at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
      	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
      	at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634)
      	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277)
      	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164)
      	at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1025)
      	at hudson.model.ResourceController.execute(ResourceController.java:97)
      	at hudson.model.Executor.run(Executor.java:428)
      

        Attachments

          Issue Links

            Activity

            Hide
            kon Kalle Niemitalo added a comment -

            From Improving instance stability with rate limiting | Bitbucket Data Center and Server 7.16 | Atlassian Documentation, it looks like rate limiting requires a Bitbucket Data Center license, so the error won't occur if the server has only a Bitbucket Server license.

            Show
            kon Kalle Niemitalo added a comment - From Improving instance stability with rate limiting | Bitbucket Data Center and Server 7.16 | Atlassian Documentation , it looks like rate limiting requires a Bitbucket Data Center license, so the error won't occur if the server has only a Bitbucket Server license.
            Hide
            shankar128 Shankar Ramasamy added a comment -

            I have added a fix on this PR.
            When ratelimit on Bitbucket server is encountered for any requests, the same API call is retried after a short wait time.

            Show
            shankar128 Shankar Ramasamy added a comment - I have added a fix on this PR. When ratelimit on Bitbucket server is encountered for any requests, the same API call is retried after a short wait time.
            Hide
            smirky Bogomil Vasilev added a comment -

            I can confirm we hit this every time with up-to-date plugins and Jenkins:

            ERROR: [Thu Jul 30 10:08:02 CEST 2020] Could not fetch branches from source fb32e1e7-96a7-4001-8fd0-fcc97610aea6
            java.io.IOException: Communication error for url: devops/jenkins/Jenkinsfile status code: 429
             at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:519)
             at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1477)
             at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99)
             at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74)
             at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
             at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieveBranches(BitbucketSCMSource.java:747)
             at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:589)
             at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
             at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
             at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:641)
             at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:278)
             at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:165)
             at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1032)
             at hudson.model.ResourceController.execute(ResourceController.java:97)
             at hudson.model.Executor.run(Executor.java:428)
            [Thu Jul 30 10:08:02 CEST 2020] Finished branch indexing. Indexing took 1.7 sec
            FATAL: Failed to recompute children of ***********
            java.io.IOException: Communication error for url: devops/jenkins/Jenkinsfile status code: 429
             at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:519)
             at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1477)
             at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99)
             at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74)
             at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
             at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieveBranches(BitbucketSCMSource.java:747)
             at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:589)
             at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
             at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
             at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:641)
             at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:278)
             at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:165)
             at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1032)
             at hudson.model.ResourceController.execute(ResourceController.java:97)
             at hudson.model.Executor.run(Executor.java:428)
            Finished: FAILURE
            

            Could we please get a fix, as it is definitely out of the question to preserve a certain count of branches without a solid reason?

            Show
            smirky Bogomil Vasilev added a comment - I can confirm we hit this every time with up-to-date plugins and Jenkins: ERROR: [Thu Jul 30 10:08:02 CEST 2020] Could not fetch branches from source fb32e1e7-96a7-4001-8fd0-fcc97610aea6 java.io.IOException: Communication error for url: devops/jenkins/Jenkinsfile status code: 429 at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:519) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1477) at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99) at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74) at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieveBranches(BitbucketSCMSource.java:747) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:589) at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283) at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:641) at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:278) at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:165) at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1032) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) [Thu Jul 30 10:08:02 CEST 2020] Finished branch indexing. Indexing took 1.7 sec FATAL: Failed to recompute children of *********** java.io.IOException: Communication error for url: devops/jenkins/Jenkinsfile status code: 429 at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:519) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1477) at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99) at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74) at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieveBranches(BitbucketSCMSource.java:747) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:589) at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283) at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:641) at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:278) at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:165) at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1032) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:428) Finished: FAILURE Could we please get a fix, as it is definitely out of the question to preserve a certain count of branches without a solid reason?
            Show
            amuniz Antonio Muñiz added a comment - BTW: https://github.com/jenkinsci/bitbucket-branch-source-plugin/issues/288
            Hide
            gnome Dzmitry Moisa added a comment -

            Antonio Muñiz yes, I've updated the description and exception stack with the actual information.

            Show
            gnome Dzmitry Moisa added a comment - Antonio Muñiz yes, I've updated the description and exception stack with the actual information.
            Hide
            amuniz Antonio Muñiz added a comment -

            Reopening, rate limit support is implemented only for Bitbucket Cloud, not Server.

            Show
            amuniz Antonio Muñiz added a comment - Reopening, rate limit support is implemented only for Bitbucket Cloud, not Server.
            Hide
            amuniz Antonio Muñiz added a comment -

            Rate limit support added in 2.2.4.

            Show
            amuniz Antonio Muñiz added a comment - Rate limit support added in 2.2.4.
            Hide
            amuniz Antonio Muñiz added a comment -

            Latest release is 2.8.0, did you try it?

            Show
            amuniz Antonio Muñiz added a comment - Latest release is 2.8.0, did you try it?

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              gnome Dzmitry Moisa
              Votes:
              5 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated: