Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62479

Rate limit reached for Bitbucket server

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We have a repository with ~120+ opened PRs and >2000 branches at own Bitbucket server (not a cloud instance) which is scanned for Jenkins files.

      The plugin hits the rate limit for Bitbucket every time it scans the repository for Jenkinsfiles. For cloud instances a fix was implemented and merged in scope of JENKINS-42458

      We need the same fix for on-premise instances.

      java.io.IOException: Communication error for url: Jenkinsfile status code: 429
      	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:493)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1478)
      	at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99)
      	at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74)
      	at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrievePullRequests(BitbucketSCMSource.java:675)
      	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:593)
      	at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
      	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
      	at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634)
      	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277)
      	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164)
      	at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1025)
      	at hudson.model.ResourceController.execute(ResourceController.java:97)
      	at hudson.model.Executor.run(Executor.java:428)
      

        Attachments

          Issue Links

            Activity

            gnome Dzmitry Moisa created issue -
            gnome Dzmitry Moisa made changes -
            Field Original Value New Value
            Priority Critical [ 2 ] Major [ 3 ]
            gnome Dzmitry Moisa made changes -
            Environment Jenkins 2.32.3
            Bitbucket-branch-source-plugin 2.1.0 (working version 1.9)
            Jenkins 2.222.1
            Bitbucket-branch-source-plugin 2.7.0
            amuniz Antonio Muñiz made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Fixed but Unreleased [ 10203 ]
            amuniz Antonio Muñiz made changes -
            Resolution Fixed [ 1 ]
            Status Fixed but Unreleased [ 10203 ] Reopened [ 4 ]
            amuniz Antonio Muñiz made changes -
            Assignee Antonio Muñiz [ amuniz ]
            gnome Dzmitry Moisa made changes -
            Description We have a team repository with about 300 repositories which is scanned for jenkins files.

            We updated Bitbucket-branch-source-plugin to 2.1.0 (working version 1.9) yesterday and now we hit the rate limit every time.

            This means that repositories that are last in the arbitrarily sorted list is never added. We think that this plugin should know about the rate limit and throttle it self. It should always handle newly updated repositories first.

            The best solution would be if Atlasian created a webhook in the team resource for repository creation. But we need a workaround in the mean time.

            {code:java}
            [Fri Mar 03 05:29:24 UTC 2017] Finished organization scan. Scan took 8 min 23 sec
            FATAL: Failed to recompute children of {Removed Job name}
            com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: 429: Unknown Status Code.
            Rate limit for this resource has been exceeded
            at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.getRequest(BitbucketCloudApiClient.java:568)
            at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.getRepository(BitbucketCloudApiClient.java:232)
            at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.isPrivate(BitbucketCloudApiClient.java:405)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrievePullRequests(BitbucketSCMSource.java:334)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:325)
            at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300)
            at jenkins.scm.api.SCMSource.fetch(SCMSource.java:254)
            at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria.recognizes(MultiBranchProjectFactory.java:263)
            at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.recognizes(OrganizationFolder.java:1266)
            at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.complete(OrganizationFolder.java:1281)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator.add(BitbucketSCMNavigator.java:212)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator.visitSources(BitbucketSCMNavigator.java:187)
            at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:399)
            at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219)
            at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:154)
            at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:850)
            at hudson.model.ResourceController.execute(ResourceController.java:98)
            at hudson.model.Executor.run(Executor.java:404)
            Finished: FAILURE
            {code}
            We have a repository with ~120+ opened PRs and >2000 branches at own Bitbucket server (not a cloud instance) which is scanned for Jenkins files.

            The plugin hits the rate limit for Bitbucket every time it scans the repository for Jenkinsfiles. For cloud instances a fix was implemented and merged in scope of JENKINS-42458

            We need the same fix for in-premise instances.
            {code:java}
            java.io.IOException: Communication error for url: Jenkinsfile status code: 429
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:493)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1478)
            at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99)
            at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74)
            at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrievePullRequests(BitbucketSCMSource.java:675)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:593)
            at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
            at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
            at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634)
            at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277)
            at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164)
            at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1025)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:428)
            {code}
            gnome Dzmitry Moisa made changes -
            Description We have a repository with ~120+ opened PRs and >2000 branches at own Bitbucket server (not a cloud instance) which is scanned for Jenkins files.

            The plugin hits the rate limit for Bitbucket every time it scans the repository for Jenkinsfiles. For cloud instances a fix was implemented and merged in scope of JENKINS-42458

            We need the same fix for in-premise instances.
            {code:java}
            java.io.IOException: Communication error for url: Jenkinsfile status code: 429
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:493)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1478)
            at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99)
            at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74)
            at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrievePullRequests(BitbucketSCMSource.java:675)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:593)
            at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
            at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
            at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634)
            at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277)
            at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164)
            at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1025)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:428)
            {code}
            We have a repository with ~120+ opened PRs and >2000 branches at own Bitbucket server (not a cloud instance) which is scanned for Jenkins files.

            The plugin hits the rate limit for Bitbucket every time it scans the repository for Jenkinsfiles. For cloud instances a fix was implemented and merged in scope of JENKINS-42458

            We need the same fix for on-premise instances.
            {code:java}
            java.io.IOException: Communication error for url: Jenkinsfile status code: 429
            at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.checkPathExists(BitbucketServerAPIClient.java:493)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$BitbucketProbeFactory$1.exists(BitbucketSCMSource.java:1478)
            at jenkins.scm.api.SCMSourceCriteria$Probe.stat(SCMSourceCriteria.java:99)
            at org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:74)
            at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrievePullRequests(BitbucketSCMSource.java:675)
            at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:593)
            at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
            at jenkins.scm.api.SCMSource.fetch(SCMSource.java:283)
            at jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:634)
            at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277)
            at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164)
            at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1025)
            at hudson.model.ResourceController.execute(ResourceController.java:97)
            at hudson.model.Executor.run(Executor.java:428)
            {code}
            kon Kalle Niemitalo made changes -
            Link This issue relates to JENKINS-64418 [ JENKINS-64418 ]
            kon Kalle Niemitalo made changes -
            Link This issue is related to JENKINS-64418 [ JENKINS-64418 ]
            kon Kalle Niemitalo made changes -
            Link This issue relates to JENKINS-64418 [ JENKINS-64418 ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              gnome Dzmitry Moisa
              Votes:
              5 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated: