Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64418

Add exponential backoff to BitBucket rate limit retry loop

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      The current implementation of retry for Bitbucket rate limits sleeps 5 seconds and tries again.

      We have Bitbucket folders that have hundreds of repos some of which have hundreds of branches. This can sometimes take hours to do a folder scan. During that time the Bitbucket rate limit is constantly at its maximum

      Adding exponential backoff (with jitter for randomness) will allow other API requests (such as to download the Jenkinsfile when a multibranch pipeline is building) to go through and will alleviate the constant pressure on the rate limit.

        Attachments

          Issue Links

            Activity

            Hide
            artheus Morten Hekkvang added a comment -

            Ben Dean Provided a PR (https://github.com/jenkinsci/bitbucket-branch-source-plugin/pull/414) for exponential back-off for the Bitbucket API Client.

            As stated, by me and Liam Newman in the PR, it would be a much more stable and reliable solution to make use of the rate limit headers of the Bitbucket API for throttling the client request rate to the API.

            Liam Newman provided a great link to the Github Branch Source plugin, showing how the rate limit throttling was solved there: https://github.com/jenkinsci/github-branch-source-plugin/blob/master/src/main/java/org/jenkinsci/plugins/github_branch_source/ApiRateLimitChecker.java

            I will look into creating a PR to control request throttling in a similar manner to the Github Branch Source plugin.

             

            Show
            artheus Morten Hekkvang added a comment - Ben Dean  Provided a PR ( https://github.com/jenkinsci/bitbucket-branch-source-plugin/pull/414)  for exponential back-off for the Bitbucket API Client. As stated, by me and Liam Newman  in the PR, it would be a much more stable and reliable solution to make use of the rate limit headers of the Bitbucket API for throttling the client request rate to the API. Liam Newman  provided a great link to the Github Branch Source plugin, showing how the rate limit throttling was solved there: https://github.com/jenkinsci/github-branch-source-plugin/blob/master/src/main/java/org/jenkinsci/plugins/github_branch_source/ApiRateLimitChecker.java I will look into creating a PR to control request throttling in a similar manner to the Github Branch Source plugin.  

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              b_dean Ben Dean
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated: