Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41112

GitHub Branch Source should throttle calls to stay below rate limit

XMLWordPrintable

      As a user, I would like to be able to use GitHub Branch Source plugin on big GitHub organizations.
      Right now, though it's a lot better, we still hit the wall of the rate limit.

      The workaround could be to specify a Repository name pattern but that wouldn't work for all our cases because we don't have naming rules that would enable that.

      I think the plugin could for example:

      • check the number of repositories of the requested organization (and the given pattern)
      • do a rough calculation of the number of API calls
      • warn the user it's going to take (far) more time than pure CPU/network capacities should enable to be able to complete the operation.

      Example:

      say I need 10 API calls per repo and I find the user has 1000. This would be 10k calls, instead of the max of 5k. Without even counting the probable other calls coming from some other plugins and so on.

      So, the plugin would regularly sleep from time to time to throttle and stay under the radar.

      Here is an example of what the scanning provoked in term of API calls after I created a new Org Folder and clicked on Save:

      $ while true
do curl -s -u USER:REDACTED_PAT https://api.github.com/rate_limit | jq -r .rate.remaining
sleep 10
done
4996
4996
4742
4518
4316
4072
3884
3707
3320
2800
2211
1895
1685
1466
1231
952
685
454
204
0
0

      And sure, when 0 was reached, the plugin started failing as expected:

      Started
[Mon Jan 16 20:50:34 UTC 2017] Starting organization scan...
[Mon Jan 16 20:50:34 UTC 2017] Updating actions...
Looking up details of ORG...
ERROR: [Mon Jan 16 20:50:34 UTC 2017] Could not refresh actions for navigator org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator@1164b181
org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded
	at org.jenkinsci.plugins.github_branch_source.Connector$1.onError(Connector.java:259)
	at org.kohsuke.github.Requester.handleApiError(Requester.java:649)
	at org.kohsuke.github.Requester._to(Requester.java:284)
	at org.kohsuke.github.Requester.to(Requester.java:225)
	at org.kohsuke.github.GitHub.checkApiUrlValidity(GitHub.java:669)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.retrieveActions(GitHubSCMNavigator.java:476)
	at jenkins.scm.api.SCMNavigator.fetchActions(SCMNavigator.java:315)
	at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:371)
	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:187)
	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:139)
	at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:814)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
[Mon Jan 16 20:50:34 UTC 2017] Consulting GitHub Organization
ERROR: [Mon Jan 16 20:50:34 UTC 2017] Could not fetch sources from navigator org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator@1164b181
org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded
	at org.jenkinsci.plugins.github_branch_source.Connector$1.onError(Connector.java:259)
	at org.kohsuke.github.Requester.handleApiError(Requester.java:649)
	at org.kohsuke.github.Requester._to(Requester.java:284)
	at org.kohsuke.github.Requester.to(Requester.java:225)
	at org.kohsuke.github.GitHub.checkApiUrlValidity(GitHub.java:669)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.visitSources(GitHubSCMNavigator.java:265)
	at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:420)
	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:187)
	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:139)
	at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:814)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
[Mon Jan 16 20:50:34 UTC 2017] Finished organization scan. Scan took 0.24 sec
FATAL: Failed to recompute children of Plugins ยป Operations center all the things
org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded
	at org.jenkinsci.plugins.github_branch_source.Connector$1.onError(Connector.java:259)
	at org.kohsuke.github.Requester.handleApiError(Requester.java:649)
	at org.kohsuke.github.Requester._to(Requester.java:284)
	at org.kohsuke.github.Requester.to(Requester.java:225)
	at org.kohsuke.github.GitHub.checkApiUrlValidity(GitHub.java:669)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.visitSources(GitHubSCMNavigator.java:265)
	at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:420)
	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:187)
	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:139)
	at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:814)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
Finished: FAILURE

            Unassigned Unassigned
            batmat Baptiste Mathus
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: