Awesome. That definitely sounds like a good change. Hopefully we can try it out some time in late January when things are supposed to settle down for us a bit.

Thanks for the info and update. I'll post here when we get the chance to try it.

I was running into this problem, I haven't run into this issue post installing this beta(and waiting for a bit so that Github API rate limits are reset)

FTR I filed JENKINS-41112 as a dupe with many details, so summarizing here. With an org with a real lot of repositories, this still fails, but I can relate this is indeed an order of magnitude better. From looking at the logs, I think we were almost done before hitting the limit. Like a gut feeling that if we had had say only 900 or 1000 Git repositories that could have worked.

We have an org with more than 1200 repositories, so at some point either the plugin should probably throttle calls, but also this seems pretty stupid that GitHub doesn't just allow us to pay for raising that limit a bit (I mean, I guess it's obvious that in general you're likely to need more calls on an org with more repositories...), but that's another discussion.

Sample of messages that can be seen in jenkins.log:

Jan 24, 2017 9:10:27 PM hudson.model.Executor finish1
SEVERE: Executor threw an exception
java.lang.Error: org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded
at org.kohsuke.github.Requester$PagingIterator.fetch(Requester.java:506)
at org.kohsuke.github.Requester$PagingIterator.hasNext(Requester.java:471)
at org.kohsuke.github.PagedIterator.fetch(PagedIterator.java:44)
at org.kohsuke.github.PagedIterator.hasNext(PagedIterator.java:32)
at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.visitSources(GitHubSCMNavigator.java:291)
at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:398)
at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219)
at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:141)
at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:849)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Caused by: org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded
at org.jenkinsci.plugins.github_branch_source.Connector$1.onError(Connector.java:177)
at org.kohsuke.github.Requester.handleApiError(Requester.java:649)
at org.kohsuke.github.Requester$PagingIterator.fetch(Requester.java:500)
... 10 more

Jan 24, 2017 9:10:53 PM com.squareup.okhttp.ConnectionPool pruneAndGetAllocationCount
WARNING: A connection to https://api.github.com/ was leaked. Did you forget to close a response body?

stephenconnolly this is still an issue. And on a scan of an Org initially, we are guaranteed to hit this limit, and never be able to complete the Org scan. How does one proceed in this event?

As a workaround, I have created multiple GitHub Organization Folders, with a Repository name pattern to limit the number of matching repositories in that folder. Then ensure the Build Periodically is set to H H * * *, which will spread the automatic re-scans throughout the day.

For my company it made sense to group repos by the team responsible for that repository, but I imagine a simple alphabetical partition would work fine too. Note that the search functionality (upper right corner) works fine no matter how the folders are organized.

I believe all of this is supposed to be resolved (no workarounds needed) in the SCM 2.0 plugins, which have recently been re-cleared for production install (but I've not tried yet myself).

morgan_goose Correct this is still an issue, but it is not as bad. I have some changes that we are testing that proactively delay operations to prevent the rate limit from being exhausted and allow the operations to complete. Those changes will resolve this issue but we decided that they were too much to add in with the change to SCM API 2.0.x's event API

The plan is to release the rate limit throttling fixes as we are confident in each fix, but the immediate priority is actually fixing JENKINS-36029 (which affects BitBucket not GitHub but is very very bad)

Yes I confirmed today - it is a whole lot better than it was (today I wasn't able to exhaust the api, which is good).

I'm having trouble following the 30+ comments all the other open issues surrounding Github rate limits, but this seems to have gotten much worse not better than me. Now when re-scanning hits the rate limit, not only can I not perform more builds or scan for new jobs, but the jobs that got scanned after the rate limit (myproject-c through myproject-g in my example) actually get deleted since the plugin concludes they no longer exist, then completes with "SUCCESS".

Is this plugin considered experimental or do you advise downgrading to 1.x? Has anyone had luck contacting Github & getting API rate limits increased? Really feeling dead in the water here as we need to disable all Github scanning for fear of losing current jobs and not being able to perform other Github operations.

Example log:
... many branches repos scanned successfully, followed by eventually many similar exceptions and finally projects that can no longer be scanned getting removed from our list of jobs:

ERROR: Failed to create or update a subproject my-project-a
org.jenkinsci.plugins.github_branch_source.RateLimitExceededException: GitHub API rate limit exceeded
	at org.jenkinsci.plugins.github_branch_source.Connector$1.onError(Connector.java:259)
	at org.kohsuke.github.Requester.handleApiError(Requester.java:649)
	at org.kohsuke.github.Requester._to(Requester.java:284)
	at org.kohsuke.github.Requester.to(Requester.java:225)
	at org.kohsuke.github.GitHub.checkApiUrlValidity(GitHub.java:669)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:414)
	at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:300)
	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:254)
	at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria.recognizes(MultiBranchProjectFactory.java:260)
	at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.recognizes(OrganizationFolder.java:1153)
	at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.complete(OrganizationFolder.java:1168)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.add(GitHubSCMNavigator.java:459)
	at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.visitSources(GitHubSCMNavigator.java:319)
	at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:398)
	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:219)
	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:141)
	at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:849)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:404)
[Fri Feb 10 03:44:40 UTC 2017] Finished organization scan. Scan took 9 min 21 sec
Evaluating orphaned items in My Organization.
Will not remove myproject-b as MyOrganization Inc. » myproject-b » master #3 is still in progress
Will remove myproject-c as it is #1 in the list
Will remove myproject-d as it is #2 in the list
Will remove myproject-e as it is #3 in the list
Will remove myproject-f as it is #4 in the list
Will remove myproject-g as it is #5 in the list
Finished: SUCCESS

Reposting our success with a quick patch of 1.1 for John and anyone in a similar situation.

https://issues.jenkins-ci.org/browse/JENKINS-36121?focusedCommentId=277600&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-277600

We greatly reduced our api usage and have had no problems with our fork for several months.

jhovell how many repos do you have in an org? are they all private repos?

michaelneale - 120 repos, all but ~5 are private, most of which do not use this plugin (yet) but many which sadly have a large number of branches (I found 1 repo with 250 branches, an artifact of some other tool). We're experimenting using branch name filters for our critical projects, but since the repo filter needs to be a regex and we have a large number of repos it seems challenging to maintain a regex that includes the right repos.

An aside - looks like each repo scanned is requiring ~10 API calls per repo per branch? I have been curling https://api.github.com/rate_limit as the repo refresh process is occurring & can watch it steadily drop.... scanning 2 branches on all repos consumed about 2000 API calls from the 5000 limit per hour github enforces for authenticated users. Still it means we need to make sure no one triggers a rescan more than once per hour at most.

spockninja - thank you I did see your comment earlier. I am not a Jenkins plugin developer & wasn't quite sure how to build & install a patch to a plugin. If there are some documentation on how to do this I'd definitely give it a try though I'm not sure what drawbacks exist (other than the obvious needing to keep your fork maintained as the project moves forward)

Thanks for the quick response! The power of this plugin is awesome, it's changed the way we work, just challenging to keep running for larger orgs.

Thanks jhovell - wow that doesn't sound too huge. I know the ci.jenkins.io project is using this itself (or trying to) with 1000's of repos (it understandably has problems).

This doesn't sound like it should be happening - 10 API calls per branch per repo stephenconnolly? Is that expected?

FYI it will be hard to downgrade to a 1.1 version (if you do, make sure you backup things etc) if you want to try it. But hopefully we can sort out the SCM 2 flavour of things, as your use case is spot on what this should work for, with no issues at all. Glad this functionality is of value to you, it does make things a whole lot easier when it works.

Having the same problem with Rate limit exceeding, although I've got like 150 repos in organization, but real long history of branches. I was reading the sources of the plugin "Github Branch Source" and found a call to retrieve collaborators when querying the particular repository which ends up to API request to Collaborators endpoint. Seems odd to me because it does not use that particular local collaboratorNames variable any further in the body of the method. It will save 150 requests for me, but even more for people with thousands of repositories.

2 cents.

Noticed the collaborators call (it's in the organization sync log as well) and thought it was strange as I wasn't sure why it would be needed. Happy to provide full logs or anything else if it's of help debugging.

Figured out that collaboratorNames is used to identify "trusted" users of the repository. Although if the repository is private it does not make sense to assume any pull request as not trusted, because only selected people have access to the repository anyway.

paulchubatyy

Although if the repository is private it does not make sense to assume any pull request as not trusted, because only selected people have access to the repository anyway.

Oh if only it were that simple. Some organizations will allow everyone in the org read access to the OPS repos but only the OPS team have write access. It is the write access to a repo that should gate whether the PR is trusted or not.

In any case, there is a solution to the rate limit problem, pro-active throttling. I have a PoC hack implementation in a Git Stash but there are some implications that need to be thought through carefully. For example: in this change to branch-api I have had to make the event handlers use synchronization in order to ensure that there are no file handle leaks. I will need to find a better fix (i.e. write more complex code) in order to work with the pro-active throttling as otherwise a rate limit throttle will effectively block all events from all sources.

As I said earlier, there is a much more critical bug in the BitBucket plugin that is currently #1 on my todo list. This issue is #2

good to hear stephenconnolly.

Ok. The data loss issue for BitBucket (JENKINS-36029) and the general data loss issue (JENKINS-42000) have now both been fixed. This issue is now number 1 on my ToDo list.

Are there any volunteers for taking experimental builds when I have them ready?

I'll have to upgrade to 2.x first, but will be happy to give it a try.

stephenconnolly I do.

Meanwhile I've got a very hacky workaround for this issue with a pipeline script that will check github organization and create the multibranch projects for all the repositories without doing any checks. See this gist for details.

Be aware that it uses Jenkins Job DSL plugin, so you will have to install it before running.

OK... Very experimental trial... use at your own risk. BACKUP EVERYTHING IMPORTANT TO YOU BEFORE INSTALLING

You will need:

1. Updated snapshot of cloudbee-folder
2. Updated snapshot of branch-api
3. Updated snapshot of github-api
4. Updated snapshot of github-branch-source

You will get:

• Log messages like this:

Started by user anonymous
Consulting GitHub Organization
Connecting to https://api.github.com with no credentials, anonymous access
API Rate Limit 60 with 0 remaining
API Rate Limit 60 with 0 remaining, sleeping until Mon Feb 20 18:15:22 GMT 2017

• I have not put the rate limit guards on every code path, so you may still end up tripping the rate limits, but the primary important paths all have the rate limit guards, so you should have your scan complete eventually.
• If another plugin is also using the same credentials, they will not be paying any attention to the branch source's needs and hence may cause the rate limit to trip anyway... but most people should not hit the limits (though an anonymous scan of even reasonable sized organizations will take hours 60 requests per hour is not much!)

cloudbees-folder.hpi branch-api.hpi github-api.hpi github-branch-source.hpi

Also note, you should see throttling kick in progressively when the rate limit is 75% consumed... it will progressively sleep for longer and longer, starting with 30 second sleeps, progressing through 1 and 5 minute sleeps until the rate limit is 99% consumed at which point it will sleep until the reset.

I am working on a different algorithm, but my aim is to proof out the approach in general.

if you need to roll back, you only need to roll back the GitHub Branch Source plugin. The other plugins should be fine to run.

cc paulchubatyy kiora morgan_goose there is some stuff to test above ^

all the images from this post are hosted on imgur so that this issue did not get trashed by attachments. Obviously I cannot add images from 3rd parties into this post, so please follow the links, they should be self-explanatory.

Testing on:

OS
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"
================

Jenkins package itself
$ apt-cache show jenkins
Package: jenkins
Architecture: all
Version: 2.47
Priority: extra
Section: devel
Maintainer: Kohsuke Kawaguchi <kk@kohsuke.org>
Installed-Size: 67107
Depends: daemon, adduser, procps, psmisc, net-tools, default-jre-headless (>= 2:1.7) | java7-runtime-headless
Conflicts: hudson
Replaces: hudson
Filename: binary/jenkins_2.47_all.deb
Size: 68329532
MD5sum: ce0ba54cf9b384af318e61330cb00b2d
SHA1: 58930ad52cf6c49044a02663cb9f2cd066a99ff3
SHA256: ddfb5eafec356eebdce011f00ca2ba37b65f5168a3825eaff8b8668e8f3495c2
SHA512: 5aa113ca649efbfa7440f8aac446d54478ed2bbe236106b80ba3361c2a4b9a850a7bfc388022f49c0c2b2da06337d682061da56a8e2c0689f066866056d8af23
Homepage: http://jenkins.io/

$ sudo apt-get install jenkins
Reading package lists... Done
Building dependency tree
Reading state information... Done
jenkins is already the newest version.

Plugins installed: Github API and Github Branch Source Plugins, Folders Plugin, Branch API Plugin

Initial organization scan

Organization settings pretty straightforward, nothing special. Trigger scan every 5 minutes. Do not build anything automatically.

Result: exception raised and not handled
Scan failed.

All screenshots on one page.

paulchubatyy thanks for that, my anon scan completed just fine (after 4 hours) so I guess I got lucky with where the org repos fell.

The anon access token is probably going to be most problematic with this as 60 is really not much and you need to wait an hour between tests if they fail (or hop VPNs )

I expect that the -SNAPSHOT should work much better when given a real rate limit (i.e. 5000/hr)

github-branch-source.hpi New improved -SNAPSHOT

$ sha1sum github-branch-source.hpi 
d7b09b1ac67dab05079fb53b92808b1063061c9d  github-branch-source.hpi

paulchubatyy this new snapshot should fix the issues for you (can still have the rate limit pulled out from under it by other plugins using the same credentials)

I have setup my Jenkins instance with 11 GitHub Organization folders and 2 "normal" Folders. My company is divided into 11 sub-teams ("verticals"), and I use a regex to identify which GitHub repos in our company are mapped to each "vertical".

After installing these plugins, my 2 "normal" folders load, one of my GitHub Organization Folders loads, and the remaining 10 GitHub Organization Folders fail to load. `jenkins.log` has this for each of the 10 failed-to-load folders:

```
(lots of these, one for each repo + branch which exists on the filesystem. Inspection by hand confirms config.xml is not there)
Feb 22, 2017 1:18:57 AM com.cloudbees.hudson.plugins.folder.AbstractFolder loadChildren
WARNING: could not find file /var/lib/jenkins/jobs/operations_vertical/jobs/chef-repo/branches/typography-nerd/config.xml
Feb 22, 2017 1:18:57 AM jenkins.InitReactorRunner$1 onTaskFailed
SEVERE: Failed Loading item analytics_vertical
java.lang.NullPointerException
at jenkins.branch.OrganizationFolder.onLoad(OrganizationFolder.java:185)
at hudson.model.Items.load(Items.java:372)
at jenkins.model.Jenkins$17.run(Jenkins.java:3060)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
at jenkins.model.Jenkins$7.runTask(Jenkins.java:1064)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
```

I just updated to the SCM 2.x series yesterday; everything had seemed to be working OK before I tried these attached plugins, but it's possible the 1.x to 2.x update left things in a fragile or broken state, and I didn't noticed until now. I'm going to try to uninstall these experimental plugins and get back to a working Jenkins instance. I've preserved all my logs if you'd like more details.

Reverted and all my folders are displaying as expected:

• github-branch-source 2.0.3
• branch-api 2.0.6
• github-api 1.84
• cloudbees-folder 5.17

danielrussell an NPE on OrganizationFolders:185 indicates that the navigator failed to deserialize. There would have either been a data migration error on the main admin screen with details or the jenkins main log would have detailed a plugin failing to load.

Absence either of those I cannot diagnose your issue further. Do you think you could try and replicate (which will either work or give us the error details from which to determine the fix... though I suspect it is a plugin version mismatch causing plugins to fail to load)

Aha! Apparently I had both `jpi`s and your `hpi`s, and the `jpi`s won. I removed the `jpi` files and everything started as expected. I'll try some regular tests now.

```
Feb 22, 2017 3:04:42 PM hudson.PluginManager$1$3$1 isDuplicate
INFO: Ignoring /var/lib/jenkins/plugins/cloudbees-folder.hpi because /var/lib/jenkins/plugins/cloudbees-folder.jpi is already loaded
Feb 22, 2017 3:04:42 PM hudson.PluginManager$1$3$1 isDuplicate
INFO: Ignoring /var/lib/jenkins/plugins/github-branch-source.hpi because /var/lib/jenkins/plugins/github-branch-source.jpi is already loaded
Feb 22, 2017 3:04:42 PM hudson.PluginManager$1$3$1 isDuplicate
INFO: Ignoring /var/lib/jenkins/plugins/github-api.hpi because /var/lib/jenkins/plugins/github-api.jpi is already loaded
Feb 22, 2017 3:04:42 PM hudson.PluginManager$1$3$1 isDuplicate
INFO: Ignoring /var/lib/jenkins/plugins/branch-api.hpi because /var/lib/jenkins/plugins/branch-api.jpi is already loaded
Feb 22, 2017 3:04:42 PM jenkins.InitReactorRunner$1 onAttained
```

Ahh cool

Trivial display-related bug ("-219 ms"):

```
15:41:56 GitHub API Usage: 261 remaining; 255 planned; 5000 per hour limit. Reset in 3.8 sec
Proposing xxxxyyyyyzzzzz
15:41:56 GitHub API Usage: 240 remaining; 254 planned; 5000 per hour limit. Reset due in 3.5 sec. Sleeping until reset.
15:42:40 GitHub API Usage: 240 remaining; 197 planned; 5000 per hour limit. Reset in -219 ms
15:42:40 Connecting to https://api.github.com using myusername/******
```

Looks like the "remaining" and "planned" jumped up, then dropped back down. I'd expect the "remaining" to possibly be a caching-related inconsistency from GH, but the "planned" seems like something that would be owned by Jenkins.

```
15:42:44 GitHub API Usage: 261 remaining; 192 planned; 5000 per hour limit. Reset in -72 ms
Ignoring apple
15:42:44 GitHub API Usage: 261 remaining; 192 planned; 5000 per hour limit. Reset in -72 ms
Ignoring banana
15:42:44 GitHub API Usage: 261 remaining; 192 planned; 5000 per hour limit. Reset in -72 ms
Proposing cucumber
15:42:44 GitHub API Usage: 4934 remaining; 4941 planned; 5000 per hour limit. Reset in 59 min. Sleeping for 24 sec.
15:43:08 GitHub API Usage: 4934 remaining; 4908 planned; 5000 per hour limit. Reset in 58 min
15:43:09 Connecting to https://api.github.com using myusername/******
Looking up myorg/dill

Getting remote pull requests...
0 pull requests were processed

Getting remote branches...

Checking branch bar
‘Jenkinsfile’ not found
Does not meet criteria

Checking branch master
‘Jenkinsfile’ found
Met criteria

0 branches were processed (query completed)

Done examining myorg/dill

15:43:13 GitHub API Usage: 261 remaining; 154 planned; 5000 per hour limit. Reset in -482 ms
Ignoring eggplant
15:43:13 GitHub API Usage: 261 remaining; 154 planned; 5000 per hour limit. Reset in -482 ms
```

Core functionality (ratelimit awareness and backoff) looks good.

I have a job which extracts the remaining API calls and dumps the number into graphite. With the old plugns, an org scan created a straight downward-sloping line. With the new plugins, the line (usage) is a bit more wobbly, which seems to indicate visually that the periodic pauses are working.

Also, with the old plugins my org scans will happily drive the remaining API calls to 0; the new plugins seem to slow down enough that they never hit 0.

Note that I have multiple instances (11) of the GitHub Organization folder, and even when I triggered a Scan Organization -> Run Now in several of them, the ratelimiting code worked as I expect (all scans progressed, but I didn't hit GH's limits).

Like you said earlier, there is likely some room for improvement in the backoff algorithm, but this seems to completely solve my use case.

Looks like the "remaining" and "planned" jumped up, then dropped back down

So remaining is the amount of rate limit remaining

planned is the amount that we have budgeted to have used by this point in time in the current quota cycle

I have renamed the terminology in my latest build which I will be attaching shortly

Please find attached the latest -SNAPSHOT

To run this -SNAPSHOT you will need:

• Upgrade CloudBees Folders Plugin to 5.18 (released today)
• Upgrade Branch API plugin to 2.0.7 (released today)
• GitHub API plugin 1.85-SNAPSHOT attached to this ticket github-api.hpi
• GitHub Branch Source plugin 2.0.4-SNAPSHOT attached to this ticket github-branch-source.hpi

$ sha1sum github-branch-source.hpi 
98284c2d1816300db4cda2d9e75c4998a44cd1a3  github-branch-source.hpi

I have tightened up some of the connection pooling that was causing the rate limit to be mis-reported in some cases and would result in a rate limit trip.

I have one WARNING: A connection to https://api.github.com/ was leaked. Did you forget to close a response body? issue from the GitHub API plugin's getCollaborators method that I am currently trying to fix. It is orthogonal to this issue so I may punt on it for this fix.

If we could get some more testing of this latest and the testing is positive then I think we are almost ready to release (but my pedantic nature wants a fix for the WARNING: A connection to https://api.github.com/ was leaked. Did you forget to close a response body? )

I updated to the 2 released and 2 latest attachments, and all looks good.

Thanks!!

WARNING: A connection to https://api.github.com/ was leaked. Did you forget to close a response body?

Oh man... that was a crazy rabbit hole! I think I have it solved now!

(Hopefully) final -SNAPSHOTs:

• github-api github-api.hpi SHA1: 5f26d128db1231c09a29cf8f573b84ce8976a0a9
• github-branch-source github-branch-source.hpi SHA1: e634bff27597cab7ae21169ce913a268c376764e

Please let me know if there are any regressions or rate limit trips with these plugins.

Currently blocked on ci_jenkinsci_org to release the github-api components, so likely next week before he's back from vacation and can cut that release

Also positive test results are very much appreciated. Everyone who can report that with these plugins the rate limit issue is gone and they have not encountered any regressions is adding to confidence in these changes.

stephenconnolly this seems to be working well! Thank you for the quick turn around here:

10:01:45 GitHub API Usage: Current quota has 3278 remaining (2 over budget). Next quota of 5000 in 48 min. Sleeping for 26 sec.

Current wild guess at an ETA for official release is 2 weeks from now then? When that happens will the plugin updater let us override this, or because it's manual it'll sorta stick? I'm happy enough with the results of this that I'm thinking of using your cuts until the real release.

morgan_goose these are -SNAPSHOTs so once the release versions are available in the update center you will see them and be able to update using the standard Jenkins UI.

I would expect to get KK to release github-api on Mon/Tue and then it's just a question of how long it takes to get final code review and whatever additional testing we decide in our test plan. Could be released as early as next Friday (no holding your breath though)

I do not foresee any issues running these -SNAPSHOTs in production, but it is your call on your system. Obviously the more shake-down these SNAPSHOTs get the more confidence we have in them and the easier the release.

Thanks for taking the time to test them

I realize this is a bit off topic but is there a separate issue for having a feature to just add a new repository (or delete a no longer needed repo) by name instead of needing to scan the entire org to pick up a new repo? I feel like this would eliminate 99% of the use cases to need to scan the organization at all. Thanks!

jhovell that would be very much a separate issue/topic/pr, but it is worth considering, as I find the same thing. It is nice to use org folders for this but sometimes you know you just want to add one... perhaps another JIRA with your use cases would be worth writing up? as I know this has been talked about before (so that things can be added directly to a computed folder).

michaelneale I created https://issues.jenkins-ci.org/browse/JENKINS-42387

I wasn't sure if it might be a duplicate.

I am not aware of how I could use folders to mitigate this issue. Is there some documentation somewhere? With the deprecated github organization folder plugin, Googling is failing me. I do not see any obvious ways to add folders or further organization to my single github organization. Thanks!

Code changed in jenkins
User: Stephen Connolly
Path:
pom.xml
src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubBuildStatusNotification.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubClosable.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubConsoleNote.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMFile.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMFileSystem.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMNavigator.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMProbe.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMSource.java
http://jenkins-ci.org/commit/github-branch-source-plugin/8a393a4231a4eda81cd4f931d9bc07804713543d
Log:
JENKINS-36121 Throttle API usage rather than trip the API limit

Code changed in jenkins
User: Stephen Connolly
Path:
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMNavigator.java
http://jenkins-ci.org/commit/github-branch-source-plugin/c16c1f4dabbe08feda6f8661772c7bad62334bbf
Log:
JENKINS-36121 Actually don't even waste the API calls listing repositories for the common event case

Code changed in jenkins
User: Stephen Connolly
Path:
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubConsoleNote.java
http://jenkins-ci.org/commit/github-branch-source-plugin/b025751381452800809847d9585f5f356a38d9ac
Log:
JENKINS-36121 Typos

Code changed in jenkins
User: Stephen Connolly
Path:
src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java
http://jenkins-ci.org/commit/github-branch-source-plugin/fcafab8f42b55bcb2dbc6383f7ce01b29810cd98
Log:
JENKINS-36121 Review comments

Code changed in jenkins
User: Stephen Connolly
Path:
src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java
http://jenkins-ci.org/commit/github-branch-source-plugin/b59c83dbab0ef203bf930e5cbde49796ddc609b9
Log:
JENKINS-36121 Clarifying comments

Code changed in jenkins
User: Stephen Connolly
Path:
pom.xml
src/main/java/org/jenkinsci/plugins/github_branch_source/Connector.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubBuildStatusNotification.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubClosable.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubConsoleNote.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMFile.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMFileSystem.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMNavigator.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMProbe.java
src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMSource.java
http://jenkins-ci.org/commit/github-branch-source-plugin/2b7a7f955978c275892c10460d2c4351ff400cb8
Log:
Merge pull request #124 from stephenc/jenkins-36121

JENKINS-36121 Do not trip the rate limit

Compare: https://github.com/jenkinsci/github-branch-source-plugin/compare/1dbf190e3c29...2b7a7f955978

Released as 2.0.4-beta-1 and available via the experimental update center.

Assuming our (CloudBees) soak tests and any other community contributed test results are successful, this should be released to the general update center in the next couple of days.

This is your last chance to provide feedback before this is incorporated into a mainline release

Works for me so far

Released as 2.0.4