The way Multibranch works is that every time it scans the repository, for each branch it gets the latest revision.
Then it re-configures each branches job to build that revision only and triggers a build if the previous configuration was for a different revision.
If you want SCM polling, you should have the scan run at the polling interval as the scan will do one efficient scan for all branches rather than having each branch scan.
If you want some branches to not build so often, you should have a rate limit branch property on those branches, etc
Now for an organization folder, we should probably provide a mechanism for defining have repository specific properties (analogous to branch specific properties) so that the foo repository can poll every 10 minutes while all other repositories poll daily or something like that.
The other thing I want to investigate is polling the events for GitHub, e.g. https://developer.github.com/v3/activity/events/#list-repository-events and https://developer.github.com/v3/activity/events/#list-events-for-an-organization as those should be a much more efficient polling algorithm for the more delta aware modern SCM API event subsystem. Probably this would work as a global setting for the GitHub server where you define the server as either being web-hook push events or being api poll-events (here is the frequency) and then we can have an admin level log of the polling of events which would then get dispatched to the event listeners. (The issue I see with poll-events is that there are a number of events that GitHub says in the documentation are only delivered by web-hook... if one of those is a critical event then we are SooL)
OK, so back to this issue.
- It is necessary for Multibranch to configure each branch job's SCM to be pinned to the revision to build
- It is confusing to users that clicking build on a branch job will not build the latest code, but rather rebuild the revision
The user confusion can probably be cleared up by tweaking the branch project's actions.
Hack Solution: If we replace(or rename) the "Build Now" button with a "Rebuild Now" button that would clarify for the user that we will be rebuilding the same revision not building the latest revision. Additionally we could inject a "Build Now" button that uses SCM API to query the head revision of the branch and reconfigure the SCM (while also setting the marker revision to suppress any delayed events from auto-triggering the job)
Real Solution: Change core so that SCMDecisionHandler gets consulted after polling has detected changes but before the job gets triggered. Then branch-api would be able to update the Multibranch tracking state and the SCM so that the polling triggered job:
1. Builds the latest revision
2. Correctly records that the latest revision has been built in order to ensure that a scan does not retrigger the build
It should be relatively easy to inject such a hook into https://github.com/jenkinsci/jenkins/blob/d111e2ac1658c8fa5fb768e7d1233613b4b9992d/core/src/main/java/hudson/triggers/SCMTrigger.java#L621-L639 and then have Branch API implement the hook to perform its requirements.
There is one final gotcha... namely concurrent builds... if there are concurrent builds of the same branch, we would be re-writing the job configuration as each job starts... that may mean that by the time the first build reaches checkout scm it actually is checking out the next build's revision... I wonder if we should actually mandate some more of implementers of the Branch API... (ok I'm rambling now)
Polling intervals shorter than the build duration could the the problem here.