Your includes rule is

master

Which means it will only build heads with the name master

What you want is an includes rule of

master, PR-*

so that heads called PR-2 etc are also allowed to be discovered

Please take a little longer to comprehend this defect. I'm challenging your assertion that it's not a defect.

The current implementation is counter intuitive to practically every other integration with GitHub pull requests. Take for example:

• ghprb-plugin not requiring snowflake branch names in order to process PRs.
• Travis CI building pull requests despite a master whitelist; see the .travis.yml file

It's not just Travis CI and ghprb-plugin which interact with GitHub pull requests. I can spend all day finding and giving you examples because there's an overwhelming amount.

The current implementation leaves a lot to be desired. The current architecture puts unnecessary burden on a development team to know and document that they have to include PR- as part of their branch name.

Proposed fix

Pull requests destined for the master branch, regardless of their branch name, should be automatically built (but keep the same forking vs internal PRs model).

What you want is an extension plugin for the GitHub Branch Source that filters in a GitHub aware way rather than the generic filter in the SCM API plugin.

It should be fairly trivial to do: basically mostly like https://github.com/jenkinsci/scm-api-plugin/blob/master/src/main/java/jenkins/scm/impl/trait/WildcardSCMHeadFilterTrait.java you just need to split the logic at https://github.com/jenkinsci/scm-api-plugin/blob/master/src/main/java/jenkins/scm/impl/trait/WildcardSCMHeadFilterTrait.java#L99-L100 so that if it is a ChangeRequestSCMHead you instead filter on https://github.com/jenkinsci/scm-api-plugin/blob/master/src/main/java/jenkins/scm/api/mixin/ChangeRequestSCMHead.java#L57 (in fact you could probably make it a more generic extension plugin as that only needs to rely on SCM API... but it should still be an extension plugin as this is a special case filtering)

Not really against the scm-api plugin as this should be in a new extension plugin
New feature should be trivial to implement. Here is an example of another extension plugin: https://github.com/shantur/github-pr-label-filter-plugin

Thanks for the second consideration stephenconnolly!

I would also like to add that checking out PRs should be easy because every repository stores pull requests under refs/pull/*.

$ git ls-remote origin | grep 'refs/pull'
e9f6d340969704646a95c20c65c45cb50b716098	refs/pull/15/head
175259a4dd7004b83c8295e197a0e2b107378a15	refs/pull/20/head
93348b757a97ed99b72e7f1275766a5edecd7907	refs/pull/56/head
7606e1d3a8df87bd8649ec8bb6e43be2b32edeb6	refs/pull/74/head
9b5d6b5f644da3589076dbdce13c59a77ac4c811	refs/pull/75/head
84d042d153020f18a29476b96d0a99462a8dbc40	refs/pull/75/merge
d122b4858b5beaf5eb23ac726074cb290534b129	refs/pull/83/head
e8db964a38d33d472e0857e2d5b35a4092c43538	refs/pull/86/head
acf6de35147d2c4459fdd2fb2c84b883198be8f1	refs/pull/92/head
f238b74394ec9f1a21a3c2c7531b3c7408fdfdfb	refs/pull/92/merge

I found a solution:

1. Using GitHub branch source
2. Under Behaviors within repository
3. Add an "Additional" behavior Specify ref specs**
4. Add the following refspec:

+refs/pull/*/head:refs/heads/PR-*

Additionally, don't forget the following:

1. Using GitHub branch source
2. Under Behaviors within repository
3. Filter by name (with wildcards) value set to:

master PR-*

I just needed to apply a little git-fu to the current expectations for how the multibranch plugin works.

stephenconnolly This may be resolved but I wonder if it would be good for users to have a shortcut branch source?  Either way, I'm good now .

I know you don't want to hear this, but what you have done will blow up in your face:

1. You now have lost all security features of being able to filter PRs based on trust. All PRs will be trusted as they will be discovered as "branches" not as PRs. Even PRs from forks. There is a reason we have the "Discover PRs from origin" and "Discover PRs from forks" behaviours.
2. You are still not getting the filtering of PRs based on targeting master, which AIUI was what you wanted. So instead you have just replicated "Discover PRs from origin", "Discover PRs from forks", and "Filter by name (with wildcards)=master PR-*" only without: the correct push event support, the ability to prevent 'Hack your Jenkins by drive-by pull request' style attacks
3. You are forcing all checkouts to pull all PRs which slows down the build.

All because you do not want to try and write a rather simple extension plugin that consists of one class (which you can almost copy and paste, changing the line https://github.com/jenkinsci/scm-api-plugin/blob/master/src/main/java/jenkins/scm/impl/trait/WildcardSCMHeadFilterTrait.java#L99-L100) and its corresponding .jelly file

I'll make this event easier for you, here is the line you want:

                if (head instanceof ChangeRequestSCMHead) {
                    head = ((ChangeRequestSCMHead)head).getTarget();
                }
                return !Pattern.matches(getPattern(getIncludes()), head.getName())
                        || (Pattern.matches(getPattern(getExcludes()), head.getName()));

With that filter configured for master then you will get only the master branch and only PRs that target master - of course you need to include the "Discover PR from ..." behaviours that match your security and trust model.

I know you think you are good, but, respectfully, you actually are not as good as you think you are

I agree with this assessment.  I appear to have very quickly hit GitHub API limits (which is strange to me because I have disabled periodically checking GitHub for changes).

18:46:53 Connecting to https://api.github.com using samrocketman/****** (GitHub user and personal access token used by multibranch pipeline jobs for the GitHub API)
18:46:53 GitHub API Usage: Current quota has 4979 remaining (35808 over budget). Next quota of 5000 in 10 hr. Sleeping for 9 hr 33 min.
18:49:53 GitHub API Usage: Still sleeping, now only 9 hr 30 min remaining.
18:52:53 GitHub API Usage: Still sleeping, now only 9 hr 27 min remaining.
18:55:53 GitHub API Usage: Still sleeping, now only 9 hr 24 min remaining.
18:58:53 GitHub API Usage: Still sleeping, now only 9 hr 21 min remaining.
19:01:53 GitHub API Usage: Still sleeping, now only 9 hr 18 min remaining.
19:04:53 GitHub API Usage: Still sleeping, now only 9 hr 15 min remaining.
19:07:53 GitHub API Usage: Still sleeping, now only 9 hr 12 min remaining.
19:10:53 GitHub API Usage: Still sleeping, now only 9 hr 9 min remaining.
19:13:53 GitHub API Usage: Still sleeping, now only 9 hr 6 min remaining.
19:16:53 GitHub API Usage: Still sleeping, now only 9 hr 3 min remaining.
19:19:53 GitHub API Usage: Still sleeping, now only 9 hr 0 min remaining.

I only have two projects configured with multi-branch so I'm considering filing a separate bug with scm-api-plugin hitting limits despite it not polling for changes.

I'll have to review your recommendations on developing on the extensions to see if I'm personally capable of it. I've not developed a plugin before; only maintained and released to Jenkins artifactory plugins that already exist. I'm not really sure how extension APIs work in Java and in the context of Jenkins.

Thanks for the tips and hints on what I need to change. I really appreciate it.

I'm going to try proposing a change which adds an option for the behavior.

I have proposed a work in progress change https://github.com/jenkinsci/scm-api-plugin/pull/48; stephenconnolly please review.

FTR we are not adding checkboxes to the existing traits in scm-api.

I have declared in multiple places, multiple times, that I view the correct direction to be people creating plugins that just contain the required @Extension s to add the new behaviour. I will accept PRs against the core plugins that enable people to write more complex extension plugins, but there was considerable effort to move from checkbox hell to the traits based UI (which is more composable, comprehensible and testable, e.g. there were 64 combinations of checkbox options with regard to SCMHead discovery in the old GitHub Branch Source UI and yet we had no place to hang things like trustability for fork PRs. The new UI makes it clearer what is being discovered - obviously modulo understanding the conventions of the new UI - and as each trait is composable, we can get better test coverage on the behaviour and you can reason more about what happens when the traits are composed together)

At some point, perhaps scm-api 3.x or something, I may move the existing traits in scm-api out into their own plugin - in order to keep the API plugin as pure API. The reason those were added there was because they are the most basic common denominator traits and as such they should stay basic common denominator traits.

The behaviour you are after would best fit a plugin with a name like scm-cr-aware-filters-impl or scm-pr-aware-filters-impl. You can copy most of the code from the existing behaviours, no need to add checkboxes, just give the copy traits a new name... this will greatly simplify the testing (because you have fewer branches) and because the functionality surface is smaller, people have less concern about upgrades... i.e. the plugin does one thing and one thing only, and it only does that when you explicitly configure your jobs to use it.

stephenconnolly Thanks for your clear and concise advice. I believe I have a working solution now. Please review https://github.com/samrocketman/scm-branch-pr-filter-plugin/commit/318277a1cbfa3783cec2056f78372e7535393282

Do you think this makes sense to open a request to have this plugin included at https://github.com/jenkinsci? I've still got more testing to do before I would contribute the plugin.

I did my best to try to separate what was originally scm-api-plugin and my plugin so that it is very clear the exact changes I made. Feedback from you is welcome. I ended up having to update the wiki on https://wiki.jenkins.io/display/JENKINS/Plugin+tutorial because the command to generate a new plugin threw errors.

I opened JENKINS-47154 to discuss my issue with the github rate limit (I commented earlier in this ticket but just formalized it into a separate issue). If you have comments about it; please discuss it there.

sag47 I am going to assign this issue to you as your plugin should address this need. I recommend filing a hosting request with Jenkins CI so that others can benefit.

Hopefully this plugin was as easy to implement as I have claimed and will be proof of my strategy

stephenconnolly how do I do the following with the filter?

EnvVars env = build.getEnvironment(listener));

I would like to inject an environment variable IS_PR_BUILD so one could use it in scripts if desired.

But you already have CHANGE_ID if the SCMHead is a PR: https://github.com/jenkinsci/branch-api-plugin/blob/master/src/main/java/jenkins/branch/BranchNameContributor.java#L65

I did not realize that. I'll add IS_PR_BUILD to my personal pipeline global library based on the existence of CHANGE_ID. The idea is for it to be true or false so shell scripts can simply:

if ${IS_PULL_REQUEST}; then
  #do something because it's a pull request
fi

Not exactly clear to me what the gain in all that hassle is over

if [ -n "${CHANGE_ID}" ] ; then
  # do something because it's a pull request
fi

It just simplifies bash scripting. It's not something that needs to be included in this plugin. I was able to update my scripts since you pointed that out to me. https://github.com/samrocketman/jervis/commit/315c20eb5e2a4ec8e468f0dac3534282ff04a4f0

stephenconnolly I went ahead and documented this in the plugin wiki page. https://wiki.jenkins.io/display/JENKINS/SCM+Branch+PR+Filter+Plugin

sag47 I suspect abayer might be able to advise you on better examples of testing for PRs to provide on your Wiki page. Certainly there is a better way for declarative style... but I also think that your non-declarative style example is over-egged

Feel free to update it; it's a wiki

This issue has been resolved with the creation of a new plugin: https://wiki.jenkins.io/display/JENKINS/SCM+Filter+Branch+PR+Plugin