There is already a separate bug report that pipeline polling does not ignore commits from specific users.

Can you provide a job definition which shows the problem you're describing? I spent last weekend investigating with a freestyle job and a multi-branch pipeline job and was unable to duplicate the type of problem you describe with those jobs. Refer to JENKINS-41906 (and my jenkins-bugs repo) for my notes on my attempt to duplicate the problem.

I doubt this is due to a recent change in the plugin, since there have been no changes in the polling area. However, the pipeline support (with the exclusions) was only added in the last 12 months, so it may never have been working the way you expect in pipeline.

Hi markewaite, I've attached some images and edited the description, hope it makes more sense. I would like to re-iterate that this is NOT a multibranch pipeline.

NOTE: I decided against multibranch pipeline because it lacks 'github hook trigger' to auto generate a build whenever a new commit/branch is pushed into the repo. Regardless, there would still be an issue because 'polling ignores commits..' functionality is broken so I can't even do a periodic polling.

I'm having a similar, and extremely frustrating time with this. I'm probably being really stupid, so I'm happy to be educated and corrected to get this working.

The problem I'm encountering is that jobs that are confined to a specific branch specifier/refspec are kicked off no matter which branch was committed to in GIT.

My configuration is this:

Jenkins 2.44
java-1.8.0-openjdk-1.8.0.121-0.b13.el6_8.x86_64

1) create a Jenkins/Freestyle Job/Git configured/branch combo:
git@github.foo.com:org/repo.git
refspec: +refs/heads/2.24.7:refs/remotes/origin/2.24.7
branch specifier: 2.24.7 (also tested with refs/heads/2.24.7)

2) create a Jenkins/Freestyle Job/Git configured/master combo:
git@github.foo.com:org/repo.git
refspec: +refs/heads/master:refs/remotes/origin/master
branch specifier: master

3) Set the trigger in each job: GitHub hook trigger for GITScm polling

4) git clone git@github.foo.com:org/repo.git && # modify a file in your repo, then do a git commit -m file BRANCH, and then

5) Tail the jenkins.log, to watch the aggravation happen:

INFO: Received PUSH for https://github.foo.com/orgfoo/buildtest1 from 18.121.155.200 ⇒ http://foo.build.iwefjiwfj.com:12031/github-webhook/
Feb 14, 2017 1:22:14 PM com.cloudbees.jenkins.GitHubPushTrigger$1 run
INFO: SCM changes detected in buildtest-master. Triggering #4
Feb 14, 2017 1:22:14 PM com.cloudbees.jenkins.GitHubPushTrigger$1 run
INFO: SCM changes detected in buildtest-2.24-only. Triggering #2

Why?? Why is this triggering a master build when I performed my actions solely on the 2.24.7 branch? I must have configured something wrong (like the endpoint URL) or something in GitHub.. The webhook is configured only for push events.

f00ser I don't think you configured your job wrong. I got mine setup and functional last week (up until friday) and somehow, it started breaking. The only thing that I remember doing is upgrading my plugins; unfortunately, I was stupid enough to not pay attention to the versions and the list of plugins I've upgraded.

Thanks for your insight, Ralph A. Did you try downgrading Git Plugin to 3.0.1 and retry? I'm thinking to stepwise/trial-and-error downgrade plugin versions until this works..

In fact, I did f00ser. I downgraded to 3.0.1 (even though 1 plugin complained) and it still didn't work. Either I had a much lower Git plugin before Friday or the plugins I upgraded together with Git are the root cause. But I like I mentioned, I didn't pay attention to the plugins I've upgraded...

I tagged this bug as Git because the feature itself shows that it is from Git; it's highly probable that another plugin working in conjunction with Git is causing the issue.

Ralph A, sorry to hear that. I'll also confirm that downgrading to Git Plugin 3.0.1 concluded with no change in brokenness (I also service-restarted Jenkins). In this case, for every branch I've created a job for a specific repo, there's a build being kicked off when I commit to a specific branch. Builds are kicked off for:

2.24.7 <-- change was committed here
2.30.1
master

My team has another legacy cluster (Jenkins 1.625, Git Plugin 2.4.0) where this is not happening. Whether I keep or omit the refspec, change the branch-specifier to be accurate, etc., I can't, for the life of me, get the individual branch-job combination. Is there something I am missing, as far as setting up a filter for the github webhook? Should the payload be modified some how - I'm not sure how to do this with GitHub Plugin and the Git Webhook still does a push event, but doesn't filter what's in the payload without a post receive hook. I was really hoping to avoid going down this path..

I attempted a variation on your bug report, creating a pipeline job on the master branch and a pipeline job on a branch named JENKINS-42020. I configured both jobs to "Poll SCM" with no polling schedule set. That allows them to honor the notifyCommit (which I believe is equivalent to the GitHub WebHook that you're using) but won't launch a job if there are no changes on the branch.

Unfortunately, my test jobs don't show the problem.

It would be a great help if you could provide a technique to duplicate the problem which does not require that I configure a Jenkins on the public internet, or that I configure a GitHub Enterprise installation. Here are the steps you can use to recreate the environment I have configured:

I'm unable to duplicate the problem you're reporting, either with a freestyle job, nor with a pipeline job. The environment I'm using to duplicate the problem can be reconstructed with the following steps:

1. Clone, build, and run the docker instance

     $ git clone https://github.com/MarkEWaite/docker JENKINS-42020
     $ cd JENKINS-42020
     $ git checkout -b lts-with-plugins origin/lts-with-plugins
     $ docker build -t jenkins:JENKINS-42020 .
     $ # Copy jdk8 from Oracle to ~/public_html/jdk/jdk-8u121-linux-x64.tar.gz
     $ # Copy ant-1.9.8 from Apache to ~/public_html/ant/apache-ant-1.9.8-bin.zip
     $ docker run -i --rm \
         --publish 8080:8080 \
         --volume ~/public_html/:/var/jenkins_home/userContent/ \
         jenkins:JENKINS-42020
   

2. Connect a web browser to that docker instance (http://localhost:8080)
3. Open the "Bugs - Individual Checks" folder
4. Open the "JENKINS-42020-master" job
5. Click "Poll now" (simulate the web hook), confirm that the job builds (because it has not been built before)
6. Click "Poll now" (simulate the web hook again), confirm that the job does not build (already built)
7. Open the "JENKINS-42020-branch" job
8. Click "Poll now" (simulate the web hook), confirm that the job builds (because it has not been built before)
9. Click "Poll now" (simulate the web hook again), confirm that the job does not build (already built)

I also checked it with a multi-branch pipeline job in that same Docker instance. You can see the behavior in the pipeline job with the following steps:

1. Open the "Bugs - Pipeline Checks" folder
2. Open the "jenkins-bugs" pipeline job
3. Click the "Scan Multibranch Pipeline" link and "Run Now" to index branches for that repository
4. Confirm that a job is created and built for the "master" branch and all other branches
5. Wait 10-15 minutes, then Click the "Scan Multibranch Pipeline" link and "Run Now" to index branches again for that repository. That will likely detect changes on one of the branches, since I usually have tests running somewhere that will commit to that repository
6. Confirm that the job named "master" did not rebuild, even though other jobs rebuilt due to changes in them

Are there other details of your environment which I missed in my attempt to duplicate the problem?

markewaite
I attempted to reproduce the issue using your docker image by following steps 1-2 and then created my own pipeline project based on the first two images that I have attached in the bug description. The github hook trigger didn't even go through so I am inclined to assume that Github hook trigger does not function properly in this setup/environment to begin with (or at least some part of the flow is broken).

In addition, when you tried to reproduce the issue related to multiple branches triggering from one branch, you used "multi-branch pipeline". I should've mentioned, if I haven't already, that I am using a standalone pipeline specifically because it does not support Github hook trigger...

As for the issue related to "polling ignore commits.." functionality, my project is configured to push a new commit to the SAME branch after a successful build. Theoretically, this should repeat indefinitely UNLESS otherwise configured, hence, I had "polling ignore commits.." enabled to ignore the commit based on specific values. But in you're docker image, the Jenkins project setup does not have this configuration enabled. The steps followed to reproduce this issue does not seem like a very good simulation to recreate the original problem.

If there is anything unclear in my comment, please let me know.

f00ser Do you have any insight in this whole thing? Have you found any workaround?

EDIT:

I'm an idiot. Of course the webhook from github won't work because I didn't have it set up..But I would like to re-iterate that the steps followed, in my opinion, do not seem like a good way to reproduce the issue.

Sadly, I am not sure if there is an easier way for you to reproduce the issue besides setting it up the way I did. Is there anyone else that you know who might be able to help or have an environment setup up similar to mine?

I have a feeling this might be related to this bug: JENKINS-41121

stephenconnolly Can you please take a look at this? It might be related to SCM-API plugin instead of Git Plugin. If you don't currently have time to, would it be possible to get versoin 2.0 somewhere? It was not in the archive so I can't download it.

NOTE: I decided against multibranch pipeline because it lacks 'github hook trigger' to auto generate a build whenever a new commit/branch is pushed into the repo.

Cough Cough... GitHub Branch Source plugin is what you are looking for

It is not related to the SCM API plugin as I explicitly left the consolidation of event handlers out of scope: https://github.com/jenkinsci/git-plugin/blob/c7809b55c42532afd13d5587076c9cf437167fe3/src/main/java/jenkins/plugins/git/GitSCMSource.java#L383 (if I had the possibility of returning true from that then that would be triggering for you, but as you can see it returns false always

As you are not using a multibranch pipeline it can have nothing to do with JENKINS-41121

I suspect your problems would mostly go away if you install the GitHub Branch Source and use that for your multibranch pipeline

We'd also need to see how your master branch pipeline is set up, because that could be set up to build multiple branches.

Have you confirmed whether the master branch pipeline is building its existing head revision or whether it is building the revision pushed to your branch?

So the GitHub plugin does not consider the branch name when deciding which jobs should have polling run: https://github.com/jenkinsci/github-plugin/blob/master/src/main/java/org/jenkinsci/plugins/github/webhook/subscriber/DefaultPushGHEventSubscriber.java#L92-L102

So what you should see is any freestyle jobs that have the `github hook trigger` enabled should have a polling triggered any time there is a push hook received to the matching repository, irrespective of the branch.

Ohhhh hang on a sec... You have some excluded behaviours so this cannot do fast polling and has to fall back to actually doing a build IIRC.

I would recommend changing your workflow

stephenconnolly
"Cough Cough... GitHub Branch Source plugin is what you are looking for"
I DO have this plugin installed and have tried multibranch pipeline but I couldn't get the trigger to function nor could I find the option to enable it. Could you maybe attach a screenshot explain how to enable this?

From your last few comments, it is safe to assume that you've seen the images and read the description.
From my understanding, anything with the matching repository to receive the webhook trigger from github but only build if there are any changes in the branch. However, whenever I push to my dev branch, master gets triggered as well (for some weird reason...)

And as I have mentioned previously, I push a new change to the same branch whenever the build is successful, therefore, I need some sort of checker to NOT build whenever the 'auto-commit' has been added to my remote repo.

I will try to create a new multibranch pipeline (again) and then post my results here if that helps anyone with the same issue

stephenconnolly
I just realized why I didn't end up using 'Github Branch Source Plugin'. Under Branch Sources, I chose Git because the options for Github were all grayed out so I thought I'm supposed to choose Git instead. I'm going to try and figure out why this is the case but feel free to jump in and leave a comment if you know the answer already

stephenconnolly
I tried Github Branch Source yesterday and, like I mentioned previously, the options are grayed out:

The repository is empty as well when clicked. Do you know why this is?

Did you check the logs to see if it is an issue with your credentials?

Not sure what steps to follow to properly produce "useful" logs but here is what I did.
1. Went to the configure page for the multibranch pipeline
2. Clicked 'add source' -> 'Github client' and got this error:
/org.jenkinsci.plugins.github_branch_source.GitHubSCMSource/fillCheckoutCredentialsIdItems java.lang.reflect.InvocationTargetException
.
.
.
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.jenkinsci.plugins.gitclient.GitClient

Does this mean github branch source is failing because of Git plugin?

or the git-client plugin... perhaps a version mismatch

These are the known versions on a system I am using to test improvements on API rate throttling

(hence the newer -SNAPSHOT of the GHBS plugin, but otherwise you should be fine substituting the latest release of GHBS)

Besides Blue Ocean, which I don't have, my version for git-related plugins are the same as yours except for github branch source plugin. I have 2.0.3

I'm stumped. I don't really don't know what I'm doing wrong....

java: 1.8.0
credentials: 2.1.11
ssh credentials: 1.13

Very strange

Something is definitely broken since this used to work last week Feb 10th for standalone pipelines (I never got multibranch pipeline working with my setup)

I should point out that the GitHub Branch Source is auto-wired to the webhook, so when you get that working you should not need a "Periodically unless otherwise ran" that is more often than once per hour... once per day being more likely (that setting should be seen as 'If an event from GitHub was not delivered, how long can I wait without it being detected'... most people are probably 1 week as typically event delivery is very reliable and there are usually follow-up events anyway)

So, I may have chimed in on a thread where Ralph A's environment is centered on multi-branch and Pipeline Plugin usage. I'm not using Pipelines (yet; more of a team choice for that) and mine was a case specific to push event triggers from GitHub affecting branches with Multiple SCM Plugin, and the triggering of multiple branch build jobs. I isolated my situation (and wrote up a ticket here: Jenkins builds all jobs when user intends for one) around the behavior I saw. I'm open to understanding that we might have an ill-suited dev process for the way Jenkins functions with is SCM-push-trigger logic, but it seems to me that when a user commits a change to a branch, only that branch job should build, and not others (regardless if their GitHub state is a mismatch according to Jenkins).

The "workaround" is that we use a single GitHub repo SCM with a submodule of the other GitHub repo that's needed (our jobs only had 2 repos). But since that 2nd repo is only needed for CI (and not dev work), this was an acceptable alternative.

I've given up and changed my previous workflow, as per Stephen's sugggestion. I would also like to apologize for my previous comment that is misleading and not entirely correct.

I originally had a working freestyle project that did everything that I needed but, in order to satisfy our developers' need, we opted to switch it to a pipeline project in order to take advantage of Jenkinsfile. My memory got mixed up with what is working and what is half-working since I tested Jenkinsfile in a pipeline project in chunks so I remember the situation incorrectly.

That aside, I would like to now focus on building a standalone pipeline project (I still could not get multibranch to work with Github webhooks) but I still need to get the polling ignore commits to function. As per markewaite "[t]here is already a separate bug report that pipeline polling does not ignore commits from specific users". Could you please comment on the issue number so I know which one to track? I found a few issue related to polling ignore commits/users but I am unsure which one you are specifically referring to

Cheers!

Refer to JENKINS-36195 or JENKINS-35988 for the pipeline specific polling bug report.

How github plugin works is written in docs. Screenshot contains wrong git configuration that unrelated to github-plugin. 

Please go to user mail list with discussing user workflows and branch-source/org-folder etc