Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-42161

Jenkins builds all jobs when user intends for one

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None

      The issue is that Jenkins builds any job configured with a repo (using Multiple SCM Plugin) when that trigger event is confined to the BRANCH. The behavior should be that the SCM push event corresponds to the intended job that should be triggered AND ONLY that job. Instead, any job that was triggered/aborted in the quiet period gets triggered, so that it appears that all jobs for a repo are triggered.

      Set-up and reproduce:

      (This process assumes you've configured an OAUTH application secret, your Jenkins user to have GitHub access, Jenkins for GitHub webhook access, and vice versa)

      Set-up:

      -Create 2 GitHub repositories: buildtest and foo (initialized with README)
      -Configure their WebHook with http://your-jenkins-url:8080/github-webhook/, your secret text, etc.
      -Select Just the push event for the WebHook configuration
      -Add Jenkins user to Collaborators & Teams and give your Jenkins user Admin access to the buildtest1 and foo1 repositories
      -Create branches: buildtest with 1.1 and 5.0 branches, foo only has master
      -Create 4 Freestyle jobs with Git and Multiple SCM Plugin, selecting GitHub hook trigger for GITScm polling, configure 2 SCMs per job:
      +FOR ALL JOBS: Add SCM for foo and branch specifier: */master (branch specifier) for all jobs
      +1 job has second SCM as buildtest and branch specifier 1.1
      +1 job has second SCM as buildtest and branch specifier 5.0
      +1 job has second SCM as buildtest and branch specifier master
      +Configure the SCM to be git@whatever-your-github-is:your_org/

      {buildtest,foo}

      and with your GitHub key for the Jenkins user
      -Build all the jobs to confirm they will expectedly function (hey! we want a clean test case =] )
      -Start committing an update to buildtest:1.1, watch the build kick off, ABORT IT during its run. (Note: I used a quiet period of 120 seconds to aid the delay)
      -Repeat this for buildtest:5.0, buildtest:master, foo:master jobs
      -After all the jobs have been aborted, the state is now that all the jobs should trigger on a change to their branch. And if a change is done against foo, that should kick off ALL the jobs.
      -Commit a change to buildtest:5.0, and watch all the builds get kicked off.

      This state is typical - we have builds that can take up to an hour for whatever reason, so we may abort the build, or the state of our SCM will change. However, if a dev pushes to a branch, ONLY that build should run, and not others that might have been aborted during the quiet period.

      Is this a problem with the use of the GitHub Plugin being expected to filter-by-branch and appropriately kick off the corresponding build (user error)? Or is it expected behavior that when a commit is done on a branch, that ONLY THAT BRANCH job is triggered?

      It seems with the SCM push even-trigger logic, that it pokes all of the builds for a git sha-Jenkins sha comparison (I'm just reading into this, would love for someone to explain more in detail) and if there's a mismatch, it'll build all of them. However, this is disconnected from the user event where only one branch in the repo needed a rebuild, not all the mismatched ones.

          [JENKINS-42161] Jenkins builds all jobs when user intends for one

          So the way that the GitHub plugin triggers builds is that it matches against repository URL only. The branch is not considered.

          So when a hook arrives, we get to this code which will trigger a polling of every job that has a git repository url matching and has the GitHub Trigger defined.

          Then via this method your job's multiple Git SCM instances will be discovered

          I suspect what is then happening is that the polling logic in multiple SCMs is picking up the aborted build differently from a regular build and consequently treating that as a revision change. To test some aspects of this theory I would recommend the following additional test cases:

          • Rather than having a quiet period of 120 seconds, use a sleep 120 so that you are aborting a build that has actually completed a checkout rather than aborting a build before it even starts. If you do not see the behaviour with this change then what that is saying is that you have a stacked up pending build and you are just awaiting polling to detect the build requirement
          • Set up two regular Jenkins job with just a single Git SCM set to build one branch only. One with a quiet period and one with a sleep 120. See if the behaviour is reproduced there. I expect the aborted during quiet period build will get kicked off by anything that triggers polling on the matching git repository. I expect the aborted during sleep build will not get kicked off.

          Hopefully we can identify where the issue is arising.

          NOTE: Ideally the GitHub plugin would be enhanced to use the SCMEvent API for triggering builds as that allows to match events against the branch that they affect, but in principle there is nothing wrong with the current GitHub plugin triggering behaviour... just that it creates unnecessary extra polling for jobs where it could be known that polling is not required

          Stephen Connolly added a comment - So the way that the GitHub plugin triggers builds is that it matches against repository URL only. The branch is not considered. So when a hook arrives, we get to this code which will trigger a polling of every job that has a git repository url matching and has the GitHub Trigger defined. Then via this method your job's multiple Git SCM instances will be discovered I suspect what is then happening is that the polling logic in multiple SCMs is picking up the aborted build differently from a regular build and consequently treating that as a revision change. To test some aspects of this theory I would recommend the following additional test cases: Rather than having a quiet period of 120 seconds, use a sleep 120 so that you are aborting a build that has actually completed a checkout rather than aborting a build before it even starts. If you do not see the behaviour with this change then what that is saying is that you have a stacked up pending build and you are just awaiting polling to detect the build requirement Set up two regular Jenkins job with just a single Git SCM set to build one branch only. One with a quiet period and one with a sleep 120 . See if the behaviour is reproduced there. I expect the aborted during quiet period build will get kicked off by anything that triggers polling on the matching git repository. I expect the aborted during sleep build will not get kicked off. Hopefully we can identify where the issue is arising. NOTE: Ideally the GitHub plugin would be enhanced to use the SCMEvent API for triggering builds as that allows to match events against the branch that they affect, but in principle there is nothing wrong with the current GitHub plugin triggering behaviour... just that it creates unnecessary extra polling for jobs where it could be known that polling is not required

          Shannon Carey added a comment - - edited

          An explanation of this behavior should be added to the inline help for the "Build when a change is pushed to GitHub" option. It's surprising that our "master" branch builds are getting triggered by pushes to non-master branches, and it seems like it previously used to work properly.

          Edit: It looks like it works right if you enable "Poll SCM" without putting a schedule in it... though the first push will usually result in a build even if it's not on the branch specified in the git settings because the poller hasn't seen the most recent commit before.

          Shannon Carey added a comment - - edited An explanation of this behavior should be added to the inline help for the "Build when a change is pushed to GitHub" option. It's surprising that our "master" branch builds are getting triggered by pushes to non-master branches, and it seems like it previously used to work properly. Edit: It looks like it works right if you enable "Poll SCM" without putting a schedule in it... though the first push will usually result in a build even if it's not on the branch specified in the git settings because the poller hasn't seen the most recent commit before.

          Is there a way to do not to build the jobs for the first time or at least to match the branch as well? I am having serious issues with this since first time all jobs are been triggered and in our infrastructure the workspaces are allocated in a Docker container which this can be deleted once the container does not exists the rest which is critical is in AWS EBS (Jenkins configuration, jobs definition and so on).

          Our jobs are triggered using regular expression for the tags which starts by ":" , right? is quite complex our process but fully automatic the CI/CD until production but I found weakness in this plugin by not matching correctly the branch or what ever pattern we set in addition to the GIT URL.

          Daniel Alejandro Hernández added a comment - Is there a way to do not to build the jobs for the first time or at least to match the branch as well? I am having serious issues with this since first time all jobs are been triggered and in our infrastructure the workspaces are allocated in a Docker container which this can be deleted once the container does not exists the rest which is critical is in AWS EBS (Jenkins configuration, jobs definition and so on). Our jobs are triggered using regular expression for the tags which starts by ":" , right? is quite complex our process but fully automatic the CI/CD until production but I found weakness in this plugin by not matching correctly the branch or what ever pattern we set in addition to the GIT URL.

            Unassigned Unassigned
            f00ser Neuronal T
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: