Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26887

git polling/web hooks will not start concurrent builds

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core, git-plugin

      I need a web hook to be able to kick off concurrent builds. Whereas right now if a commit comes in to a job that is already building that commit will just be ignored and will never be built even if concurrent builds is turned on.

      In addition to web hooks not working when a job is already running. Polling will not even be attempted if a job is currently running. Which makes since for most builds, but not for builds with concurrent builds enabled.

          [JENKINS-26887] git polling/web hooks will not start concurrent builds

          It will execute concurrent builds only if they are kicked off manually. If a build is already running and an new commit comes in to the repository and Jenkins gets notified via web hook Jenkins will just ignore it and continue running the current build instead of starting a concurrent build to build the new commit.

          Jeremiah Norris added a comment - It will execute concurrent builds only if they are kicked off manually. If a build is already running and an new commit comes in to the repository and Jenkins gets notified via web hook Jenkins will just ignore it and continue running the current build instead of starting a concurrent build to build the new commit.

          Also it looks like polling won't even be attempted if a build is already running. It doesn't take concurrent builds in to consideration at all.

          Jeremiah Norris added a comment - Also it looks like polling won't even be attempted if a build is already running. It doesn't take concurrent builds in to consideration at all.

          Jan Klass added a comment -

          I still don’t see why a polling workspace would be necessary.
          Most of the use-case described here seems like a bug report rather than an improvement.

          What are the actions to be taken that would make a workspace or repository clone necessary?

          Jan Klass added a comment - I still don’t see why a polling workspace would be necessary. Most of the use-case described here seems like a bug report rather than an improvement. What are the actions to be taken that would make a workspace or repository clone necessary?

          Perhaps you are right I don't think that we need to have a separate workspace for polling to resolve this issue. I've updated this issue and converted it to a bug.

          Jeremiah Norris added a comment - Perhaps you are right I don't think that we need to have a separate workspace for polling to resolve this issue. I've updated this issue and converted it to a bug.

          I'm having the same problem, and it looks like a regression (It used work well for me in the past till the recent upgrade of jenkins ? some plugins ?)

          I've opened an issue for github plugin component, thinking the problem is there.

          https://issues.jenkins-ci.org/browse/JENKINS-47136

          Lior Goikhburg added a comment - I'm having the same problem, and it looks like a regression (It used work well for me in the past till the recent upgrade of jenkins ? some plugins ?) I've opened an issue for github plugin component, thinking the problem is there. https://issues.jenkins-ci.org/browse/JENKINS-47136

          I've found a workaround that works for me:

          0. Use env injector plugin.

          1. Select "Prepare an environment for the run"

          2. In "Properties Content" enter: rev=

          3. In SCM configuration, in "Branches to build" set "Branch Specifier" to: $rev

           

          I'm not sure how, but this fixes the problem, builds triggered by hook now run concurrently. Maybe a bug in SCM component ?

          Attaching a config of working demo job. config.xml

           

           

           

          Lior Goikhburg added a comment - I've found a workaround that works for me: 0. Use env injector plugin. 1. Select "Prepare an environment for the run" 2. In "Properties Content" enter: rev= 3. In SCM configuration, in "Branches to build" set "Branch Specifier" to: $rev   I'm not sure how, but this fixes the problem, builds triggered by hook now run concurrently. Maybe a bug in SCM component ? Attaching a config of working demo job.  config.xml      

          Mark Waite added a comment -

          zerthimon I believe you're describing something different than this bug is requesting.  At its core, the git plugin will block polling of a freestyle job while an instance of the job is running, even if "allow concurrent builds" is enabled.  It has behaved that way for many years.

          Can you describe in more detail the specific case you're using?  For example, are you using a freestyle job or some other type of job?  Are you trying to concurrently build a single project (job), or is it multiple projects?

          Mark Waite added a comment - zerthimon I believe you're describing something different than this bug is requesting.  At its core, the git plugin will block polling of a freestyle job while an instance of the job is running, even if "allow concurrent builds" is enabled.  It has behaved that way for many years. Can you describe in more detail the specific case you're using?  For example, are you using a freestyle job or some other type of job?  Are you trying to concurrently build a single project (job), or is it multiple projects?

          markewaite The use case is that I have a really "busy" repository on github and only one job (freestyle project) that has "allow concurrent builds" enabled. I would like it to build every single commit (regardless of it's branch) immediately (I don't want the job to wait for the previous build (instance of the job) to finish, as the build takes a long time).

          The job (config) I've attached is similar to the job I have and it works exactly as I need e.g: spawns additional job instance for every commit without wait.

          Lior Goikhburg added a comment - markewaite  The use case is that I have a really "busy" repository on github and only one job (freestyle project) that has "allow concurrent builds" enabled. I would like it to build every single commit (regardless of it's branch) immediately (I don't want the job to wait for the previous build (instance of the job) to finish, as the build takes a long time). The job (config) I've attached is similar to the job I have and it works exactly as I need e.g: spawns additional job instance for every commit without wait.

          Lior Goikhburg added a comment - - edited

          jjno91

          if a commit comes in to a job that is already building that commit will just be ignored and will never be built even if concurrent builds is turned on.

          What I observed is little different: if a commit comes in to a job that is already building, that commit will wait until the previous commit finishes building and once its finishes then it's built. Can you confirm that in your case it isn't so ?

          Lior Goikhburg added a comment - - edited jjno91 if a commit comes in to a job that is already building that commit will just be ignored and will never be built even if concurrent builds is turned on. What I observed is little different: if a commit comes in to a job that is already building, that commit will wait until the previous commit finishes building and once its finishes then it's built. Can you confirm that in your case it isn't so ?

          zerthimon there have been many changes since I originally submitted this issue. Currently I believe webhooks now work properly across the board as I have concurrent builds build multiple commits in parallel now.

          On polling you can still have missed commits, but it is much more rare now. The last testing I did the only scenario where commits get missed is if while a build is running multiple commits happen on the repo. In this scenario only the most recent commit will be built and the additional commits that happened between the building commit and the latest commit will be skipped.

          Additionally concurrent builds will still never be triggered by polling alone because polling does not run while a build is running. To get the behavior you will have to use a web hook.

          Jeremiah Norris added a comment - zerthimon  there have been many changes since I originally submitted this issue. Currently I believe webhooks now work properly across the board as I have concurrent builds build multiple commits in parallel now. On polling you can still have missed commits, but it is much more rare now. The last testing I did the only scenario where commits get missed is if while a build is running multiple commits happen on the repo. In this scenario only the most recent commit will be built and the additional commits that happened between the building commit and the latest commit will be skipped. Additionally concurrent builds will still never be triggered by polling alone because polling does not run while a build is running. To get the behavior you will have to use a web hook.

            Unassigned Unassigned
            jjno91 Jeremiah Norris
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: