• Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin
    • None

      Provide a job option that allows jobs found from polling to be added to the queue even when there is another job running.

      Current Behavior:
      Poll finds job and starts it.
      If job is still running the next time a poll occurs no jobs are added to the queue and the following message is reported: _ "Build in progress, polling delayed."_

      Desired Behavior:
      Poll finds job and starts it.
      If job is still running the next time a poll occurs a job is queued.

      Current Workaround:

      • Shorter polling periods so there is less delay after the previous job completes.
      • Set 'Execute concurrent builds if necessary' and use Perforce change-submit triggers to trigger Perforce job.

          [JENKINS-41849] Optionally queue polled jobs

          p4karl With long running pipelines in docker containers, for us this is actually more than a "minor" issue. I would really like to get this done I order to get triggered builds per change.

          Staffan Forsell added a comment - p4karl With long running pipelines in docker containers, for us this is actually more than a "minor" issue. I would really like to get this done I order to get triggered builds per change.

          Karl Wirth added a comment -

          Hi fkykko  I've increased the priority on this. We are in the middle of implementing a couple of large features but will add this to the backlog of things to be looked into.

          Karl Wirth added a comment - Hi fkykko   I've increased the priority on this. We are in the middle of implementing a couple of large features but will add this to the backlog of things to be looked into.

          Staffan Forsell added a comment - - edited

          p4karl Ok, thanks. Also note that the "Workaround" does not work for pipeline jobs since it does not have the checkbox "Execute concurrent builds if necessary"

          Staffan Forsell added a comment - - edited p4karl Ok, thanks. Also note that the "Workaround" does not work for pipeline jobs since it does not have the checkbox "Execute concurrent builds if necessary"

          p4karl or p4paul: Any news on this? This is blocking having our long running pipelines execute for each submitted changelist. This could simplify finding the exact changelist that broke the build.

          Staffan Forsell added a comment - p4karl or p4paul : Any news on this? This is blocking having our long running pipelines execute for each submitted changelist. This could simplify finding the exact changelist that broke the build.

          Karl Wirth added a comment -

          Have discussed with Paul and in the current Jenkins framework we have some ideas of how this can be implemented. We'll take a look.

          Karl Wirth added a comment - Have discussed with Paul and in the current Jenkins framework we have some ideas of how this can be implemented. We'll take a look.

          Great! This would reduce the need for failFast in our parallel stages and give us a better indication of which commit broke the build. Awesome if it can be resolved.

          I realize that there is some complications if multiple syncs are used (We don't typically), but I guess that other scm plugins must have similar problems. 

          Staffan Forsell added a comment - Great! This would reduce the need for failFast in our parallel stages and give us a better indication of which commit broke the build. Awesome if it can be resolved. I realize that there is some complications if multiple syncs are used (We don't typically), but I guess that other scm plugins must have similar problems. 

          Robby Pocase added a comment -

          p4paul p4karl I was looking at getting this implemented in JENKINS-34052 and didn't realize this ticket existed. What's the use case for this being optional? It seems like you're either concurrent or not, so only polling if concurrent builds are enabled would be sufficient. I almost have a fix for the other description of this problem assuming supporting concurrent is the new functionality.

          Robby Pocase added a comment - p4paul p4karl I was looking at getting this implemented in JENKINS-34052 and didn't realize this ticket existed. What's the use case for this being optional? It seems like you're either concurrent or not, so only polling if concurrent builds are enabled would be sufficient. I almost have a fix for the other description of this problem assuming supporting concurrent is the new functionality.

          Karl Wirth added a comment -

          rpocase First thanks for looking at this.

          Not in front of a Jenkins system at the moment. What happens to the job if concurrent is disabled. Does the next job coming in get queued or thrown away? I think I marked this as optional because I want the following options to be available:

          (1) I want to create a build every time I poll. These jobs should be queued to minimise resource usage.

          (2) I want a new build to be available. This is triggered by a poll, but if a build is already running then ignore further build requests till it stops.

          (3) I want to create a build for every poll. I have lots of build capacity so these builds can be concurrent.

           

          The other reason for making it optional is there are probably users that already have worked around this functionallity. Updating to a version of the plugin where this behavior changes may break the existing build functionallity (just my 2c).

          Karl Wirth added a comment - rpocase First thanks for looking at this. Not in front of a Jenkins system at the moment. What happens to the job if concurrent is disabled. Does the next job coming in get queued or thrown away? I think I marked this as optional because I want the following options to be available: (1) I want to create a build every time I poll. These jobs should be queued to minimise resource usage. (2) I want a new build to be available. This is triggered by a poll, but if a build is already running then ignore further build requests till it stops. (3) I want to create a build for every poll. I have lots of build capacity so these builds can be concurrent.   The other reason for making it optional is there are probably users that already have worked around this functionallity. Updating to a version of the plugin where this behavior changes may break the existing build functionallity (just my 2c).

          Robby Pocase added a comment - - edited

          (1) I want to create a build every time I poll. These jobs should be queued to minimise resource usage.

          This option isn't possible as described (with freestyle). The first "poll with change" will get queued, but will won't start until after the currently running build is finished. For pipeline, you could handle this using throttle concurrent builds plugin.

          (2) I want a new build to be available. This is triggered by a poll, but if a build is already running then ignore further build requests till it stops.

          To be as close to in line with current functionality, I believe using a condition of "if concurrent is disabled and currently building, don't bother polling". For freestyle users this doesn't change anything, but pipeline has a property to disableConcurrentBuilds.

          (3) I want to create a build for every poll. I have lots of build capacity so these builds can be concurrent.

          This would be the new default .

          I'd personally prefer breaking existing functionality in favor of "principal of least surprise" now that pipeline (and therefore concurrent builds) is the recommended default for Jenkins jobs.

          Robby Pocase added a comment - - edited (1) I want to create a build every time I poll. These jobs should be queued to minimise resource usage. This option isn't possible as described (with freestyle). The first "poll with change" will get queued, but will won't start until after the currently running build is finished. For pipeline, you could handle this using throttle concurrent builds plugin. (2) I want a new build to be available. This is triggered by a poll, but if a build is already running then ignore further build requests till it stops. To be as close to in line with current functionality, I believe using a condition of "if concurrent is disabled and currently building, don't bother polling". For freestyle users this doesn't change anything, but pipeline has a property to disableConcurrentBuilds. (3) I want to create a build for every poll. I have lots of build capacity so these builds can be concurrent. This would be the new default . I'd personally prefer breaking existing functionality in favor of "principal of least surprise" now that pipeline (and therefore concurrent builds) is the recommended default for Jenkins jobs.

            Unassigned Unassigned
            p4karl Karl Wirth
            Votes:
            6 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: