Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37588

Multibranch pipeline does not allow quiet time or throttle

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • branch-api-plugin
    • None
    • Running jenkins 2.15 with pipeline-multibranch 2.8.
    • branch-api-plugin-2.6.0

      Currently, for an individual pipeline job I can set a quiet time to avoid one-for-one commit-to-build ratio when there are many commits in a short time. I like to use the multibranch plugin to allow branch automation, but there does not appear to be a way for a multibranch config (either the meta-job itself or from within a Jenkinsfile) to define a quiet time or some other ability to throttle job runs.

          [JENKINS-37588] Multibranch pipeline does not allow quiet time or throttle

          Oleg Nenashev added a comment -

          Changed the component.
          Actually there are many throttling options in Pipeline, but Quite Period is definitely not one of them

          Oleg Nenashev added a comment - Changed the component. Actually there are many throttling options in Pipeline, but Quite Period is definitely not one of them

          Brian Sipos added a comment -

          My issue is with the multibranch pipeline though. I don't see any way to configure the multibranch job which will make use of job throttling on the auto-generated branch jobs.

          I see an option to "suppress automatic SCM triggering" but then the jobs will always need to be manually triggered. I don't see a way to have the auto-created per-branch job contain a "build periodically" or similar parameter. I just have a read-only view on the job configuration with unchecked options for "Build Triggers" section.

          Brian Sipos added a comment - My issue is with the multibranch pipeline though. I don't see any way to configure the multibranch job which will make use of job throttling on the auto-generated branch jobs. I see an option to "suppress automatic SCM triggering" but then the jobs will always need to be manually triggered. I don't see a way to have the auto-created per-branch job contain a "build periodically" or similar parameter. I just have a read-only view on the job configuration with unchecked options for "Build Triggers" section.

          Andras Gaal added a comment -

          Oleg could please share some documentation regarding the throttling options of multipipeline?

          Andras Gaal added a comment - Oleg could please share some documentation regarding the throttling options of multipipeline?

          Jesse Glick added a comment -

          try: Pipeline Syntax » properties » Throttle builds (have not personally ever used it)

          Jesse Glick added a comment - try: Pipeline Syntax » properties » Throttle builds (have not personally ever used it)

          Andras Gaal added a comment -

          Thanks Jesse,

          I missed "Pipeline Syntax » properties" somehow. I will try to test "Throttle builds".

          However not all the options are listed there. For example I could set up 'upstream' project. With this code:

          properties([
            pipelineTriggers([
              upstream(
                threshold: hudson.model.Result.SUCCESS,
                upstreamProjects: 'jobB'
              )
            ])
          ])
          

          I got this code from a comment somewhere.

          Is there a documentation somewhere? Or where is the related code where I can find it out?

          Thanks,
          Andras

          Andras Gaal added a comment - Thanks Jesse, I missed "Pipeline Syntax » properties" somehow. I will try to test "Throttle builds". However not all the options are listed there. For example I could set up 'upstream' project. With this code: properties([ pipelineTriggers([ upstream( threshold: hudson.model.Result.SUCCESS, upstreamProjects: 'jobB' ) ]) ]) I got this code from a comment somewhere. Is there a documentation somewhere? Or where is the related code where I can find it out? Thanks, Andras

          Jesse Glick added a comment -

          not all the options are listed there

          I am not sure I understand what options you are expecting to see and what you actually saw. Sounds like an independent bug at any rate.

          Documentation is sparse to say the least.

          The code is in workflow-job-plugin but there is not much to it: basically, any JobProperty compatible with Pipeline can be specified, and should appear as an option under properties. In turn the trigger property should offer any Trigger compatible with Pipeline. upstream refers to ReverseBuildTrigger in Jenkins core (for 1.x users the syntax is different).

          Jesse Glick added a comment - not all the options are listed there I am not sure I understand what options you are expecting to see and what you actually saw. Sounds like an independent bug at any rate. Documentation is sparse to say the least. The code is in workflow-job-plugin but there is not much to it: basically, any JobProperty compatible with Pipeline can be specified, and should appear as an option under properties . In turn the trigger property should offer any Trigger compatible with Pipeline. upstream refers to ReverseBuildTrigger in Jenkins core (for 1.x users the syntax is different).

          Pete Hale added a comment -

          I am trying to figure out how to apply quietPeriod to our multibranch pipeline jobs. As describe above, it is supposed to be possible to do this using properties DSL.

          If I follow the source for properties DSL (JobPropertyStep), it takes a list of JobProperty instances and applies those to the branch job.

          The quiet period "property" is defined in two ways; a global quiet period (GlobalQuietPeriodConfiguration) and also on the job (AbstractProject). 

          AFAIK, there does not seem to be a path to the property quietPeriod via properties DSL.

          Has anyone figured out how to set quiet period on a multibranch job?

          Pete Hale added a comment - I am trying to figure out how to apply quietPeriod to our multibranch pipeline jobs. As describe above, it is supposed to be possible to do this using properties DSL. If I follow the source for properties DSL (JobPropertyStep), it takes a list of JobProperty instances and applies those to the branch job. The quiet period "property" is defined in two ways; a global quiet period (GlobalQuietPeriodConfiguration) and also on the job (AbstractProject).  AFAIK, there does not seem to be a path to the property quietPeriod via properties DSL. Has anyone figured out how to set quiet period on a multibranch job?

          Jesse Glick added a comment -

          AFAIK there is no support for quiet periods in branch projects. Generally speaking, this system was introduced in the early days of Hudson to work around limitations in CVS.

          Jesse Glick added a comment - AFAIK there is no support for quiet periods in branch projects. Generally speaking, this system was introduced in the early days of Hudson to work around limitations in CVS.

          GitHub is sometimes sending multiple webhook notifications for a single push within a second, and causing multiple jobs to spawn for us in the multibranch pipeline. Having a quiet period would have been a nice workaround.

          Richard Bullington-McGuire added a comment - GitHub is sometimes sending multiple webhook notifications for a single push within a second, and causing multiple jobs to spawn for us in the multibranch pipeline. Having a quiet period would have been a nice workaround.

          Adam Krapfl added a comment -

          I feel there are legitimate use cases for this.  jglick can you explain why this is no longer needed?  I'm struggling to come up with alternative approaches in a Jenkinsfile without a quiet period.

          Adam Krapfl added a comment - I feel there are legitimate use cases for this.  jglick can you explain why this is no longer needed?  I'm struggling to come up with alternative approaches in a Jenkinsfile without a quiet period.

          Felix Drueke added a comment - - edited

          Kohsuke described the quiet feature in this (old) article.

          I still find that it is valid and needed (especially for longer builds).

          Felix Drueke added a comment - - edited Kohsuke described the quiet feature in this (old) article . I still find that it is valid and needed (especially for longer builds).

          Semyon Levin added a comment -

          We have quite complex dependencies and really need throttling options in Jenkinsfile...

          Semyon Levin added a comment - We have quite complex dependencies and really need throttling options in Jenkinsfile...

          We have one project which needs a large quiet time while multiple commits are manually handled. But all the other projects have simple atomic commits. So we're in this battle of setting the global quiet time to 6 minutes vs 1 minute.

          So this feature would help. I'd run two Jenkins masters to solve the issue, but that hits other snags. We have several Jenkins worker machines that are needed for all these projects and the workers can't be shared among multiple masters.

           

           

          John Muczynski added a comment - We have one project which needs a large quiet time while multiple commits are manually handled. But all the other projects have simple atomic commits. So we're in this battle of setting the global quiet time to 6 minutes vs 1 minute. So this feature would help. I'd run two Jenkins masters to solve the issue, but that hits other snags. We have several Jenkins worker machines that are needed for all these projects and the workers can't be shared among multiple masters.    

          Jesse Glick added a comment -

          Was already implemented for Declarative Pipeline (JENKINS-51227), but is apparently not available in Scripted.

          Jesse Glick added a comment - Was already implemented for Declarative Pipeline ( JENKINS-51227 ), but is apparently not available in Scripted.

          Jesse Glick added a comment -

          Rate limiting is possible, though, as far as I know.

          Jesse Glick added a comment - Rate limiting is possible, though, as far as I know .

          We do already use rate limiting (1 build at a time).

          Well, the Declarative Pipeline support might work for me. I'll have to get a new Jenkins version ...

          We're using multibranch pipelines.

           

          pipeline {
              agent any
              options {| | | | quietPeriod(15)| | | | }

          John Muczynski added a comment - We do already use rate limiting (1 build at a time). Well, the Declarative Pipeline support might work for me. I'll have to get a new Jenkins version ... We're using multibranch pipelines.   pipeline {     agent any     options {| | | | quietPeriod(15)| | | | }

          Ah, okay, as Jesse commented on the other issue: "Nothing to do with Declarative vs. Scripted, only multibranch vs. standalone."

           

          options {
              buildDiscarder(logRotator(numToKeepStr: '90', artifactNumToKeepStr: '5'))
              retry(1)
              skipDefaultCheckout()
              disableConcurrentBuilds()
              quietPeriod(600)
          }

           
          org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
          WorkflowScript: 350: Invalid option type "quietPeriod". Valid option types: [buildDiscarder, catchError, disableConcurrentBuilds, overrideIndexTriggers, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, timestamps, waitUntil, withContext, withCredentials, withEnv, ws] @ line 350, column 9.
          quietPeriod(600)
          ^

          General error during semantic analysis: There's no @DataBoundConstructor on any constructor of class jenkins.model.GlobalQuietPeriodConfiguration

          org.kohsuke.stapler.NoStaplerConstructorException: There's no @DataBoundConstructor on any constructor of class jenkins.model.GlobalQuietPeriodConfiguration
          at org.kohsuke.stapler.ClassDescriptor.loadConstructorParamNames(ClassDescriptor.java:265)
          at org.jenkinsci.plugins.structs.describable.DescribableModel.<init>(DescribableModel.java:122)

          John Muczynski added a comment - Ah, okay, as Jesse commented on the other issue: "Nothing to do with Declarative vs. Scripted, only multibranch vs. standalone."   options { buildDiscarder(logRotator(numToKeepStr: '90', artifactNumToKeepStr: '5')) retry(1) skipDefaultCheckout() disableConcurrentBuilds() quietPeriod(600) }   org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: WorkflowScript: 350: Invalid option type "quietPeriod". Valid option types: [buildDiscarder, catchError, disableConcurrentBuilds, overrideIndexTriggers, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, timestamps, waitUntil, withContext, withCredentials, withEnv, ws] @ line 350, column 9. quietPeriod(600) ^ General error during semantic analysis: There's no @DataBoundConstructor on any constructor of class jenkins.model.GlobalQuietPeriodConfiguration org.kohsuke.stapler.NoStaplerConstructorException: There's no @DataBoundConstructor on any constructor of class jenkins.model.GlobalQuietPeriodConfiguration at org.kohsuke.stapler.ClassDescriptor.loadConstructorParamNames(ClassDescriptor.java:265) at org.jenkinsci.plugins.structs.describable.DescribableModel.<init>(DescribableModel.java:122)

          Ooph, my bad.  I didn't make sure to update Declarative to version 1.3.2

          It looks like it is accepting the quietPeriod now, after updating blue ocean.

          John Muczynski added a comment - Ooph, my bad.  I didn't make sure to update Declarative to version 1.3.2 It looks like it is accepting the quietPeriod now, after updating blue ocean.

          Yan Corneille added a comment -

          As noticed by Jesse Glick comment-350957 this does not seem to be available in Scripted pipelines. Are there any plans for that ?

          Yan Corneille added a comment - As noticed by Jesse Glick  comment-350957 this does not seem to be available in Scripted pipelines. Are there any plans for that ?

          Oliver Mueller added a comment - - edited

          We are using Jenkins 2.195, Multibranch 2.21 and Declarative version 1.3.9.

          Option quietPeriod seems to be accepted, but it does NOT seem to work.

          We are using a declarative multibranch pipeline with these options: 

          pipeline {
            agent any
            options {
              quietPeriod 1200
              timeout(time: 4, unit: 'HOURS')
              disableConcurrentBuilds()
            }
          ...

          The pipeline will get executed immediately.

          Oliver Mueller added a comment - - edited We are using Jenkins 2.195, Multibranch 2.21 and Declarative version 1.3.9. Option quietPeriod seems to be accepted, but it does NOT seem to work. We are using a declarative multibranch pipeline with these options:  pipeline { agent any options { quietPeriod 1200 timeout(time: 4, unit: 'HOURS' ) disableConcurrentBuilds() } ... The pipeline will get executed immediately.

          Javi Martin added a comment -

          we are facing the same problem (Jenkins 2.190.1, Multibranch 2.21 and Declarative 1.3.9).

          The quietPeriod option seems to be working when calling the job from another job by sending the branch name in the URL "/projects/Build-MultibranchProject/master", but it does not work when the webhook calls the Multibranch project. 

          Multibranch plugin seems to be executing the required branch job doing a "Build now" and it bypasses the quietPeriod options.

           

           

          Javi Martin added a comment - we are facing the same problem (Jenkins 2.190.1, Multibranch 2.21 and Declarative 1.3.9). The quietPeriod option seems to be working when calling the job from another job by sending the branch name in the URL "/projects/Build-MultibranchProject/master", but it does not work when the webhook calls the Multibranch project.  Multibranch plugin seems to be executing the required branch job doing a "Build now" and it bypasses the quietPeriod options.    

          Successfully tested with Declarative (Jenkins 2.200) but had no luck to set it in Scripted fashion. As someone stated before, there's no way neither to get it generated with pipeline-syntax generator, nor trying something like this:

          properties([
              pipelineTriggers([
          				[
          					$class: 'HudsonStartupTrigger',
          					quietPeriod: '2'
          				]
          			])
          ])
          

          will produce:

           no known implementation of class hudson.triggers.Trigger is named HudsonStartupTrigger
          

          vincenzo cerbone added a comment - Successfully tested with Declarative (Jenkins 2.200) but had no luck to set it in Scripted fashion. As someone stated before, there's no way neither to get it generated with pipeline-syntax generator, nor trying something like this: properties([ pipelineTriggers([ [ $class: 'HudsonStartupTrigger' , quietPeriod: '2' ] ]) ]) will produce: no known implementation of class hudson.triggers.Trigger is named HudsonStartupTrigger

          Dibakar Aditya added a comment - - edited

          vincenzo_cerbone, did you see any success using this in a multibranch pipeline project? In our setup, the Declarative syntax with standalone pipeline project works as intended, but not in multibranch yet. This functionality is much needed, especially with the maven release plugin.

          Dibakar Aditya added a comment - - edited vincenzo_cerbone , did you see any success using this in a multibranch pipeline project? In our setup, the Declarative syntax with standalone pipeline project works as intended, but not in multibranch yet. This functionality is much needed, especially with the maven release plugin.

          addib I tested it (Declarative) in multibranch as ogmueller reported and I had luck to see it working. Unfortunately I needed it to work in scripted mode because JTE (Jenking Template Engine) does not support Declarative.
          I ended up using this instruction and it works!

          }}{{currentBuild.rawBuild.getParent().setQuietPeriod(10){{}}

          About maven-release-plugin, I prefer not to use it anymor. Now I'm using :
          https://github.com/aleksandr-m/gitflow-maven-plugin{{}}

          vincenzo cerbone added a comment - addib I tested it (Declarative) in multibranch as ogmueller reported and I had luck to see it working. Unfortunately I needed it to work in scripted mode because JTE (Jenking Template Engine) does not support Declarative. I ended up using this instruction and it works! }}{{currentBuild.rawBuild.getParent().setQuietPeriod(10) {{}} About maven-release-plugin, I prefer not to use it anymor. Now I'm using : https://github.com/aleksandr-m/gitflow-maven-plugin {{}}

          Dibakar Aditya added a comment - - edited

          vincenzo_cerbone, I have tried both the Declarative and Scripted approach that you suggested on Jenkins 2.204.1 with the latest version of plugins. Though I can see the quiet period being set in the job configuration page of the branch, multibranch still chooses to ignore it.

          Dibakar Aditya added a comment - - edited vincenzo_cerbone , I have tried both the Declarative and Scripted approach that you suggested on Jenkins 2.204.1 with the latest version of plugins. Though I can see the quiet period being set in the job configuration page of the branch, multibranch still chooses to ignore it.

          Joe Cavanaugh added a comment - - edited

          I know I got this working when I was using Polling on the Multibranch pipeline. The quietPeriod was respected... however, when I moved to webhooks to kick off the pipeline through Bitbucket the quietPeriod(60) is no longer respected. I have gotten multiple complaints from Developers on this one as well.

              options {
                   buildDiscarder logRotator(numToKeepStr: '5')
                   disableConcurrentBuilds()
                   disableResume()
                   quietPeriod 60
                   timeout(time: 3, unit: 'HOURS')
                   skipStagesAfterUnstable()
                   parallelsAlwaysFailFast()
               }

          Joe Cavanaugh added a comment - - edited I know I got this working when I was using Polling on the Multibranch pipeline. The quietPeriod was respected... however, when I moved to webhooks to kick off the pipeline through Bitbucket the quietPeriod(60) is no longer respected. I have gotten multiple complaints from Developers on this one as well.     options {         buildDiscarder logRotator(numToKeepStr:  '5' )         disableConcurrentBuilds()         disableResume()         quietPeriod 60         timeout(time: 3, unit:  'HOURS' )         skipStagesAfterUnstable()         parallelsAlwaysFailFast()     }

          This would be useful in GitHub if one has a GitHub Action that pushes a commit to one's PR – to prevent the the PR from building prior to the GH Actions adding the new commit.

           

          Brian J Murrell added a comment - This would be useful in GitHub if one has a GitHub Action that pushes a commit to one's PR – to prevent the the PR from building prior to the GH Actions adding the new commit.  

          Chris Hemp added a comment -

          ^ Related to Github - With dependabot being enabled for a repo to update dependencies, it can be common to have a few PRs merge at once.  This quiet period would be helpful to have in those situations. 

          Chris Hemp added a comment - ^ Related to Github - With dependabot being enabled for a repo to update dependencies, it can be common to have a few PRs merge at once.  This quiet period would be helpful to have in those situations. 

          Deiwin Sarjas added a comment -

          Would it be possible possible to configure this once for the entire GitHub Organization? I read the thread and the PRs but I don't see how one could do that.

          Deiwin Sarjas added a comment - Would it be possible possible to configure this once for the entire GitHub Organization ? I read the thread and the PRs but I don't see how one could do that.

          Yogya Raj added a comment -

          Would it be possible to trigger one to one build for every commit with GitHub branch source plugin ,as i can see its triggering only one build for two PR merge

          Yogya Raj added a comment - Would it be possible to trigger one to one build for every commit with GitHub branch source plugin ,as i can see its triggering only one build for two PR merge

          Lenny added a comment -

          Quiet period does not work for ORG pipeline projects

          Lenny added a comment - Quiet period does not work for ORG pipeline projects

          Erich Mauerböck added a comment - - edited

          This still does not work with me, latest LTS jenkins 2.375.2 & branch-api-plugin 2.1051.v9985666b_f6cc

          I have the following in my declarative multibranch pipeline:

          options {
              quietPeriod 5
          }

          Unfortunately when viewing the configuration, the option is not checked, and obviously does not work either, the job is started immediately.

          yes, the job has run numerous times since the pipeline change.

          Btw. I need this only for a job triggered by some upstream job, like so:

          triggers {
              upstream 'Build Parent/develop'
          } 

          Erich Mauerböck added a comment - - edited This still does not work with me, latest LTS jenkins 2.375.2 & branch-api-plugin 2.1051.v9985666b_f6cc I have the following in my declarative multibranch pipeline: options { quietPeriod 5 } Unfortunately when viewing the configuration, the option is not checked, and obviously does not work either, the job is started immediately. yes, the job has run numerous times since the pipeline change. Btw. I need this only for a job triggered by some upstream job, like so: triggers { upstream 'Build Parent/develop' }

          Dennis added a comment -

          Thank you very much. Your post provided me with a wealth of information, allowing me to broaden my horizons. 

          redactle unlimited

           

          Dennis added a comment - Thank you very much. Your post provided me with a wealth of information, allowing me to broaden my horizons.  redactle unlimited  

            Unassigned Unassigned
            bsipos Brian Sipos
            Votes:
            51 Vote for this issue
            Watchers:
            59 Start watching this issue

              Created:
              Updated: