[JENKINS-16384] Blocking on downstream projects

Alex Vesely created issue - 2013-01-17 06:03

cjo9900 added a comment - 2013-01-17 11:42

Can you add the config.xml files from the 4 jobs, in particular the <publishers> ... </publishers> section.

And as the jobs are separate and there is no actual lick between them other than the upstream/downstream relationship there is no cause to prevent the jobs from running at the same time unless that advanced option is enabled.

See Advanced section in project A config and enable the block when downstream is building item and see if that fixes the problem.

cjo9900 added a comment - 2013-01-17 11:42 Can you add the config.xml files from the 4 jobs, in particular the <publishers> ... </publishers> section. And as the jobs are separate and there is no actual lick between them other than the upstream/downstream relationship there is no cause to prevent the jobs from running at the same time unless that advanced option is enabled. See Advanced section in project A config and enable the block when downstream is building item and see if that fixes the problem.

Alex Vesely added a comment - 2013-01-18 11:26 - edited

Yes, I forgot to mention that "Block when downstream is building" is of course enabled in A.

Posting <publishers> for all 5 jobs:

job_A:

  <publishers>
    <hudson.plugins.parameterizedtrigger.BuildTrigger>
      <configs>
        <hudson.plugins.parameterizedtrigger.BuildTriggerConfig>
          <configs>
            <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
          </configs>
          <projects>job_B</projects>
          <condition>SUCCESS</condition>
          <triggerWithNoParameters>false</triggerWithNoParameters>
        </hudson.plugins.parameterizedtrigger.BuildTriggerConfig>
      </configs>
    </hudson.plugins.parameterizedtrigger.BuildTrigger>
    <hudson.plugins.emailext.ExtendedEmailPublisher>
      <recipientList>$DEFAULT_RECIPIENTS</recipientList>
      <configuredTriggers>
        <hudson.plugins.emailext.plugins.trigger.FailureTrigger>
          <email>
            <recipientList></recipientList>
            <subject>$PROJECT_DEFAULT_SUBJECT</subject>
            <body>$PROJECT_DEFAULT_CONTENT</body>
            <sendToDevelopers>true</sendToDevelopers>
            <sendToRequester>false</sendToRequester>
            <includeCulprits>false</includeCulprits>
            <sendToRecipientList>true</sendToRecipientList>
          </email>
        </hudson.plugins.emailext.plugins.trigger.FailureTrigger>
      </configuredTriggers>
      <contentType>default</contentType>
      <defaultSubject>$DEFAULT_SUBJECT</defaultSubject>
      <defaultContent>$DEFAULT_CONTENT</defaultContent>
      <attachmentsPattern></attachmentsPattern>
      <presendScript></presendScript>
    </hudson.plugins.emailext.ExtendedEmailPublisher>
  </publishers>

job_B:

  <publishers>
    <hudson.plugins.logparser.LogParserPublisher>
      <unstableOnWarning>false</unstableOnWarning>
      <failBuildOnError>true</failBuildOnError>
      <parsingRulesPath>make.error</parsingRulesPath>
    </hudson.plugins.logparser.LogParserPublisher>
    <hudson.plugins.parameterizedtrigger.BuildTrigger>
      <configs>
        <hudson.plugins.parameterizedtrigger.BuildTriggerConfig>
          <configs>
            <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
          </configs>
          <projects>job_C, job_D</projects>
          <condition>SUCCESS</condition>
          <triggerWithNoParameters>false</triggerWithNoParameters>
        </hudson.plugins.parameterizedtrigger.BuildTriggerConfig>
      </configs>
    </hudson.plugins.parameterizedtrigger.BuildTrigger>
    <join.JoinTrigger>
      <joinProjects></joinProjects>
      <joinPublishers>
        <hudson.plugins.parameterizedtrigger.BuildTrigger>
          <configs>
            <hudson.plugins.parameterizedtrigger.BuildTriggerConfig>
              <configs>
                <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
                  <properties>A_CUSTOM_PARAMETER=HELLO</properties>
                </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters>
                <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/>
              </configs>
              <projects>job_E, </projects>
              <condition>SUCCESS</condition>
              <triggerWithNoParameters>false</triggerWithNoParameters>
            </hudson.plugins.parameterizedtrigger.BuildTriggerConfig>
          </configs>
        </hudson.plugins.parameterizedtrigger.BuildTrigger>
      </joinPublishers>
      <evenIfDownstreamUnstable>false</evenIfDownstreamUnstable>
    </join.JoinTrigger>
    <hudson.notify.UpDownStreamNumberSychronizedNotify>
      <times>10</times>
      <itimes>10</itimes>
    </hudson.notify.UpDownStreamNumberSychronizedNotify>
    <hudson.plugins.emailext.ExtendedEmailPublisher>
      <recipientList>$DEFAULT_RECIPIENTS</recipientList>
      <configuredTriggers/>
      <contentType>default</contentType>
      <defaultSubject>$DEFAULT_SUBJECT</defaultSubject>
      <defaultContent>$DEFAULT_CONTENT</defaultContent>
      <attachmentsPattern></attachmentsPattern>
      <presendScript></presendScript>
      <matrixTriggerMode>ONLY_PARENT</matrixTriggerMode>
    </hudson.plugins.emailext.ExtendedEmailPublisher>
  </publishers>

job_C:

  <publishers>
    <hudson.tasks.ArtifactArchiver>
      <artifacts>**/test/**/*.log</artifacts>
      <latestOnly>false</latestOnly>
    </hudson.tasks.ArtifactArchiver>
    <hudson.plugins.emailext.ExtendedEmailPublisher>
      <recipientList>$DEFAULT_RECIPIENTS</recipientList>
      <configuredTriggers>
        <hudson.plugins.emailext.plugins.trigger.FailureTrigger>
          <email>
            <recipientList></recipientList>
            <subject>$PROJECT_DEFAULT_SUBJECT</subject>
            <body>$PROJECT_DEFAULT_CONTENT</body>
            <sendToDevelopers>true</sendToDevelopers>
            <sendToRequester>false</sendToRequester>
            <includeCulprits>true</includeCulprits>
            <sendToRecipientList>true</sendToRecipientList>
          </email>
        </hudson.plugins.emailext.plugins.trigger.FailureTrigger>
      </configuredTriggers>
      <contentType>default</contentType>
      <defaultSubject>$DEFAULT_SUBJECT</defaultSubject>
      <defaultContent>$DEFAULT_CONTENT</defaultContent>
      <attachmentsPattern></attachmentsPattern>
      <presendScript></presendScript>
      <matrixTriggerMode>ONLY_PARENT</matrixTriggerMode>
    </hudson.plugins.emailext.ExtendedEmailPublisher>
  </publishers>

job_D:

  <publishers>
    <hudson.plugins.emailext.ExtendedEmailPublisher>
      <recipientList>$DEFAULT_RECIPIENTS</recipientList>
      <configuredTriggers/>
      <contentType>default</contentType>
      <defaultSubject>$DEFAULT_SUBJECT</defaultSubject>
      <defaultContent>$DEFAULT_CONTENT</defaultContent>
      <attachmentsPattern></attachmentsPattern>
      <presendScript></presendScript>
      <matrixTriggerMode>ONLY_PARENT</matrixTriggerMode>
    </hudson.plugins.emailext.ExtendedEmailPublisher>
  </publishers>

job_E:

  <publishers>
    <hudson.plugins.emailext.ExtendedEmailPublisher>
      <recipientList>$JENKINS_MAIL_LIST</recipientList>
      <configuredTriggers>
        <hudson.plugins.emailext.plugins.trigger.FailureTrigger>
          <email>
            <recipientList></recipientList>
            <subject>$PROJECT_DEFAULT_SUBJECT</subject>
            <body>$PROJECT_DEFAULT_CONTENT</body>
            <sendToDevelopers>true</sendToDevelopers>
            <sendToRequester>false</sendToRequester>
            <includeCulprits>false</includeCulprits>
            <sendToRecipientList>true</sendToRecipientList>
          </email>
        </hudson.plugins.emailext.plugins.trigger.FailureTrigger>
      </configuredTriggers>
      <contentType>default</contentType>
      <defaultSubject>[Jenkins-Maintenance] Warning, job failed!</defaultSubject>
      <defaultContent>$DEFAULT_CONTENT</defaultContent>
      <attachmentsPattern></attachmentsPattern>
      <presendScript></presendScript>
    </hudson.plugins.emailext.ExtendedEmailPublisher>
  </publishers>

Alex Vesely added a comment - 2013-01-18 11:26 - edited Yes, I forgot to mention that "Block when downstream is building" is of course enabled in A. Posting <publishers> for all 5 jobs: job_A: <publishers> <hudson.plugins.parameterizedtrigger.BuildTrigger> <configs> <hudson.plugins.parameterizedtrigger.BuildTriggerConfig> <configs> <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/> </configs> <projects>job_B</projects> <condition>SUCCESS</condition> <triggerWithNoParameters>false</triggerWithNoParameters> </hudson.plugins.parameterizedtrigger.BuildTriggerConfig> </configs> </hudson.plugins.parameterizedtrigger.BuildTrigger> <hudson.plugins.emailext.ExtendedEmailPublisher> <recipientList>$DEFAULT_RECIPIENTS</recipientList> <configuredTriggers> <hudson.plugins.emailext.plugins.trigger.FailureTrigger> <email> <recipientList></recipientList> <subject>$PROJECT_DEFAULT_SUBJECT</subject> <body>$PROJECT_DEFAULT_CONTENT</body> <sendToDevelopers>true</sendToDevelopers> <sendToRequester>false</sendToRequester> <includeCulprits>false</includeCulprits> <sendToRecipientList>true</sendToRecipientList> </email> </hudson.plugins.emailext.plugins.trigger.FailureTrigger> </configuredTriggers> <contentType>default</contentType> <defaultSubject>$DEFAULT_SUBJECT</defaultSubject> <defaultContent>$DEFAULT_CONTENT</defaultContent> <attachmentsPattern></attachmentsPattern> <presendScript></presendScript> </hudson.plugins.emailext.ExtendedEmailPublisher> </publishers> job_B: <publishers> <hudson.plugins.logparser.LogParserPublisher> <unstableOnWarning>false</unstableOnWarning> <failBuildOnError>true</failBuildOnError> <parsingRulesPath>make.error</parsingRulesPath> </hudson.plugins.logparser.LogParserPublisher> <hudson.plugins.parameterizedtrigger.BuildTrigger> <configs> <hudson.plugins.parameterizedtrigger.BuildTriggerConfig> <configs> <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/> </configs> <projects>job_C, job_D</projects> <condition>SUCCESS</condition> <triggerWithNoParameters>false</triggerWithNoParameters> </hudson.plugins.parameterizedtrigger.BuildTriggerConfig> </configs> </hudson.plugins.parameterizedtrigger.BuildTrigger> <join.JoinTrigger> <joinProjects></joinProjects> <joinPublishers> <hudson.plugins.parameterizedtrigger.BuildTrigger> <configs> <hudson.plugins.parameterizedtrigger.BuildTriggerConfig> <configs> <hudson.plugins.parameterizedtrigger.PredefinedBuildParameters> <properties>A_CUSTOM_PARAMETER=HELLO</properties> </hudson.plugins.parameterizedtrigger.PredefinedBuildParameters> <hudson.plugins.parameterizedtrigger.CurrentBuildParameters/> </configs> <projects>job_E, </projects> <condition>SUCCESS</condition> <triggerWithNoParameters>false</triggerWithNoParameters> </hudson.plugins.parameterizedtrigger.BuildTriggerConfig> </configs> </hudson.plugins.parameterizedtrigger.BuildTrigger> </joinPublishers> <evenIfDownstreamUnstable>false</evenIfDownstreamUnstable> </join.JoinTrigger> <hudson.notify.UpDownStreamNumberSychronizedNotify> <times>10</times> <itimes>10</itimes> </hudson.notify.UpDownStreamNumberSychronizedNotify> <hudson.plugins.emailext.ExtendedEmailPublisher> <recipientList>$DEFAULT_RECIPIENTS</recipientList> <configuredTriggers/> <contentType>default</contentType> <defaultSubject>$DEFAULT_SUBJECT</defaultSubject> <defaultContent>$DEFAULT_CONTENT</defaultContent> <attachmentsPattern></attachmentsPattern> <presendScript></presendScript> <matrixTriggerMode>ONLY_PARENT</matrixTriggerMode> </hudson.plugins.emailext.ExtendedEmailPublisher> </publishers> job_C: <publishers> <hudson.tasks.ArtifactArchiver> <artifacts>**/test/**/*.log</artifacts> <latestOnly>false</latestOnly> </hudson.tasks.ArtifactArchiver> <hudson.plugins.emailext.ExtendedEmailPublisher> <recipientList>$DEFAULT_RECIPIENTS</recipientList> <configuredTriggers> <hudson.plugins.emailext.plugins.trigger.FailureTrigger> <email> <recipientList></recipientList> <subject>$PROJECT_DEFAULT_SUBJECT</subject> <body>$PROJECT_DEFAULT_CONTENT</body> <sendToDevelopers>true</sendToDevelopers> <sendToRequester>false</sendToRequester> <includeCulprits>true</includeCulprits> <sendToRecipientList>true</sendToRecipientList> </email> </hudson.plugins.emailext.plugins.trigger.FailureTrigger> </configuredTriggers> <contentType>default</contentType> <defaultSubject>$DEFAULT_SUBJECT</defaultSubject> <defaultContent>$DEFAULT_CONTENT</defaultContent> <attachmentsPattern></attachmentsPattern> <presendScript></presendScript> <matrixTriggerMode>ONLY_PARENT</matrixTriggerMode> </hudson.plugins.emailext.ExtendedEmailPublisher> </publishers> job_D: <publishers> <hudson.plugins.emailext.ExtendedEmailPublisher> <recipientList>$DEFAULT_RECIPIENTS</recipientList> <configuredTriggers/> <contentType>default</contentType> <defaultSubject>$DEFAULT_SUBJECT</defaultSubject> <defaultContent>$DEFAULT_CONTENT</defaultContent> <attachmentsPattern></attachmentsPattern> <presendScript></presendScript> <matrixTriggerMode>ONLY_PARENT</matrixTriggerMode> </hudson.plugins.emailext.ExtendedEmailPublisher> </publishers> job_E: <publishers> <hudson.plugins.emailext.ExtendedEmailPublisher> <recipientList>$JENKINS_MAIL_LIST</recipientList> <configuredTriggers> <hudson.plugins.emailext.plugins.trigger.FailureTrigger> <email> <recipientList></recipientList> <subject>$PROJECT_DEFAULT_SUBJECT</subject> <body>$PROJECT_DEFAULT_CONTENT</body> <sendToDevelopers>true</sendToDevelopers> <sendToRequester>false</sendToRequester> <includeCulprits>false</includeCulprits> <sendToRecipientList>true</sendToRecipientList> </email> </hudson.plugins.emailext.plugins.trigger.FailureTrigger> </configuredTriggers> <contentType>default</contentType> <defaultSubject>[Jenkins-Maintenance] Warning, job failed!</defaultSubject> <defaultContent>$DEFAULT_CONTENT</defaultContent> <attachmentsPattern></attachmentsPattern> <presendScript></presendScript> </hudson.plugins.emailext.ExtendedEmailPublisher> </publishers>

cjo9900 added a comment - 2013-01-22 11:58

I have managed to reproduce what I think is the issue.
and the steps are

Trigger A1
A1 triggers B1
Other Jobs triggers E1
Trigger A2
B1 triggers C1, D1
C1, D1 trigger E2
E1 runs
A2 runs
E2 waits until B2,C2,D2 have finished.

In this case I have enabled the Block when upstream is building on project E.
and seem to see the same issue as JENKINS-14918

Where E2 is seen as Blocked because E1 is building
The the queue is processed in order with A2 checked first and E2 is still classed as blocked so A2 is run.
Then E2 is checked and still blocked due A2 running.

Can you check the settings for Job E and see if this is the case.

cjo9900 added a comment - 2013-01-22 11:58 I have managed to reproduce what I think is the issue. and the steps are Trigger A1 A1 triggers B1 Other Jobs triggers E1 Trigger A2 B1 triggers C1, D1 C1, D1 trigger E2 E1 runs A2 runs E2 waits until B2,C2,D2 have finished. In this case I have enabled the Block when upstream is building on project E. and seem to see the same issue as JENKINS-14918 Where E2 is seen as Blocked because E1 is building The the queue is processed in order with A2 checked first and E2 is still classed as blocked so A2 is run. Then E2 is checked and still blocked due A2 running. Can you check the settings for Job E and see if this is the case.

Alex Vesely added a comment - 2013-01-23 09:29

Do you mean if job E has 'block if upstream is building' enabled? No, it has both the 'block' flags unchecked.

Can the issue be connected to the "Check job prerequisites" script that job E runs before finding a good slave for execution?

Alex Vesely added a comment - 2013-01-23 09:29 Do you mean if job E has 'block if upstream is building' enabled? No, it has both the 'block' flags unchecked. Can the issue be connected to the "Check job prerequisites" script that job E runs before finding a good slave for execution?

cjo9900 added a comment - 2013-01-23 16:19

I have added the Slave Prerequisites plugin to Job E (without the block upstream enabled) and that seems to block job A as expected even if job E is started from other places.

This is because the plugin blocks at the executor level and the block on downstream check does include queued jobs.

So cannot see a particular cause of why Job A should run if Job E is in the queue, unless it is a timing issue in that when the join trigger schedules job E it is not seen by Job A when it does its checks which is probably Queue related item.
Which I cannot really investigate unless there is a particular case that can be seen causing job A to run instead of job E.

If you can get logs on why Job E is being blocked in the queue, before Job A is started then a particular case might be found that can be invistigated.

cjo9900 added a comment - 2013-01-23 16:19 I have added the Slave Prerequisites plugin to Job E (without the block upstream enabled) and that seems to block job A as expected even if job E is started from other places. This is because the plugin blocks at the executor level and the block on downstream check does include queued jobs. So cannot see a particular cause of why Job A should run if Job E is in the queue, unless it is a timing issue in that when the join trigger schedules job E it is not seen by Job A when it does its checks which is probably Queue related item. Which I cannot really investigate unless there is a particular case that can be seen causing job A to run instead of job E. If you can get logs on why Job E is being blocked in the queue, before Job A is started then a particular case might be found that can be invistigated.

Alex Vesely added a comment - 2013-01-24 08:06

The issue happens very rarely (so rarely that I even thought I'd solved it by rolling back to the LTS), so these logs seem impossible to catch. All I get is a failed job every once in a while, when it's already too late to collect blocking logs.

Is there a way to collect the required kind of logs automatically?

Alex Vesely added a comment - 2013-01-24 08:06 The issue happens very rarely (so rarely that I even thought I'd solved it by rolling back to the LTS), so these logs seem impossible to catch. All I get is a failed job every once in a while, when it's already too late to collect blocking logs. Is there a way to collect the required kind of logs automatically?

ikedam added a comment - 2014-02-27 07:39

Sounds an issue of join plugin. Removed parameterized-plugin.
You'd better to update the description as "Block when downstream is building" is very important for this problem.

Throttle concurrent build plugin may help you.

ikedam added a comment - 2014-02-27 07:39 Sounds an issue of join plugin. Removed parameterized-plugin. You'd better to update the description as "Block when downstream is building" is very important for this problem. Throttle concurrent build plugin may help you.

ikedam made changes - 2014-02-27 07:39

Component/s

Original: parameterized-trigger [ 15592 ]

R. Tyler Croy made changes - 2016-07-25 23:50

Workflow

Original: JNJira [ 147202 ]

New: JNJira + In-Review [ 176837 ]

Jenkins

Details

Description

Attachments

Activity

Collapse comment: cjo9900 added a comment - 2013-01-17 11:42

Expand comment: cjo9900 added a comment - 2013-01-17 11:42

Collapse comment: Alex Vesely added a comment - 2013-01-18 11:26, Edited by Alex Vesely - 2013-01-18 11:30

Expand comment: Alex Vesely added a comment - 2013-01-18 11:26, Edited by Alex Vesely - 2013-01-18 11:30

Collapse comment: cjo9900 added a comment - 2013-01-22 11:58

Expand comment: cjo9900 added a comment - 2013-01-22 11:58

Collapse comment: Alex Vesely added a comment - 2013-01-23 09:29

Expand comment: Alex Vesely added a comment - 2013-01-23 09:29

Collapse comment: cjo9900 added a comment - 2013-01-23 16:19

Expand comment: cjo9900 added a comment - 2013-01-23 16:19

Collapse comment: Alex Vesely added a comment - 2013-01-24 08:06

Expand comment: Alex Vesely added a comment - 2013-01-24 08:06

Collapse comment: ikedam added a comment - 2014-02-27 07:39

Expand comment: ikedam added a comment - 2014-02-27 07:39

People

Dates