-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
Jenkins 1.532.1
Hi,
We have an upstream project(with configuration matrix) which trigers downstream one. Downstream project has "Throttle Concurrent Builds" option enabled with "Throttle this project alone", "Maximum Total Concurrent Builds=2", "Maximum Concurrent Builds Per Node=2"
Both upstream and downstream projects have "xUnit" related steps in "Post-build Actions"
Issue there is when we have triggered 10 concurrent builds(#100 - #109) of the downstream project and the third one (#102) requires more time to finish - all further builds (#103 - #109) will be locked until this third build finish.
Other example, when two builds there were executing simultaneously, one of them failed and took long time to finish.
It looks like successful one was waiting for unsuccessful one.
Just compare end of logs for these pairs of tests.
Pay attention on time stamps: gap between "Stopping recording" and "Sending e-mails":
jenkins/view/QA/job/_SIP/job/pjsip_c/2900/console
jenkins/view/QA/job/_SIP/job/pjsip_c/2901/console
11:07:30 [xUnit] [INFO] - Starting to record.
11:07:30 [xUnit] [INFO] - Processing JUnit
11:07:30 [xUnit] [INFO] - [JUnit] - 3 test report file(s) were found with the pattern 'results-pjsip/*.xml' relative to '/tmp/workspace/_SIP/pjsip_c' for the testing framework 'JUnit'.
11:07:35 [xUnit] [INFO] - Check 'Failed Tests' threshold.
11:07:35 [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value.
11:07:35 [xUnit] [INFO] - Setting the build status to FAILURE
11:07:35 [xUnit] [INFO] - Stopping recording.
11:07:35 Build step 'Publish xUnit test result report' changed build result to FAILURE
11:07:35 Sending e-mails to:
11:08:05 Notifying upstream projects of job completion
11:08:05 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys,Version=390
11:08:05 Finished: FAILURE
VS
10:56:40 [xUnit] [INFO] - Setting the build status to SUCCESS
10:56:40 [xUnit] [INFO] - Stopping recording.
11:07:34 Sending e-mails to:
11:08:04 Notifying upstream projects of job completion
11:08:04 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys_ora,Version=380
11:08:04 Finished: SUCCESS
And:
jenkins/view/QA/job/_SIP/job/pjsip_c/2904/console
jenkins/view/QA/job/_SIP/job/pjsip_c/2905/console
10:34:52 [xUnit] [INFO] - Starting to record.
10:34:52 [xUnit] [INFO] - Processing JUnit
10:34:53 [xUnit] [INFO] - [JUnit] - 3 test report file(s) were found with the pattern 'results-pjsip/*.xml' relative to '/tmp/workspace/_SIP/pjsip_c' for the testing framework 'JUnit'.
10:34:57 [xUnit] [INFO] - Check 'Failed Tests' threshold.
10:34:57 [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value.
10:34:57 [xUnit] [INFO] - Setting the build status to FAILURE
10:34:57 [xUnit] [INFO] - Stopping recording.
10:34:57 Build step 'Publish xUnit test result report' changed build result to FAILURE
10:34:57 Sending e-mails to:
10:35:36 Notifying upstream projects of job completion
10:35:36 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys_ora,Version=390
10:35:36 Finished: FAILURE
VS
10:21:28 [xUnit] [INFO] - Setting the build status to SUCCESS
10:21:28 [xUnit] [INFO] - Stopping recording.
10:34:57 Sending e-mails to:
10:35:31 Notifying upstream projects of job completion
10:35:31 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys,Version=371
10:35:31 Finished: SUCCESS
Could you please advise how we can workaround this issue. Any help is appreciated.
- is related to
-
JENKINS-16376 BuildStepMonitor.BUILD makes concurrent builds wait, could be changed to BuildStepMonitor.NONE?
-
- Resolved
-
[JENKINS-21145] Several concurrent builds waits for each other
Environment | Original: Gentoo | New: Jenkins 1.532.1 |
Description |
Original:
Hi, We have an upstream project(with configuration matrix) which tigers downstream one. Downstream project has "Throttle Concurrent Builds" tick enabled with "Throttle this project alone", "Maximum Total Concurrent Builds=2", "Maximum Concurrent Builds Per Nod=2" Both upstream and downstream projects have "xUnit" related steps in "Post-build Actions" Issue there is when we have triggered 10 concurrent builds(#100 - #109) of the downstream project and the third one (#102) requires more time to finish - all further builds (#103 - #109) will be locked until this third build finish. Other example, when two builds there were executing simultaneously, one of them failed and took long time to finish. It looks like successful one was waiting for unsuccessful one. Just compare end of logs for these pairs of tests: https://jenkins/view/QA/job/_SIP/job/pjsip_c/2900/console https://jenkins/view/QA/job/_SIP/job/pjsip_c/2901/console 11:07:30 [xUnit] [INFO] - Starting to record. 11:07:30 [xUnit] [INFO] - Processing JUnit 11:07:30 [xUnit] [INFO] - [JUnit] - 3 test report file(s) were found with the pattern 'results-pjsip/*.xml' relative to '/tmp/workspace/_SIP/pjsip_c' for the testing framework 'JUnit'. 11:07:35 [xUnit] [INFO] - Check 'Failed Tests' threshold. 11:07:35 [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value. 11:07:35 [xUnit] [INFO] - Setting the build status to FAILURE 11:07:35 [xUnit] [INFO] - Stopping recording. 11:07:35 Build step 'Publish xUnit test result report' changed build result to FAILURE 11:07:35 Sending e-mails to: 11:08:05 Notifying upstream projects of job completion 11:08:05 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys,Version=390 11:08:05 Finished: FAILURE VS 10:56:40 [xUnit] [INFO] - Setting the build status to SUCCESS 10:56:40 [xUnit] [INFO] - Stopping recording. 11:07:34 Sending e-mails to: 11:08:04 Notifying upstream projects of job completion 11:08:04 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys_ora,Version=380 11:08:04 Finished: SUCCESS And: https://jenkins/view/QA/job/_SIP/job/pjsip_c/2904/console https://jenkins/view/QA/job/_SIP/job/pjsip_c/2905/console 10:34:52 [xUnit] [INFO] - Starting to record. 10:34:52 [xUnit] [INFO] - Processing JUnit 10:34:53 [xUnit] [INFO] - [JUnit] - 3 test report file(s) were found with the pattern 'results-pjsip/*.xml' relative to '/tmp/workspace/_SIP/pjsip_c' for the testing framework 'JUnit'. 10:34:57 [xUnit] [INFO] - Check 'Failed Tests' threshold. 10:34:57 [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value. 10:34:57 [xUnit] [INFO] - Setting the build status to FAILURE 10:34:57 [xUnit] [INFO] - Stopping recording. 10:34:57 Build step 'Publish xUnit test result report' changed build result to FAILURE 10:34:57 Sending e-mails to: 10:35:36 Notifying upstream projects of job completion 10:35:36 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys_ora,Version=390 10:35:36 Finished: FAILURE VS 10:21:28 [xUnit] [INFO] - Setting the build status to SUCCESS 10:21:28 [xUnit] [INFO] - Stopping recording. 10:34:57 Sending e-mails to: 10:35:31 Notifying upstream projects of job completion 10:35:31 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys,Version=371 10:35:31 Finished: SUCCESS Could you prompt us how to overcome this issue? Any help is appreciated. |
New:
Hi, We have an upstream project(with configuration matrix) which trigers downstream one. Downstream project has "Throttle Concurrent Builds" option enabled with "Throttle this project alone", "Maximum Total Concurrent Builds=2", "Maximum Concurrent Builds Per Node=2" Both upstream and downstream projects have "xUnit" related steps in "Post-build Actions" Issue there is when we have triggered 10 concurrent builds(#100 - #109) of the downstream project and the third one (#102) requires more time to finish - all further builds (#103 - #109) will be locked until this third build finish. Other example, when two builds there were executing simultaneously, one of them failed and took long time to finish. It looks like successful one was waiting for unsuccessful one. Just compare end of logs for these pairs of tests. Pay attention on time stamps: gap between "Stopping recording" and "Sending e-mails": jenkins/view/QA/job/_SIP/job/pjsip_c/2900/console jenkins/view/QA/job/_SIP/job/pjsip_c/2901/console 11:07:30 [xUnit] [INFO] - Starting to record. 11:07:30 [xUnit] [INFO] - Processing JUnit 11:07:30 [xUnit] [INFO] - [JUnit] - 3 test report file(s) were found with the pattern 'results-pjsip/*.xml' relative to '/tmp/workspace/_SIP/pjsip_c' for the testing framework 'JUnit'. 11:07:35 [xUnit] [INFO] - Check 'Failed Tests' threshold. 11:07:35 [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value. 11:07:35 [xUnit] [INFO] - Setting the build status to FAILURE 11:07:35 [xUnit] [INFO] - Stopping recording. 11:07:35 Build step 'Publish xUnit test result report' changed build result to FAILURE 11:07:35 Sending e-mails to: 11:08:05 Notifying upstream projects of job completion 11:08:05 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys,Version=390 11:08:05 Finished: FAILURE VS 10:56:40 [xUnit] [INFO] - Setting the build status to SUCCESS 10:56:40 [xUnit] [INFO] - Stopping recording. 11:07:34 Sending e-mails to: 11:08:04 Notifying upstream projects of job completion 11:08:04 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys_ora,Version=380 11:08:04 Finished: SUCCESS And: jenkins/view/QA/job/_SIP/job/pjsip_c/2904/console jenkins/view/QA/job/_SIP/job/pjsip_c/2905/console 10:34:52 [xUnit] [INFO] - Starting to record. 10:34:52 [xUnit] [INFO] - Processing JUnit 10:34:53 [xUnit] [INFO] - [JUnit] - 3 test report file(s) were found with the pattern 'results-pjsip/*.xml' relative to '/tmp/workspace/_SIP/pjsip_c' for the testing framework 'JUnit'. 10:34:57 [xUnit] [INFO] - Check 'Failed Tests' threshold. 10:34:57 [xUnit] [INFO] - The total number of tests for this category exceeds the specified 'failure' threshold value. 10:34:57 [xUnit] [INFO] - Setting the build status to FAILURE 10:34:57 [xUnit] [INFO] - Stopping recording. 10:34:57 Build step 'Publish xUnit test result report' changed build result to FAILURE 10:34:57 Sending e-mails to: 10:35:36 Notifying upstream projects of job completion 10:35:36 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys_ora,Version=390 10:35:36 Finished: FAILURE VS 10:21:28 [xUnit] [INFO] - Setting the build status to SUCCESS 10:21:28 [xUnit] [INFO] - Stopping recording. 10:34:57 Sending e-mails to: 10:35:31 Notifying upstream projects of job completion 10:35:31 Join notifier cannot find upstream project: _SIP/pjsip_cron/VM=etsys,Version=371 10:35:31 Finished: SUCCESS Could you please advise how we can workaround this issue. Any help is appreciated. |
Component/s | New: xunit [ 15636 ] | |
Component/s | Original: core [ 15593 ] | |
Assignee | New: Gregory Boissinot [ gbois ] |
Assignee | Original: Gregory Boissinot [ gbois ] | |
Resolution | New: Cannot Reproduce [ 5 ] | |
Status | Original: Open [ 1 ] | New: Resolved [ 5 ] |
Link |
New:
This issue is related to |
Component/s | New: email-ext [ 15538 ] | |
Component/s | Original: xunit [ 15636 ] |
Post-Build actions (publishers) define the locking behavior in concurrent builds on their own. I've re-assigned the issue to the xunit plugin