-
New Feature
-
Resolution: Unresolved
-
Major
-
distributed matrix setup
It will be good to have possibility for automatically set order of execution of matrix subtasks based on tracked time(in execution history) - longer subtask must be executed earlier then shorter. I think, it could help to decrease execution time for full matrix task.
F.e. I have 2 nodes and 3 matrix configurations(1,2,3) and estimation time of execution based on history is 1h ,2h, 3h. I suggest to write plugin which set execution order: 3->2->1. In this case, full execution time will be 3h (node 1 - task 3, node 2 - task1+task2). In any other order full execution time will be longer.
[JENKINS-13820] please create possibility to automatically set order of execution of matrix subtasks based on previously tracked time
Roman G
created issue -
Lucie Votypkova
made changes -
| Resolution | New: Fixed [ 1 ] | |
| Status | Original: Open [ 1 ] | New: Resolved [ 5 ] |
Roman G
added a comment - looks like plugin allows execution on only one slave node sequentially and doesn't
allow parallel execution on few nodes
Roman G
added a comment - looks like plugin allows execution on only one slave node sequentially and doesn't
allow parallel execution on few nodes Roman G
added a comment - cannot execute on three nodes in one time Roman G
made changes -
| Resolution | Original: Fixed [ 1 ] | |
| Status | Original: Resolved [ 5 ] | New: Reopened [ 4 ] |
Lucie Votypkova
added a comment - Yes, it is not for parallel execution. If the execution is parallel, the building of configurations is started at the same time(there is no need to define order). If some configuration in parallel execution is not building in the same time as others, it is due to waiting for appropriate node (no appropriate node is available) not due to its order.
Can you explain me why you need some order for parallel execution? I have no idea what is good for - if you need some order for example configuration a must be executed after the execution of configuration b is finished, it is nonsense to do it in parallel execution.
Lucie Votypkova
added a comment - Yes, it is not for parallel execution. If the execution is parallel, the building of configurations is started at the same time(there is no need to define order). If some configuration in parallel execution is not building in the same time as others, it is due to waiting for appropriate node (no appropriate node is available) not due to its order.
Can you explain me why you need some order for parallel execution? I have no idea what is good for - if you need some order for example configuration a must be executed after the execution of configuration b is finished, it is nonsense to do it in parallel execution. Roman G
added a comment - My issue:
I have matrix which generate about 13 configurations and only 5 nodes of executions. All 5 nodes absolutely same(VM).
4 configurations needs comparable more times then other.
I want to execute them earlier.
After enabling your plugin I see that execution start only on one node and 4 other are not used.
Roman G
added a comment - My issue:
I have matrix which generate about 13 configurations and only 5 nodes of executions. All 5 nodes absolutely same(VM).
4 configurations needs comparable more times then other.
I want to execute them earlier.
After enabling your plugin I see that execution start only on one node and 4 other are not used. Lucie Votypkova
added a comment - Ok your problem is that you have more than one configuration which is executed on the same node and you want to execute the longest first. But if you have 3 configuration, which takes 1h, 2h and 3h, for the same node, it does not matter in which order are executed 
- it takes 6h, so the shorter time is not what you need, am I right? I must understand what is the advantage to do it right . What you obtain with this? Can you give me link on that job?
Thanks
Lucie Votypkova
added a comment - Ok your problem is that you have more than one configuration which is executed on the same node and you want to execute the longest first. But if you have 3 configuration, which takes 1h, 2h and 3h, for the same node, it does not matter in which order are executed - it takes 6h, so the shorter time is not what you need, am I right? I must understand what is the advantage to do it right . What you obtain with this? Can you give me link on that job?
Thanks Roman G
added a comment - I agree that common CPU time will be same. My goal is decrease execution time of full taks.
I have 13 jobs and 5 nodes and 4 very slow configurations, 5 fast and 4 medium.
Decrease my environment for example. 2 nodes and 5 configurations by 1,2,3,4,5 execution hours. Common CPU time will be 1+2+3+4+5.
We have 2 nodes.
If configurations are executing randomly we can have bad case: max((5+4) and (3+2+1))=9 hours. 1 node will be free 2 hours.
The best choice is max((5+2) (4+3+1)) = 8 hours, 1 hours 2 node will be free.
For other distributions bonus could be more significant.
Roman G
added a comment - I agree that common CPU time will be same. My goal is decrease execution time of full taks.
I have 13 jobs and 5 nodes and 4 very slow configurations, 5 fast and 4 medium.
Decrease my environment for example. 2 nodes and 5 configurations by 1,2,3,4,5 execution hours. Common CPU time will be 1+2+3+4+5.
We have 2 nodes.
If configurations are executing randomly we can have bad case: max((5+4) and (3+2+1))=9 hours. 1 node will be free 2 hours.
The best choice is max((5+2) (4+3+1)) = 8 hours, 1 hours 2 node will be free.
For other distributions bonus could be more significant.
I did new release so it should be available after next restart of Jenkins