-
Bug
-
Resolution: Fixed
-
Major
-
Powered by SuggestiMate
Hello,
I set up a simple Maven 2/3 job which is supposed to be build on all slave nodes sharing a label. I try to achieve this using the NodeLabel Parameter Plugin version 1.5.1. Concurrent/sequential execution - I don't care however all nodes with the label are supposed to be targeted by the same build instance.
However, when I build, only one node is ever targeted, regardless of whether the other nodes are online/offline. The one node changes between iterations.
I include screenshots of my setup as attachments. Jenkins version is 1.554.
[JENKINS-22185] Job with "NodeLabel Parameter Plugin" does not build in parallel on all nodes which share a label.
Hello Christian,
I have dealt with this issue. I am actually giving a presentation on it later this month.
I don't believe it's a bug/issue as much as a design decision, with the idea being that if a single machine can pass a build, why run it on an identical machine?
Here is my solution:
Summary: We have a parent job that finds all online nodes that match a certain label, then foreach node, pass a downstream job a label specific to that node. We have the NodeLabel Parameter plugin installed.
Details:
We have 40 nodes named 'hl-combo-##' where ## is 01-40. Each of those nodes is also labelled 'harnesslab' and 'hl-combo-##'. In a parent job, using groovy, we search all online nodes that have that label:
jenkins.model.Jenkins.instance.nodes.each { if (it.toComputer().online && it.assignedLabels.contains(new hudson.model.labels.LabelAtom('harnesslab'))) { harnesslabOnlineNodes += it
Then after some other code, we do:
harnesslabOnlineNodes.each { println 'Starting configure job for: ' + it.name def nodeLabel = it.name // Since each of our nodes has a label with its name. If this isn't the case for you, you can do some regex on the it.name to convert each node's name into whatever the unique label is. params_with_label += new org.jvnet.jenkins.plugins.nodelabelparameter.LabelParameterValue('NODE_LABEL', nodeLabel) def paramsAction = new hudson.model.ParametersAction(params_with_label) // Then other code to get the cause and job boolean targetBuildQueued = job.scheduleBuild(5, cause, paramsAction);
This way, we end up queueing 40 parallel jobs, each pointing at 1 node, but if we were change anything about the node configuration or number, we wouldn't need to alter any code.
Hope this helps!
Hi Josh,
Thanks for sharing your solution, I will look into it as an workaround.
I still think that no manual code or parent job should be needed and that this is an bug.
If I have a label "someLabel" that I have applied to nodeA, nodeB and nodeC.
If I enable "Run on all nodes matching this label" with the description "The job will run on all nodes matching the label" for the label "someLabel" I expect the job to run on nodeA, nodeB and nodeC.
Ah, if you want that functionality, I recommend the Matrix Project Plugin. This lets you create a job where you check off a series of labels, and it will run the build on 1 machine per label. So, for example, if you have 5 boxes, each with a unique label, and you check those 5 unique labels, it can run in parallel on those 5 boxes. The only issue with this plugin is that the list is static, and if you want to add nodes you'll have to re-configure the job.
I have solved this using a trigger job and a BuildParameterFactory as described on the wiki: https://wiki.jenkins-ci.org/display/JENKINS/NodeLabel+Parameter+Plugin
It would be nice if this could be solved by only using one single jenkins job without any downstream triggering.
When using a Node parameter with multiple nodes selected it will run the job on all selected nodes and the Allow multi node selection for concurrent builds option is chosen. I would expect the same to happen for a Label parameter when checking Run on all nodes matching the label otherwise this checkbox is pretty useless since there is the Restrict where this project can be run option. Running the same job on all machines with the same label without having to add a trigger job is extremely useful, it can be used to limit test jobs to a subset of the test slaves e.g. all Windows 7 test slaves. Unfortunately the Node parameter requires much more maintenance compared to using labels.
I tried for both Node and Label parameter with all variations of settings.
The build doesn't happen on all the systems.
Is there any solution to this?
Hello Aswin,
Did you try the steps I wrote out above? The critical understanding is that any combination of nodes/labels will only run on a single machine which matches those criteria.
Good luck,
Josh
I too discovered this issue today and am disappointed that when you select run on all nodes matching this label that it only executes on 1 node...
Minimally, the confusing wording should be changed ASAP to avoid confusion to something like: run on only 1 random node that matches the label
Ideally, it should do what it states and run on all nodes that match the label.
As an improvement, supporting label regex matching would make life even easier for some
PR sent with a fix: https://github.com/jenkinsci/nodelabelparameter-plugin/pull/9
Code changed in jenkins
User: Antonio Muñiz
Path:
src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/LabelParameterDefinition.java
src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/LabelParameterValue.java
http://jenkins-ci.org/commit/nodelabelparameter-plugin/24cc8f069fd274ea1ad19465d0290fcbe189ad63
Log:
JENKINS-22185 Run the build on all matching nodes
Code changed in jenkins
User: Antonio Muñiz
Path:
src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/wrapper/TriggerNextBuildWrapper.java
http://jenkins-ci.org/commit/nodelabelparameter-plugin/b235b9e86cc7f1f9cdfc08b9c01550b78e1b0506
Log:
JENKINS-22185 Avoid to run twice in the same node
Hello
Please see https://issues.jenkins-ci.org/browse/JENKINS-32645 which I raised recently. Changing the behaviour of your plugin actually is the opposite of what I was using it for and expecting it to do, so when I upgraded it, my build environment was unusable.
Is there a way I can upgrade the plugin, but retain the old behaviour?
Thanks
You were relying in the existence of a bug. There is nothing to fix in the plugin but in your configuration.
I have replied on my own ticket, but would just like to say, is changing the main behaviour of this plugin after it working one way for three/four years not rather drastic?
I am one of the users who complained that the plug-in was not doing what was originally intended and what the documentation clearly stated it would do. I applaud that the bug was corrected. If a plug-in states that when the ALL option is specific it should execute on all servers matching a label. If it don't then it's broke...
We began to use the "fixed" version in our environment now that it work as advertised.
By the way, this plug-in helps us execute jobs on all hosts matching a label w/o having to memorize "lists" of targets supporting specific features... Makes adding/deleted slaves to a feature much easier!
Eric L. Edberg
My slaves are all identical and so is the job they run, I just run another Jenkins job with a for loop in it which triggered another job which was set to run on all, this would spawn out the jobs as many times as it needed to slaves... it was quite a nice way to control running the same job with different parameters across a load of slaves... I didn't see this 22185 as a bug, I just wrote code to make the plugin do what I wanted.
I agree with the fix for the reasons ericledberg noted. Before the patch I created a Pipeline job intending to rely on the documented functionality of the label, "broadcasting" execution to every node with the label expression but then had to write a bunch of extra Jenkins API logic to discover the matching nodes. Ie the target node pool is constructed automatically, but I had to write unnecessary (IMO) logic to do so. Plus explicitly specify the parallel execution when the pipeline script should have been a 1- to 2-liner at most.
Any update on this issue? We are also experiencing the same problem.
200$ is up for grabs for solving this issue, see https://freedomsponsors.org/issue/609/job-with-nodelabel-parameter-plugin-does-not-build-in-parallel-on-all-nodes-which-share-a-label?alert=SPONSOR