Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22185

Job with "NodeLabel Parameter Plugin" does not build in parallel on all nodes which share a label.

      Hello,

      I set up a simple Maven 2/3 job which is supposed to be build on all slave nodes sharing a label. I try to achieve this using the NodeLabel Parameter Plugin version 1.5.1. Concurrent/sequential execution - I don't care however all nodes with the label are supposed to be targeted by the same build instance.

      However, when I build, only one node is ever targeted, regardless of whether the other nodes are online/offline. The one node changes between iterations.

      I include screenshots of my setup as attachments. Jenkins version is 1.554.

          [JENKINS-22185] Job with "NodeLabel Parameter Plugin" does not build in parallel on all nodes which share a label.

          Any update on this issue? We are also experiencing the same problem.

          200$ is up for grabs for solving this issue, see https://freedomsponsors.org/issue/609/job-with-nodelabel-parameter-plugin-does-not-build-in-parallel-on-all-nodes-which-share-a-label?alert=SPONSOR

          Christian Bremer added a comment - Any update on this issue? We are also experiencing the same problem. 200$ is up for grabs for solving this issue, see https://freedomsponsors.org/issue/609/job-with-nodelabel-parameter-plugin-does-not-build-in-parallel-on-all-nodes-which-share-a-label?alert=SPONSOR

          Josh Shapiro added a comment - - edited

          Hello Christian,

          I have dealt with this issue. I am actually giving a presentation on it later this month.

          I don't believe it's a bug/issue as much as a design decision, with the idea being that if a single machine can pass a build, why run it on an identical machine?

          Here is my solution:
          Summary: We have a parent job that finds all online nodes that match a certain label, then foreach node, pass a downstream job a label specific to that node. We have the NodeLabel Parameter plugin installed.

          Details:
          We have 40 nodes named 'hl-combo-##' where ## is 01-40. Each of those nodes is also labelled 'harnesslab' and 'hl-combo-##'. In a parent job, using groovy, we search all online nodes that have that label:

          jenkins.model.Jenkins.instance.nodes.each {
          	if (it.toComputer().online && it.assignedLabels.contains(new hudson.model.labels.LabelAtom('harnesslab'))) {
          		harnesslabOnlineNodes += it
          

          Then after some other code, we do:

          harnesslabOnlineNodes.each {
          		println 'Starting configure job for: ' + it.name
          		def nodeLabel = it.name // Since each of our nodes has a label with its name. If this isn't the case for you, you can do some regex on the it.name to convert each node's name into whatever the unique label is.
                          params_with_label += new org.jvnet.jenkins.plugins.nodelabelparameter.LabelParameterValue('NODE_LABEL', nodeLabel)
                          def paramsAction = new hudson.model.ParametersAction(params_with_label)
                          // Then other code to get the cause and job
                          boolean targetBuildQueued = job.scheduleBuild(5, cause, paramsAction);
          

          This way, we end up queueing 40 parallel jobs, each pointing at 1 node, but if we were change anything about the node configuration or number, we wouldn't need to alter any code.

          Hope this helps!

          Josh Shapiro added a comment - - edited Hello Christian, I have dealt with this issue. I am actually giving a presentation on it later this month. I don't believe it's a bug/issue as much as a design decision, with the idea being that if a single machine can pass a build, why run it on an identical machine? Here is my solution: Summary: We have a parent job that finds all online nodes that match a certain label, then foreach node, pass a downstream job a label specific to that node. We have the NodeLabel Parameter plugin installed. Details: We have 40 nodes named 'hl-combo-##' where ## is 01-40. Each of those nodes is also labelled 'harnesslab' and 'hl-combo-##'. In a parent job, using groovy, we search all online nodes that have that label: jenkins.model.Jenkins.instance.nodes.each { if (it.toComputer().online && it.assignedLabels.contains( new hudson.model.labels.LabelAtom( 'harnesslab' ))) { harnesslabOnlineNodes += it Then after some other code, we do: harnesslabOnlineNodes.each { println 'Starting configure job for : ' + it.name def nodeLabel = it.name // Since each of our nodes has a label with its name. If this isn 't the case for you, you can do some regex on the it.name to convert each node' s name into whatever the unique label is. params_with_label += new org.jvnet.jenkins.plugins.nodelabelparameter.LabelParameterValue( 'NODE_LABEL' , nodeLabel) def paramsAction = new hudson.model.ParametersAction(params_with_label) // Then other code to get the cause and job boolean targetBuildQueued = job.scheduleBuild(5, cause, paramsAction); This way, we end up queueing 40 parallel jobs, each pointing at 1 node, but if we were change anything about the node configuration or number, we wouldn't need to alter any code. Hope this helps!

          Hi Josh,

          Thanks for sharing your solution, I will look into it as an workaround.

          I still think that no manual code or parent job should be needed and that this is an bug.

          If I have a label "someLabel" that I have applied to nodeA, nodeB and nodeC.
          If I enable "Run on all nodes matching this label" with the description "The job will run on all nodes matching the label" for the label "someLabel" I expect the job to run on nodeA, nodeB and nodeC.

          Christian Bremer added a comment - Hi Josh, Thanks for sharing your solution, I will look into it as an workaround. I still think that no manual code or parent job should be needed and that this is an bug. If I have a label "someLabel" that I have applied to nodeA, nodeB and nodeC. If I enable "Run on all nodes matching this label" with the description "The job will run on all nodes matching the label" for the label "someLabel" I expect the job to run on nodeA, nodeB and nodeC.

          Josh Shapiro added a comment - - edited

          Ah, if you want that functionality, I recommend the Matrix Project Plugin. This lets you create a job where you check off a series of labels, and it will run the build on 1 machine per label. So, for example, if you have 5 boxes, each with a unique label, and you check those 5 unique labels, it can run in parallel on those 5 boxes. The only issue with this plugin is that the list is static, and if you want to add nodes you'll have to re-configure the job.

          Josh Shapiro added a comment - - edited Ah, if you want that functionality, I recommend the Matrix Project Plugin. This lets you create a job where you check off a series of labels, and it will run the build on 1 machine per label. So, for example, if you have 5 boxes, each with a unique label, and you check those 5 unique labels, it can run in parallel on those 5 boxes. The only issue with this plugin is that the list is static, and if you want to add nodes you'll have to re-configure the job.

          Christian Bremer added a comment - - edited

          I have solved this using a trigger job and a BuildParameterFactory as described on the wiki: https://wiki.jenkins-ci.org/display/JENKINS/NodeLabel+Parameter+Plugin

          It would be nice if this could be solved by only using one single jenkins job without any downstream triggering.

          Christian Bremer added a comment - - edited I have solved this using a trigger job and a BuildParameterFactory as described on the wiki: https://wiki.jenkins-ci.org/display/JENKINS/NodeLabel+Parameter+Plugin It would be nice if this could be solved by only using one single jenkins job without any downstream triggering.

          When using a Node parameter with multiple nodes selected it will run the job on all selected nodes and the Allow multi node selection for concurrent builds option is chosen. I would expect the same to happen for a Label parameter when checking Run on all nodes matching the label otherwise this checkbox is pretty useless since there is the Restrict where this project can be run option. Running the same job on all machines with the same label without having to add a trigger job is extremely useful, it can be used to limit test jobs to a subset of the test slaves e.g. all Windows 7 test slaves. Unfortunately the Node parameter requires much more maintenance compared to using labels.

          Gert Ceulemans added a comment - When using a Node parameter with multiple nodes selected it will run the job on all selected nodes and the Allow multi node selection for concurrent builds option is chosen. I would expect the same to happen for a Label parameter when checking Run on all nodes matching the label otherwise this checkbox is pretty useless since there is the Restrict where this project can be run option. Running the same job on all machines with the same label without having to add a trigger job is extremely useful, it can be used to limit test jobs to a subset of the test slaves e.g. all Windows 7 test slaves. Unfortunately the Node parameter requires much more maintenance compared to using labels.

          I tried for both Node and Label parameter with all variations of settings.
          The build doesn't happen on all the systems.
          Is there any solution to this?

          Aswin Karthik Ramachandran Venkatapathy added a comment - I tried for both Node and Label parameter with all variations of settings. The build doesn't happen on all the systems. Is there any solution to this?

          Josh Shapiro added a comment -

          Hello Aswin,

          Did you try the steps I wrote out above? The critical understanding is that any combination of nodes/labels will only run on a single machine which matches those criteria.

          Good luck,
          Josh

          Josh Shapiro added a comment - Hello Aswin, Did you try the steps I wrote out above? The critical understanding is that any combination of nodes/labels will only run on a single machine which matches those criteria. Good luck, Josh

          Eric Edberg added a comment -

          I too discovered this issue today and am disappointed that when you select run on all nodes matching this label that it only executes on 1 node...

          Minimally, the confusing wording should be changed ASAP to avoid confusion to something like: run on only 1 random node that matches the label

          Ideally, it should do what it states and run on all nodes that match the label.
          As an improvement, supporting label regex matching would make life even easier for some

          Eric Edberg added a comment - I too discovered this issue today and am disappointed that when you select run on all nodes matching this label that it only executes on 1 node... Minimally, the confusing wording should be changed ASAP to avoid confusion to something like: run on only 1 random node that matches the label Ideally, it should do what it states and run on all nodes that match the label. As an improvement, supporting label regex matching would make life even easier for some

          Antonio Muñiz added a comment - PR sent with a fix: https://github.com/jenkinsci/nodelabelparameter-plugin/pull/9

          Code changed in jenkins
          User: Antonio Muñiz
          Path:
          src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/LabelParameterDefinition.java
          src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/LabelParameterValue.java
          http://jenkins-ci.org/commit/nodelabelparameter-plugin/24cc8f069fd274ea1ad19465d0290fcbe189ad63
          Log:
          JENKINS-22185 Run the build on all matching nodes

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Antonio Muñiz Path: src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/LabelParameterDefinition.java src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/LabelParameterValue.java http://jenkins-ci.org/commit/nodelabelparameter-plugin/24cc8f069fd274ea1ad19465d0290fcbe189ad63 Log: JENKINS-22185 Run the build on all matching nodes

          Code changed in jenkins
          User: Antonio Muñiz
          Path:
          src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/wrapper/TriggerNextBuildWrapper.java
          http://jenkins-ci.org/commit/nodelabelparameter-plugin/b235b9e86cc7f1f9cdfc08b9c01550b78e1b0506
          Log:
          JENKINS-22185 Avoid to run twice in the same node

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Antonio Muñiz Path: src/main/java/org/jvnet/jenkins/plugins/nodelabelparameter/wrapper/TriggerNextBuildWrapper.java http://jenkins-ci.org/commit/nodelabelparameter-plugin/b235b9e86cc7f1f9cdfc08b9c01550b78e1b0506 Log: JENKINS-22185 Avoid to run twice in the same node

          released with version 1.7

          Dominik Bartholdi added a comment - released with version 1.7

          Duncan Bell added a comment -

          Hello

          Please see https://issues.jenkins-ci.org/browse/JENKINS-32645 which I raised recently. Changing the behaviour of your plugin actually is the opposite of what I was using it for and expecting it to do, so when I upgraded it, my build environment was unusable.

          Is there a way I can upgrade the plugin, but retain the old behaviour?

          Thanks

          Duncan Bell added a comment - Hello Please see https://issues.jenkins-ci.org/browse/JENKINS-32645 which I raised recently. Changing the behaviour of your plugin actually is the opposite of what I was using it for and expecting it to do, so when I upgraded it, my build environment was unusable. Is there a way I can upgrade the plugin, but retain the old behaviour? Thanks

          You were relying in the existence of a bug. There is nothing to fix in the plugin but in your configuration.

          Antonio Muñiz added a comment - You were relying in the existence of a bug. There is nothing to fix in the plugin but in your configuration.

          Duncan Bell added a comment -

          I have replied on my own ticket, but would just like to say, is changing the main behaviour of this plugin after it working one way for three/four years not rather drastic?

          Duncan Bell added a comment - I have replied on my own ticket, but would just like to say, is changing the main behaviour of this plugin after it working one way for three/four years not rather drastic?

          Eric Edberg added a comment -

          I am one of the users who complained that the plug-in was not doing what was originally intended and what the documentation clearly stated it would do. I applaud that the bug was corrected. If a plug-in states that when the ALL option is specific it should execute on all servers matching a label. If it don't then it's broke...

          We began to use the "fixed" version in our environment now that it work as advertised.

          By the way, this plug-in helps us execute jobs on all hosts matching a label w/o having to memorize "lists" of targets supporting specific features... Makes adding/deleted slaves to a feature much easier!

          Eric L. Edberg

          Eric Edberg added a comment - I am one of the users who complained that the plug-in was not doing what was originally intended and what the documentation clearly stated it would do. I applaud that the bug was corrected. If a plug-in states that when the ALL option is specific it should execute on all servers matching a label. If it don't then it's broke... We began to use the "fixed" version in our environment now that it work as advertised. By the way, this plug-in helps us execute jobs on all hosts matching a label w/o having to memorize "lists" of targets supporting specific features... Makes adding/deleted slaves to a feature much easier! Eric L. Edberg

          Duncan Bell added a comment -

          My slaves are all identical and so is the job they run, I just run another Jenkins job with a for loop in it which triggered another job which was set to run on all, this would spawn out the jobs as many times as it needed to slaves... it was quite a nice way to control running the same job with different parameters across a load of slaves... I didn't see this 22185 as a bug, I just wrote code to make the plugin do what I wanted.

          Duncan Bell added a comment - My slaves are all identical and so is the job they run, I just run another Jenkins job with a for loop in it which triggered another job which was set to run on all, this would spawn out the jobs as many times as it needed to slaves... it was quite a nice way to control running the same job with different parameters across a load of slaves... I didn't see this 22185 as a bug, I just wrote code to make the plugin do what I wanted.

          Brian Ray added a comment -

          I agree with the fix for the reasons ericledberg noted. Before the patch I created a Pipeline job intending to rely on the documented functionality of the label, "broadcasting" execution to every node with the label expression but then had to write a bunch of extra Jenkins API logic to discover the matching nodes. Ie the target node pool is constructed automatically, but I had to write unnecessary (IMO) logic to do so. Plus explicitly specify the parallel execution when the pipeline script should have been a 1- to 2-liner at most.

          Brian Ray added a comment - I agree with the fix for the reasons ericledberg noted. Before the patch I created a Pipeline job intending to rely on the documented functionality of the label, "broadcasting" execution to every node with the label expression but then had to write a bunch of extra Jenkins API logic to discover the matching nodes. Ie the target node pool is constructed automatically, but I had to write unnecessary (IMO) logic to do so. Plus explicitly specify the parallel execution when the pipeline script should have been a 1- to 2-liner at most.

            amuniz Antonio Muñiz
            david_tha_dude David Koch
            Votes:
            5 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: