Loading...

Type: Bug
Resolution: Unresolved
Priority: Minor
Component/s: core
Labels:
- regression

Similar Issues:

Show

In Jenkins 2.340 we observed that in some cases the node() pipeline step was giving us a node that did not have the requested label. We dynamically modify labels on our nodes, and the node assigned previously held the requested label, but did not currently have it. I believe I have traced this to the “Only apply trimLabels operation to affected nodes when adding or removing them” ~~JENKINS-67099~~ change introduced in Jenkins 2.321. From what I can tell, the Jenkins.trimLabels(Node) method is not properly resetting labels that are removed from an updated node. If Jenkins.trimLabels() is called, the labels appear to be properly updated, but this is not happening for all node label changes, but would eventually be cleaned up by a periodic run of trimLabels(). The workaround is to specifically invoke Jenkins.trimLabels() after making a label change.

I have written the following script to demonstrate how the behavior of node label updates differs between 2.320, 2.321, and finally 2.341.

println "Jenkins version: " + jenkins.model.Jenkins.instance.VERSION

void printStatus(def node) {
	println "---"
	println "Node: " + node.getNodeName()
	println "Label String: " + node.getLabelString()	
	println "Labels: " + node.getAssignedLabels()
	println "Nodes with label alpha: " + hudson.model.Label.get("alpha").getNodes()
	println "Nodes with label beta: " + hudson.model.Label.get("beta").getNodes()
	println "Nodes with label alpha&&beta: " + hudson.model.Label.get("alpha&&beta").getNodes()
	println "---"
}

void setLabel(def node, def label) {
	println "Setting label on " + node.getNodeName() + " to '" + label + "'"
	node.setLabelString(label)
	Jenkins.instance.updateNode(node)
}

println "trimLabels()..."
jenkins.model.Jenkins.instance.trimLabels()

def nodeName = "JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu"
def node = jenkins.model.Jenkins.instance.getNode(nodeName)

setLabel(node, "alpha")
printStatus(node)

setLabel(node, "alpha beta")
printStatus(node)

setLabel(node, "beta")
printStatus(node)

try { 
	println "trimLabels(node)..."
	jenkins.model.Jenkins.instance.trimLabels(node)
	printStatus(node)
} catch (Exception e) {
	println "method unavailable."
}

println "trimLabels()..."
jenkins.model.Jenkins.instance.trimLabels()
printStatus(node)

setLabel(node, "alpha beta")
printStatus(node)

println "trimLabels()..."
jenkins.model.Jenkins.instance.trimLabels()
printStatus(node)

Jenkins version: 2.320
trimLabels()...
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: []
Nodes with label alpha&&beta: []
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: []
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
trimLabels(node)...
method unavailable.
trimLabels()...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: []
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---
trimLabels()...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---

Jenkins version: 2.321
trimLabels()...
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: []
Nodes with label alpha&&beta: []
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
trimLabels(node)...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
trimLabels()...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: []
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
trimLabels()...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---

Jenkins version: 2.341
trimLabels()...
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: []
Nodes with label alpha&&beta: []
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---
trimLabels(node)...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---
trimLabels()...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, beta]
Nodes with label alpha: []
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: []
---
Setting label on JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu to 'alpha beta'
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---
trimLabels()...
---
Node: JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu
Label String: alpha beta
Labels: [JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu, alpha, beta]
Nodes with label alpha: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
Nodes with label alpha&&beta: [org.jenkinsci.plugins.vSphereCloudProvisionedSlave[JenkinsDevel_RHEL82.IPV6.BIOS_8nl5bsa9jn2ypb7lv3vc6z3zu]]
---

is caused by

JENKINS-67099 trimLabels calling Cloud#canProvision too often

Closed

Details

Description

Attachments

Issue Links

Activity

People

Dates