We see an interesting behavior with the node config.xml with latest LTS Jenkins version in combination with Java17 and Java21.
In short:
The config.xml of a node gets corrupted after saving the configurations if a specific path is followed (see repro steps).
Repro steps:
Jenkins running with Java 17 (or even 21) version.
1.) Connect node to Jenkins
1.1) -> config.xml still fine
2.) Set node temporarily offline in Jenkins UI (no cause provided = null)
2.1) -> config.xml still fine
3.) In terminal cut / disconnect node (kill java connection)
3.1) -> config.xml still fine (in node UI overview has changed to "agent broken connection")
4.) Open node configuration and just press save. Save works, but throws an exception in the system log. The file is now corrupted.
4.1) -> config.xml file of node is corrupted.
Checkt config.xml path: <jenkins_url>/computer/<node>/config.xml
Corrupted config.xml of node:
<?xml version="1.1" encoding="UTF-8"?> <slave> <temporaryOfflineCause class=" hudson.slaves.OfflineCause$ChannelTermination "> <timestamp>1732105207072</timestamp> <cause class="java.nio.channels.ClosedChannelException"
non-valid xml format in this case. (Not more is saved to the file).
Exception in Jenkins System Log:
Unable to complete save, temporary offline status will not be persisted: java.lang.RuntimeException: Failed to serialize hudson.model.Node#temporaryOfflineCause for class hudson.slaves.DumbSlave
Exception:
java.lang.reflect.InaccessibleObjectException: Unable to make field private static final long java.nio.channels.ClosedChannelException.serialVersionUID accessible: module java.base does not "opens java.nio.channels" to unnamed module @28194a50 ... (full exception to long to enter here)
Works still fine with Java11. But since enforcement of Java 17 from the latest LTS version, we run into this problem.
Longer Jenkins system log exception:
java.lang.reflect.InaccessibleObjectException: Unable to make field private static final long java.nio.channels.ClosedChannelException.serialVersionUID accessible: module java.base does not "opens java.nio.channels" to unnamed module @533377b at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354) at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297) at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178) at java.base/java.lang.reflect.Field.setAccessible(Field.java:172) at com.thoughtworks.xstream.converters.reflection.FieldDictionary.buildDictionaryEntryForClass(FieldDictionary.java:176) at com.thoughtworks.xstream.converters.reflection.FieldDictionary.buildMap(FieldDictionary.java:142) at com.thoughtworks.xstream.converters.reflection.FieldDictionary.fieldsFor(FieldDictionary.java:80) at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:167) at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:206) at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:163) at com.thoughtworks.xstream.converters.extended.ThrowableConverter.marshal(ThrowableConverter.java:62) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:68) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:83) at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:283) at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:270) Caused: java.lang.RuntimeException: Failed to serialize hudson.slaves.OfflineCause$ChannelTermination#cause for class hudson.slaves.OfflineCause$ChannelTermination at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:274) at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:241) at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:174) at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:226) at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:163) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:68) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:83) at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:283) at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:270) Caused: java.lang.RuntimeException: Failed to serialize hudson.model.Node#temporaryOfflineCause for class hudson.slaves.DumbSlave at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:274) at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:241)
I can't duplicate the issue with Jenkins 2.486 (weekly) or with Jenkins 2.479.1 LTS. Can you provide more details so that others can duplicate the problem from a fresh Jenkins installation? Plugins and their versions may be especially important in this case. The plugin versions script can generate that report
My current local setup, where i run in the same issue:
Our prod had Jenkins version: LTS 2.462.3
Jenkins: 2.479.1 OS: Linux - 5.15.0-126-generic Java: 17.0.13 - Ubuntu (OpenJDK 64-Bit Server VM) --- antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 asm-api:9.7.1-97.v4cc844130d97 bootstrap5-api:5.3.3-1 bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_ branch-api:2.1200.v4b_a_3da_2eb_db_4 build-name-setter:2.4.3 build-timeout:1.33 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.2.1 cloudbees-folder:6.955.v81e2a_35c08d3 commons-lang3-api:3.17.0-84.vb_b_938040b_078 commons-text-api:1.12.0-129.v99a_50df237f7 configuration-as-code:1887.v9e47623cb_043 copyartifact:757.v05365583a_455 credentials:1389.vd7a_b_f5fa_50a_2 credentials-binding:687.v619cb_15e923f data-tables-api:2.1.8-1 display-url-api:2.209.v582ed814ff2f durable-task:577.v2a_8a_4b_7c0247 echarts-api:5.5.1-4 eddsa-api:0.3.0-4.v84c6f0f4969e email-ext:1849.v6dd03b_f6e423 font-awesome-api:6.6.0-2 git:5.6.0 git-client:6.1.0 gson-api:2.11.0-85.v1f4e87273c33 instance-identity:201.vd2a_b_5a_468a_a_6 ionicons-api:74.v93d5eb_813d5f jackson2-api:2.17.0-379.v02de8ec9f64c jakarta-activation-api:2.1.3-1 jakarta-mail-api:2.1.3-1 javax-activation-api:1.2.0-7 javax-mail-api:1.6.2-10 jaxb:2.3.9-1 jobConfigHistory:1283.veb_dfb_00b_5ec0 joda-time-api:2.13.0-93.v9934da_29b_a_e9 jquery3-api:3.7.1-2 json-api:20240303-101.v7a_8666713110 json-path-api:2.9.0-118.v7f23ed82a_8b_8 junit:1307.vdd5b_2646279e ldap:770.vb_455e934581a_ lockable-resources:1327.ved786b_a_197e0 mailer:489.vd4b_25144138f matrix-auth:3.2.3 matrix-project:840.v812f627cb_578 mina-sshd-api-common:2.14.0-133.vcc091215a_358 mina-sshd-api-core:2.14.0-133.vcc091215a_358 pam-auth:1.11 pipeline-build-step:540.vb_e8849e1a_b_d8 pipeline-graph-analysis:216.vfd8b_ece330ca_ pipeline-groovy-lib:744.v5b_556ee7c253 pipeline-input-step:495.ve9c153f6067b_ pipeline-milestone-step:119.vdfdc43fc3b_9a_ pipeline-model-api:2.2214.vb_b_34b_2ea_9b_83 pipeline-model-definition:2.2214.vb_b_34b_2ea_9b_83 pipeline-model-extensions:2.2214.vb_b_34b_2ea_9b_83 pipeline-rest-api:2.34 pipeline-stage-step:312.v8cd10304c27a_ pipeline-stage-tags-metadata:2.2214.vb_b_34b_2ea_9b_83 pipeline-stage-view:2.34 plain-credentials:183.va_de8f1dd5a_2b_ plugin-util-api:5.1.0 prism-api:1.29.0-17 rebuild:332.va_1ee476d8f6d resource-disposer:0.25 role-strategy:743.v142ea_b_d5f1d3 scm-api:698.v8e3b_c788f0a_6 script-security:1367.vdf2fc45f229c snakeyaml-api:2.3-123.v13484c65210a_ ssh-credentials:343.v884f71d78167 ssh-slaves:2.973.v0fa_8c0dea_f9f structs:338.v848422169819 timestamper:1.28 token-macro:400.v35420b_922dcb_ trilead-api:2.147.vb_73cc728a_32e variant:60.v7290fc0eb_b_cd workflow-aggregator:600.vb_57cdd26fdd7 workflow-api:1336.vee415d95c521 workflow-basic-steps:1058.vcb_fc1e3a_21a_9 workflow-cps:3990.vd281dd77a_388 workflow-durable-task-step:1371.vb_7cec8f3b_95e workflow-job:1460.v28178c1ef6e6 workflow-multibranch:795.ve0cb_1f45ca_9a_ workflow-scm-step:427.v4ca_6512e7df1 workflow-step-api:678.v3ee58b_469476 workflow-support:932.vb_555de1b_a_b_94 ws-cleanup:0.48
Thanks for the feedback, i check on fresh installation + update all kind of plugins first to see if that helps.
I'll try to provide a more detailed way of what i did to run into the issue, maybe i miss something.
Same issue with LTS 2.479.1 and fresh installation with "default" installed plugins
openjdk 17.0.13 2024-10-15
OpenJDK Runtime Environment (build 17.0.13+11-Ubuntu-2ubuntu122.04)
OpenJDK 64-Bit Server VM (build 17.0.13+11-Ubuntu-2ubuntu122.04, mixed mode, sharing)
Dummy start "./start_jenkins.sh" bash.
#!/bin/bash echo "Starting Jenkins ..." export JENKINS_HOME=/var/jenkins_home echo "JENKINS_HOME=${JENKINS_HOME}" java -jar jenkins.war
Create a new (permanent) node in Jenkins
Open new Node view and execute in terminal:
curl -sO <my_jenkins_url>/jnlpJars/agent.jar
java -jar agent.jar -url <my_jenkins_url> -secret <my_secret> -name blub -webSocket -workDir "/var/jenkins_home/test123"
Go back to Jenkins UI, set the connected node via UI temporarily offline.
Cut the node connection in the terminal with "Ctrl+C".
Agent UI switches to:
Go to "Configure > Save (without any changes)" which works - no Jenkins error just exception in system log.
Checking
/computer/blub/config.xml
shows corrupted xml.
Since it's related to Java it might be openJDK version, but there i'm not sure and not deep enough into Java topics in general.
// Edit: Just as reference, "Bring this node back online" or "Update offline reason" writes back a valid config.xml. But as long as this does not happen the config.xml stays broken.
Just to have everything together, with a "fresh" Jenkins installation:
Jenkins: 2.479.1
OS: Linux - 5.15.0-126-generic
Java: 17.0.13 - Ubuntu (OpenJDK 64-Bit Server VM)
---
ant:511.v0a_a_1a_334f41b_
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
asm-api:9.7.1-97.v4cc844130d97
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_
branch-api:2.1200.v4b_a_3da_2eb_db_4
build-timeout:1.33
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.1
cloudbees-folder:6.959.v4ed5cc9e2dd4
commons-lang3-api:3.17.0-84.vb_b_938040b_078
commons-text-api:1.12.0-129.v99a_50df237f7
credentials:1389.vd7a_b_f5fa_50a_2
credentials-binding:687.v619cb_15e923f
dark-theme:479.v661b_1b_911c01
display-url-api:2.209.v582ed814ff2f
durable-task:577.v2a_8a_4b_7c0247
echarts-api:5.5.1-4
eddsa-api:0.3.0-4.v84c6f0f4969e
email-ext:1855.vd9e491cb_de1e
font-awesome-api:6.6.0-2
git:5.6.0
git-client:6.1.0
github:1.40.0
github-api:1.321-478.vc9ce627ce001
github-branch-source:1807.v50351eb_7dd13
gradle:2.13.1
gson-api:2.11.0-85.v1f4e87273c33
instance-identity:201.vd2a_b_5a_468a_a_6
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javax-activation-api:1.2.0-7
jaxb:2.3.9-1
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
joda-time-api:2.13.0-93.v9934da_29b_a_e9
jquery3-api:3.7.1-2
json-api:20240303-101.v7a_8666713110
json-path-api:2.9.0-118.v7f23ed82a_8b_8
junit:1309.v0078b_fecd6ed
ldap:770.vb_455e934581a_
mailer:489.vd4b_25144138f
matrix-auth:3.2.3
matrix-project:840.v812f627cb_578
metrics:4.2.21-458.vcf496cb_839e4
mina-sshd-api-common:2.14.0-133.vcc091215a_358
mina-sshd-api-core:2.14.0-133.vcc091215a_358
okhttp-api:4.11.0-181.v1de5b_83857df
pam-auth:1.11
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:61.v629f2cc41d83
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-graph-view:382.vb_9a_27b_7b_ea_71
pipeline-groovy-lib:744.v5b_556ee7c253
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2218.v56d0cda_37c72
pipeline-model-definition:2.2218.v56d0cda_37c72
pipeline-model-extensions:2.2218.v56d0cda_37c72
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2218.v56d0cda_37c72
plain-credentials:183.va_de8f1dd5a_2b_
plugin-util-api:5.1.0
resource-disposer:0.25
scm-api:698.v8e3b_c788f0a_6
script-security:1369.v9b_98a_4e95b_2d
snakeyaml-api:2.3-123.v13484c65210a_
ssh-credentials:343.v884f71d78167
ssh-slaves:2.973.v0fa_8c0dea_f9f
structs:338.v848422169819
theme-manager:262.vc57ee4a_eda_5d
timestamper:1.28
token-macro:400.v35420b_922dcb_
trilead-api:2.147.vb_73cc728a_32e
variant:60.v7290fc0eb_b_cd
workflow-aggregator:600.vb_57cdd26fdd7
workflow-api:1336.vee415d95c521
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:3993.v3e20a_37282f8
workflow-durable-task-step:1378.v6a_3e903058a_3
workflow-job:1468.vcf4f5ee92395
workflow-multibranch:795.ve0cb_1f45ca_9a_
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:678.v3ee58b_469476
workflow-support:932.vb_555de1b_a_b_94
ws-cleanup:0.48
It seems this only affect the config.xml that is delivered via the url. The config.xml that is found in the file system at JENKINS_HOME/nodes/<name>/config.xml looks good.
It seems this only affect the config.xml that is delivered via the url. The config.xml that is found in the file system at JENKINS_HOME/nodes/<name>/config.xml looks good.
That's right, the config.xml on the controller / file system looks good but misses the
<temporaryOfflineCause ...>
tag. So the offline cause tag just vanishes in this case. If we restart Jenkins the cause is missing but via url a valid config.xml is returned.
Missed that, had to be clearer that the config.xml retrieved via the Jenkins url is corrupted.
Result of config.xml with Jenkins running with Java 11:
<?xml version="1.1" encoding="UTF-8"?>
<slave>
<temporaryOfflineCause class="hudson.slaves.OfflineCause$ChannelTermination">
<timestamp>1732108674193</timestamp>
<cause class="java.nio.channels.ClosedChannelException">
<stackTrace>
<trace>org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:155)</trace>
<trace>org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:143)</trace>
<trace>org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:789)</trace>
<trace>jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)</trace>
<trace>jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)</trace>
<trace>jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51)</trace>
<trace>java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)</trace>
<trace>java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)</trace>
<trace>java.base/java.lang.Thread.run(Thread.java:829)</trace>
</stackTrace>
<suppressedExceptions class="empty-list"/>
</cause>
</temporaryOfflineCause>
<name>Foobar</name>
<description></description>
<remoteFS>/home/jenkins/tools/jenkins-node/workspace1</remoteFS>
<numExecutors>1</numExecutors>
<mode>NORMAL</mode>
<retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>
<launcher class="hudson.slaves.JNLPLauncher">
<workDirSettings>
<disabled>false</disabled>
<internalDir>remoting</internalDir>
<failIfWorkDirIsMissing>false</failIfWorkDirIsMissing>
</workDirSettings>
<webSocket>false</webSocket>
</launcher>
<label>windows swarm</label>
<nodeProperties/>
</slave>