Environment: Deployed Jenkins instances onto our Kubernetes cluster.
Our Jenkins instances use the Kubernetes plugin and we add some custom configuration in the <clouds> section of the config.xml file related to the Kubernetes cloud.
Problem: After upgrading the Jenkins LTS core version, and upgrading the Kubernetes plugin, the entire <cloud> section of the config.xml is missing. We have to manually add the configuration back to the config.xml and restart the instance.
What is causing this and how can we fix it?
How to reproduce:
If possible, deploy Jenkins instance onto Kubernetes cluster using the Jenkins helm chart with image tag jenkins:jenkins/2.375.2. Add the Kubernetes plugin, version 1.30.11.
In the JENKINS_HOME/config.xml file, add <cloud> configuration similar to the following (this is a customer configuration we add upon Jenkins instance creation). You will need to replace the cloud name, namespace and instance name according to your environment.
<clouds><org.csanchez.jenkins.plugins.kubernetes.KubernetesCloudplugin="kubernetes@1.30.11"><name>cloud-name</name><defaultsProviderTemplate></defaultsProviderTemplate><templates><org.csanchez.jenkins.plugins.kubernetes.PodTemplate><id>jenkins-slave-k8s-namespace-pod-name-be6daa98f9ed6bfbf1d25655c0864a5a</id><inheritFrom></inheritFrom><name>jenkins-slave</name><namespace>k8s-namespace</namespace><privileged>false</privileged><supplementalGroups></supplementalGroups><capOnlyOnAlivePods>false</capOnlyOnAlivePods><alwaysPullImage>false</alwaysPullImage><instanceCap>2147483647</instanceCap><slaveConnectTimeout>100</slaveConnectTimeout><idleMinutes>0</idleMinutes><activeDeadlineSeconds>0</activeDeadlineSeconds><label>jenkins-slave-k8s-namespace-pod-name</label><serviceAccount>k8s-namespace-pod-name-sa</serviceAccount><nodeSelector>kubernetes.io/os=linux</nodeSelector><nodeUsageMode>NORMAL</nodeUsageMode><hostNetwork>false</hostNetwork><workspaceVolumeclass="org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume"><memory>true</memory></workspaceVolume><volumes/><containers><org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate><name>jnlp</name><image>jenkins/jnlp-slave:alpine</image><privileged>false</privileged><alwaysPullImage>false</alwaysPullImage><workingDir>/home/jenkins</workingDir><command></command><args>${computer.jnlpmac} ${computer.name}</args><ttyEnabled>true</ttyEnabled><resourceRequestCpu></resourceRequestCpu><resourceRequestMemory></resourceRequestMemory><resourceLimitCpu></resourceLimitCpu><resourceLimitMemory></resourceLimitMemory><envVars><org.csanchez.jenkins.plugins.kubernetes.model.KeyValueEnvVar><key>JENKINS_URL</key><value>http://k8s-namespace-pod-name-svc.jenkins-master-ns.svc.cluster.local:8080/k8s-namespace-pod-name</value></org.csanchez.jenkins.plugins.kubernetes.model.KeyValueEnvVar></envVars><ports/><livenessProbe><execArgs></execArgs><timeoutSeconds>0</timeoutSeconds><initialDelaySeconds>0</initialDelaySeconds><failureThreshold>0</failureThreshold><periodSeconds>0</periodSeconds><successThreshold>0</successThreshold></livenessProbe></org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate></containers><envVars/><annotations/><imagePullSecrets/><nodeProperties/><yamlMergeStrategyclass="org.csanchez.jenkins.plugins.kubernetes.pod.yaml.Overrides"/><showRawYaml>true</showRawYaml><podRetentionclass="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/></org.csanchez.jenkins.plugins.kubernetes.PodTemplate></templates><serverUrl>https://k8s-cluster-url/</serverUrl><useJenkinsProxy>false</useJenkinsProxy><skipTlsVerify>true</skipTlsVerify><addMasterProxyEnvVars>false</addMasterProxyEnvVars><capOnlyOnAlivePods>false</capOnlyOnAlivePods><namespace>k8s-namespace</namespace><webSocket>false</webSocket><directConnection>false</directConnection><jenkinsUrl>http://k8s-namespace-pod-name-svc.jenkins-master-ns.svc.cluster.local:8080/k8s-namespace-pod-name</jenkinsUrl><jenkinsTunnel>k8s-namespace-pod-name-svc-slave.jenkins-master-ns.svc.cluster.local:50000</jenkinsTunnel><containerCap>10</containerCap><retentionTimeout>5</retentionTimeout><connectTimeout>5</connectTimeout><readTimeout>15</readTimeout><podLabels/><usageRestricted>false</usageRestricted><maxRequestsPerHost>32</maxRequestsPerHost><waitForPodSec>600</waitForPodSec><podRetentionclass="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/></org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud>
</clouds>
Then upgrade the instance by modifying the 'image' tag in the helm chart to: jenkins:jenkins/2.387.2.
Also upgrade the Kubernetes plugin to version 3937.vd7b_82db_e347b_, and check the JENKINS_HOME/config.xml again - and you will see the <cloud> section missing.
[JENKINS-71450] Kubernetes Cloud Config Missing after upgrading Jenkins LTS version
Description |
Original:
Environment: Deployed Jenkins instances onto our Kubernetes cluster.
Our instances use the Kubernetes plugin and we add some custom configuration in the <clouds> section of the config.xml file related to the Kubernetes cloud. Problem: After upgrading the Jenkins LTS core version, and upgrading the Kubernetes plugin, the entire <cloud> section of the config.xml is missing. We have to manually add the configuration back to the config.xml and restart the instance. What is causing this and how can we fix it? |
New:
Environment: Deployed Jenkins instances onto our Kubernetes cluster.
Our Jenkins instances use the Kubernetes plugin and we add some custom configuration in the <clouds> section of the config.xml file related to the Kubernetes cloud. Problem: After upgrading the Jenkins LTS core version, and upgrading the Kubernetes plugin, the entire <cloud> section of the config.xml is missing. We have to manually add the configuration back to the config.xml and restart the instance. What is causing this and how can we fix it? How to reproduce: If possible, deploy Jenkins instance onto Kubernetes cluster using the Jenkins helm chart with image tag jenkins:jenkins/2.375.2. Add the Kubernetes plugin, version 1.30.11. In the JENKINS_HOME/config.xml file, add <cloud> configuration similar to the following (this is a customer configuration we add upon Jenkins instance creation). You will need to replace the cloud name, namespace and instance name according to your environment. {{{}<clouds>{}}}{{{}<org.csanchez.jenkins.plugins.kubernetes.KubernetesCloudplugin="kubernetes@1.30.11">{}}}{{{}<name>cloud-name</name>{}}}{{{}<defaultsProviderTemplate></defaultsProviderTemplate>{}}}{{{}<templates>{}}}{{{}<org.csanchez.jenkins.plugins.kubernetes.PodTemplate>{}}}{{{}<id>jenkins-slave-k8s-namespace-pod-name-be6daa98f9ed6bfbf1d25655c0864a5a</id>{}}}{{{}<inheritFrom></inheritFrom>{}}}{{{}<name>jenkins-slave</name>{}}}{{{}<namespace>k8s-namespace</namespace>{}}}{{{}<privileged>false</privileged>{}}}{{{}<supplementalGroups></supplementalGroups>{}}}{{{}<capOnlyOnAlivePods>false</capOnlyOnAlivePods>{}}}{{{}<alwaysPullImage>false</alwaysPullImage>{}}}{{{}<instanceCap>2147483647</instanceCap>{}}}{{{}<slaveConnectTimeout>100</slaveConnectTimeout>{}}}{{{}<idleMinutes>0</idleMinutes>{}}}{{{}<activeDeadlineSeconds>0</activeDeadlineSeconds>{}}}{{{}<label>jenkins-slave-k8s-namespace-pod-name</label>{}}}{{{}<serviceAccount>k8s-namespace-pod-name-sa</serviceAccount>{}}}{{{}<nodeSelector>kubernetes.io/os=linux</nodeSelector>{}}}{{{}<nodeUsageMode>NORMAL</nodeUsageMode>{}}}{{{}<hostNetwork>false</hostNetwork>{}}}{{{}<workspaceVolumeclass="org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume">{}}}{{{}<memory>true</memory>{}}}{{{}</workspaceVolume>{}}}{{{}<volumes/>{}}}{{{}<containers>{}}}{{{}<org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>{}}}{{{}<name>jnlp</name>{}}}{{{}<image>jenkins/jnlp-slave:alpine</image>{}}}{{{}<privileged>false</privileged>{}}}{{{}<alwaysPullImage>false</alwaysPullImage>{}}}{{{}<workingDir>/home/jenkins</workingDir>{}}}{{{}<command></command>{}}}{{{}<args>${computer.jnlpmac} ${computer.name}</args>{}}}{{{}<ttyEnabled>true</ttyEnabled>{}}}{{{}<resourceRequestCpu></resourceRequestCpu>{}}}{{{}<resourceRequestMemory></resourceRequestMemory>{}}}{{{}<resourceLimitCpu></resourceLimitCpu>{}}}{{{}<resourceLimitMemory></resourceLimitMemory>{}}}{{{}<envVars>{}}}{{{}<org.csanchez.jenkins.plugins.kubernetes.model.KeyValueEnvVar>{}}}{{{}<key>JENKINS_URL</key>{}}}{{{}<value>http://k8s-namespace-pod-name-svc.jenkins-master-ns.svc.cluster.local:8080/k8s-namespace-pod-name</value>{}}}{{{}</org.csanchez.jenkins.plugins.kubernetes.model.KeyValueEnvVar>{}}}{{{}</envVars>{}}}{{{}<ports/>{}}}{{{}<livenessProbe>{}}}{{{}<execArgs></execArgs>{}}}{{{}<timeoutSeconds>0</timeoutSeconds>{}}}{{{}<initialDelaySeconds>0</initialDelaySeconds>{}}}{{{}<failureThreshold>0</failureThreshold>{}}}{{{}<periodSeconds>0</periodSeconds>{}}}{{{}<successThreshold>0</successThreshold>{}}}{{{}</livenessProbe>{}}}{{{}</org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>{}}}{{{}</containers>{}}}{{{}<envVars/>{}}}{{{}<annotations/>{}}}{{{}<imagePullSecrets/>{}}}{{{}<nodeProperties/>{}}}{{{}<yamlMergeStrategyclass="org.csanchez.jenkins.plugins.kubernetes.pod.yaml.Overrides"/>{}}}{{{}<showRawYaml>true</showRawYaml>{}}}{{{}<podRetentionclass="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/>{}}}{{{}</org.csanchez.jenkins.plugins.kubernetes.PodTemplate>{}}}{{{}</templates>{}}}{{{}<serverUrl>https://k8s-cluster-url/</serverUrl>{}}}{{{}<useJenkinsProxy>false</useJenkinsProxy>{}}}{{{}<skipTlsVerify>true</skipTlsVerify>{}}}{{{}<addMasterProxyEnvVars>false</addMasterProxyEnvVars>{}}}{{{}<capOnlyOnAlivePods>false</capOnlyOnAlivePods>{}}}{{{}<namespace>k8s-namespace</namespace>{}}}{{{}<webSocket>false</webSocket>{}}}{{{}<directConnection>false</directConnection>{}}}{{{}<jenkinsUrl>http://k8s-namespace-pod-name-svc.jenkins-master-ns.svc.cluster.local:8080/k8s-namespace-pod-name</jenkinsUrl>{}}}{{{}<jenkinsTunnel>k8s-namespace-pod-name-svc-slave.jenkins-master-ns.svc.cluster.local:50000</jenkinsTunnel>{}}}{{{}<containerCap>10</containerCap>{}}}{{{}<retentionTimeout>5</retentionTimeout>{}}}{{{}<connectTimeout>5</connectTimeout>{}}}{{{}<readTimeout>15</readTimeout>{}}}{{{}<podLabels/>{}}}{{{}<usageRestricted>false</usageRestricted>{}}}{{{}<maxRequestsPerHost>32</maxRequestsPerHost>{}}}{{{}<waitForPodSec>600</waitForPodSec>{}}}{{{}<podRetentionclass="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/>{}}}{{{}</org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud>{}}} {{</clouds>}} Then upgrade the instance by modifying the 'image' tag in the helm chart to: jenkins:jenkins/2.387.2. Also upgrade the Kubernetes plugin to version 3937.vd7b_82db_e347b_, and check the JENKINS_HOME/config.xml again - and you will see the <cloud> section missing. |
Environment |
Original:
Jenkins version: 2.375.2, 2.387.2
Kubernetes plugin versions: 1.30.11, 3937.vd7b_82db_e347b_ |
New:
Jenkins version: 2.375.2, 2.387.2
Kubernetes plugin versions: 1.30.11, 3937.vd7b_82db_e347b_ Kubernetes cluster where instance is running: v 1.22 Running Jenkins in a container, deployed using the Jenkins helm chart in our Kubernetes cluster. |