Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-71450

Kubernetes Cloud Config Missing after upgrading Jenkins LTS version

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • kubernetes-plugin
    • None

      Environment: Deployed Jenkins instances onto our Kubernetes cluster. 

      Our Jenkins instances use the Kubernetes plugin and we add some custom configuration in the <clouds> section of the config.xml file related to the Kubernetes cloud. 

      Problem: After upgrading the Jenkins LTS core version, and upgrading the Kubernetes plugin, the entire <cloud> section of the config.xml is missing. We have to manually add the configuration back to the config.xml and restart the instance. 

      What is causing this and how can we fix it? 

       

      How to reproduce: 

      If possible, deploy Jenkins instance onto Kubernetes cluster using the Jenkins helm chart with image tag jenkins:jenkins/2.375.2. Add the Kubernetes plugin, version 1.30.11. 

      In the JENKINS_HOME/config.xml file, add <cloud> configuration similar to the following (this is a customer configuration we add upon Jenkins instance creation). You will need to replace the cloud name, namespace and instance name according to your environment.  
       
      <clouds><org.csanchez.jenkins.plugins.kubernetes.KubernetesCloudplugin="kubernetes@1.30.11"><name>cloud-name</name><defaultsProviderTemplate></defaultsProviderTemplate><templates><org.csanchez.jenkins.plugins.kubernetes.PodTemplate><id>jenkins-slave-k8s-namespace-pod-name-be6daa98f9ed6bfbf1d25655c0864a5a</id><inheritFrom></inheritFrom><name>jenkins-slave</name><namespace>k8s-namespace</namespace><privileged>false</privileged><supplementalGroups></supplementalGroups><capOnlyOnAlivePods>false</capOnlyOnAlivePods><alwaysPullImage>false</alwaysPullImage><instanceCap>2147483647</instanceCap><slaveConnectTimeout>100</slaveConnectTimeout><idleMinutes>0</idleMinutes><activeDeadlineSeconds>0</activeDeadlineSeconds><label>jenkins-slave-k8s-namespace-pod-name</label><serviceAccount>k8s-namespace-pod-name-sa</serviceAccount><nodeSelector>kubernetes.io/os=linux</nodeSelector><nodeUsageMode>NORMAL</nodeUsageMode><hostNetwork>false</hostNetwork><workspaceVolumeclass="org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume"><memory>true</memory></workspaceVolume><volumes/><containers><org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate><name>jnlp</name><image>jenkins/jnlp-slave:alpine</image><privileged>false</privileged><alwaysPullImage>false</alwaysPullImage><workingDir>/home/jenkins</workingDir><command></command><args>${computer.jnlpmac} ${computer.name}</args><ttyEnabled>true</ttyEnabled><resourceRequestCpu></resourceRequestCpu><resourceRequestMemory></resourceRequestMemory><resourceLimitCpu></resourceLimitCpu><resourceLimitMemory></resourceLimitMemory><envVars><org.csanchez.jenkins.plugins.kubernetes.model.KeyValueEnvVar><key>JENKINS_URL</key><value>http://k8s-namespace-pod-name-svc.jenkins-master-ns.svc.cluster.local:8080/k8s-namespace-pod-name</value></org.csanchez.jenkins.plugins.kubernetes.model.KeyValueEnvVar></envVars><ports/><livenessProbe><execArgs></execArgs><timeoutSeconds>0</timeoutSeconds><initialDelaySeconds>0</initialDelaySeconds><failureThreshold>0</failureThreshold><periodSeconds>0</periodSeconds><successThreshold>0</successThreshold></livenessProbe></org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate></containers><envVars/><annotations/><imagePullSecrets/><nodeProperties/><yamlMergeStrategyclass="org.csanchez.jenkins.plugins.kubernetes.pod.yaml.Overrides"/><showRawYaml>true</showRawYaml><podRetentionclass="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/></org.csanchez.jenkins.plugins.kubernetes.PodTemplate></templates><serverUrl>https://k8s-cluster-url/</serverUrl><useJenkinsProxy>false</useJenkinsProxy><skipTlsVerify>true</skipTlsVerify><addMasterProxyEnvVars>false</addMasterProxyEnvVars><capOnlyOnAlivePods>false</capOnlyOnAlivePods><namespace>k8s-namespace</namespace><webSocket>false</webSocket><directConnection>false</directConnection><jenkinsUrl>http://k8s-namespace-pod-name-svc.jenkins-master-ns.svc.cluster.local:8080/k8s-namespace-pod-name</jenkinsUrl><jenkinsTunnel>k8s-namespace-pod-name-svc-slave.jenkins-master-ns.svc.cluster.local:50000</jenkinsTunnel><containerCap>10</containerCap><retentionTimeout>5</retentionTimeout><connectTimeout>5</connectTimeout><readTimeout>15</readTimeout><podLabels/><usageRestricted>false</usageRestricted><maxRequestsPerHost>32</maxRequestsPerHost><waitForPodSec>600</waitForPodSec><podRetentionclass="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/></org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud>
      </clouds>
       
      Then upgrade the instance by modifying the 'image' tag in the helm chart to: jenkins:jenkins/2.387.2. 

      Also upgrade the Kubernetes plugin to version 3937.vd7b_82db_e347b_, and check the JENKINS_HOME/config.xml again - and you will see the <cloud> section missing.

          [JENKINS-71450] Kubernetes Cloud Config Missing after upgrading Jenkins LTS version

          Mark Waite added a comment -

          sslk you'll need to provide more details so that others can duplicate the issue that you're having. We regularly upgrade ci.jenkins.io to the most recent LTS version and its kubernetes plugin configuration is retained across those version upgrades. See "How to report an issue" for the types of information that need to be provided.

          Mark Waite added a comment - sslk you'll need to provide more details so that others can duplicate the issue that you're having. We regularly upgrade ci.jenkins.io to the most recent LTS version and its kubernetes plugin configuration is retained across those version upgrades. See "How to report an issue" for the types of information that need to be provided.

          Dann added a comment -

          My case

          I deployed jenkins(jenkins/jenkins:lts-jdk11) on omv docker(24.0.7) and installed kubernetes-plugin(kubernetes@4151.v6fa_f0fb_0b_4c9)

          Then I configured in manage jenkins > clouds > New cloud

          It works properly. I can run all my pipelines in a rke2 kubernetes cluster but after restart or recreate jenkins container it loses cloud configurations. JENKINS_HOME is well configured in a persistent volume so any other configs or pipelines are identitcal but clouds configures in a manage jenkins.

          It's same in jenkins/jenkins:lts-jdk17, too.

           

          Dann added a comment - My case I deployed jenkins(jenkins/jenkins:lts-jdk11) on omv docker(24.0.7) and installed kubernetes-plugin(kubernetes@4151.v6fa_f0fb_0b_4c9) Then I configured in manage jenkins > clouds > New cloud It works properly. I can run all my pipelines in a rke2 kubernetes cluster but after restart or recreate jenkins container it loses cloud configurations. JENKINS_HOME is well configured in a persistent volume so any other configs or pipelines are identitcal but clouds configures in a manage jenkins. It's same in jenkins/jenkins:lts-jdk17, too.  

            Unassigned Unassigned
            sslk Surya
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: