Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62646

Failed to initialize Kubernetes secret provider

XMLWordPrintable

      Hi,

      We've been seeing the exception below for a while now. Since upgrading to Jenkins 2.240 and Kubernetes Plugin 1.26 it has started to add a permanent notification every N minutes. When I logged in today, there were about 9480 notifications and it was keeping on adding new notifications. Those won't go away until you restart Jenkins. 

      Failed to initialize Kubernetes secret providerio.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://10.43.0.1/api/v1/namespaces/jenkins/secrets?labelSelector=jenkins.io%2Fcredentials-type. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. secrets is forbidden: User "system:serviceaccount:jenkins:default" cannot list resource "secrets" in API group "" in the namespace "jenkins".
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:568)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:505)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:471)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:430)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:412)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:151)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:621)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:70)
	at com.cloudbees.jenkins.plugins.kubernetes_credentials_provider.KubernetesCredentialProvider.startWatchingForSecrets(KubernetesCredentialProvider.java:115)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
	at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1131)
	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

      I'm not sure why is it complaining, since executing a curl request to the above endpoint using the jenkin's user token gives a successful result (it simply has no secrets configured in the namespace):

      $  curl -k "https://10.43.0.1/api/v1/namespaces/jenkins/secrets?labelSelector=jenkins.io%2Fcredentials-type" -H "Authorization: Bearer $JENKINS_USER_TOKEN"
{
  "kind": "SecretList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/namespaces/jenkins/secrets",
    "resourceVersion": "63369926"
  },
  "items": []
}

       

        1. image-2020-07-21-20-53-33-987.png
          image-2020-07-21-20-53-33-987.png
          71 kB
        2. image-2020-07-17-16-39-06-763.png
          image-2020-07-17-16-39-06-763.png
          38 kB
        3. image-2020-07-15-11-02-43-434.png
          image-2020-07-15-11-02-43-434.png
          0.3 kB

          [JENKINS-62646] Failed to initialize Kubernetes secret provider

          Steve Todorov created issue -
          Steve Todorov made changes -
          Assignee New: Vincent Latombe [ vlatombe ]
          Vincent Latombe made changes -
          Labels Original: kubernetes-plugin security
          Vincent Latombe made changes -
          Component/s New: kubernetes-credentials-provider-plugin [ 23531 ]
          Component/s Original: kubernetes-plugin [ 20639 ]
          Vincent Latombe made changes -
          Environment Original: Jenkins: 2.240 (running on k8s in a dedicated namespace)
          Kubernetes Plugin: 1.26.0           		New: Jenkins: 2.240 (running on k8s in a dedicated namespace)
          Vincent Latombe made changes -
          Assignee Original: Vincent Latombe [ vlatombe ] New: James Nord [ teilo ]
          LUCIANO BEJA made changes -
          Attachment New: image-2020-07-15-11-02-43-434.png [ 51913 ]
          LUCIANO BEJA made changes -
          Attachment New: image-2020-07-17-16-39-06-763.png [ 51937 ]
          Steve Todorov made changes -
          Attachment New: image-2020-07-21-20-53-33-987.png [ 51955 ]

            teilo James Nord
            stodorov Steve Todorov
            Votes:
            5 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: