Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58143

Periodically finish in: Exec Failure: HTTP 401, Status: 401 - Unauthorized

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • kubernetes-plugin
    • None

      Hello,

       

      Jun 21, 2019 2:10:17 PM FINE io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager scheduleReconnectSubmitting reconnect task to the executor
      Jun 21, 2019 2:10:17 PM WARNING io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2 onFailureExec Failure: HTTP 401, Status: 401 - Unauthorized
      java.net.ProtocolException: Expected HTTP 101 response but was '401 Unauthorized'
      	at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:219)
      	at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:186)
      	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153)
      	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      
      Jun 21, 2019 2:10:17 PM FINE io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$3 executeScheduling reconnect task
      
      

       

       

      It happens on from time to time. We have other Jenkins master in the same project. The other master works good. 

      Kubernetes plugins in: 1.16.1

      Jenkins master is: Jenkins ver. 2.164.2

          [JENKINS-58143] Periodically finish in: Exec Failure: HTTP 401, Status: 401 - Unauthorized

          have you tried running with a lower number of seconds in -Dorg.csanchez.jenkins.plugins.kubernetes.clients.cacheExpiration=86400
          https://github.com/jenkinsci/kubernetes-plugin/pull/429

          Carlos Sanchez added a comment - have you tried running with a lower number of seconds in -Dorg.csanchez.jenkins.plugins.kubernetes.clients.cacheExpiration=86400 https://github.com/jenkinsci/kubernetes-plugin/pull/429

          I added the option + restarted pod. Monitoring how it behaves. csanchez thank you!

          Andrei Stepanov added a comment - I added the option + restarted pod. Monitoring how it behaves. csanchez thank you!

          The solution provided by csanchez helped. The ticket can be closed. csanchez thank you!

          Andrei Stepanov added a comment - The solution provided by csanchez helped. The ticket can be closed. csanchez thank you!

          Jesse Glick added a comment -

          Good that there was a workaround, but you should not need to set an obscure system property just to make the system work. Any known way to reproduce from scratch?

          Jesse Glick added a comment - Good that there was a workaround, but you should not need to set an obscure system property just to make the system work. Any known way to reproduce from scratch?

          Jesse Glick added a comment -

          I am guessing you are using some sort of time-limited authentication to the cluster. What specifically do your credentials look like? The current “fix” merely refreshes the client after a day, which is better than nothing, but not very discoverable. Possible resolutions include

          • Printing a meaningful error about an expired token somewhere that an administrator will see it (not just the system log).
          • Automatically asking to refresh the token by some means.

          Jesse Glick added a comment - I am guessing you are using some sort of time-limited authentication to the cluster. What specifically do your credentials look like? The current “fix” merely refreshes the client after a day, which is better than nothing, but not very discoverable. Possible resolutions include Printing a meaningful error about an expired token somewhere that an administrator will see it (not just the system log). Automatically asking to refresh the token by some means.

          Jesse Glick added a comment -

          Discussed in kubernetes #548.

          Jesse Glick added a comment - Discussed in kubernetes #548.

          The client is re-created after 10 minutes to cope with this class of failure.

          Vincent Latombe added a comment - The client is re-created after 10 minutes to cope with this class of failure.

            Unassigned Unassigned
            astepano Andrei Stepanov
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: