Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-21566

Periodic Authentication Failure - Token Caching?

XMLWordPrintable

      I am using the jenkins-jclouds plugin to create single use slaves to run jenkins jobs. After ~24 Hours jenkins fails to create any new slaves and logs 401 Unauthorized errors.

      If I restart jenkins or open the main configuration page and save it, then Jenkins can again create slaves. My guess is that jenkins or jclouds is storing an authentication token which is expiring, restarting/updating config causes that token to be flushed and temporarily solves the problem.

      Example stack trace:

      Jan 27, 2014 9:14:46 AM hudson.triggers.SafeTimerTask run
      SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@4bdbdaec failed
      org.jclouds.rest.AuthorizationException: GET https://lon.servers.api.rackspacecloud.com/v2/********/servers/detail HTTP/
      1.1 -> HTTP/1.1 401 Unauthorized
      at org.jclouds.openstack.nova.v2_0.handlers.NovaErrorHandler.handleError(NovaErrorHandler.java:96)
      at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:67)
      at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:1
      80)
      at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:150)
      at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.invoke(InvokeSyncToAsyncHttpMethod.java:131)
      at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:97)
      at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:58)
      at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
      at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:70)
      at com.sun.proxy.$Proxy102.listInDetail(Unknown Source)
      at org.jclouds.openstack.nova.v2_0.compute.NovaComputeServiceAdapter.listNodes(NovaComputeServiceAdapter.java:199)
      at org.jclouds.compute.strategy.impl.AdaptingComputeServiceStrategies.listDetailsOnNodesMatching(AdaptingComputeServiceStrategies.java:124)
      at org.jclouds.compute.strategy.impl.AdaptingComputeServiceStrategies.listNodes(AdaptingComputeServiceStrategies.java:114)
      at org.jclouds.compute.internal.BaseComputeService.listNodes(BaseComputeService.java:337)
      at sun.reflect.GeneratedMethodAccessor388.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:622)
      at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:40)
      at com.sun.proxy.$Proxy100.listNodes(Unknown Source)
      at jenkins.plugins.jclouds.compute.JCloudsCloud.getRunningNodesCount(JCloudsCloud.java:298)
      at jenkins.plugins.jclouds.compute.JCloudsCloud.provision(JCloudsCloud.java:209)
      at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281)
      at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51)
      at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368)
      at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:701)
      Caused by: org.jclouds.http.HttpResponseException: request: GET https://lon.servers.api.rackspacecloud.com/v2/******/servers/detail HTTP/1.1 failed with response: HTTP/1.1 401 Unauthorized
      at org.jclouds.openstack.nova.v2_0.handlers.NovaErrorHandler.handleError(NovaErrorHandler.java:81)
      ... 32 more

            Unassigned Unassigned
            hughsaunders Hugh Saunders
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: