I am using the jenkins-jclouds plugin to create single use slaves to run jenkins jobs. After ~24 Hours jenkins fails to create any new slaves and logs 401 Unauthorized errors.
If I restart jenkins or open the main configuration page and save it, then Jenkins can again create slaves. My guess is that jenkins or jclouds is storing an authentication token which is expiring, restarting/updating config causes that token to be flushed and temporarily solves the problem.
Example stack trace:
Jan 27, 2014 9:14:46 AM hudson.triggers.SafeTimerTask run
SEVERE: Timer task hudson.slaves.NodeProvisioner$NodeProvisionerInvoker@4bdbdaec failed
org.jclouds.rest.AuthorizationException: GET https://lon.servers.api.rackspacecloud.com/v2/********/servers/detail HTTP/
1.1 -> HTTP/1.1 401 Unauthorized
at org.jclouds.openstack.nova.v2_0.handlers.NovaErrorHandler.handleError(NovaErrorHandler.java:96)
at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:67)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:1
80)
at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:150)
at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.invoke(InvokeSyncToAsyncHttpMethod.java:131)
at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:97)
at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:58)
at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:70)
at com.sun.proxy.$Proxy102.listInDetail(Unknown Source)
at org.jclouds.openstack.nova.v2_0.compute.NovaComputeServiceAdapter.listNodes(NovaComputeServiceAdapter.java:199)
at org.jclouds.compute.strategy.impl.AdaptingComputeServiceStrategies.listDetailsOnNodesMatching(AdaptingComputeServiceStrategies.java:124)
at org.jclouds.compute.strategy.impl.AdaptingComputeServiceStrategies.listNodes(AdaptingComputeServiceStrategies.java:114)
at org.jclouds.compute.internal.BaseComputeService.listNodes(BaseComputeService.java:337)
at sun.reflect.GeneratedMethodAccessor388.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:40)
at com.sun.proxy.$Proxy100.listNodes(Unknown Source)
at jenkins.plugins.jclouds.compute.JCloudsCloud.getRunningNodesCount(JCloudsCloud.java:298)
at jenkins.plugins.jclouds.compute.JCloudsCloud.provision(JCloudsCloud.java:209)
at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:281)
at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51)
at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:368)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Caused by: org.jclouds.http.HttpResponseException: request: GET https://lon.servers.api.rackspacecloud.com/v2/******/servers/detail HTTP/1.1 failed with response: HTTP/1.1 401 Unauthorized
at org.jclouds.openstack.nova.v2_0.handlers.NovaErrorHandler.handleError(NovaErrorHandler.java:81)
... 32 more