• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • other
    • None
    • Platform: All, OS: Windows XP

      I get the following thread after running a job on windows. I don't get where
      hudson spends the cycle but it's obviously just sitting and waiting for
      something. The job fails in about 3 minutes and then hudson has been sitting
      there for 8 hours. Since there's a single executor, that pretty much puts the
      node off-line.

      ...
      BUILD FAILED
      Total time: 2 minutes 55 seconds

      Thread Dump
      Thread[main,5,main] (RUNNABLE)
      java.net.PlainSocketImpl.socketAccept(Native Method)
      java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
      java.net.ServerSocket.implAccept(ServerSocket.java:450)
      java.net.ServerSocket.accept(ServerSocket.java:421)
      org.apache.catalina.core.StandardServer.await(StandardServer.java:389)
      org.apache.catalina.startup.Catalina.await(Catalina.java:630)
      org.apache.catalina.startup.Catalina.start(Catalina.java:590)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      java.lang.reflect.Method.invoke(Method.java:585)
      org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
      org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
      ThreadExecutor #0 for opends-xp,5,main (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      hudson.util.OneShotEvent.block(OneShotEvent.java:28)
      hudson.model.Queue.pop(Queue.java:340)
      hudson.model.Executor.run(Executor.java:51)
      Thread[http-8080-2,5,main] (RUNNABLE)
      java.lang.Thread.dumpThreads(Native Method)
      java.lang.Thread.getAllStackTraces(Thread.java:1434)
      hudson.Functions.dumpAllThreads(Functions.java:492)
      sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      java.lang.reflect.Method.invoke(Method.java:585)
      org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:255)
      org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
      org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:82)
      org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:56)
      org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:50)
      org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:79)
      org.apache.commons.jelly.expression.jexl.JexlExpression.evaluate(JexlExpression.java:69)
      org.apache.commons.jelly.expression.ExpressionSupport.evaluateRecurse(ExpressionSupport.java:61)
      org.apache.commons.jelly.expression.ExpressionSupport.evaluateAsIterator(ExpressionSupport.java:94)
      org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:89)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.tags.core.IfTag.doTag(IfTag.java:42)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.tags.core.JellyTag.doTag(JellyTag.java:45)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.DynamicTag.doTag(DynamicTag.java:80)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.impl.StaticTag.doTag(StaticTag.java:65)
      org.apache.commons.jelly.impl.StaticTagScript.run(StaticTagScript.java:112)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.tags.core.JellyTag.doTag(JellyTag.java:45)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.DynamicTag.doTag(DynamicTag.java:80)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:186)
      org.apache.commons.jelly.tags.core.JellyTag.doTag(JellyTag.java:45)
      org.apache.commons.jelly.impl.TagScript.run(TagScript.java:262)
      org.kohsuke.stapler.jelly.JellyClassTearOff.invokeScript(JellyClassTearOff.java:78)
      org.kohsuke.stapler.MetaClass$3.dispatch(MetaClass.java:111)
      org.kohsuke.stapler.Stapler.invoke(Stapler.java:307)
      org.kohsuke.stapler.Stapler.invoke(Stapler.java:244)
      org.kohsuke.stapler.Stapler.service(Stapler.java:92)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      hudson.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:79)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
      org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
      org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
      org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:261)
      org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
      org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:581)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
      java.lang.Thread.run(Thread.java:595)
      Thread[Java2D Disposer,10,system] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
      java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
      sun.java2d.Disposer.run(Disposer.java:125)
      java.lang.Thread.run(Thread.java:595)
      Thread[C:\Users\arnaud\Documents\apache-tomcat-6.0.13\temp\hudson9824.bat:
      stdout copier,5,main] (RUNNABLE)
      java.io.FileInputStream.readBytes(Native Method)
      java.io.FileInputStream.read(FileInputStream.java:194)
      java.io.BufferedInputStream.read1(BufferedInputStream.java:254)
      java.io.BufferedInputStream.read(BufferedInputStream.java:313)
      java.io.FilterInputStream.read(FilterInputStream.java:90)
      hudson.Util.copyStream(Util.java:250)
      hudson.util.StreamCopyThread.run(StreamCopyThread.java:27)
      Thread[TP-Processor1,5,main] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:658)
      java.lang.Thread.run(Thread.java:595)
      Thread[TCP slave agent listener port=0,5,main] (RUNNABLE)
      java.net.PlainSocketImpl.socketAccept(Native Method)
      java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
      java.net.ServerSocket.implAccept(ServerSocket.java:450)
      java.net.ServerSocket.accept(ServerSocket.java:421)
      hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:96)
      Thread[Thread-1,5,main] (TIMED_WAITING)
      java.lang.Object.wait(Native Method)
      hudson.model.ViewJob$ReloadThread.getNext(ViewJob.java:136)
      hudson.model.ViewJob$ReloadThread.run(ViewJob.java:152)
      Thread[C:\Users\arnaud\Documents\apache-tomcat-6.0.13\temp\hudson9824.bat:
      stderr copier,5,main] (RUNNABLE)
      java.io.FileInputStream.readBytes(Native Method)
      java.io.FileInputStream.read(FileInputStream.java:177)
      hudson.Util.copyStream(Util.java:250)
      hudson.util.StreamCopyThread.run(StreamCopyThread.java:27)
      ThreadExecutor #0 for master,5,main (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Thread.join(Thread.java:1095)
      java.lang.Thread.join(Thread.java:1148)
      hudson.Proc$LocalProc.join(Proc.java:101)
      hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:57)
      hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:35)
      hudson.model.Build$RunnerImpl.build(Build.java:147)
      hudson.model.Build$RunnerImpl.doRun(Build.java:127)
      hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:129)
      hudson.model.Run.run(Run.java:576)
      hudson.model.Build.run(Build.java:107)
      hudson.model.ResourceController.execute(ResourceController.java:66)
      hudson.model.Executor.run(Executor.java:59)
      ThreadExecutor #1 for opends-solaris-x86,5,main (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      hudson.util.OneShotEvent.block(OneShotEvent.java:28)
      hudson.model.Queue.pop(Queue.java:340)
      hudson.model.Executor.run(Executor.java:51)
      Thread[TP-Monitor,5,main] (TIMED_WAITING)
      java.lang.Object.wait(Native Method)
      org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run(ThreadPool.java:561)
      java.lang.Thread.run(Thread.java:595)
      Thread[http-8080-4,5,main] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:416)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:442)
      java.lang.Thread.run(Thread.java:595)
      Thread[Signal Dispatcher,9,system] (RUNNABLE)
      Thread[http-8080-3,5,main] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:416)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:442)
      java.lang.Thread.run(Thread.java:595)
      Thread[TP-Processor4,5,main] (RUNNABLE)
      java.net.PlainSocketImpl.socketAccept(Native Method)
      java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
      java.net.ServerSocket.implAccept(ServerSocket.java:450)
      java.net.ServerSocket.accept(ServerSocket.java:421)
      org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:306)
      org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:660)
      org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:870)
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:686)
      java.lang.Thread.run(Thread.java:595)
      ThreadExecutor #0 for opends-linux,5,main (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      hudson.util.OneShotEvent.block(OneShotEvent.java:28)
      hudson.model.Queue.pop(Queue.java:340)
      hudson.model.Executor.run(Executor.java:51)
      Thread[http-8080-1,5,main] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:416)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:442)
      java.lang.Thread.run(Thread.java:595)
      Thread[TP-Processor3,5,main] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:658)
      java.lang.Thread.run(Thread.java:595)
      Thread[Finalizer,8,system] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
      java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
      java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
      Thread[Reference Handler,10,system] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
      Thread[ContainerBackgroundProcessor[StandardEngine[Catalina]],5,main]
      (TIMED_WAITING)
      java.lang.Thread.sleep(Native Method)
      org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1579)
      java.lang.Thread.run(Thread.java:595)
      Thread[Hudson cron thread,5,main] (TIMED_WAITING)
      java.lang.Object.wait(Native Method)
      java.util.TimerThread.mainLoop(Timer.java:509)
      java.util.TimerThread.run(Timer.java:462)
      Thread[AWT-Windows,6,main] (RUNNABLE)
      sun.awt.windows.WToolkit.eventLoop(Native Method)
      sun.awt.windows.WToolkit.run(WToolkit.java:269)
      java.lang.Thread.run(Thread.java:595)
      ThreadExecutor #0 for opends-solaris-x86,5,main (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      hudson.util.OneShotEvent.block(OneShotEvent.java:28)
      hudson.model.Queue.pop(Queue.java:340)
      hudson.model.Executor.run(Executor.java:51)
      Thread[TP-Processor2,5,main] (WAITING)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Object.java:474)
      org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:658)
      java.lang.Thread.run(Thread.java:595)
      Thread[http-8080-Acceptor-0,5,main] (RUNNABLE)
      java.net.PlainSocketImpl.socketAccept(Native Method)
      java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
      java.net.ServerSocket.implAccept(ServerSocket.java:450)
      java.net.ServerSocket.accept(ServerSocket.java:421)
      org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
      org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:310)
      java.lang.Thread.run(Thread.java:595)

          [JENKINS-770] job loops endlessly after build is finished

          The cause of this issue is most likely as described in
          http://hudson.gotdns.com/wiki/display/JENKINS/Spawning+processes+from+build

          I guess the underlying problem is that Hudson cannot really diagnose whether
          this is happening or not, so the failure mode doesn't really point out what the
          problem is.

          So in 1.136, I improved Hudson so that it detects this situation (by doing a
          timed wait) and issue a warning.

          Kohsuke Kawaguchi added a comment - The cause of this issue is most likely as described in http://hudson.gotdns.com/wiki/display/JENKINS/Spawning+processes+from+build I guess the underlying problem is that Hudson cannot really diagnose whether this is happening or not, so the failure mode doesn't really point out what the problem is. So in 1.136, I improved Hudson so that it detects this situation (by doing a timed wait) and issue a warning.

            Unassigned Unassigned
            arnaud_lacour arnaud_lacour
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: