• Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Minor Minor
    • core, winstone-jetty
    • None

      Multiple reports (like JENKINS-42058) indicate the default timeouts are too aggressive when dealing with terrible connections.

          [JENKINS-42513] Increase default keep-alive timeouts

          Daniel Beck created issue -
          Josh Soref made changes -
          Link New: This issue relates to JENKINS-46094 [ JENKINS-46094 ]
          Josh Soref made changes -
          Link New: This issue relates to JENKINS-39909 [ JENKINS-39909 ]
          Josh Soref made changes -
          Link New: This issue relates to JENKINS-39909 [ JENKINS-39909 ]
          Josh Soref made changes -
          Link Original: This issue relates to JENKINS-39909 [ JENKINS-39909 ]
          Josh Soref made changes -
          Link New: This issue relates to JENKINS-38623 [ JENKINS-38623 ]
          Josh Soref made changes -
          Link New: This issue relates to JENKINS-42058 [ JENKINS-42058 ]

          Josh Soref added a comment -

          Note: --httpKeepAliveTimeout=10000 appears to be honored whether winstone means jetty or actually winstone...

          This is what we hit:

          java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 5000/5000 ms
                  at org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:235)
                  at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:216)
                  at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:514)
                  at net.bull.javamelody.internal.web.FilterServletOutputStream.write(FilterServletOutputStream.java:88)
                  at net.bull.javamelody.internal.web.CounterResponseStream.write(CounterResponseStream.java:81)
                  at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:604)
                  at org.kohsuke.stapler.ResponseImpl.serveFile(ResponseImpl.java:216)
                  at hudson.model.DirectoryBrowserSupport.serveFile(DirectoryBrowserSupport.java:349)
                  at hudson.model.DirectoryBrowserSupport.generateResponse(DirectoryBrowserSupport.java:141)
                  at org.kohsuke.stapler.HttpResponseRenderer$Default.handleHttpResponse(HttpResponseRenderer.java:124)
                  at org.kohsuke.stapler.HttpResponseRenderer$Default.generateResponse(HttpResponseRenderer.java:69)
                  at org.kohsuke.stapler.Function.renderResponse(Function.java:164)
                  at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:147)
                  at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:537)
                  at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
                  at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870)
                  at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:458)
                  at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870)
                  at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:282)
                  at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
                  at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668)
                  at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
                  at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
                  at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
                  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
                  at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at com.cloudbees.jenkins.support.slowrequest.SlowRequestFilter.doFilter(SlowRequestFilter.java:37)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at com.smartcodeltd.jenkinsci.plugin.assetbundler.filters.LessCSS.doFilter(LessCSS.java:47)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239)
                  at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215)
                  at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:88)
                  at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:114)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
                  at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
                  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
                  at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:105)
                  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
                  at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at jenkins.security.BasicHeaderProcessor.success(BasicHeaderProcessor.java:140)
                  at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:82)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
                  at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
                  at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
                  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
                  at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
                  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
                  at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
                  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
                  at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
                  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
                  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
                  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
                  at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
                  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
                  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
                  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
                  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
                  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
                  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
                  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
                  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
                  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
                  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
                  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
                  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
                  at org.eclipse.jetty.server.Server.handle(Server.java:503)
                  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
                  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
                  at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
                  at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
                  at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
                  at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
                  at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
                  at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
                  at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
                  at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
                  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
                  at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
                  at java.lang.Thread.run(Thread.java:748)
          Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 5000/5000 ms
                  at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
                  at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
                  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                  ... 1 more 

          I'm going to try this for a week and see if it causes our problems to go away.

          Afaict, this wasn't a problem until a recent jenkins (or build node) upgrade. But recently it's become a significant problem.

          Josh Soref added a comment - Note: --httpKeepAliveTimeout=10000 appears to be honored whether winstone means jetty or actually winstone... This is what we hit: java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 5000/5000 ms at org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:235) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:216) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:514) at net.bull.javamelody.internal.web.FilterServletOutputStream.write(FilterServletOutputStream.java:88) at net.bull.javamelody.internal.web.CounterResponseStream.write(CounterResponseStream.java:81) at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:604) at org.kohsuke.stapler.ResponseImpl.serveFile(ResponseImpl.java:216) at hudson.model.DirectoryBrowserSupport.serveFile(DirectoryBrowserSupport.java:349) at hudson.model.DirectoryBrowserSupport.generateResponse(DirectoryBrowserSupport.java:141) at org.kohsuke.stapler.HttpResponseRenderer$Default.handleHttpResponse(HttpResponseRenderer.java:124) at org.kohsuke.stapler.HttpResponseRenderer$Default.generateResponse(HttpResponseRenderer.java:69) at org.kohsuke.stapler.Function.renderResponse(Function.java:164) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:147) at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:537) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870) at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:458) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870) at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:282) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at com.cloudbees.jenkins.support.slowrequest.SlowRequestFilter.doFilter(SlowRequestFilter.java:37) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at com.smartcodeltd.jenkinsci.plugin.assetbundler.filters.LessCSS.doFilter(LessCSS.java:47) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:239) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:215) at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:88) at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:114) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:105) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.success(BasicHeaderProcessor.java:140) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:82) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang. Thread .run( Thread .java:748) Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 5000/5000 ms at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166) at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more I'm going to try this for a week and see if it causes our problems to go away. Afaict, this wasn't a problem until a recent jenkins (or build node) upgrade. But recently it's become a significant problem.

          It's not just "terrible connections"; it could also be a consumer being slow for another reason.
          We just ran into this issue with a shell script doing:

            curl -f $JENKINS/$JOB/artifact/binaries.tar.gz | tar xz

          Sometimes unpacking was too slow (due to high load on the VM) and Jenkins terminated the download.

          Daniel Grunwald added a comment - It's not just "terrible connections"; it could also be a consumer being slow for another reason. We just ran into this issue with a shell script doing:   curl -f $JENKINS/$JOB/artifact/binaries.tar.gz | tar xz Sometimes unpacking was too slow (due to high load on the VM) and Jenkins terminated the download.
          Daniel Beck made changes -
          Assignee Original: Daniel Beck [ danielbeck ]

            Unassigned Unassigned
            danielbeck Daniel Beck
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: