Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10524

winstone.ClientSocketException: Failed to write to client

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • Windows Server 2003, JDK 1.6.0_23_64, Jenkins 1.415

      Slaves are connected via JNLP

      winstone.ClientSocketException: Failed to write to client
      at winstone.ClientOutputStream.write(ClientOutputStream.java:41)
      at winstone.WinstoneOutputStream.commit(WinstoneOutputStream.java:181)
      at winstone.WinstoneOutputStream.commit(WinstoneOutputStream.java:123)
      at winstone.WinstoneOutputStream.write(WinstoneOutputStream.java:112)
      at net.bull.javamelody.FilterServletOutputStream.write(FilterServletOutputStream.java:70)
      at net.bull.javamelody.CounterResponseStream.write(CounterResponseStream.java:83)
      at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:410)
      at org.kohsuke.stapler.Stapler.serveStaticResource(Stapler.java:241)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:149)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
      at winstone.ServletConfiguration.execute(ServletConfiguration.java:249)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:335)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:378)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:157)
      at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:131)
      at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:82)
      at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
      at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:52)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
      at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:65)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:97)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at winstone.FilterConfiguration.execute(FilterConfiguration.java:195)
      at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:368)
      at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
      at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:244)
      at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
      at java.lang.Thread.run(Thread.java:662)
      Caused by: java.net.SocketException: Software caused connection abort: socket write error
      at java.net.SocketOutputStream.socketWrite0(Native Method)
      at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
      at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
      at winstone.ClientOutputStream.write(ClientOutputStream.java:39)
      ... 55 more

          [JENKINS-10524] winstone.ClientSocketException: Failed to write to client

          robsimon created issue -

          The exception indicates that the HTTP client has shut down the connection without fully reading the response, which is actually quite normal.

          In Jenkins, the code is written such that we simply swallow this exception, but if you are seeing it in console, it must be that someone is catching it and dumping it. I suspect the monitoring plugin is causing this.

          Kohsuke Kawaguchi added a comment - The exception indicates that the HTTP client has shut down the connection without fully reading the response, which is actually quite normal. In Jenkins, the code is written such that we simply swallow this exception, but if you are seeing it in console, it must be that someone is catching it and dumping it. I suspect the monitoring plugin is causing this.

          Jesse Glick added a comment -

          I have been tormented by similar exceptions (usually java.net.SocketException: Connection reset, but sometimes also java.io.IOException: Broken pipe from Jetty in 1.457 IIRC), which pollute the log file with junk.

          org.kohsuke.stapler.compression.CompressionFilter.reportException is what actually sends the log record. Not sure what “code is written such that we simply swallow this exception”.

          Unfortunately I have not yet managed to reproduce the problem in a controlled environment. Have tried installing a plugin with

          @Extension
          public class XXX extends PageDecorator {
              public void sleep() throws InterruptedException, IOException {
                  Stapler.getCurrentResponse().flushBuffer();
                  Thread.sleep(5000);
              }
          }
          

          and a matching footer.jelly

          <?xml version="1.0" encoding="UTF-8"?>
          <?jelly escape-by-default='true'?>
          <j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:i="jelly:fmt">
              <hr/>
              Sleeping…
              ${it.sleep()}
              done.
          </j:jelly>
          

          which makes a page take a long time to load, but halting it halfway prints nothing to the Jenkins console. Installing the Monitoring plugin changes nothing. Tried Jenkins trunk, 1.480.3, 1.466.2. Tried Stop button in Chrome, as well as running curl and interrupting the process before the page finishes loading.

          So if I can ever figure out how to reproduce this, I will try to patch reportException to check for exceptions (incl. nested causes) that seem to indicate a broken socket, and if so not send them to LOGGER (or use FINE rather than WARNING).

          Jesse Glick added a comment - I have been tormented by similar exceptions (usually java.net.SocketException: Connection reset , but sometimes also java.io.IOException: Broken pipe from Jetty in 1.457 IIRC), which pollute the log file with junk. org.kohsuke.stapler.compression.CompressionFilter.reportException is what actually sends the log record. Not sure what “code is written such that we simply swallow this exception”. Unfortunately I have not yet managed to reproduce the problem in a controlled environment. Have tried installing a plugin with @Extension public class XXX extends PageDecorator { public void sleep() throws InterruptedException, IOException { Stapler.getCurrentResponse().flushBuffer(); Thread .sleep(5000); } } and a matching footer.jelly <?xml version= "1.0" encoding= "UTF-8" ?> <?jelly escape-by-default= 'true' ?> <j:jelly xmlns:j = "jelly:core" xmlns:st = "jelly:stapler" xmlns:d = "jelly:define" xmlns:i = "jelly:fmt" > <hr/> Sleeping… ${it.sleep()} done. </j:jelly> which makes a page take a long time to load, but halting it halfway prints nothing to the Jenkins console. Installing the Monitoring plugin changes nothing. Tried Jenkins trunk, 1.480.3, 1.466.2. Tried Stop button in Chrome, as well as running curl and interrupting the process before the page finishes loading. So if I can ever figure out how to reproduce this, I will try to patch reportException to check for exceptions (incl. nested causes) that seem to indicate a broken socket, and if so not send them to LOGGER (or use FINE rather than WARNING ).

          Jesse Glick added a comment -

          Also java.net.SocketException: Broken pipe from Tomcat.

          Jesse Glick added a comment - Also java.net.SocketException: Broken pipe from Tomcat.
          Jesse Glick made changes -
          Assignee New: Jesse Glick [ jglick ]
          James Hardwick made changes -
          Link New: This issue is related to JENKINS-14825 [ JENKINS-14825 ]

          I've been trying to find a pattern as well for reproducing this (and JENKINS-14825) but have had no luck. The randomness is making it a really stickler.

          I'll keep trying and hopefully we can come up with something. Thanks for looking into it Jesse!

          James Hardwick added a comment - I've been trying to find a pattern as well for reproducing this (and JENKINS-14825 ) but have had no luck. The randomness is making it a really stickler. I'll keep trying and hopefully we can come up with something. Thanks for looking into it Jesse!

          @Kohsuke I also suspected the monitoring plugin, so I uninstalled it entirely and removed JENKINS_HOME/monitoring. Still no luck.

          The odd thing about this is that it seems to result in HTTP 404 errors on the front end, sometimes for an entire page and other times just for certain ajax calls. I'd expect a 500ish error.

          James Hardwick added a comment - @Kohsuke I also suspected the monitoring plugin, so I uninstalled it entirely and removed JENKINS_HOME/monitoring. Still no luck. The odd thing about this is that it seems to result in HTTP 404 errors on the front end, sometimes for an entire page and other times just for certain ajax calls. I'd expect a 500ish error.
          Jesse Glick made changes -
          Link New: This issue is duplicated by JENKINS-14825 [ JENKINS-14825 ]
          Jesse Glick made changes -
          Link Original: This issue is related to JENKINS-14825 [ JENKINS-14825 ]

            jglick Jesse Glick
            robsimon robsimon
            Votes:
            10 Vote for this issue
            Watchers:
            22 Start watching this issue

              Created:
              Updated:
              Resolved: