Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-11130

java.io.IOException: Unexpected termination of the channel - SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar (works on older Hudson version)

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • cli
    • None
    • * Running on SLES9 Linux server with 4 CPUs and plenty of diskspace.
      * Tomcat 7.0.14
      * JDK 1.6.0_14
      * Only ONE Master configuration - no slaves are configured
      * 3 Executors - (one less than the max number of CPUs)

      We are currently seeing the following issue using the jenkins-cli.jar
      with Jenkins 1.430. Tried going to 1.431, same issue.

      Each time we insure we have the latest matching jenkins-cli.jar using the link on the Manage Jenkins page.

      From inside a Jenkins job (I'll call it the parent) I'm trying to
      execute multiple jobs using the client jar. On the last job, I
      include the -s parameter so the parent job will not finish until the
      last job in the list is complete. The problem is that after about 15
      secs of waiting, the parent job ends (fails) with the EOF exception
      below.

      IMPORTANT INFO: When performing this exact same build using an older hudson-cli.jar with an older (pre-fork) version of Hudson v1.339, THIS WORKS. We want to move all of our builds over to the latest Jenkins version, and this is blocking us at the moment.

      We have searched all day and tried numerous things including
      reducing the number of jobs and also trying to run them sequentially.

      Stack trace seen in Console at failure:

      [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.14_jenkins/temp/
      hudson4690948305077288103.sh
      + WAR_FOLDER=/tmp/trunk/UIToolkit1.1.2
      + rm -rf /tmp/trunk/UIToolkit1.1.2/calendar.war /tmp/trunk/
      UIToolkit1.1.2/Pay.war /tmp/trunk/UIToolkit1.1.2/OrgChart.war
      + /opt/Sun/jdk1.6.0_14/bin/java jar /opt/Sun/jdk1.6.0_14/lib/jenkins
      cli.jar s http://jenkins-host:8082/jenkins/ build XYZ_Platform
      Calendar -p SVN_PATH=trunk -p STATIC_PATH=UIToolkit1.1.2
      + /opt/Sun/jdk1.6.0_14/bin/java jar /opt/Sun/jdk1.6.0_14/lib/jenkins
      cli.jar -s http://jenkins-host:8082/jenkins/ build XYZ_Platform-Pay -p
      SVN_PATH=trunk -p STATIC_PATH=UIToolkit1.1.2
      + /opt/Sun/jdk1.6.0_14/bin/java jar /opt/Sun/jdk1.6.0_14/lib/jenkins
      cli.jar s http://jenkins-host:8082/jenkins/ build XYZ_Platform
      OrgChart -p SVN_PATH=trunk -p STATIC_PATH=UIToolkit1.1.2
      + /opt/Sun/jdk1.6.0_14/bin/java jar /opt/Sun/jdk1.6.0_14/lib/jenkins
      cli.jar s http://jenkins-host:8082/jenkins/ build XYZ_Platform
      PolicyAck -s -p SVN_PATH=trunk -p STATIC_PATH=UIToolkit1.1.2
      Sep 16, 2011 4:35:37 PM hudson.remoting.Channel$ReaderThread run
      SEVERE: I/O error in channel Chunked connection to http://jenkins-host:8082/jenkins/cli
      java.io.IOException: Unexpected termination of the channel
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:1093)
      Caused by: java.io.EOFException
      at java.io.ObjectInputStream
      $BlockDataInputStream.peekByte(ObjectInputStream.java:2498)
      at
      java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:
      348)
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:1087)
      Exception in thread "main" hudson.remoting.RequestAbortedException:
      hudson.remoting.RequestAbortedException: java.io.IOException:
      Unexpected termination of the channel
      at hudson.remoting.Request.call(Request.java:149)
      at hudson.remoting.Channel.call(Channel.java:681)
      at
      hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java­:
      158)
      at $Proxy2.main(Unknown Source)
      at hudson.cli.CLI.execute(CLI.java:171)
      at hudson.cli.CLI._main(CLI.java:297)
      at hudson.cli.CLI.main(CLI.java:216)
      Caused by: hudson.remoting.RequestAbortedException:
      java.io.IOException: Unexpected termination of the channel
      at hudson.remoting.Request.abort(Request.java:273)
      at hudson.remoting.Channel.terminate(Channel.java:732)
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:1117)
      Caused by: java.io.IOException: Unexpected termination of the channel
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:1093)
      Caused by: java.io.EOFException
      at java.io.ObjectInputStream
      $BlockDataInputStream.peekByte(ObjectInputStream.java:2498)
      at
      java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1273)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:
      348)
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:1087)
      Build step 'Execute shell' marked build as failure
      Notifying upstream projects of job completion
      Finished: FAILURE

          [JENKINS-11130] java.io.IOException: Unexpected termination of the channel - SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar (works on older Hudson version)

          @jordanh I couldn't reproduce a failure with your testcase. But it did give me a little bit of food for thought and I have now reproduced @mark streit's problem using the cli. Basically if I force the connection to use a HTTP connection (rather than the cli port) then I can reproduce the problem. So I'm guessing that the cli port is not visible to the external cli program and hence it falls back to a HTTP based connection.

          I can make the HTTP based connection work properly if I reduce the PingThread timeout from 30 seconds to 5 or 10 seconds. I'll generate a patch/pull request later today.

          Richard Mortimer added a comment - @jordanh I couldn't reproduce a failure with your testcase. But it did give me a little bit of food for thought and I have now reproduced @mark streit's problem using the cli. Basically if I force the connection to use a HTTP connection (rather than the cli port) then I can reproduce the problem. So I'm guessing that the cli port is not visible to the external cli program and hence it falls back to a HTTP based connection. I can make the HTTP based connection work properly if I reduce the PingThread timeout from 30 seconds to 5 or 10 seconds. I'll generate a patch/pull request later today.

          jordanh added a comment -

          That sounds like some progress. I want to reiterate that the use of the "chunked" transfer encoding header appears to be dynamic. In my testing it is always used when I try to read xml for the main Jenkins dashboard. That is probably due to the fact that I have around 90-100 projects in my instance of Jenkins right now. It seems that if the xml to be sent is large enough then the "chunked" header is used. Why it things go wonky when "chunked" is used is probably more the issue and it sounds like maybe you have an idea about that. Cheers.

          jordanh added a comment - That sounds like some progress. I want to reiterate that the use of the "chunked" transfer encoding header appears to be dynamic. In my testing it is always used when I try to read xml for the main Jenkins dashboard. That is probably due to the fact that I have around 90-100 projects in my instance of Jenkins right now. It seems that if the xml to be sent is large enough then the "chunked" header is used. Why it things go wonky when "chunked" is used is probably more the issue and it sounds like maybe you have an idea about that. Cheers.

          mark streit added a comment -

          So my group's workaround was to move off Tomcat and use Winstone and the problem was solved. I don't like having to run on something with less control than Tomcat... but it works for now. As I have stated - something changed with Jenkins CLI because with Hudson CLI and older Hudson 1.339 and Tomcat 6.0.18 ... no problems...it worked. We moved to Jenkins and its CLI and started seeing this.

          mark streit added a comment - So my group's workaround was to move off Tomcat and use Winstone and the problem was solved. I don't like having to run on something with less control than Tomcat... but it works for now. As I have stated - something changed with Jenkins CLI because with Hudson CLI and older Hudson 1.339 and Tomcat 6.0.18 ... no problems...it worked. We moved to Jenkins and its CLI and started seeing this.

          @jordanh yes the "chunked" transfer is dynamic and that is part of HTTP/1.1. I'm going to describe my findings in detail and hopefully that will allow you to determine if you are seeing the same issue as here.

          @mark streit I do not intend to dig around and find out how/why it works for older Hudson or Tomcat instances. I am sure that a few things have changed that could have avoided timeouts in the past. It could be as simple as the CLI client being able to connect via a direct socket rather than having to use HTTP.

          Richard Mortimer added a comment - @jordanh yes the "chunked" transfer is dynamic and that is part of HTTP/1.1. I'm going to describe my findings in detail and hopefully that will allow you to determine if you are seeing the same issue as here. @mark streit I do not intend to dig around and find out how/why it works for older Hudson or Tomcat instances. I am sure that a few things have changed that could have avoided timeouts in the past. It could be as simple as the CLI client being able to connect via a direct socket rather than having to use HTTP.

          Short story - Tomcat times out the upload data connection. Jenkins sees this as the CLI client going away. Then Jenkins closes the download connection. CLI sees the download connection unexpectedly close and it generates the error message that we see.

          Long story

          1. First Cli downloads the regular HTML base webpage of the Jenkins instance.
            • from this it looks at the HTTP headers and tries to find the X-Jenkins-CLI-Port or X-Hudson-CLI-Port headers.
            • in this case it does not find the header or maybe it fails to connect due to firewall/ip address binding issues.
          2. At this point the CLI client falls-back to a full duplex connection using two HTTP connections: one for upload and one for download of data. Both of these use HTTP POST requests with chunked encoding out of necessity.
            • the download side POSTs a small amount of setup data in the HTTP headers and a zero length data stream. Then the server sends a 200 response code and starts sending remote protocol data to the CLI client.
            • The upload side does a HTTP POST and sends the chunked remote protocol data as the post data body.
          3. From this point the remote protocol uses the stream to pass serialized code/data between CLI client and Jenkins server.
          4. The thing that we must remember is that these are just regular webserver HTTP connections and the Tomcat/Winstone/whatever is not aware that there is any special significance in these streams and hence it applies the same logic that any HTTP connection would get. In particular inactivity timeouts are applied.
          5. In this case the upload side (which is technically just sending HTTP POST data) is subject to timeouts waiting for the full data to arrive. Without these timeouts clients could mount a denial of service attack against the webserver.

          So once the build has started the CLI client has no need to send any data to the server so the upload side sits idle. After 30 seconds Tomcat timesout the upload and we end up with an error.

          This is where thing get complicated because the error reported is that the download side closed unexpectedly. To explain this we have to consider the whole sequence of failure events:

          • cli upload side. Is sat idle and does/sees nothing. If it did try to send data it would find that the upstream side is closed and that it has a HTTP 200 response/body to deal with.
          • server upload side. Jenkins sees an EOF from the client in the upload data stream. This is effectively faked by Tomcat because it has timed-out out the upload. Indeed we get an error in the Jenkins log to that effect.

            03-Nov-2011 19:19:31 hudson.remoting.Channel$ReaderThread run
            SEVERE: I/O error in channel HTTP full-duplex channel 6e5b90d4-767d-4f8b-b1d8-9c
            ec92d539d2
            java.net.SocketTimeoutException: Read timed out

          • At this point Jenkins starts to close down the download side of the connection because the client has gone away.
          • Now the CLI download side sees what to it is an unexpected EOF because Jenkins just closes the connection.

          Hence the net result is that the CLI client thinks that the server has broken but this was actually caused by inactivity in the upload datastream from itself!

          So to resolve the issue Jenkins needs to issue a remoting keepalive ping or the timeouts in Tomcat need to be increased.

          Richard Mortimer added a comment - Short story - Tomcat times out the upload data connection. Jenkins sees this as the CLI client going away. Then Jenkins closes the download connection. CLI sees the download connection unexpectedly close and it generates the error message that we see. Long story First Cli downloads the regular HTML base webpage of the Jenkins instance. from this it looks at the HTTP headers and tries to find the X-Jenkins-CLI-Port or X-Hudson-CLI-Port headers. in this case it does not find the header or maybe it fails to connect due to firewall/ip address binding issues. At this point the CLI client falls-back to a full duplex connection using two HTTP connections: one for upload and one for download of data. Both of these use HTTP POST requests with chunked encoding out of necessity. the download side POSTs a small amount of setup data in the HTTP headers and a zero length data stream. Then the server sends a 200 response code and starts sending remote protocol data to the CLI client. The upload side does a HTTP POST and sends the chunked remote protocol data as the post data body. From this point the remote protocol uses the stream to pass serialized code/data between CLI client and Jenkins server. The thing that we must remember is that these are just regular webserver HTTP connections and the Tomcat/Winstone/whatever is not aware that there is any special significance in these streams and hence it applies the same logic that any HTTP connection would get. In particular inactivity timeouts are applied. In this case the upload side (which is technically just sending HTTP POST data) is subject to timeouts waiting for the full data to arrive. Without these timeouts clients could mount a denial of service attack against the webserver. So once the build has started the CLI client has no need to send any data to the server so the upload side sits idle. After 30 seconds Tomcat timesout the upload and we end up with an error. This is where thing get complicated because the error reported is that the download side closed unexpectedly. To explain this we have to consider the whole sequence of failure events: cli upload side. Is sat idle and does/sees nothing. If it did try to send data it would find that the upstream side is closed and that it has a HTTP 200 response/body to deal with. server upload side. Jenkins sees an EOF from the client in the upload data stream. This is effectively faked by Tomcat because it has timed-out out the upload. Indeed we get an error in the Jenkins log to that effect. 03-Nov-2011 19:19:31 hudson.remoting.Channel$ReaderThread run SEVERE: I/O error in channel HTTP full-duplex channel 6e5b90d4-767d-4f8b-b1d8-9c ec92d539d2 java.net.SocketTimeoutException: Read timed out At this point Jenkins starts to close down the download side of the connection because the client has gone away. Now the CLI download side sees what to it is an unexpected EOF because Jenkins just closes the connection. Hence the net result is that the CLI client thinks that the server has broken but this was actually caused by inactivity in the upload datastream from itself! So to resolve the issue Jenkins needs to issue a remoting keepalive ping or the timeouts in Tomcat need to be increased.

          One workaround is to set the Tomcat Connector connectionUploadTimeout and disableUploadTimeout attributes in the Tomcat server.xml file. Setting it to 60 seconds allows the existing ping at 30 seconds duration to work properly.

              <Connector port="8080" protocol="HTTP/1.1"
                         connectionTimeout="20000" 
                         disableUploadTimeout="false"
                         connectionUploadTimeout="60000"
                         URIEncoding="UTF-8"
                         redirectPort="8443" />
          

          Richard Mortimer added a comment - One workaround is to set the Tomcat Connector connectionUploadTimeout and disableUploadTimeout attributes in the Tomcat server.xml file. Setting it to 60 seconds allows the existing ping at 30 seconds duration to work properly. <Connector port= "8080" protocol= "HTTP/1.1" connectionTimeout= "20000" disableUploadTimeout= " false " connectionUploadTimeout= "60000" URIEncoding= "UTF-8" redirectPort= "8443" />

          Richard Mortimer added a comment - https://github.com/jenkinsci/jenkins/pull/304

          Code changed in jenkins
          User: Richard Mortimer
          Path:
          changelog.html
          cli/src/main/java/hudson/cli/CLI.java
          http://jenkins-ci.org/commit/jenkins/92496f99e4c941de9d7dab628466cadcde70d808
          Log:
          [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar

          When using HTTP full duplex connections the upstream channel can be idle for
          long periods of time and the webserver will time the connection out. Jenkins
          sees this and terminates the downstream channel causing the unexpected closure
          of the read channel in the CLI client.

          Timeout of 15 seconds is based on the 20 second default timeout as seen in
          Ubuntu Oneiric Tomcat 6 installations.

          Compare: https://github.com/jenkinsci/jenkins/compare/9730359...92496f9

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Richard Mortimer Path: changelog.html cli/src/main/java/hudson/cli/CLI.java http://jenkins-ci.org/commit/jenkins/92496f99e4c941de9d7dab628466cadcde70d808 Log: [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar When using HTTP full duplex connections the upstream channel can be idle for long periods of time and the webserver will time the connection out. Jenkins sees this and terminates the downstream channel causing the unexpected closure of the read channel in the CLI client. Timeout of 15 seconds is based on the 20 second default timeout as seen in Ubuntu Oneiric Tomcat 6 installations. Compare: https://github.com/jenkinsci/jenkins/compare/9730359...92496f9

          dogfood added a comment -

          Integrated in jenkins_main_trunk #1271
          [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar

          Kohsuke Kawaguchi : 92496f99e4c941de9d7dab628466cadcde70d808
          Files :

          • changelog.html
          • cli/src/main/java/hudson/cli/CLI.java

          dogfood added a comment - Integrated in jenkins_main_trunk #1271 [FIXED JENKINS-11130] SEVERE: I/O error in channel Chunked connection when using jenkins-cli.jar Kohsuke Kawaguchi : 92496f99e4c941de9d7dab628466cadcde70d808 Files : changelog.html cli/src/main/java/hudson/cli/CLI.java

          mark streit added a comment -

          We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438).

          Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone.

          Checked Manifest of CLI jar downloaded with the 1.441 WAR installation:

          Manifest-Version: 1.0
          Archiver-Version: Plexus Archiver
          Created-By: Apache Maven
          Built-By: kohsuke
          Build-Jdk: 1.6.0_26
          Main-Class: hudson.cli.CLI
          Jenkins-CLI-Version: 1.441

          Under Tomcat 7, we get this stacktrace:

          Started by command line
          [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh
          + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
          Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run
          SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
          java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
          Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request.call(Request.java:149)
          at hudson.remoting.Channel.call(Channel.java:681)
          at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
          at $Proxy2.main(Unknown Source)
          at hudson.cli.CLI.execute(CLI.java:200)
          at hudson.cli.CLI._main(CLI.java:330)
          at hudson.cli.CLI.main(CLI.java:245)
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Request.abort(Request.java:273)
          at hudson.remoting.Channel.terminate(Channel.java:732)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
          Caused by: java.io.IOException: Unexpected termination of the channel
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115)
          Caused by: java.io.EOFException
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
          Build step 'Execute shell' marked build as failure
          Notifying upstream projects of job completion
          Finished: FAILURE

          Under Winstone, we get this stacktrace - it's somewhat different:

          Started by command line
          [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh
          + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk
          Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run
          SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli
          java.io.IOException: Premature EOF
          at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
          at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
          at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
          at java.io.FilterInputStream.read(FilterInputStream.java:116)
          at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
          at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
          at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
          at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
          at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
          Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
          at hudson.remoting.Request.call(Request.java:149)
          at hudson.remoting.Channel.call(Channel.java:681)
          at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
          at $Proxy2.main(Unknown Source)
          at hudson.cli.CLI.execute(CLI.java:200)
          at hudson.cli.CLI._main(CLI.java:330)
          at hudson.cli.CLI.main(CLI.java:245)
          Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF
          at hudson.remoting.Request.abort(Request.java:273)
          at hudson.remoting.Channel.terminate(Channel.java:732)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139)
          Caused by: java.io.IOException: Premature EOF
          at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) (http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29)
          at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) (http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29)
          at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) (http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29)
          at java.io.FilterInputStream.read(FilterInputStream.java:116)
          at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) (http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29)
          at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) (http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29)
          at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) (http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29)
          at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
          at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
          at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109)
          Build step 'Execute shell' marked build as failure
          Notifying upstream projects of job completion
          Finished: FAILURE

          mark streit added a comment - We have downloaded and installed the LATEST 1.441 release that lists the fix for this problem. Currently we were running 1.438 on Winstone only (since with Tomcat 6 or 7, we had experienced the error HOWEVER yet under Winstone, it worked OK so that was our workaround - while running 1.438). Now with Jenkins 1.441 - we are getting the ERROR again and NOW WITH BOTH Winstone and the Tomcat configurations). We have left the Jenkins 1.441 WAR file in place running on Winstone, and reverted the CLI jar file back to the 1.438 version for now and that appears to work again with Winstone. Checked Manifest of CLI jar downloaded with the 1.441 WAR installation: Manifest-Version: 1.0 Archiver-Version: Plexus Archiver Created-By: Apache Maven Built-By: kohsuke Build-Jdk: 1.6.0_26 Main-Class: hudson.cli.CLI Jenkins-CLI-Version: 1.441 Under Tomcat 7, we get this stacktrace: Started by command line [workspace] $ /bin/bash -xe /opt/apache-tomcat-7.0.22_jenkins/temp/hudson32817888834817830.sh + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk Dec 5, 2011 12:59:11 PM hudson.remoting.Channel$ReaderThread run SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109) Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.call(Request.java:149) at hudson.remoting.Channel.call(Channel.java:681) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158) at $Proxy2.main(Unknown Source) at hudson.cli.CLI.execute(CLI.java:200) at hudson.cli.CLI._main(CLI.java:330) at hudson.cli.CLI.main(CLI.java:245) Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:273) at hudson.remoting.Channel.terminate(Channel.java:732) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Channel$ReaderThread.run(Channel.java:1115) Caused by: java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109) Build step 'Execute shell' marked build as failure Notifying upstream projects of job completion Finished: FAILURE Under Winstone, we get this stacktrace - it's somewhat different: Started by command line [workspace] $ /bin/bash -xe /tmp/hudson10791816374444704.sh + /opt/Sun/jdk1.6.0_14/bin/java -jar /opt/Sun/jdk1.6.0_14/lib/jenkins-cli.jar -s http://11.22.33.44:8082/jenkins/ build XYZ_Project-SharedLibs -s -p SVN_PATH=trunk Dec 5, 2011 1:18:22 PM hudson.remoting.Channel$ReaderThread run SEVERE: I/O error in channel Chunked connection to http://11.22.33.44:8082/jenkins/cli java.io.IOException: Premature EOF at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) ( http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29 ) at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) ( http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29 ) at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) ( http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29 ) at java.io.FilterInputStream.read(FilterInputStream.java:116) at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) ( http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29 ) at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) ( http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29 ) at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) ( http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29 ) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249) at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109) Exception in thread "main" hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF at hudson.remoting.Request.call(Request.java:149) at hudson.remoting.Channel.call(Channel.java:681) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158) at $Proxy2.main(Unknown Source) at hudson.cli.CLI.execute(CLI.java:200) at hudson.cli.CLI._main(CLI.java:330) at hudson.cli.CLI.main(CLI.java:245) Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Premature EOF at hudson.remoting.Request.abort(Request.java:273) at hudson.remoting.Channel.terminate(Channel.java:732) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1139) Caused by: java.io.IOException: Premature EOF at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:538) ( http://www.http.ChunkedInputStream.readAheadBlocking%28ChunkedInputStream.java:538%29 ) at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:582) ( http://www.http.ChunkedInputStream.readAhead%28ChunkedInputStream.java:582%29 ) at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:669) ( http://www.http.ChunkedInputStream.read%28ChunkedInputStream.java:669%29 ) at java.io.FilterInputStream.read(FilterInputStream.java:116) at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2504) ( http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2504%29 ) at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2499) ( http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2499%29 ) at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2488) ( http://www.protocol.http.HttpURLConnection$HttpInputStream.read%28HttpURLConnection.java:2488%29 ) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249) at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:1109) Build step 'Execute shell' marked build as failure Notifying upstream projects of job completion Finished: FAILURE

            oldelvet Richard Mortimer
            mcs13099 mark streit
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: