Randomly some builds fails on ssh-slaves because of a StreamCorruptedException.

      It was not happening at all a couple of version ago.

      Here is the log:

      Construction à distance sur XXXXXXX
      FATAL: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 4B
      hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 4B
      at hudson.remoting.Request.call(Request.java:137)
      at hudson.remoting.Channel.call(Channel.java:629)
      at hudson.FilePath.act(FilePath.java:745)
      at hudson.FilePath.act(FilePath.java:738)
      at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:683)
      at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:632)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:1171)
      at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:501)
      at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:415)
      at hudson.model.Run.run(Run.java:1362)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:145)
      Caused by: hudson.remoting.RequestAbortedException: java.io.StreamCorruptedException: invalid type code: 4B
      at hudson.remoting.Request.abort(Request.java:257)
      at hudson.remoting.Channel.terminate(Channel.java:680)
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:971)
      Caused by: java.io.StreamCorruptedException: invalid type code: 4B
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1355)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:947)

          [JENKINS-8856] StreamCorruptedException

          Indra Gunawan added a comment -

          Indra Gunawan added a comment - Is there a RedHat rpm available for this version? http://repo.jenkins-ci.org/releases/org/jenkins-ci/main/jenkins-war/1.509.2.JENKINS-8856-diag/jenkins-war-1.509.2.JENKINS-8856-diag.war Thanks. -Indra

          Jesse Glick added a comment -

          @ingunawa: uh, no. This build is for experimentation only. If you have an RPM installation and want to try it, just overwrite your existing jenkins.war and be prepared to revert.

          Jesse Glick added a comment - @ingunawa: uh, no. This build is for experimentation only. If you have an RPM installation and want to try it, just overwrite your existing jenkins.war and be prepared to revert.

          Jesse Glick added a comment -

          One user reported that the stream corruption was in fact the text Killed, and that /var/log/messages mentioned kernel: java invoked oom-killer, i.e. the slave OS (Linux) had run out of memory and was killing off slave agents. Unfortunately this message seems to have been dumped into the remoting channel and thus confused the master (which saw the K and died). The SSH Slaves plugin is supposed to disconnect the original stdio streams from the channel once the connection is set up, but perhaps this is not working.

          Jesse Glick added a comment - One user reported that the stream corruption was in fact the text Killed , and that /var/log/messages mentioned kernel: java invoked oom-killer , i.e. the slave OS (Linux) had run out of memory and was killing off slave agents. Unfortunately this message seems to have been dumped into the remoting channel and thus confused the master (which saw the K and died). The SSH Slaves plugin is supposed to disconnect the original stdio streams from the channel once the connection is set up, but perhaps this is not working.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/hudson/remoting/HexDump.java
          src/test/java/hudson/remoting/HexDumpTest.java
          http://jenkins-ci.org/commit/remoting/d0334ba25b503038e17488cdbd5dd9bb5cb8121c
          Log:
          JENKINS-8856 Diagnostic improvement: show ‘'i' 'l' 'l' 'e' 'd' 0x0a’ rather than ‘696c6c65640a’, so it is more obviously the rest of ‘Killed’.

          Compare: https://github.com/jenkinsci/remoting/compare/09524af34b57...d0334ba25b50

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/remoting/HexDump.java src/test/java/hudson/remoting/HexDumpTest.java http://jenkins-ci.org/commit/remoting/d0334ba25b503038e17488cdbd5dd9bb5cb8121c Log: JENKINS-8856 Diagnostic improvement: show ‘'i' 'l' 'l' 'e' 'd' 0x0a’ rather than ‘696c6c65640a’, so it is more obviously the rest of ‘Killed’. Compare: https://github.com/jenkinsci/remoting/compare/09524af34b57...d0334ba25b50

          Code changed in jenkins
          User: Daniel Beck
          Path:
          src/main/java/hudson/remoting/HexDump.java
          src/test/java/hudson/remoting/HexDumpTest.java
          http://jenkins-ci.org/commit/remoting/aae9b457c15f88b7a8607978177f1673db2ec2ff
          Log:
          JENKINS-8856 Nicer logging of ASCII text.

          It's no longer a 'hex' dump, but much more readable for all known error cases (JVM dying and writing to standard out).

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: src/main/java/hudson/remoting/HexDump.java src/test/java/hudson/remoting/HexDumpTest.java http://jenkins-ci.org/commit/remoting/aae9b457c15f88b7a8607978177f1673db2ec2ff Log: JENKINS-8856 Nicer logging of ASCII text. It's no longer a 'hex' dump, but much more readable for all known error cases (JVM dying and writing to standard out).

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/hudson/remoting/HexDump.java
          src/test/java/hudson/remoting/HexDumpTest.java
          http://jenkins-ci.org/commit/remoting/8b509574e538061ecce717510688bb18acfc3be3
          Log:
          Merge pull request #16 from daniel-beck/master

          More readable diagnostic for JENKINS-8856

          Compare: https://github.com/jenkinsci/remoting/compare/a56730603577...8b509574e538

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/remoting/HexDump.java src/test/java/hudson/remoting/HexDumpTest.java http://jenkins-ci.org/commit/remoting/8b509574e538061ecce717510688bb18acfc3be3 Log: Merge pull request #16 from daniel-beck/master More readable diagnostic for JENKINS-8856 Compare: https://github.com/jenkinsci/remoting/compare/a56730603577...8b509574e538

          Daniel Beck added a comment -

          IMO this should be closed. Any new occurrences (1.527+ and future 1.532.x LTS) will be accompanied by much more information and should probably be a new Jira issue about the specific failure indicated by diagnostic output.

          Daniel Beck added a comment - IMO this should be closed. Any new occurrences (1.527+ and future 1.532.x LTS) will be accompanied by much more information and should probably be a new Jira issue about the specific failure indicated by diagnostic output.

          Jesse Glick added a comment -

          Agreed, newer reports will probably be more useful.

          Jesse Glick added a comment - Agreed, newer reports will probably be more useful.

          Jesse Glick added a comment -

          Jesse Glick added a comment - New remoting fix in: https://github.com/jenkinsci/jenkins/commit/4137c46182a0652d2eed2897a1e797bdc3f8c7f9

          Code changed in jenkins
          User: Oliver Gondža
          Path:
          http://jenkins-ci.org/commit/jenkins/5cf3e28c4885a86a4dad3463952e1aae34932f9c
          Log:
          [FIXED JENKINS-20093 JENKINS-19453 JENKINS-19004 JENKINS-8856] Noting retroactively

          Backported as 23424ea7dbced486963a73d6bf56e8367ab779dd

          Compare: https://github.com/jenkinsci/jenkins/compare/7d602d52b445...5cf3e28c4885

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oliver Gondža Path: http://jenkins-ci.org/commit/jenkins/5cf3e28c4885a86a4dad3463952e1aae34932f9c Log: [FIXED JENKINS-20093 JENKINS-19453 JENKINS-19004 JENKINS-8856] Noting retroactively Backported as 23424ea7dbced486963a73d6bf56e8367ab779dd Compare: https://github.com/jenkinsci/jenkins/compare/7d602d52b445...5cf3e28c4885

            kohsuke Kohsuke Kawaguchi
            ebannn i b
            Votes:
            3 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: