Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-3077

Hudson 1.284 streams many NullPointerExceptions

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • other
    • None
    • Platform: All, OS: All

      After Hudson 1.284 executes a couple of builds (on a slave node?), it starts
      streaming 25-30 NullPointerExceptions that look like:

      Feb 18, 2009 3:24:58 PM hudson.remoting.Channel$ReaderThread run
      SEVERE: Failed to execute command Pipe.EOF(0)
      java.lang.NullPointerException
      at hudson.remoting.ProxyOutputStream$EOF.execute(ProxyOutputStream.java:229)
      at hudson.remoting.Channel$ReaderThread.run(Channel.java:670)
      Feb 18, 2009 3:24:58 PM hudson.remoting.Channel$ReaderThread run
      SEVERE: This command is created here
      Command Pipe.EOF(0) created at
      at hudson.remoting.Command.<init>(Command.java:24)
      at hudson.remoting.ProxyOutputStream$EOF.<init>(ProxyOutputStream.java:169)
      at hudson.remoting.ProxyOutputStream.doClose(ProxyOutputStream.java:104)
      at hudson.remoting.ProxyOutputStream.close(ProxyOutputStream.java:100)
      at hudson.remoting.RemoteOutputStream.close(RemoteOutputStream.java:93)
      at hudson.FilePath$23.invoke(FilePath.java:874)
      at hudson.FilePath$23.invoke(FilePath.java:869)
      at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1278)
      at hudson.remoting.UserRequest.perform(UserRequest.java:69)
      at hudson.remoting.UserRequest.perform(UserRequest.java:23)
      at hudson.remoting.Request$2.run(Request.java:213)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
      at java.util.concurrent.FutureTask.run(FutureTask.java:123)
      at
      java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
      at
      java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
      at java.lang.Thread.run(Thread.java:595)

      I'm not sure what, if any, effect this has on Hudson or build results besides
      spamming the server log (although my Hudson server currently only lives about 2
      hours anyway due to issue 3076.) Both master and slaves are RHEL5 machines.

          [JENKINS-3077] Hudson 1.284 streams many NullPointerExceptions

          bbarlow created issue -

          bbarlow added a comment -

          Note: Judging by the timestamps on the ls /prod/PID/fd output, the .tmp files
          are last touched about the time the build completes. There also seem to be about
          100 .tmp files per job.

          bbarlow added a comment - Note: Judging by the timestamps on the ls /prod/PID/fd output, the .tmp files are last touched about the time the build completes. There also seem to be about 100 .tmp files per job.

          bbarlow added a comment -

          (ignore my previous comment - I intended to post that to issue 3076 instead, sorry)

          bbarlow added a comment - (ignore my previous comment - I intended to post that to issue 3076 instead, sorry)

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/core/src/test/java/hudson/FilePathTest.java
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/RemoteOutputStream.java
          http://fisheye4.cenqua.com/changelog/hudson/?cs=15446
          Log:
          JENKINS-3077
          Adding more probes to understand where the invalid data creeps in

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/test/java/hudson/FilePathTest.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java trunk/hudson/main/remoting/src/main/java/hudson/remoting/RemoteOutputStream.java http://fisheye4.cenqua.com/changelog/hudson/?cs=15446 Log: JENKINS-3077 Adding more probes to understand where the invalid data creeps in

          vijayj added a comment -

          I am running into the same issue. I have the following information all over my
          log file. Builds are running just fine. One thing I have noticed, I am not able
          to view any files when click on the workspace icon. Not sure these are all
          related this issue...

          Feb 18, 2009 11:08:23 AM hudson.remoting.Channel$ReaderThread run
          SEVERE: Failed to execute command Pipe.EOF(0)
          java.lang.NullPointerException
          at hudson.remoting.ProxyOutputStream$EOF.execute(ProxyOutputStream.java:229)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:670)
          Feb 18, 2009 11:08:23 AM hudson.remoting.Channel$ReaderThread run
          SEVERE: This command is created here
          Command Pipe.EOF(0) created at
          at hudson.remoting.Command.<init>(Command.java:24)
          at hudson.remoting.ProxyOutputStream$EOF.<init>(ProxyOutputStream.java:169)
          at hudson.remoting.ProxyOutputStream.doClose(ProxyOutputStream.java:104)
          at hudson.remoting.ProxyOutputStream.close(ProxyOutputStream.java:100)
          at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:196)
          at hudson.FilePath$18.call(FilePath.java:771)
          at hudson.FilePath$18.call(FilePath.java:762)
          at hudson.remoting.UserRequest.perform(UserRequest.java:69)
          at hudson.remoting.UserRequest.perform(UserRequest.java:23)
          at hudson.remoting.Request$2.run(Request.java:213)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
          at java.util.concurrent.FutureTask.run(FutureTask.java:138)
          at
          java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at
          java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:619)
          Feb 18, 2009 11:08:31 AM hudson.remoting.Channel$ReaderThread run
          SEVERE: Failed to execute command Pipe.EOF(0)
          java.lang.NullPointerException
          at hudson.remoting.ProxyOutputStream$EOF.execute(ProxyOutputStream.java:229)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:670)
          Feb 18, 2009 11:08:31 AM hudson.remoting.Channel$ReaderThread run

          vijayj added a comment - I am running into the same issue. I have the following information all over my log file. Builds are running just fine. One thing I have noticed, I am not able to view any files when click on the workspace icon. Not sure these are all related this issue... Feb 18, 2009 11:08:23 AM hudson.remoting.Channel$ReaderThread run SEVERE: Failed to execute command Pipe.EOF(0) java.lang.NullPointerException at hudson.remoting.ProxyOutputStream$EOF.execute(ProxyOutputStream.java:229) at hudson.remoting.Channel$ReaderThread.run(Channel.java:670) Feb 18, 2009 11:08:23 AM hudson.remoting.Channel$ReaderThread run SEVERE: This command is created here Command Pipe.EOF(0) created at at hudson.remoting.Command.<init>(Command.java:24) at hudson.remoting.ProxyOutputStream$EOF.<init>(ProxyOutputStream.java:169) at hudson.remoting.ProxyOutputStream.doClose(ProxyOutputStream.java:104) at hudson.remoting.ProxyOutputStream.close(ProxyOutputStream.java:100) at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:196) at hudson.FilePath$18.call(FilePath.java:771) at hudson.FilePath$18.call(FilePath.java:762) at hudson.remoting.UserRequest.perform(UserRequest.java:69) at hudson.remoting.UserRequest.perform(UserRequest.java:23) at hudson.remoting.Request$2.run(Request.java:213) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Feb 18, 2009 11:08:31 AM hudson.remoting.Channel$ReaderThread run SEVERE: Failed to execute command Pipe.EOF(0) java.lang.NullPointerException at hudson.remoting.ProxyOutputStream$EOF.execute(ProxyOutputStream.java:229) at hudson.remoting.Channel$ReaderThread.run(Channel.java:670) Feb 18, 2009 11:08:31 AM hudson.remoting.Channel$ReaderThread run

          sydpolk added a comment -

          I am being hit by this bug, so I want to see progress.

          sydpolk added a comment - I am being hit by this bug, so I want to see progress.

          Jesse Glick added a comment -

          BTW the newly created FilePathTest seems to fail randomly on my machine.
          Sometimes passes with no messages. Sometimes prints:

          Feb 19, 2009 9:10:39 PM hudson.remoting.Channel$CloseCommand execute
          SEVERE: close command failed on The other side of the channel
          java.io.IOException: Pipe closed
          at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:244)
          at java.io.PipedInputStream.receive(PipedInputStream.java:210)
          at java.io.PipedOutputStream.write(PipedOutputStream.java:132)
          at
          java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1838)
          at
          java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1747)
          at java.io.ObjectOutputStream.reset(ObjectOutputStream.java:475)
          at hudson.remoting.Channel.send(Channel.java:352)
          at hudson.remoting.Channel.close(Channel.java:617)
          at hudson.remoting.Channel$CloseCommand.execute(Channel.java:576)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:670)
          Feb 19, 2009 9:10:39 PM hudson.remoting.Channel$CloseCommand execute
          INFO: close command created at
          Command close created at
          at hudson.remoting.Command.<init>(Command.java:47)
          at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:573)
          at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:573)
          at hudson.remoting.Channel.close(Channel.java:617)
          at hudson.FilePathTest.tearDown(FilePathTest.java:73)
          at junit.framework.TestCase.runBare(TestCase.java:140)
          at junit.framework.TestResult$1.protect(TestResult.java:110)
          at junit.framework.TestResult.runProtected(TestResult.java:128)
          at junit.framework.TestResult.run(TestResult.java:113)
          at junit.framework.TestCase.run(TestCase.java:124)
          at junit.framework.TestSuite.runTest(TestSuite.java:232)
          at junit.framework.TestSuite.run(TestSuite.java:227)
          at
          org.junit.internal.runners.OldTestClassRunner.run(OldTestClassRunner.java:76)
          at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:32)
          at
          org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421)
          Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.34 sec

          and usually also gets an error which makes the test fail:

          SEVERE: I/O error in channel This side of the channel
          java.io.EOFException
          at
          java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
          at hudson.remoting.Channel$ReaderThread.run(Channel.java:660)

          sometimes also something in tearDown during french.close which I cannot now
          reproduce.

          If I comment out the body of tearDown then the test passes.

          Jesse Glick added a comment - BTW the newly created FilePathTest seems to fail randomly on my machine. Sometimes passes with no messages. Sometimes prints: Feb 19, 2009 9:10:39 PM hudson.remoting.Channel$CloseCommand execute SEVERE: close command failed on The other side of the channel java.io.IOException: Pipe closed at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:244) at java.io.PipedInputStream.receive(PipedInputStream.java:210) at java.io.PipedOutputStream.write(PipedOutputStream.java:132) at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1838) at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1747) at java.io.ObjectOutputStream.reset(ObjectOutputStream.java:475) at hudson.remoting.Channel.send(Channel.java:352) at hudson.remoting.Channel.close(Channel.java:617) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:576) at hudson.remoting.Channel$ReaderThread.run(Channel.java:670) Feb 19, 2009 9:10:39 PM hudson.remoting.Channel$CloseCommand execute INFO: close command created at Command close created at at hudson.remoting.Command.<init>(Command.java:47) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:573) at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:573) at hudson.remoting.Channel.close(Channel.java:617) at hudson.FilePathTest.tearDown(FilePathTest.java:73) at junit.framework.TestCase.runBare(TestCase.java:140) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.junit.internal.runners.OldTestClassRunner.run(OldTestClassRunner.java:76) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:32) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421) Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.34 sec and usually also gets an error which makes the test fail: SEVERE: I/O error in channel This side of the channel java.io.EOFException at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2554) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at hudson.remoting.Channel$ReaderThread.run(Channel.java:660) sometimes also something in tearDown during french.close which I cannot now reproduce. If I comment out the body of tearDown then the test passes.

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/Channel.java
          http://fisheye4.cenqua.com/changelog/hudson/?cs=15498
          Log:
          JENKINS-3077 Fixed a random failure.

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/remoting/src/main/java/hudson/remoting/Channel.java http://fisheye4.cenqua.com/changelog/hudson/?cs=15498 Log: JENKINS-3077 Fixed a random failure.

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java
          trunk/www/changelog.html
          http://fisheye4.cenqua.com/changelog/hudson/?cs=15767
          Log:
          [FIXED JENKINS-3077]
          This was caused by the serialization format change, masked by the manual serialVersionUID for the given class. This is only an issue when the version of slave.jar on the slave and the master disagrees, but then, that is often a common case when people upgrade the master.

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/remoting/src/main/java/hudson/remoting/ProxyOutputStream.java trunk/www/changelog.html http://fisheye4.cenqua.com/changelog/hudson/?cs=15767 Log: [FIXED JENKINS-3077] This was caused by the serialization format change, masked by the manual serialVersionUID for the given class. This is only an issue when the version of slave.jar on the slave and the master disagrees, but then, that is often a common case when people upgrade the master.
          SCM/JIRA link daemon made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

            Unassigned Unassigned
            bbarlow bbarlow
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: