Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18152

Thread interruption in remoting proxy results in UndeclaredThrowableException

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Minor
    • Resolution: Fixed
    • remoting
    • None
    • master: CentOS 6.3, Tomcat 7, Jenkins 1.514
      slaves: CentOS 6.3, Jenkins 1.514

    Description

      Periodically in the tomcat logs (and always on start up) I see the following stack trace. It seems to imply a version mismatch between the slave and master (can't deserialize an object)?

      May 30, 2013 4:03:44 PM hudson.node_monitors.AbstractNodeMonitorDescriptor$Record run
WARNING: Failed to monitor h2_deuterium for Free Temp Space
hudson.util.IOException2: remote file operation failed: /home/jenkins/hydrogen at hudson.remoting.Channel@d48ff87:h2_deuterium
        at hudson.FilePath.act(FilePath.java:900)
        at hudson.FilePath.act(FilePath.java:877)
        at hudson.node_monitors.TemporarySpaceMonitor$1.getFreeSpace(TemporarySpaceMonitor.java:73)
        at hudson.node_monitors.DiskSpaceMonitorDescriptor.monitor(DiskSpaceMonitorDescriptor.java:169)
        at hudson.node_monitors.DiskSpaceMonitorDescriptor.monitor(DiskSpaceMonitorDescriptor.java:49)
        at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:246)
Caused by: java.io.IOException: Remote call on h2_deuterium failed
        at hudson.remoting.Channel.call(Channel.java:681)
        at hudson.FilePath.act(FilePath.java:893)
        ... 5 more
Caused by: java.lang.Error: Failed to deserialize the Callable object.
        at hudson.remoting.UserRequest.perform(UserRequest.java:104)
        at hudson.remoting.UserRequest.perform(UserRequest.java:48)
        at hudson.remoting.Request$2.run(Request.java:326)
        at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.reflect.UndeclaredThrowableException
        at $Proxy5.fetch2(Unknown Source)
        at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:127)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2442)
        at java.lang.Class.getDeclaredMethod(Class.java:1952)
        at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1411)
        at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:69)
        at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:481)
        at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:455)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:455)
        at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:352)
        at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:589)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
        at hudson.remoting.UserRequest.deserialize(UserRequest.java:182)
        at hudson.remoting.UserRequest.perform(UserRequest.java:98)
        ... 8 more
Caused by: java.lang.InterruptedException
        at java.lang.Object.wait(Native Method)
        at hudson.remoting.Request.call(Request.java:146)
        at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:160)
        ... 34 more

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-18438 Node monitoring should run in parallel

          • Major - Major loss of function.
          • Resolved

          Activity

            Ascending order - Click to sort in descending order
            kohsuke Kohsuke Kawaguchi added a comment -

            The actual issue is that if a thread that's making a remote call through a type-safe proxy gets interrupted, and if the method that's being called doesn't have "throws InterruptedException", then the proxy code throws UndeclaredThrowableException as opposed to the raw InterruptedException.

            Since JVM does allow an undeclared checked exception to be thrown (even though javac won't allow it), it would have been better for us if the InterruptedException gets thrown directly, which then gets caught and handled properly.

            While this happens more often in node monitors, as they tend to make calls on nodes that may be hanging, this is actually an universal problem. So I'm adjusting the summary accordingly.

            Fixing this that way would require us to find a better proxy library. Alternatively, we can wrap it in something like RequestAbortedException after checking if the signature allows InterruptedException.

            kohsuke Kohsuke Kawaguchi added a comment - The actual issue is that if a thread that's making a remote call through a type-safe proxy gets interrupted, and if the method that's being called doesn't have "throws InterruptedException", then the proxy code throws UndeclaredThrowableException as opposed to the raw InterruptedException . Since JVM does allow an undeclared checked exception to be thrown (even though javac won't allow it), it would have been better for us if the InterruptedException gets thrown directly, which then gets caught and handled properly. While this happens more often in node monitors, as they tend to make calls on nodes that may be hanging, this is actually an universal problem. So I'm adjusting the summary accordingly. Fixing this that way would require us to find a better proxy library. Alternatively, we can wrap it in something like RequestAbortedException after checking if the signature allows InterruptedException .
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            src/main/java/hudson/remoting/RemoteInvocationHandler.java
            http://jenkins-ci.org/commit/remoting/c8c0e818b7ee9d7351ec0424027010c94efbd8ef
            Log:
            [FIXED JENKINS-18152]

            Avoid UndeclaredThrowableException by being smart about what exception to propagate and what to wrap.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/RemoteInvocationHandler.java http://jenkins-ci.org/commit/remoting/c8c0e818b7ee9d7351ec0424027010c94efbd8ef Log: [FIXED JENKINS-18152] Avoid UndeclaredThrowableException by being smart about what exception to propagate and what to wrap.

            People

              Unassigned Unassigned
              bullhornrelease Bullhorn RelEng
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: