Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52125

jenkins master: Cannot contact <slave name>: java.lang.InterruptedException without heavy load or obvious network issue

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Cannot Reproduce
    • Component/s: remoting
    • Labels:
      None
    • Environment:
    • Similar Issues:

      Description

      This error, Cannot contact <slave name>: java.lang.InterruptedException,

      • no other messages, just one line.
      • happen most often during network I/O intense process like downloading.. also seen during shell script running, building software and tarring files, git merge... while other metrics like cpu usage, memory usage and disk usage are pretty normal/stable, not high load.
      • This error will/will not lead the job to fail. 
      • This message will show in build logs and it happens on many slaves, some of the slaves are VMs on server, some of them are physical server.
      • many affected jobs have parallel branches.
      • network seems working well.
      • also it might be correlated to, (shown in slave's log on master machine, this one does not happen often, but at least one time it synced with the error in question): 
        • INFO: Failed to synchronize IO streams on the channel hudson.remoting.Channel@762efe5d:channel java.lang.InterruptedException     at java.lang.Object.wait(Native Method) at hudson.remoting.Request.call(Request.java:172) at hudson.remoting.Channel.call(Channel.java:951) at hudson.remoting.Channel.syncIO(Channel.java:1679) at hudson.Launcher$RemoteLaunchCallable$1.join(Launcher.java:1315) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:927) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:901) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:850) at hudson.remoting.UserRequest.perform(UserRequest.java:210) at hudson.remoting.UserRequest.perform(UserRequest.java:53) at hudson.remoting.Request$2.run(Request.java:364) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

           

        Attachments

          Issue Links

            Activity

            Hide
            wynet321 Dennis Wang added a comment -

            Still happen on 2.121.1

            Show
            wynet321 Dennis Wang added a comment - Still happen on 2.121.1
            Hide
            venkatemadi venkat reddy added a comment -

            I too see this issue : Cannot contact <machine>: java.lang.InterruptedException. 
            When this error encountered, job hangs at this state. 

            No other error messages are shown in console log.

            There is no network/ssh/hardware issue from slave machine. Since slave machine is online in Jenkins Master and reachable.

            Show
            venkatemadi venkat reddy added a comment - I too see this issue : Cannot contact <machine>: java.lang.InterruptedException.  When this error encountered, job hangs at this state.  No other error messages are shown in console log. There is no network/ssh/hardware issue from slave machine. Since slave machine is online in Jenkins Master and reachable.
            Hide
            jthompson Jeff Thompson added a comment -

            Unfortunately, there is not enough information provided to be able to diagnose or reproduce this problem. If you can determine more information about how to isolate or characterize it, we could look into it in more detail. 

            This presentation contains further information on diagnosing Remoting connectivity issues. You might be able to find some information in there useful for your troubleshooting exercise. https://speakerdeck.com/onenashev/day-of-jenkins-2017-dealing-with-agent-connectivity-issues

            Show
            jthompson Jeff Thompson added a comment - Unfortunately, there is not enough information provided to be able to diagnose or reproduce this problem. If you can determine more information about how to isolate or characterize it, we could look into it in more detail.  This presentation contains further information on diagnosing Remoting connectivity issues. You might be able to find some information in there useful for your troubleshooting exercise.  https://speakerdeck.com/onenashev/day-of-jenkins-2017-dealing-with-agent-connectivity-issues
            Hide
            jthompson Jeff Thompson added a comment -

            As there hasn't been any response on this for a long time, there is insufficient information to reproduce, and responses were provided to the submitter I'm going to close this out.

            Show
            jthompson Jeff Thompson added a comment - As there hasn't been any response on this for a long time, there is insufficient information to reproduce, and responses were provided to the submitter I'm going to close this out.
            Hide
            riteshm ritesh mahendrakar added a comment -

            I get this error only when I start the Jenkins as a Service and run a pipeline job which transfers data between Master and Slave.

            The same job works fine when I Enable / Launch the save using client.

            I am using Jenkins 2.138.2

            Show
            riteshm ritesh mahendrakar added a comment - I get this error only when I start the Jenkins as a Service and run a pipeline job which transfers data between Master and Slave. The same job works fine when I Enable / Launch the save using client. I am using Jenkins 2.138.2
            Hide
            tiendungitd Dennis Tran added a comment - - edited

            I got the same issue in version 2.164.2. I think the build job produce a heavy load, sometimes it also encountered the error 

            The forked VM terminated without properly saying goodbye. VM crash or System.exit called
            Show
            tiendungitd Dennis Tran added a comment - - edited I got the same issue in version 2.164.2. I think the build job produce a heavy load, sometimes it also encountered the error  The forked VM terminated without properly saying goodbye. VM crash or System .exit called
            Hide
            jthompson Jeff Thompson added a comment -

            I've seen something like what Dennis Tran reports a few times associated with an OutOfMemoryError. It can take a bit of digging to actually see the errors in these cases. Because the channel or VM fails, the errors may not reach the logs.

            Show
            jthompson Jeff Thompson added a comment - I've seen something like what Dennis Tran reports a few times associated with an OutOfMemoryError. It can take a bit of digging to actually see the errors in these cases. Because the channel or VM fails, the errors may not reach the logs.

              People

              Assignee:
              jthompson Jeff Thompson
              Reporter:
              wocow Zichuan Zou
              Votes:
              4 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: