Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33852

Exception while copying in thread - causes Slave disconnection

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core, remoting
    • Windows Server 2008 R2 (both Slaves and Master)
      Jenkins ver. 1.654

      It's been happening several weeks already, that a Java process that being executed within Maven command, get killed randomly, as the Jenkins slave disconnects the connection unexpectedly, with error:

      WARNING: Exception while copying in thread: Copier for /127.0.0.1:{RANDOM PORT}
      java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(Unknown Source)
      at java.net.SocketInputStream.read(Unknown Source)
      at java.io.FilterInputStream.read(Unknown Source)
      at java.io.FilterInputStream.read(Unknown Source)
      at hudson.remoting.forward.CopyThread.run(CopyThread.java:30)

      Looking at Job console, we see that the Maven process completes, but the inner Java thread is terminated:

      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD FAILURE
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 02:26 h
      [INFO] Finished at: 2016-03-28T17:23:46+03:00
      [INFO] Final Memory: 33M/220M
      [INFO] ------------------------------------------------------------------------
      [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.19.1:integration-test (failsafe-integration-tests) on project AF: Execution failsafe-integration-tests of goal org.apache.maven.plugins:maven-failsafe-plugin:2.19.1:integration-test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
      [ERROR] Command was cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_65\jre\bin\java" -Xmx4096m -jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefirebooter4689747360754285342.jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire1687639200913429704tmp c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire_07436515838238741397tmp"
      [ERROR] -> [Help 1]
      org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.19.1:integration-test (failsafe-integration-tests) on project AF: Execution failsafe-integration-tests of goal org.apache.maven.plugins:maven-failsafe-plugin:2.19.1:integration-test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
      Command was cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_65\jre\bin\java" -Xmx4096m -jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefirebooter4689747360754285342.jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire1687639200913429704tmp c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire_07436515838238741397tmp"
      at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
      at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
      at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
      at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
      at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
      at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
      at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
      at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
      at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
      at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
      at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
      at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
      at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:497)
      at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
      at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
      at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
      at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
      Caused by: org.apache.maven.plugin.PluginExecutionException: Execution failsafe-integration-tests of goal org.apache.maven.plugins:maven-failsafe-plugin:2.19.1:integration-test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
      Command was cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_65\jre\bin\java" -Xmx4096m -jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefirebooter4689747360754285342.jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire1687639200913429704tmp c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire_07436515838238741397tmp"
      at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
      at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
      ... 20 more
      Caused by: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
      Command was cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_65\jre\bin\java" -Xmx4096m -jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefirebooter4689747360754285342.jar c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire1687639200913429704tmp c:\jenkins-slave\workspace\Automation_Matrix_New\55a7bc48\surefire\surefire_07436515838238741397tmp"
      at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:590)
      at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:460)
      at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:229)
      at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:201)
      at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1026)
      at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:862)
      at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:755)
      at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
      ... 21 more

          [JENKINS-33852] Exception while copying in thread - causes Slave disconnection

          Noam Manos added a comment -

          Many attempts to find cause in Maven configuration, or in Java implementation, or to set any known Jenkins configuration, to workaround this problem, did not succeed, and this unpredictable error kept on happening, until...
          I asked our IT guys to increase CPU and RAM (from 4GB to 16GB), and walla - the problem disappeared.
          It's still a bug, but at least now with a workaround.

          Noam Manos added a comment - Many attempts to find cause in Maven configuration, or in Java implementation, or to set any known Jenkins configuration, to workaround this problem, did not succeed, and this unpredictable error kept on happening, until... I asked our IT guys to increase CPU and RAM (from 4GB to 16GB), and walla - the problem disappeared. It's still a bug, but at least now with a workaround.

          Oleg Nenashev added a comment -

          Should be re-tested with Jenkins 2.32.1+, which includes the https://github.com/kohsuke/winp/issues/22 fix in Windows process management library.

          Oleg Nenashev added a comment - Should be re-tested with Jenkins 2.32.1+, which includes the https://github.com/kohsuke/winp/issues/22 fix in Windows process management library.

            Unassigned Unassigned
            nmanos Noam Manos
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: