Seems that JAR caching also takes DLL files on Windows.
      Thanks to Windows architecture (), it leads to access conflicts, because DDLs may be locked by other process.

      The issue is being heavily reproduced during the process. After several hours, any process termination fails with the FileNotFoundException exception (see the log below).

      Probably, the high frequency of the issue is somehow related to "Cygwin Process Killer" plugin. On my installation it calls additional remote call on the node before any process termination (it is an interim workaround for JENKINS-19156 and JENKINS-20289 in the custom 1.509.4 core build).

      Dec 07, 2013 7:02:28 AM org.jvnet.winp.Native load
      WARNING: Failed to write winp.x64.dll
      java.io.FileNotFoundException: C:\Users\XXX\.jenkins\cache\jars\01\winp.x64.dll (The process cannot access the file because it is being used by another process)
      at java.io.FileOutputStream.open(Native Method)
      at java.io.FileOutputStream.<init>(Unknown Source)
      at java.io.FileOutputStream.<init>(Unknown Source)
      at org.jvnet.winp.Native.load(Native.java:81)
      at org.jvnet.winp.Native.<clinit>(Native.java:52)
      at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:200)
      at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:470)
      at hudson.util.ProcessTree.get(ProcessTree.java:335)
      at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:899)
      at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:890)
      at hudson.remoting.UserRequest.perform(UserRequest.java:118)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:326)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
      at java.util.concurrent.FutureTask.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at hudson.remoting.Engine$1$1.run(Engine.java:63)
      at java.lang.Thread.run(Unknown Source)

      Dec 07, 2013 7:02:35 AM hudson.util.ProcessTree get
      WARNING: Failed to load winp. Reverting to the default
      java.lang.UnsatisfiedLinkError: Native Library C:\Users\XXX\.jenkins\cache\jars\01\winp.x64.dll already loaded in another classloader
      at java.lang.ClassLoader.loadLibrary1(Unknown Source)
      at java.lang.ClassLoader.loadLibrary0(Unknown Source)
      at java.lang.ClassLoader.loadLibrary(Unknown Source)
      at java.lang.Runtime.load0(Unknown Source)
      at java.lang.System.load(Unknown Source)
      at org.jvnet.winp.Native.loadDll(Native.java:158)
      at org.jvnet.winp.Native.load(Native.java:90)
      at org.jvnet.winp.Native.<clinit>(Native.java:52)
      at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:200)
      at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:470)
      at hudson.util.ProcessTree.get(ProcessTree.java:335)
      at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:899)
      at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:890)
      at hudson.remoting.UserRequest.perform(UserRequest.java:118)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:326)
      at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
      at java.util.concurrent.FutureTask.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at hudson.remoting.Engine$1$1.run(Engine.java:63)
      at java.lang.Thread.run(Unknown Source)

          [JENKINS-20913] [jar caching] - winp.x64.dll access conflicts

          boris ivan added a comment -

          Yeah, this one is killing me. This is an exact duplicate of the bug I logged previously: 20759. Hoping to see some activity on this soon, it's extraordinarily painful. Curious, do your remote build agents run as a process or from the command line (cmd/powershell/etc)?

          boris ivan added a comment - Yeah, this one is killing me. This is an exact duplicate of the bug I logged previously: 20759. Hoping to see some activity on this soon, it's extraordinarily painful. Curious, do your remote build agents run as a process or from the command line (cmd/powershell/etc)?

          boris ivan added a comment -

          One additional note: with other CI servers (TC) with the same JRE running the same Maven goal (integration-test) with the same version of Maven on a build agent running the same version of windows, etc... this has never occurred. Therefore I don't think it's something that is contained within the Maven build itself. It's either plugin related or something else Jenkins specific that interacts with this that causes the scenario to occur. But on each of my 6 remote Jenkins build slaves running various different Maven projects, this always occurs 1-2 times a week.

          boris ivan added a comment - One additional note: with other CI servers (TC) with the same JRE running the same Maven goal (integration-test) with the same version of Maven on a build agent running the same version of windows, etc... this has never occurred. Therefore I don't think it's something that is contained within the Maven build itself. It's either plugin related or something else Jenkins specific that interacts with this that causes the scenario to occur. But on each of my 6 remote Jenkins build slaves running various different Maven projects, this always occurs 1-2 times a week.

          kishorerp added a comment - - edited

          We are seeing this on win8/8.1 slaves once in a week!!
          slave.jar crashes with the above error
          we are on Jenkins LTS 1.532.1 running on CentOs6.5
          also, at the end of each job, the above issue occurs everytime!!

          kishorerp added a comment - - edited We are seeing this on win8/8.1 slaves once in a week!! slave.jar crashes with the above error we are on Jenkins LTS 1.532.1 running on CentOs6.5 also, at the end of each job, the above issue occurs everytime!!

          Oleg Nenashev added a comment -

          @Kishore

          If I understand toy correctly, you see the following issue:

          Unable to render embedded object: File (..can this be fixed please) not found.

          If yes, I suppose that this is an another issue. Please create ticket for it

          Oleg Nenashev added a comment - @Kishore If I understand toy correctly, you see the following issue: Unable to render embedded object: File (..can this be fixed please) not found. If yes, I suppose that this is an another issue. Please create ticket for it

          kishorerp added a comment -

          @Oleg

          Not sure how that embedded object got added.
          We are seeing the same [jar caching] - winp.x64.dll access conflicts mentioned above

          kishorerp added a comment - @Oleg Not sure how that embedded object got added. We are seeing the same [jar caching] - winp.x64.dll access conflicts mentioned above

          This issue is killing the whole continuous integration process. Do we have any workaround or any time line to get this fixed?

          Murali Devakumar added a comment - This issue is killing the whole continuous integration process. Do we have any workaround or any time line to get this fixed?

          Oleg Nenashev added a comment -

          @Murali
          Currently there is no solutions or ETAs.

          As a workaround, you can disable Jar caching by calling a "hudson.remoting.Engine.current().setJarCache(null)" to disable the cache (e.g. call a System Groovy script on the node).

          // Probably, it is possible to pass null via "-jar-cache" CLI parameter.

          Oleg Nenashev added a comment - @Murali Currently there is no solutions or ETAs. As a workaround, you can disable Jar caching by calling a "hudson.remoting.Engine.current().setJarCache(null)" to disable the cache (e.g. call a System Groovy script on the node). // Probably, it is possible to pass null via "-jar-cache" CLI parameter.

          @Oleg,
          Thanks for quick response.

          Can you please be more elaborate on either of options and I am running slave as windows service, which limits me with CLI parameters.

          Murali Devakumar added a comment - @Oleg, Thanks for quick response. Can you please be more elaborate on either of options and I am running slave as windows service, which limits me with CLI parameters.

          kishorerp added a comment -

          @Oleg

          Thanks for your quick response, could you please mention the groovy script that you have told above, this is very predominant in our environment and our ci is going for a toss!

          kishorerp added a comment - @Oleg Thanks for your quick response, could you please mention the groovy script that you have told above, this is very predominant in our environment and our ci is going for a toss!

          Oleg Nenashev added a comment -

          Disclaimer: This approach works on my installation (1.509.4), but it actually corrupts the normal flow and may lead to serious classloading issues.

          You will need a Groovy plugin...

          A simple manual approach:
          1) Go ${JENKINS_URL}/computer/${NODE_NAME}/script
          2) Call "http://arcjenkinsdev/computer/ru20-hw-farm23/script"

          After that, the Jar caching will be disabled till the node's reconnection. Please note that it may affect the performance of node.

          In order to setup an automatic cache disabling, just use Startup Trigger together with a System Groovy Script build step.
          https://wiki.jenkins-ci.org/display/JENKINS/Startup+Trigger

          Oleg Nenashev added a comment - Disclaimer: This approach works on my installation (1.509.4), but it actually corrupts the normal flow and may lead to serious classloading issues. You will need a Groovy plugin... A simple manual approach: 1) Go ${JENKINS_URL}/computer/${NODE_NAME}/script 2) Call "http://arcjenkinsdev/computer/ru20-hw-farm23/script" After that, the Jar caching will be disabled till the node's reconnection. Please note that it may affect the performance of node. In order to setup an automatic cache disabling, just use Startup Trigger together with a System Groovy Script build step. https://wiki.jenkins-ci.org/display/JENKINS/Startup+Trigger

          kishorerp added a comment - - edited

          @Oleg
          thanks for your response
          not sure i can access the URL of the script mentioned, can you please check.
          when you say this affects performance of a node, could you be a little specific in terms what would be affected, (network bandwidth or cpu cycles?)
          Meanwhile i tried
          to pass null via "-jar-cache" CLI parameter
          It is not possible to do so, the slave doesn't launch via jnlp

          kishorerp added a comment - - edited @Oleg thanks for your response not sure i can access the URL of the script mentioned, can you please check. when you say this affects performance of a node, could you be a little specific in terms what would be affected, (network bandwidth or cpu cycles?) Meanwhile i tried to pass null via "-jar-cache" CLI parameter It is not possible to do so, the slave doesn't launch via jnlp

          Oleg Nenashev added a comment -

          @Kishore
          > not sure i can access the URL of the script mentioned
          There should be a "Script Console" action on the slave's main page

          > when you say this affects performance of a node, could you be a little specific in terms what would be affected, (network bandwidth or cpu cycles?)
          The JAR caching will be disabled, so your slave will have do download all classed from the master node. So the first run of jobs/etc may take more time, but all other runs won't be affected. Actually, it affects distributed systems only.

          Oleg Nenashev added a comment - @Kishore > not sure i can access the URL of the script mentioned There should be a "Script Console" action on the slave's main page > when you say this affects performance of a node, could you be a little specific in terms what would be affected, (network bandwidth or cpu cycles?) The JAR caching will be disabled, so your slave will have do download all classed from the master node. So the first run of jobs/etc may take more time, but all other runs won't be affected. Actually, it affects distributed systems only.

          kishorerp added a comment -

          Thanks Oleg, have run it once, will observe if the issue occurs again

          kishorerp added a comment - Thanks Oleg, have run it once, will observe if the issue occurs again

          @Oleg
          I'm following your recommendation:
          1) Added a freestyle job triggered via the Startup trigger plugin.
          2) In the build section, Execute system Groovy script -> Groovy command
          3) Just a single line in that field: hudson.remoting.Engine.current().setJarCache(null)

          Running the job throws me a "FATAL: Cannot invoke method setJarCache() on null object".
          Doing a print of hudson.remoting.Engine.current() in the same build job shows it as null.

          If I do this through "There should be a "Script Console" action on the slave's main page" everything runs fine (and I see that hudson.remoting.Engine.current() is not null).

          Would you have any idea why this is?

          Horia Constantin added a comment - @Oleg I'm following your recommendation: 1) Added a freestyle job triggered via the Startup trigger plugin. 2) In the build section, Execute system Groovy script -> Groovy command 3) Just a single line in that field: hudson.remoting.Engine.current().setJarCache(null) Running the job throws me a "FATAL: Cannot invoke method setJarCache() on null object". Doing a print of hudson.remoting.Engine.current() in the same build job shows it as null. If I do this through "There should be a "Script Console" action on the slave's main page" everything runs fine (and I see that hudson.remoting.Engine.current() is not null). Would you have any idea why this is?

          Oleg Nenashev added a comment -

          @Horia Constantin
          It was a my mistake.
          System Groovy script always executes on the master instead of the target slave.

          You can use the Scriptler plugin:
          1) Create a Scriptler script

          • Write down the groovy call into the script window
          • Allow execution by users having "Run Script Permission"
          • Check that "Run always on master" is disabled
            2) Call this script as a build step inside your job

          Oleg Nenashev added a comment - @Horia Constantin It was a my mistake. System Groovy script always executes on the master instead of the target slave. You can use the Scriptler plugin: 1) Create a Scriptler script Write down the groovy call into the script window Allow execution by users having "Run Script Permission" Check that "Run always on master" is disabled 2) Call this script as a build step inside your job

          Oleg Nenashev added a comment -

          Added the lts-candidate label.
          The issue makes all Windows process termination flows unreliable

          Oleg Nenashev added a comment - Added the lts-candidate label. The issue makes all Windows process termination flows unreliable

          @Oleg, your latest suggestion worked.

          The job that runs great for me: using the Startup trigger plugin, triggered for a specific label. But, due to some issues in the Startup trigger plugin, with some tweaking:
          install manually version 2.4
          configure the job "This build is parameterized", with a label parameter (the same label as before) and with "Run on all nodes matching the label" checked.

          Horia Constantin added a comment - @Oleg, your latest suggestion worked. The job that runs great for me: using the Startup trigger plugin, triggered for a specific label. But, due to some issues in the Startup trigger plugin, with some tweaking: install manually version 2.4 configure the job "This build is parameterized", with a label parameter (the same label as before) and with "Run on all nodes matching the label" checked.

          Oleg Nenashev added a comment -

          Created a PR, which allows to disable JAR caching via the CLI option
          https://github.com/jenkinsci/remoting/pull/21

          BTW, it is just a workaround. The complete resolution requires some tweaks in the caching procedures.

          Oleg Nenashev added a comment - Created a PR, which allows to disable JAR caching via the CLI option https://github.com/jenkinsci/remoting/pull/21 BTW, it is just a workaround. The complete resolution requires some tweaks in the caching procedures.

          I've made the change in winp 1.19 to be more intelligent about overwriting the file.

          Historically winp uses timestamp based up-to-date check, but because remoting uses timestamp of jar files for LRU checks, this overwrite check was broken. In 1.19 I've switched to checksum-based up-to-date check, which resolves the problem.

          But as I was doing that, I noticed a deeper issue here. "Native Library ...\winp.x64.dll already loaded in another classloader " is likely because JNLP slave is retrying after it lost a connection with the master. During the previous connection, the slave has loaded one copy of winp.dll, and after reconnection it's trying to load one more, which is obviously failing.

          To properly solve this requires a change in the client (and possibly winsw), so that after losing a connection the slave will have winsw relaunch a new service, so that JVM will start fresh.

          This has been a long standing TODO, but it's a good change as it would prevent memory leak and static state clutter over time.

          Kohsuke Kawaguchi added a comment - I've made the change in winp 1.19 to be more intelligent about overwriting the file. Historically winp uses timestamp based up-to-date check, but because remoting uses timestamp of jar files for LRU checks, this overwrite check was broken. In 1.19 I've switched to checksum-based up-to-date check, which resolves the problem. But as I was doing that, I noticed a deeper issue here. "Native Library ...\winp.x64.dll already loaded in another classloader " is likely because JNLP slave is retrying after it lost a connection with the master. During the previous connection, the slave has loaded one copy of winp.dll, and after reconnection it's trying to load one more, which is obviously failing. To properly solve this requires a change in the client (and possibly winsw), so that after losing a connection the slave will have winsw relaunch a new service, so that JVM will start fresh. This has been a long standing TODO, but it's a good change as it would prevent memory leak and static state clutter over time.

          I suppose another simple workaround fix would be to copy winp.dll into another file and load it. It'd work around "already loaded in another classloader" problem.

          Kohsuke Kawaguchi added a comment - I suppose another simple workaround fix would be to copy winp.dll into another file and load it. It'd work around "already loaded in another classloader" problem.

          kishorerp added a comment -

          @Kohsuke, Do we have a rough idea on which LTS release would have this fix. We are using LTS 1.532 on CentOS for our CI which is expected to be online 24x7x365, and we see this issue that seems to pop up on around 30 of our slaves every week or so.
          @Oleg, we would not want the workaround mentioned since all our slaves are not expected to have any overhead whatsoever due to any other running process.

          kishorerp added a comment - @Kohsuke, Do we have a rough idea on which LTS release would have this fix. We are using LTS 1.532 on CentOS for our CI which is expected to be online 24x7x365, and we see this issue that seems to pop up on around 30 of our slaves every week or so. @Oleg, we would not want the workaround mentioned since all our slaves are not expected to have any overhead whatsoever due to any other running process.

          Oleg Nenashev added a comment - - edited

          @Koshuke
          I've also noticed that I've selected an improper way.

          > To properly solve this requires a change in the client (and possibly winsw), so that after losing a connection the slave will have winsw relaunch a new service, so that JVM will start fresh.
          I suppose it slightly conflicts with "autonomous slaves" approach (slaves may continue operations on temporary disconnects), which usually appears during the infrastructure reliability discussions.

          What about checking if the DLL has been already loaded? http://stackoverflow.com/questions/1007861/how-do-i-get-a-list-of-jni-libraries-which-are-loaded/
          I suppose that the slave may just log a warning in such case

          @Kishore
          I'd prefer to recall the proposed Groovy script above, because it may lead to NPEs on uncached resource loads after restarts or plugin updates.
          Such solution should be used only on long-run slaves with "warm up" jobs and without class unloading.

          Oleg Nenashev added a comment - - edited @Koshuke I've also noticed that I've selected an improper way. > To properly solve this requires a change in the client (and possibly winsw), so that after losing a connection the slave will have winsw relaunch a new service, so that JVM will start fresh. I suppose it slightly conflicts with "autonomous slaves" approach (slaves may continue operations on temporary disconnects), which usually appears during the infrastructure reliability discussions. What about checking if the DLL has been already loaded? http://stackoverflow.com/questions/1007861/how-do-i-get-a-list-of-jni-libraries-which-are-loaded/ I suppose that the slave may just log a warning in such case @Kishore I'd prefer to recall the proposed Groovy script above, because it may lead to NPEs on uncached resource loads after restarts or plugin updates. Such solution should be used only on long-run slaves with "warm up" jobs and without class unloading.

          This bug is not even marked as fixed yet, so it's way too early to predict LTS backporting plan. We'd first have to fix this in the trunk.

          Knowing whether the library is already loaded or not doesn't really help either, because if it's loaded through another classloader, we won't be able to reuse currently loaded copy.

          Kohsuke Kawaguchi added a comment - This bug is not even marked as fixed yet, so it's way too early to predict LTS backporting plan. We'd first have to fix this in the trunk. Knowing whether the library is already loaded or not doesn't really help either, because if it's loaded through another classloader, we won't be able to reuse currently loaded copy.

          I'm now marking this issue fixed with winp 1.19.

          The other part of the issue (of not reusing the same JVM for multiple reconnections to the master) will be tracked in JENKINS-19055,

          Kohsuke Kawaguchi added a comment - I'm now marking this issue fixed with winp 1.19. The other part of the issue (of not reusing the same JVM for multiple reconnections to the master) will be tracked in JENKINS-19055 ,

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          core/pom.xml
          http://jenkins-ci.org/commit/jenkins/4d969004c121bcef07189dfe89fe3347646e741a
          Log:
          JENKINS-20913 Use winp 1.19

          Updated to a newer version

          (cherry picked from commit 6b350af44dee8e1e6cad4cce67941b3ebf88ac52)

          Conflicts:
          core/pom.xml

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/pom.xml http://jenkins-ci.org/commit/jenkins/4d969004c121bcef07189dfe89fe3347646e741a Log: JENKINS-20913 Use winp 1.19 Updated to a newer version (cherry picked from commit 6b350af44dee8e1e6cad4cce67941b3ebf88ac52) Conflicts: core/pom.xml

          Arie Belenky added a comment -

          Hi,
          This issue started to reproduce during the last week on our AWS Windows Server 2012 R2 base machines.
          Jenkins ver. 1.640
          Traceback:
          SEVERE: I/O error in channel channel
          java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(Unknown Source)
          at java.net.SocketInputStream.read(Unknown Source)
          at java.io.BufferedInputStream.fill(Unknown Source)
          at java.io.BufferedInputStream.read(Unknown Source)
          at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82)
          at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:72)
          at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103)
          at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
          at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
          at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

          Dec 31, 2015 5:17:38 PM hudson.remoting.Request$2 run
          SEVERE: Failed to send back a reply
          java.net.SocketException: Connection reset by peer: socket write error
          at java.net.SocketOutputStream.socketWrite0(Native Method)
          at java.net.SocketOutputStream.socketWrite(Unknown Source)
          at java.net.SocketOutputStream.write(Unknown Source)
          at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
          at java.io.BufferedOutputStream.write(Unknown Source)
          at hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:94)
          at hudson.remoting.ChunkedOutputStream.drain(ChunkedOutputStream.java:89)
          at hudson.remoting.ChunkedOutputStream.write(ChunkedOutputStream.java:58)
          at java.io.OutputStream.write(Unknown Source)
          at hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:45)
          at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:45)
          at hudson.remoting.Channel.send(Channel.java:582)
          at hudson.remoting.Request$2.run(Request.java:340)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at hudson.remoting.Engine$1$1.run(Engine.java:62)
          at java.lang.Thread.run(Unknown Source)

          Dec 31, 2015 5:17:38 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Terminated
          Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Locating server among https://jenkins.interjunk.com/
          Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Handshaking
          Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connecting to jenkins.interjunk.com:15400
          Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Trying protocol: JNLP2-connect
          Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status
          INFO: Connected
          Dec 31, 2015 5:19:00 PM hudson.util.ProcessTree get
          WARNING: Failed to load winp. Reverting to the default
          java.lang.UnsatisfiedLinkError: Native Library C:\Users\automation\.jenkins\cache\jars\4A\winp.x64.22D9AB310A3FA2D96B6E03A836A47724.dll
          already loaded in another classloader
          at java.lang.ClassLoader.loadLibrary1(Unknown Source)
          at java.lang.ClassLoader.loadLibrary0(Unknown Source)
          at java.lang.ClassLoader.loadLibrary(Unknown Source)
          at java.lang.Runtime.load0(Unknown Source)
          at java.lang.System.load(Unknown Source)
          at org.jvnet.winp.Native.loadDll(Native.java:190)
          at org.jvnet.winp.Native.load(Native.java:122)
          at org.jvnet.winp.Native.<clinit>(Native.java:56)
          at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:212)
          at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:494)
          at hudson.util.ProcessTree.get(ProcessTree.java:345)
          at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:965)
          at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:956)
          at hudson.remoting.UserRequest.perform(UserRequest.java:120)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:326)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at hudson.remoting.Engine$1$1.run(Engine.java:62)
          at java.lang.Thread.run(Unknown Source)

          Arie Belenky added a comment - Hi, This issue started to reproduce during the last week on our AWS Windows Server 2012 R2 base machines. Jenkins ver. 1.640 Traceback: SEVERE: I/O error in channel channel java.net.SocketException: Connection reset at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82) at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:72) at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103) at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48) Dec 31, 2015 5:17:38 PM hudson.remoting.Request$2 run SEVERE: Failed to send back a reply java.net.SocketException: Connection reset by peer: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(Unknown Source) at java.net.SocketOutputStream.write(Unknown Source) at java.io.BufferedOutputStream.flushBuffer(Unknown Source) at java.io.BufferedOutputStream.write(Unknown Source) at hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:94) at hudson.remoting.ChunkedOutputStream.drain(ChunkedOutputStream.java:89) at hudson.remoting.ChunkedOutputStream.write(ChunkedOutputStream.java:58) at java.io.OutputStream.write(Unknown Source) at hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:45) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:45) at hudson.remoting.Channel.send(Channel.java:582) at hudson.remoting.Request$2.run(Request.java:340) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:62) at java.lang.Thread.run(Unknown Source) Dec 31, 2015 5:17:38 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Terminated Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Locating server among https://jenkins.interjunk.com/ Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Handshaking Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connecting to jenkins.interjunk.com:15400 Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Trying protocol: JNLP2-connect Dec 31, 2015 5:17:48 PM hudson.remoting.jnlp.Main$CuiListener status INFO: Connected Dec 31, 2015 5:19:00 PM hudson.util.ProcessTree get WARNING: Failed to load winp. Reverting to the default java.lang.UnsatisfiedLinkError: Native Library C:\Users\automation\.jenkins\cache\jars\4A\winp.x64.22D9AB310A3FA2D96B6E03A836A47724.dll already loaded in another classloader at java.lang.ClassLoader.loadLibrary1(Unknown Source) at java.lang.ClassLoader.loadLibrary0(Unknown Source) at java.lang.ClassLoader.loadLibrary(Unknown Source) at java.lang.Runtime.load0(Unknown Source) at java.lang.System.load(Unknown Source) at org.jvnet.winp.Native.loadDll(Native.java:190) at org.jvnet.winp.Native.load(Native.java:122) at org.jvnet.winp.Native.<clinit>(Native.java:56) at org.jvnet.winp.WinProcess.enableDebugPrivilege(WinProcess.java:212) at hudson.util.ProcessTree$Windows.<clinit>(ProcessTree.java:494) at hudson.util.ProcessTree.get(ProcessTree.java:345) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:965) at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:956) at hudson.remoting.UserRequest.perform(UserRequest.java:120) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at hudson.remoting.Engine$1$1.run(Engine.java:62) at java.lang.Thread.run(Unknown Source)

          Hi, this issue is reproduced on our servers.

          Jenkins ver. 1.625.1 :
          Master on CentOS 6
          Node in Windows 7, slave running as service, working !! (ie. NO java.lang.UnsatisfiedLinkError raised with winp dll)
          Node in Windows 7, slave running via CLA, NOT working (ie. java.lang.UnsatisfiedLinkError raised with winp dll)
          Node in Windows 8.1, not working in both cases ( as service, or via CLI)

          Pierre-Henri Cazes added a comment - Hi, this issue is reproduced on our servers. Jenkins ver. 1.625.1 : Master on CentOS 6 Node in Windows 7, slave running as service, working !! (ie. NO java.lang.UnsatisfiedLinkError raised with winp dll) Node in Windows 7, slave running via CLA, NOT working (ie. java.lang.UnsatisfiedLinkError raised with winp dll) Node in Windows 8.1, not working in both cases ( as service, or via CLI)

          ci_jenkinsci_org, this bug seems to be caused by winp library issue https://github.com/kohsuke/winp/issues/26. Could you please promote a new release of fixed winp and update Jenkins to it in next LTS?

          Grigory Merkin added a comment - ci_jenkinsci_org , this bug seems to be caused by winp library issue https://github.com/kohsuke/winp/issues/26 . Could you please promote a new release of fixed winp and update Jenkins to it in next LTS?

          Ben Hines added a comment -

          We are also seeing this error pretty often.

          Ben Hines added a comment - We are also seeing this error pretty often.

          Oleg Nenashev added a comment -

          Additional fix has been integrated towards 2.34

          Oleg Nenashev added a comment - Additional fix has been integrated towards 2.34

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/pom.xml
          http://jenkins-ci.org/commit/jenkins/63c2f6c5d7d154a3a0f58c54f04f9b1a25ea5385
          Log:
          Update winp to 1.24. In particular, it addresses issues like JENKINS-20913(https://issues.jenkins-ci.org/browse/JENKINS-20913) (#2619)

              1. Changes to be picked
              1. 1.24

          Release date: Nov 2, 2016

              1. 1.23

          Release date: Fev 16, 2015

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/pom.xml http://jenkins-ci.org/commit/jenkins/63c2f6c5d7d154a3a0f58c54f04f9b1a25ea5385 Log: Update winp to 1.24. In particular, it addresses issues like JENKINS-20913 ( https://issues.jenkins-ci.org/browse/JENKINS-20913 ) (#2619) Changes to be picked 1.24 Release date: Nov 2, 2016 Issue #22 ( https://github.com/kohsuke/winp/issues/22 ) - Winp sometimes kills random processes when using killRecursive. ( PR #23 ( https://github.com/kohsuke/winp/pull/23 )) [WINP-10] ( https://java.net/jira/browse/WINP-10 ) - Fix for `getCmdLineAndEnvVars()` which fails on x64 versions of Windows. ( PR #20 ( https://github.com/kohsuke/winp/pull/20 )) Issue #24 ( https://github.com/kohsuke/winp/issues/24 ) - Wrong folder when using the `winp.folder.preferred` system property (parent instead of the actual folder). ( PR #25 ( https://github.com/kohsuke/winp/pull/25 )) Issue #26 ( https://github.com/kohsuke/winp/issues/26 ), JENKINS-20913 ( https://issues.jenkins-ci.org/browse/JENKINS-20913 ) - Native class now tries loading DLLs via the temp location. ( PR #27 ( https://github.com/kohsuke/winp/pull/27 )) 1.23 Release date: Fev 16, 2015 Migrate native components to Visual Studio Community 2013. ( PR #14 ( https://github.com/kohsuke/winp/pull/14 )) Provide a `winp.unpack.dll.to.parent.dir` property, which disables DLL unpacking to the parent dir. ( PR #14 ( https://github.com/kohsuke/winp/pull/12 ))

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/pom.xml
          http://jenkins-ci.org/commit/jenkins/e66bfbafa99fc092c6f9fe51f8bf5be267340557
          Log:
          Update winp to 1.24. In particular, it addresses issues like JENKINS-20913(https://issues.jenkins-ci.org/browse/JENKINS-20913) (#2619)

              1. Changes to be picked
              1. 1.24

          Release date: Nov 2, 2016

              1. 1.23

          Release date: Fev 16, 2015

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/pom.xml http://jenkins-ci.org/commit/jenkins/e66bfbafa99fc092c6f9fe51f8bf5be267340557 Log: Update winp to 1.24. In particular, it addresses issues like JENKINS-20913 ( https://issues.jenkins-ci.org/browse/JENKINS-20913 ) (#2619) Changes to be picked 1.24 Release date: Nov 2, 2016 Issue #22 ( https://github.com/kohsuke/winp/issues/22 ) - Winp sometimes kills random processes when using killRecursive. ( PR #23 ( https://github.com/kohsuke/winp/pull/23 )) [WINP-10] ( https://java.net/jira/browse/WINP-10 ) - Fix for `getCmdLineAndEnvVars()` which fails on x64 versions of Windows. ( PR #20 ( https://github.com/kohsuke/winp/pull/20 )) Issue #24 ( https://github.com/kohsuke/winp/issues/24 ) - Wrong folder when using the `winp.folder.preferred` system property (parent instead of the actual folder). ( PR #25 ( https://github.com/kohsuke/winp/pull/25 )) Issue #26 ( https://github.com/kohsuke/winp/issues/26 ), JENKINS-20913 ( https://issues.jenkins-ci.org/browse/JENKINS-20913 ) - Native class now tries loading DLLs via the temp location. ( PR #27 ( https://github.com/kohsuke/winp/pull/27 )) 1.23 Release date: Fev 16, 2015 Migrate native components to Visual Studio Community 2013. ( PR #14 ( https://github.com/kohsuke/winp/pull/14 )) Provide a `winp.unpack.dll.to.parent.dir` property, which disables DLL unpacking to the parent dir. ( PR #14 ( https://github.com/kohsuke/winp/pull/12 )) (cherry picked from commit 63c2f6c5d7d154a3a0f58c54f04f9b1a25ea5385)

            oleg_nenashev Oleg Nenashev
            oleg_nenashev Oleg Nenashev
            Votes:
            14 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated:
              Resolved: