I have a pipeline project which should run at a EC2 instance node.

      I have configured an EC2 connection and starting EC2 t3.medium Windows 10 instances automatically. This all works fine.

      But, the first build at an EC2 instance always performs very bad (slow!!). The next build at the same instance (without rebott etc) is much more faster.

       

      @Library('BMS-Libraries')
      import static bms.mail.Email.*
      import static bms.nexus.Nexus.*
      import static bms.utils.Utils.*
      
      node('AWS_VS2017') {
      		stage('Cleanup Build Machine'){
      			//deleting current workspace directory
      			deleteDir()
      		}
      		
      		stage('Preparing Build machine...'){
      	retrieveAndExtractBuildTools(this)
      		}
      
      //Do some more .......
      }
      

      I attached a screenshot of the runtime of the different pipeline steps.

       

      I connected via RDP to the instance during first build nad task-manager didn't display a high CPU or Memory consumption

          [JENKINS-62158] Bad performance on EC2 instance for first build

          hungvotrung did adding the jenkins\cache folder to whitelist of windows defender work for you with better performance?

          Daniel Hoerner added a comment - hungvotrung did adding the jenkins\cache folder to whitelist of windows defender work for you with better performance?

          Hung Vo added a comment -

          unfortunately, it doesn't help at all. There is still a 4 - 5 min delay. 

          Hung Vo added a comment - unfortunately, it doesn't help at all. There is still a 4 - 5 min delay. 

          Hung Vo added a comment -

          I'm not sure if it related but I can see there seem connection issue between master and agent. I did netstat when there was 6 windows agents and master open total 1744 connections. Not sure how it work behind the scene as i haven't dig more but total amount seem too high imo

          Hung Vo added a comment - I'm not sure if it related but I can see there seem connection issue between master and agent. I did netstat when there was 6 windows agents and master open total 1744 connections. Not sure how it work behind the scene as i haven't dig more but total amount seem too high imo

          Are you using WinRM by any chance?

          Raihaan Shouhell added a comment - Are you using WinRM by any chance?

          The EC2 Plugin always use WinRM for Windows Slaves at AWS.

          Daniel Hoerner added a comment - The EC2 Plugin always use WinRM for Windows Slaves at AWS.

          By default yes, there is a way to use ssh.

          Have you tried the latest of the plugin by any chance?

          Raihaan Shouhell added a comment - By default yes, there is a way to use ssh. Have you tried the latest of the plugin by any chance?

          I just tried the latest version (1.54 of the plugin) with Jenkins 2.249.3 and nothing has changed.

          I also have some issues with a broken connection during the build with following output:

          Node Log:

          Agent successfully connected and online
          ERROR: Connection terminated
          java.io.EOFException
          	at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2763)
          	at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3258)
          	at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:873)
          	at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:350)
          	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
          	at hudson.remoting.Command.readFrom(Command.java:142)
          	at hudson.remoting.Command.readFrom(Command.java:128)
          	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
          	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
          Caused: java.io.IOException: Unexpected termination of the channel
          	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)

          Build Log:

          EC2 (AWS Cloud) - AWS Windows 10 with VS 2017 (i-xxxxxxxxxxxxxxxxxxxxxxxx) was marked offline: Connection was broken: java.io.EOFExceptionEC2 (AWS Cloud) - AWS Windows 10 with VS 2017 (i-xxxxxxxxxxxxxxxxxxxxxxxx) was marked offline: Connection was broken: java.io.EOFException at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2763) at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3258) at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:873) at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:350) at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) at hudson.remoting.Command.readFrom(Command.java:142) at hudson.remoting.Command.readFrom(Command.java:128) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)Caused: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)
          [Pipeline] }[Pipeline] // node[Pipeline] End of Pipelinejava.io.IOException: Unable to create live FilePath for EC2 (AWS Cloud) - AWS Windows 10 with VS 2017 (i-xxxxxxxxxxxxxxxxxxxxxxxxxx) at org.jenkinsci.plugins.workflow.support.steps.FilePathDynamicContext.get(FilePathDynamicContext.java:64) at org.jenkinsci.plugins.workflow.support.steps.FilePathDynamicContext.get(FilePathDynamicContext.java:47) at org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:94) at org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:138) at org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:135) at org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297) at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:74) at hudson.plugins.emailext.EmailExtStep$EmailExtStepExecution.run(EmailExtStep.java:231) at hudson.plugins.emailext.EmailExtStep$EmailExtStepExecution.run(EmailExtStep.java:174) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47) at hudson.security.ACL.impersonate(ACL.java:367) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)Finished: FAILURE
          

          Daniel Hoerner added a comment - I just tried the latest version (1.54 of the plugin) with Jenkins 2.249.3 and nothing has changed. I also have some issues with a broken connection during the build with following output: Node Log: Agent successfully connected and online ERROR: Connection terminated java.io.EOFException at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2763) at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3258) at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:873) at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:350) at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) at hudson.remoting.Command.readFrom(Command.java:142) at hudson.remoting.Command.readFrom(Command.java:128) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63) Caused: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77) Build Log: EC2 (AWS Cloud) - AWS Windows 10 with VS 2017 (i-xxxxxxxxxxxxxxxxxxxxxxxx) was marked offline: Connection was broken: java.io.EOFExceptionEC2 (AWS Cloud) - AWS Windows 10 with VS 2017 (i-xxxxxxxxxxxxxxxxxxxxxxxx) was marked offline: Connection was broken: java.io.EOFException at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2763) at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3258) at java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:873) at java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:350) at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) at hudson.remoting.Command.readFrom(Command.java:142) at hudson.remoting.Command.readFrom(Command.java:128) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)Caused: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77) [Pipeline] }[Pipeline] // node[Pipeline] End of Pipelinejava.io.IOException: Unable to create live FilePath for EC2 (AWS Cloud) - AWS Windows 10 with VS 2017 (i-xxxxxxxxxxxxxxxxxxxxxxxxxx) at org.jenkinsci.plugins.workflow.support.steps.FilePathDynamicContext.get(FilePathDynamicContext.java:64) at org.jenkinsci.plugins.workflow.support.steps.FilePathDynamicContext.get(FilePathDynamicContext.java:47) at org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:94) at org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:138) at org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:135) at org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297) at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:74) at hudson.plugins.emailext.EmailExtStep$EmailExtStepExecution.run(EmailExtStep.java:231) at hudson.plugins.emailext.EmailExtStep$EmailExtStepExecution.run(EmailExtStep.java:174) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47) at hudson.security.ACL.impersonate(ACL.java:367) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang. Thread .run( Thread .java:834)Finished: FAILURE

          Steven Foster added a comment -

          raihaan

          > By default yes, there is a way to use ssh.

          can you elaborate on this? looking at the code I only see a WinRM connection method, and an SSH connection method that relies on unix tools

          Steven Foster added a comment - raihaan > By default yes, there is a way to use ssh. can you elaborate on this? looking at the code I only see a WinRM connection method, and an SSH connection method that relies on unix tools

          Daniel H added a comment -

          raihaan can you please answer how to use SSH with a Windows Agent? The performance issues still exists also with latest version of the plugin!

          Daniel H added a comment - raihaan can you please answer how to use SSH with a Windows Agent? The performance issues still exists also with latest version of the plugin!

          Yoshihiro added a comment - - edited

          I am currently having similar problems.
          Does anyone have any new information on this?

          Only at first, file access seems unusually slow.

          Yoshihiro added a comment - - edited I am currently having similar problems. Does anyone have any new information on this? Only at first, file access seems unusually slow.

            mramonleon Ramon Leon
            dhoerner Daniel Hoerner
            Votes:
            2 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated: