Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41199

Provisioned agent ECS Slave ecs-quitest failed to launch

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Minor Minor
    • amazon-ecs-plugin
    • Jenkins 2.32.1
      ECS Agent 1.13.1

      Hi support
      I'm using ECS for Jenkins slave, when I build test, I saw tasks running in ECS, but in Jenkins said ECS offline. The connection between jenkins and ECS is ok, please advise me how to figure out this issue.
      Thanks.

      --------------
      Making ecs-cloud-14121d927e9f offline because it's not responding
      Jan 19, 2017 3:38:36 AM WARNING hudson.node_monitors.ResponseTimeMonitor$1 monitor
      Making ecs-cloud-3421ccae4c82 offline because it's not responding
      Jan 19, 2017 3:38:46 AM WARNING hudson.model.listeners.ItemListener forAll
      failed to send event to listener of class org.jenkinsci.plugins.pipeline.milestone.MilestoneStepExecution$CleanupJobsOnDelete
      java.lang.NullPointerException
      at org.jenkinsci.plugins.pipeline.milestone.MilestoneStepExecution$CleanupJobsOnDelete.onDeleted(MilestoneStepExecution.java:348)
      at hudson.model.listeners.ItemListener$4.apply(ItemListener.java:205)
      at hudson.model.listeners.ItemListener$4.apply(ItemListener.java:203)
      at hudson.model.listeners.ItemListener.forAll(ItemListener.java:167)
      at hudson.model.listeners.ItemListener.fireOnDeleted(ItemListener.java:203)
      at jenkins.model.Jenkins.onDeleted(Jenkins.java:2924)
      at jenkins.model.Jenkins.onDeleted(Jenkins.java:325)
      at hudson.model.AbstractItem.delete(AbstractItem.java:591)
      at hudson.model.Job.delete(Job.java:683)
      at hudson.model.AbstractItem.doDoDelete(AbstractItem.java:550)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:335)
      at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:52)
      at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
      at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:175)
      at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:108)
      at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:124)
      at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
      at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:80)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      at org.eclipse.jetty.server.Server.handle(Server.java:499)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
      at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)

      Jan 19, 2017 3:42:22 AM WARNING com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call
      ECS Slave ecs-cloud-364b79cfdce9 (ecs task arn:aws:ecs:eu-central-1:280052068306:task/a65941bb-c874-43a6-a008-bcefe37ba60b) not connected since 1/19/17 3:27 AM seconds
      Jan 19, 2017 3:42:31 AM WARNING hudson.slaves.NodeProvisioner$2 run
      Provisioned agent ECS Slave ecs-cloud failed to launch
      java.lang.IllegalStateException: ECS Slave ecs-cloud-364b79cfdce9 (ecs task arn:aws:ecs:eu-central-1:280052068306:task/a65941bb-c874-43a6-a008-bcefe37ba60b) not connected since 1/19/17 3:27 AM seconds
      at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:294)
      at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:234)
      at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)

      Jan 19, 2017 3:50:12 AM WARNING com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call
      ECS Slave ecs-cloud-36b8e927adcc (ecs task arn:aws:ecs:eu-central-1:280052068306:task/397f3ec8-55d0-4ead-96af-e871e129e552) not connected since 1/19/17 3:35 AM seconds
      Jan 19, 2017 3:50:21 AM WARNING hudson.slaves.NodeProvisioner$2 run
      Provisioned agent ECS Slave ecs-quitest failed to launch
      java.lang.IllegalStateException: ECS Slave ecs-cloud-36b8e927adcc (ecs task arn:aws:ecs:eu-central-1:280052068306:task/397f3ec8-55d0-4ead-96af-e871e129e552) not connected since 1/19/17 3:35 AM seconds
      at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:294)
      at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:234)
      at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)

      Jan 19, 2017 4:01:35 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud
      Create cloud ecs-cloud on ECS cluster arn:aws:ecs:eu-central-1:280052068306:cluster/jenkins-slave on the region eu-central-1
      Jan 19, 2017 4:01:35 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSTaskTemplate setOwner
      Slave ecs - Created Task Definition: arn:aws:ecs:eu-central-1:280052068306:task-definition/ecs-cloud-ecs:4
      Jan 19, 2017 4:02:01 AM INFO hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
      Started provisioning ECS Slave ecs from ecs-cloud with 1 executors. Remaining excess workload: 0
      Jan 19, 2017 4:02:01 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources
      Found 2 instances
      Jan 19, 2017 4:02:01 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources
      Resources found in instance arn:aws:ecs:eu-central-1:280052068306:container-instance/046e5f71-ec94-4d46-b59c-e1361c22fcc1: [

      {Name: CPU,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 2048,StringSetValue: []}

      ,

      {Name: MEMORY,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 3954,StringSetValue: []}

      ,

      {Name: PORTS,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: [22, 2376, 2375, 51678, 51679]}

      ,

      {Name: PORTS_UDP,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: []}

      ]
      Jan 19, 2017 4:02:01 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources
      Instance arn:aws:ecs:eu-central-1:280052068306:container-instance/046e5f71-ec94-4d46-b59c-e1361c22fcc1 has 3,954mb of free memory. 512mb are required
      Jan 19, 2017 4:02:01 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources
      Instance arn:aws:ecs:eu-central-1:280052068306:container-instance/046e5f71-ec94-4d46-b59c-e1361c22fcc1 has 2,048 units of free cpu. 1 units are required
      Jan 19, 2017 4:02:01 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call
      Created Slave: ecs-cloud-382fc26759cf
      Jan 19, 2017 4:02:01 AM INFO com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call
      Slave ecs-cloud-382fc26759cf - Slave Task Started : arn:aws:ecs:eu-central-1:280052068306:task/2f4e77e5-ff09-495c-a93c-b7c6a93fbbb3
      Jan 19, 2017 4:02:50 AM WARNING hudson.node_monitors.ResponseTimeMonitor$1 monitor
      Making ecs-cloud-14121d927e9f offline because it's not responding
      Jan 19, 2017 4:02:50 AM WARNING hudson.node_monitors.ResponseTimeMonitor$1 monitor
      Making ecs-cloud-3421ccae4c82 offline because it's not responding

            Unassigned Unassigned
            tiendungitd Dennis Tran
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: