Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54893

Error provisioning slaves with ECS plugin 1.18

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: amazon-ecs-plugin
    • Labels:
      None
    • Environment:
      amazon-ecs 1.18
      jenkins 2.38.3
    • Similar Issues:

      Description

      After upgrading the Jenkins ECS plugin to 1.18, slaves no longer provision properly.

      I receive the following error:

      Error in provisioning; agent=com.cloudbees.jenkins.plugins.amazonecs.ECSSlave[ecs-fargate-d3jvq]

      java.lang.NullPointerException at com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher.getDockerRunCommand(ECSLauncher.java:247)

       

      Additional logging:

      Nov 27, 2018 6:49:05 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplateNov 27, 2018 6:49:05 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService registerTemplateINFO: Created Task Definition: {TaskDefinitionArn: arn:aws:ecs:us-west-2:XXX:task-definition/ecs-fargate-jenkins-node-8:3385,ContainerDefinitions: [{Name: ecs-fargate-jenkins-node-8,Image: 202306134469.dkr.ecr.us-west-2.amazonaws.com/ookla-jnlp-nodejs-8,Cpu: 2048,Memory: 4096,Links: [],PortMappings: [],Essential: true,EntryPoint: [],Command: [],Environment: [],MountPoints: [],VolumesFrom: [],User: jenkins,Privileged: false,DnsServers: [],DnsSearchDomains: [],ExtraHosts: [],DockerSecurityOptions: [],Ulimits: [],LogConfiguration: {LogDriver: awslogs,Options: {awslogs-group=fargate, awslogs-region=us-west-2, awslogs-stream-prefix=node-8}},}],Family: ecs-fargate-jenkins-node-8,ExecutionRoleArn: arn:aws:iam::XXX:role/ecsTaskExecutionRole,NetworkMode: awsvpc,Revision: 3385,Volumes: [],Status: ACTIVE,RequiresAttributes: [{Name: com.amazonaws.ecs.capability.docker-remote-api.1.17,}, {Name: ecs.capability.execution-role-ecr-pull,}, {Name: com.amazonaws.ecs.capability.docker-remote-api.1.18,}, {Name: ecs.capability.task-eni,}, {Name: com.amazonaws.ecs.capability.ecr-auth,}, {Name: ecs.capability.execution-role-awslogs,}, {Name: com.amazonaws.ecs.capability.logging-driver.awslogs,}, {Name: com.amazonaws.ecs.capability.docker-remote-api.1.19,}],PlacementConstraints: [],Compatibilities: [EC2, FARGATE],RequiresCompatibilities: [FARGATE],Cpu: 2048,Memory: 4096}Nov 27, 2018 6:49:05 PM com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher runECSTaskINFO: [ecs-fargate-d3jvq]: Starting agent with task definition arn:aws:ecs:us-west-2:XXX:task-definition/ecs-fargate-jenkins-node-8:3385}Nov 27, 2018 6:49:05 PM com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher launchWARNING: [ecs-fargate-d3jvq]: Error in provisioning; agent=com.cloudbees.jenkins.plugins.amazonecs.ECSSlave[ecs-fargate-d3jvq]java.lang.NullPointerException at com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher.getDockerRunCommand(ECSLauncher.java:247) at com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher.runECSTask(ECSLauncher.java:220) at com.cloudbees.jenkins.plugins.amazonecs.ECSLauncher.launch(ECSLauncher.java:108) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
      

      Downgrading to 1.17 resolves the issue. These are FARGATE ECS tasks (not EC2)

       

        Attachments

          Issue Links

            Activity

            Hide
            jaredrsowers Jared Sowers added a comment -

            We are having the same issue using EC2 and not Fargate. Downgrading to 1.17 fixed the issue.

             

            This only happens when there are multiple builds trying to spin up instances, if i cancel all but 1 pending build then it can successfully create the container, but if there are multiple it fails with this error.

            Show
            jaredrsowers Jared Sowers added a comment - We are having the same issue using EC2 and not Fargate. Downgrading to 1.17 fixed the issue.   This only happens when there are multiple builds trying to spin up instances, if i cancel all but 1 pending build then it can successfully create the container, but if there are multiple it fails with this error.
            Hide
            mignaulo Olivier Mignault added a comment -

            It doesn't seem obvious to me that this is a duplicate of JENKINS-55078 as the errors are different, but it's possible that the fix for 55078 also fixes this. When 1.19 is released, I'll test and report back.

            Thanks!

            Show
            mignaulo Olivier Mignault added a comment - It doesn't seem obvious to me that this is a duplicate of  JENKINS-55078  as the errors are different, but it's possible that the fix for 55078 also fixes this. When 1.19 is released, I'll test and report back. Thanks!
            Hide
            pgarbe Philipp Garbe added a comment -

            Yes, it's the same fix, that's why I closed this one. You can try out the 1.19-beta-1 already, if you want. 

            Show
            pgarbe Philipp Garbe added a comment - Yes, it's the same fix, that's why I closed this one. You can try out the 1.19-beta-1 already, if you want. 
            Hide
            mignaulo Olivier Mignault added a comment -

            Thank you Philipp Garbe, I can confirm 1.19 fixed our issues.

            Show
            mignaulo Olivier Mignault added a comment - Thank you Philipp Garbe , I can confirm 1.19 fixed our issues.

              People

              Assignee:
              pgarbe Philipp Garbe
              Reporter:
              mignaulo Olivier Mignault
              Votes:
              3 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: