Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48502

image.inside stopped working on ECS agents

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Hi,

      We are running Jenkins with ECS cloud, meaning the agents are docker based agents.

      Some of our pipelines are using image.inside to run arbitrary build tools which are not part of the slave images.

      Before upgrading to amazon-ecs-agent-1.16.0 we got the following message in the log files:

       

      ecs-cloud-33394f9575f0e2 seems to be running inside container 01b572fc3c671c11087fcb8166d686792d2b725e570af941b37ff3f28986762f 
      

      After the upgrade, we get:

      ecs-cloud-dc46d9e888c4 does not seem to be running inside a container

      Looking at The DockerClient code I see that the way the plugin understands it is running within docker container, is by matching docker cgroups pattern with the default hierarchy name docker.

      The problem is that AWS ECS changed their default hierarchy to be ecs

      root@20f3571647fc:/usr/local/tomcat# cat /proc/self/cgroup
       9:perf_event:/ecs/0410eff2-7e59-4111-823e-1e0d98ef7f30/20f3571647fc2a6c718909b85f2bfd79ffae4f6afe8c8121c8b8415f0619f013
       8:memory:/ecs/0410eff2-7e59-4111-823e-1e0d98ef7f30/20f3571647fc2a6c718909b85f2bfd79ffae4f6afe8c8121c8b8415f0619f013
       7:hugetlb:/ecs/0410eff2-7e59-4111-823e-1e0d98ef7f30/20f3571647fc2a6c718909b85f2bfd79ffae4f6afe8c8121c8b8415f0619f013
      ...

      Because of that, image.inside step stopped working and we get the following error each build.

      sh: can't create /var/jenkins_home/workspace/...@tmp/durable-9cddf800/jenkins-log.txt: nonexistent directory

       

      I believe a quick fix would be to change the regex to match ecs as well, or add a cgroups hierarchy configuration parameter to the plugin.

      Best regards,

      Ohad

        Attachments

          Issue Links

            Activity

            odavid Ohad David created issue -
            odavid Ohad David made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-47947 [ JENKINS-47947 ]
            ndeloof Nicolas De Loof made changes -
            Assignee Nicolas De Loof [ ndeloof ]
            ndeloof Nicolas De Loof made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Closed [ 6 ]

              People

              Assignee:
              ndeloof Nicolas De Loof
              Reporter:
              odavid Ohad David
              Votes:
              2 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: