Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47947

CGROUP_MATCHER_PATTERN does not work with Kubernetes >= 1.6

    XMLWordPrintable

Details

    Description

      Since version 1.6 kubernetes is using a custom cgroup format:

      Enhance Kubelet QoS:
      Pods are placed under a new cgroup hierarchy by default. This feature requires draining and restarting the node as part of upgrades. To opt-out set --cgroups-per-qos=false.

      The new cgroup looks something like "x:cpu:/kubepod/qos-level/container_id", here is an example from my kubernetes cluster:

      cat /proc/9/cgroup
11:hugetlb:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
10:perf_event:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
9:net_cls,net_prio:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
8:cpuset:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
7:devices:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
6:pids:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
5:memory:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
4:cpu,cpuacct:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
3:freezer:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
2:blkio:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
1:name=systemd:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59

      The matcher does not recognize this as a cgroup of running in a container and the docker plugin behaves as if Jenkins is not running in a container, i.e. volume mounts fail.

      It would be awesome if the matcher could be improved to recognize this so we do not have to run custom options on the kubelet just for Jenkins.

       

       

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-48502 image.inside stopped working on ECS agents

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            wduda Wojciech Duda created issue -
            wduda Wojciech Duda added a comment -

            I have found a PR for this: https://github.com/jenkinsci/docker-workflow-plugin/pull/120

            So having this merged would actually fix that ticket.

            wduda Wojciech Duda added a comment - I have found a PR for this: https://github.com/jenkinsci/docker-workflow-plugin/pull/120 So having this merged would actually fix that ticket.
            wduda Wojciech Duda added a comment -

            Actually it turns out it would not necessarily, as my example shows another cgroup format used by kubernetes.

            wduda Wojciech Duda added a comment - Actually it turns out it would not necessarily, as my example shows another cgroup format used by kubernetes.
            wduda Wojciech Duda added a comment -

            After some changes the PR mentioned above should cover this usecase as well.

            wduda Wojciech Duda added a comment - After some changes the PR mentioned above should cover this usecase as well.
            odavid Ohad David added a comment - - edited

            Hi,

            I've opened same case, but this time for ecs  https://issues.jenkins-ci.org/browse/JENKINS-48502.

            Is it possible to add ecs to the regex in the PR, to reduce the time of creating a new PR?

            Regards,

            Ohad

            odavid Ohad David added a comment - - edited Hi, I've opened same case, but this time for ecs  https://issues.jenkins-ci.org/browse/JENKINS-48502 . Is it possible to add ecs to the regex in the PR, to reduce the time of creating a new PR? Regards, Ohad
            odavid Ohad David made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-48502 [ JENKINS-48502 ]
            jglick Jesse Glick made changes -
            Labels Docker kuberentes kubernetes

            People

              Unassigned Unassigned
              wduda Wojciech Duda
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: