Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47947

CGROUP_MATCHER_PATTERN does not work with Kubernetes >= 1.6

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • docker-workflow-plugin
    • Jenkins 2.73.3, docker-workflow-plugin 1.14, kubernetes 1.7.5

      Since version 1.6 kubernetes is using a custom cgroup format:

      Enhance Kubelet QoS:
      Pods are placed under a new cgroup hierarchy by default. This feature requires draining and restarting the node as part of upgrades. To opt-out set --cgroups-per-qos=false.

      The new cgroup looks something like "x:cpu:/kubepod/qos-level/container_id", here is an example from my kubernetes cluster:

      cat /proc/9/cgroup
      11:hugetlb:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      10:perf_event:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      9:net_cls,net_prio:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      8:cpuset:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      7:devices:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      6:pids:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      5:memory:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      4:cpu,cpuacct:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      3:freezer:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      2:blkio:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      1:name=systemd:/kubepods/besteffort/pod60070ae4-c63a-11e7-92b3-0adc1ac11520/931fc44a96d53e4acbcbb037e22b7a5f55da235ca9b336105294d4f109cd8c59
      

      The matcher does not recognize this as a cgroup of running in a container and the docker plugin behaves as if Jenkins is not running in a container, i.e. volume mounts fail.

      It would be awesome if the matcher could be improved to recognize this so we do not have to run custom options on the kubelet just for Jenkins.

       

       

          [JENKINS-47947] CGROUP_MATCHER_PATTERN does not work with Kubernetes >= 1.6

          Wojciech Duda added a comment -

          I have found a PR for this: https://github.com/jenkinsci/docker-workflow-plugin/pull/120

          So having this merged would actually fix that ticket.

          Wojciech Duda added a comment - I have found a PR for this: https://github.com/jenkinsci/docker-workflow-plugin/pull/120 So having this merged would actually fix that ticket.

          Wojciech Duda added a comment -

          Actually it turns out it would not necessarily, as my example shows another cgroup format used by kubernetes.

          Wojciech Duda added a comment - Actually it turns out it would not necessarily, as my example shows another cgroup format used by kubernetes.

          Wojciech Duda added a comment -

          After some changes the PR mentioned above should cover this usecase as well.

          Wojciech Duda added a comment - After some changes the PR mentioned above should cover this usecase as well.

          Ohad David added a comment - - edited

          Hi,

          I've opened same case, but this time for ecs  https://issues.jenkins-ci.org/browse/JENKINS-48502.

          Is it possible to add ecs to the regex in the PR, to reduce the time of creating a new PR?

          Regards,

          Ohad

          Ohad David added a comment - - edited Hi, I've opened same case, but this time for ecs  https://issues.jenkins-ci.org/browse/JENKINS-48502 . Is it possible to add ecs to the regex in the PR, to reduce the time of creating a new PR? Regards, Ohad

            Unassigned Unassigned
            wduda Wojciech Duda
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: