[JENKINS-70287] Kubernetes plugin doesn't respect privileged=false in jnlp container if it is defined differently in multiple Pod Templates and inheriting from a parent with privileged=true

Type: Bug
Resolution: Unresolved
Priority: Major
Component/s: kubernetes-plugin
Labels:
- jenkins
- kubernetes
Environment:
Jenkins 2.361.1
Kubernetes 3718.ve44878b_12184

Similar Issues:
Powered by SuggestiMate

Show

on the same Kubernetes cloud I've multiple Pod Templates

a Template "jnlp", with 1 Container Template "jnlp" having Run in privileged mode = true.
a Template "jnlp-child" inheriting from pod template "jnlp", with 2 Containers Templates: 1 having Run in privileged mode = true and 1 having Run in privileged mode = false

I run a pipeline on label of "jnlp-child"

I check the containers in K8s: both have privileged mode = true

It seems the root cause is that the templates are inheriting the privileged mode from the parent, and overriding the flag on the child

IMHO, a Boolean flag shall not be inherited from parent, since when unchecking it in the child cannot override it if it is checked in the parent

N.B.: if I set privileged mode = false in the parent, and run the same scenario, I get 1 child container with privileged mode = true and 1 with privileged mode = false - as expected

links to

Fix Kubernetes plugin doesn't respect privileged=false in jnlp container if it is defined differently in multiple Pod Templates and inheriting from a parent with privileged=true

Michael created issue - 2022-12-15 11:42

Michael made changes - 2022-12-15 11:43

Summary

Original: Kubernetes plugin doesn't respect privileged=true in jnlp container if it is defined differently in multiple Pod Templates

New: Kubernetes plugin doesn't respect privileged=false in jnlp container if it is defined differently in multiple Pod Templates

Vincent Latombe added a comment - 2023-01-28 10:23

There is not enough information here in order to reproduce the problem.
Please an explicit step-by-step reproducer.

Vincent Latombe added a comment - 2023-01-28 10:23 There is not enough information here in order to reproduce the problem. Please an explicit step-by-step reproducer.

Vincent Latombe made changes - 2023-01-28 10:23

Resolution		New: Not A Defect [ 7 ]
Status	Original: Open [ 1 ]	New: Closed [ 6 ]

Michael made changes - 2023-01-28 18:21

Assignee		New: Michael [ mifitous ]
Resolution	Original: Not A Defect [ 7 ]
Status	Original: Closed [ 6 ]	New: Reopened [ 4 ]

Michael made changes - 2023-02-13 18:23

Description

New: on the same Kubernetes cloud I've multiple Pod Templates
* some have privileged = true
* some have privileged = false

I run the required template using its label

if I select the template having privileged = false, it runs with privileged = true

the only workaround was to set for ALL Pod Templates privileged = false

*example:*

I run this to get the cloud templates having one container with privileged=false:

{{result = true}}
{{if (Jenkins.instance.clouds) {}}
{{ Jenkins.instance.clouds.each { cloud -> }}
{{ cloud.templates.each { templ ->}}
{{ if (templ.containers.find \{it.privileged == false}) {}}
{{ println "name: ${templ.label}"}}
{{ templ.containers.each { cont ->}}
{{ println "name: ${templ.label} - ${cont.image} - ${cont.privileged}"}}
{{ }}}
{{ }}}
{{ }}}
{{ }}}
{{}}}
{{return result}}

result:

{{name: de-mx-sonar-isolated}}
{{name: de-mx-sonar-isolated - artifactory.sddc.mobileye.com/data-eng-docker-local/de-jenkins-slave-dind:latest - true}}
{{name: de-mx-sonar-isolated - artifactory.sddc.mobileye.com/data-eng-docker-local/de-jnlp-mx-sonar:latest - false}}
{{Result: true}}

but when I run with the label 'de-mx-sonar-isolated' I get
apiVersion: "v1"
kind: "Pod"
metadata:
  labels:
    jenkins: "slave"
    jenkins/label-digest: "b1144c48a76fa6681d5957195ca5e6d11e3223a7"
    jenkins/label: "de-mx-sonar-isolated"
  name: "jnlp-dind-sonar-mf0m1"
  namespace: "de-ci"
spec:
  containers:
  - env:
    - name: "DOCKER_HOST"
      value: "tcp://localhost:2375"
    - name: "DOCKER_TLS_CERTDIR"
      value: ""
    - name: "HOME"
      value: "/homes/sys_av_data_int"
    image: "artifactory.sddc.mobileye.com/data-eng-docker-local/de-jenkins-slave-dind:latest"
    imagePullPolicy: "Always"
    name: "jnlp-dind"
    resources:
      limits:
        memory: "8Gi"
      requests:
        memory: "1.5Gi"
        cpu: "1"
    securityContext:
      privileged: true
    tty: true
    volumeMounts:
    - mountPath: "/homes/sys_av_data_int"
      name: "volume-0"
      readOnly: false
    - mountPath: "/mobileye:shared"
      name: "volume-3"
      readOnly: false
    - mountPath: "/etc/resolv.conf"
      name: "volume-1"
      readOnly: false
    - mountPath: "/homes/swlab"
      name: "volume-2"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
    workingDir: "/home/jenkins/agent"
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_AGENT_NAME"
      value: "jnlp-dind-sonar-mf0m1"
    - name: "DOCKER_HOST"
      value: "tcp://localhost:2375"
    - name: "JENKINS_NAME"
      value: "jnlp-dind-sonar-mf0m1"
    - name: "JENKINS_AGENT_WORKDIR"
      value: "/home/jenkins/agent"
    - name: "JENKINS_URL"
      value: "[http://10.155.17.249/]"
    - name: "HOME"
      value: "/homes/sys_av_data_int"
    image: "artifactory.sddc.mobileye.com/data-eng-docker-local/de-jnlp-mx-sonar:latest"
    imagePullPolicy: "Always"
    name: "jnlp"
    resources:
      limits:
        memory: "8Gi"
        cpu: "2.5"
      requests:
        memory: "1.5Gi"
        cpu: "2"
    securityContext:
      privileged: true
      runAsGroup: 1000
      runAsUser: 600002
    tty: true
    volumeMounts:
    - mountPath: "/homes/sys_av_data_int"
      name: "volume-0"
      readOnly: false
    - mountPath: "/mobileye:shared"
      name: "volume-3"
      readOnly: false
    - mountPath: "/etc/resolv.conf"
      name: "volume-1"
      readOnly: false
    - mountPath: "/homes/swlab"
      name: "volume-2"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
    workingDir: "/home/jenkins/agent"
  hostNetwork: false
  nodeSelector:
    kubernetes.io/os: "linux"
  restartPolicy: "Never"
  securityContext:
    supplementalGroups:
    - 1053
    - 1065
    - 1298
    - 16472
    - 17545
    - 1055
    - 1186
    - 1020
    - 17536
    - 1187
    - 1066
    - 1057
    - 48008
    - 1016
  volumes:
  - hostPath:
      path: "/homes/sys_av_data_int"
    name: "volume-0"
  - hostPath:
      path: "/mobileye/sw"
    name: "volume-2"
  - hostPath:
      path: "/etc/resolv.conf"
    name: "volume-1"
  - emptyDir:
      medium: ""
    name: "workspace-volume"
  - hostPath:
      path: "/mobileye"
    name: "volume-3"

Michael made changes - 2023-02-13 18:25

Description

Original: on the same Kubernetes cloud I've multiple Pod Templates
* some have privileged = true
* some have privileged = false

I run the required template using its label

if I select the template having privileged = false, it runs with privileged = true

the only workaround was to set for ALL Pod Templates privileged = false

*example:*

I run this to get the cloud templates having one container with privileged=false:

{{result = true}}
{{if (Jenkins.instance.clouds) {}}
{{ Jenkins.instance.clouds.each { cloud -> }}
{{ cloud.templates.each { templ ->}}
{{ if (templ.containers.find \{it.privileged == false}) {}}
{{ println "name: ${templ.label}"}}
{{ templ.containers.each { cont ->}}
{{ println "name: ${templ.label} - ${cont.image} - ${cont.privileged}"}}
{{ }}}
{{ }}}
{{ }}}
{{ }}}
{{}}}
{{return result}}

result:

{{name: de-mx-sonar-isolated}}
{{name: de-mx-sonar-isolated - artifactory.sddc.mobileye.com/data-eng-docker-local/de-jenkins-slave-dind:latest - true}}
{{name: de-mx-sonar-isolated - artifactory.sddc.mobileye.com/data-eng-docker-local/de-jnlp-mx-sonar:latest - false}}
{{Result: true}}

but when I run with the label 'de-mx-sonar-isolated' I get
apiVersion: "v1"
kind: "Pod"
metadata:
  labels:
    jenkins: "slave"
    jenkins/label-digest: "b1144c48a76fa6681d5957195ca5e6d11e3223a7"
    jenkins/label: "de-mx-sonar-isolated"
  name: "jnlp-dind-sonar-mf0m1"
  namespace: "de-ci"
spec:
  containers:
  - env:
    - name: "DOCKER_HOST"
      value: "tcp://localhost:2375"
    - name: "DOCKER_TLS_CERTDIR"
      value: ""
    - name: "HOME"
      value: "/homes/sys_av_data_int"
    image: "artifactory.sddc.mobileye.com/data-eng-docker-local/de-jenkins-slave-dind:latest"
    imagePullPolicy: "Always"
    name: "jnlp-dind"
    resources:
      limits:
        memory: "8Gi"
      requests:
        memory: "1.5Gi"
        cpu: "1"
    securityContext:
      privileged: true
    tty: true
    volumeMounts:
    - mountPath: "/homes/sys_av_data_int"
      name: "volume-0"
      readOnly: false
    - mountPath: "/mobileye:shared"
      name: "volume-3"
      readOnly: false
    - mountPath: "/etc/resolv.conf"
      name: "volume-1"
      readOnly: false
    - mountPath: "/homes/swlab"
      name: "volume-2"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
    workingDir: "/home/jenkins/agent"
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_AGENT_NAME"
      value: "jnlp-dind-sonar-mf0m1"
    - name: "DOCKER_HOST"
      value: "tcp://localhost:2375"
    - name: "JENKINS_NAME"
      value: "jnlp-dind-sonar-mf0m1"
    - name: "JENKINS_AGENT_WORKDIR"
      value: "/home/jenkins/agent"
    - name: "JENKINS_URL"
      value: "[http://10.155.17.249/]"
    - name: "HOME"
      value: "/homes/sys_av_data_int"
    image: "artifactory.sddc.mobileye.com/data-eng-docker-local/de-jnlp-mx-sonar:latest"
    imagePullPolicy: "Always"
    name: "jnlp"
    resources:
      limits:
        memory: "8Gi"
        cpu: "2.5"
      requests:
        memory: "1.5Gi"
        cpu: "2"
    securityContext:
      privileged: true
      runAsGroup: 1000
      runAsUser: 600002
    tty: true
    volumeMounts:
    - mountPath: "/homes/sys_av_data_int"
      name: "volume-0"
      readOnly: false
    - mountPath: "/mobileye:shared"
      name: "volume-3"
      readOnly: false
    - mountPath: "/etc/resolv.conf"
      name: "volume-1"
      readOnly: false
    - mountPath: "/homes/swlab"
      name: "volume-2"
      readOnly: false
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
    workingDir: "/home/jenkins/agent"
  hostNetwork: false
  nodeSelector:
    kubernetes.io/os: "linux"
  restartPolicy: "Never"
  securityContext:
    supplementalGroups:
    - 1053
    - 1065
    - 1298
    - 16472
    - 17545
    - 1055
    - 1186
    - 1020
    - 17536
    - 1187
    - 1066
    - 1057
    - 48008
    - 1016
  volumes:
  - hostPath:
      path: "/homes/sys_av_data_int"
    name: "volume-0"
  - hostPath:
      path: "/mobileye/sw"
    name: "volume-2"
  - hostPath:
      path: "/etc/resolv.conf"
    name: "volume-1"
  - emptyDir:
      medium: ""
    name: "workspace-volume"
  - hostPath:
      path: "/mobileye"
    name: "volume-3"

New: on the same Kubernetes cloud I've multiple Pod Templates
* some have privileged = true
* some have privileged = false

I run the required template using its label

if I select the template having privileged = false, it runs with privileged = true

the only workaround was to set for ALL Pod Templates privileged = false

*example:*

I run this to get the cloud templates having one container with privileged=false:

{{result = true}}
{{if (Jenkins.instance.clouds) {}}
{{ Jenkins.instance.clouds.each { cloud -> }}
{{ cloud.templates.each { templ ->}}
{{ if (templ.containers.find \{it.privileged == false}) {}}
{{ println "name: ${templ.label}"}}
{{ templ.containers.each { cont ->}}
{{ println "name: ${templ.label} - ${cont.image} - ${cont.privileged}"}}
{\{ }}}
{\{ }}}
{\{ }}}
{\{ }}}
{{}}}
{{return result}}

result:

{{name: de-mx-sonar-isolated}}
{{name: de-mx-sonar-isolated - .../data-eng-docker-local/de-jenkins-slave-dind:latest - +true+}}
{{name: de-mx-sonar-isolated - .../data-eng-docker-local/de-jnlp-mx-sonar:latest - +false+}}
{{Result: true}}

but when I run with the label 'de-mx-sonar-isolated' I get
apiVersion: "v1"
kind: "Pod"
metadata:
labels:
jenkins: "slave"
jenkins/label-digest: "b1144c48a76fa6681d5957195ca5e6d11e3223a7"
jenkins/label: "de-mx-sonar-isolated"
name: "jnlp-dind-sonar-mf0m1"
namespace: "de-ci"
spec:
containers:
- env:
- name: "DOCKER_HOST"
value: "tcp://localhost:2375"
- name: "DOCKER_TLS_CERTDIR"
value: ""
- name: "HOME"
value: "/homes/sys_av_data_int"
image: ".../data-eng-docker-local/de-jenkins-slave-dind:latest"
imagePullPolicy: "Always"
name: "jnlp-dind"
resources:
limits:
memory: "8Gi"
requests:
memory: "1.5Gi"
cpu: "1"
securityContext:
+privileged: true+
tty: true
volumeMounts:
- mountPath: "/homes/sys_av_data_int"
name: "volume-0"
readOnly: false
- mountPath: "/...:shared"
name: "volume-3"
readOnly: false
- mountPath: "/etc/resolv.conf"
name: "volume-1"
readOnly: false
- mountPath: "/homes/swlab"
name: "volume-2"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
workingDir: "/home/jenkins/agent"
- env:
- name: "JENKINS_SECRET"
value: "********"
- name: "JENKINS_AGENT_NAME"
value: "jnlp-dind-sonar-mf0m1"
- name: "DOCKER_HOST"
value: "tcp://localhost:2375"
- name: "JENKINS_NAME"
value: "jnlp-dind-sonar-mf0m1"
- name: "JENKINS_AGENT_WORKDIR"
value: "/home/jenkins/agent"
- name: "JENKINS_URL"
value: "[http://10.155.17.249/]"
- name: "HOME"
value: "/homes/sys_av_data_int"
image: ".../data-eng-docker-local/de-jnlp-mx-sonar:latest"
imagePullPolicy: "Always"
name: "jnlp"
resources:
limits:
memory: "8Gi"
cpu: "2.5"
requests:
memory: "1.5Gi"
cpu: "2"
securityContext:
+privileged: true+
runAsGroup: 1000
runAsUser: 600002
tty: true
volumeMounts:
- mountPath: "/homes/sys_av_data_int"
name: "volume-0"
readOnly: false
- mountPath: "/...:shared"
name: "volume-3"
readOnly: false
- mountPath: "/etc/resolv.conf"
name: "volume-1"
readOnly: false
- mountPath: "/homes/swlab"
name: "volume-2"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
workingDir: "/home/jenkins/agent"
hostNetwork: false
nodeSelector:
kubernetes.io/os: "linux"
restartPolicy: "Never"
securityContext:
supplementalGroups:
- 1053
- 1065
- 1298
- 16472
- 17545
- 1055
- 1186
- 1020
- 17536
- 1187
- 1066
- 1057
- 48008
- 1016
volumes:
- hostPath:
path: "/homes/sys_av_data_int"
name: "volume-0"
- hostPath:
path: "/.../sw"
name: "volume-2"
- hostPath:
path: "/etc/resolv.conf"
name: "volume-1"
- emptyDir:
medium: ""
name: "workspace-volume"
- hostPath:
path: "/..."
name: "volume-3"

Michael made changes - 2023-02-13 20:56

Remote Link

New: This issue links to "Fix Kubernetes plugin doesn't respect privileged=false in jnlp container if it is defined differently in multiple Pod Templates #1317 (Web Link)" [ 28480 ]

Michael made changes - 2023-04-20 09:12

Description

Original: on the same Kubernetes cloud I've multiple Pod Templates
* some have privileged = true
* some have privileged = false

I run the required template using its label

if I select the template having privileged = false, it runs with privileged = true

the only workaround was to set for ALL Pod Templates privileged = false

*example:*

I run this to get the cloud templates having one container with privileged=false:

{{result = true}}
{{if (Jenkins.instance.clouds) {}}
{{ Jenkins.instance.clouds.each { cloud -> }}
{{ cloud.templates.each { templ ->}}
{{ if (templ.containers.find \{it.privileged == false}) {}}
{{ println "name: ${templ.label}"}}
{{ templ.containers.each { cont ->}}
{{ println "name: ${templ.label} - ${cont.image} - ${cont.privileged}"}}
{\{ }}}
{\{ }}}
{\{ }}}
{\{ }}}
{{}}}
{{return result}}

result:

{{name: de-mx-sonar-isolated}}
{{name: de-mx-sonar-isolated - .../data-eng-docker-local/de-jenkins-slave-dind:latest - +true+}}
{{name: de-mx-sonar-isolated - .../data-eng-docker-local/de-jnlp-mx-sonar:latest - +false+}}
{{Result: true}}

but when I run with the label 'de-mx-sonar-isolated' I get
apiVersion: "v1"
kind: "Pod"
metadata:
labels:
jenkins: "slave"
jenkins/label-digest: "b1144c48a76fa6681d5957195ca5e6d11e3223a7"
jenkins/label: "de-mx-sonar-isolated"
name: "jnlp-dind-sonar-mf0m1"
namespace: "de-ci"
spec:
containers:
- env:
- name: "DOCKER_HOST"
value: "tcp://localhost:2375"
- name: "DOCKER_TLS_CERTDIR"
value: ""
- name: "HOME"
value: "/homes/sys_av_data_int"
image: ".../data-eng-docker-local/de-jenkins-slave-dind:latest"
imagePullPolicy: "Always"
name: "jnlp-dind"
resources:
limits:
memory: "8Gi"
requests:
memory: "1.5Gi"
cpu: "1"
securityContext:
+privileged: true+
tty: true
volumeMounts:
- mountPath: "/homes/sys_av_data_int"
name: "volume-0"
readOnly: false
- mountPath: "/...:shared"
name: "volume-3"
readOnly: false
- mountPath: "/etc/resolv.conf"
name: "volume-1"
readOnly: false
- mountPath: "/homes/swlab"
name: "volume-2"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
workingDir: "/home/jenkins/agent"
- env:
- name: "JENKINS_SECRET"
value: "********"
- name: "JENKINS_AGENT_NAME"
value: "jnlp-dind-sonar-mf0m1"
- name: "DOCKER_HOST"
value: "tcp://localhost:2375"
- name: "JENKINS_NAME"
value: "jnlp-dind-sonar-mf0m1"
- name: "JENKINS_AGENT_WORKDIR"
value: "/home/jenkins/agent"
- name: "JENKINS_URL"
value: "[http://10.155.17.249/]"
- name: "HOME"
value: "/homes/sys_av_data_int"
image: ".../data-eng-docker-local/de-jnlp-mx-sonar:latest"
imagePullPolicy: "Always"
name: "jnlp"
resources:
limits:
memory: "8Gi"
cpu: "2.5"
requests:
memory: "1.5Gi"
cpu: "2"
securityContext:
+privileged: true+
runAsGroup: 1000
runAsUser: 600002
tty: true
volumeMounts:
- mountPath: "/homes/sys_av_data_int"
name: "volume-0"
readOnly: false
- mountPath: "/...:shared"
name: "volume-3"
readOnly: false
- mountPath: "/etc/resolv.conf"
name: "volume-1"
readOnly: false
- mountPath: "/homes/swlab"
name: "volume-2"
readOnly: false
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
workingDir: "/home/jenkins/agent"
hostNetwork: false
nodeSelector:
kubernetes.io/os: "linux"
restartPolicy: "Never"
securityContext:
supplementalGroups:
- 1053
- 1065
- 1298
- 16472
- 17545
- 1055
- 1186
- 1020
- 17536
- 1187
- 1066
- 1057
- 48008
- 1016
volumes:
- hostPath:
path: "/homes/sys_av_data_int"
name: "volume-0"
- hostPath:
path: "/.../sw"
name: "volume-2"
- hostPath:
path: "/etc/resolv.conf"
name: "volume-1"
- emptyDir:
medium: ""
name: "workspace-volume"
- hostPath:
path: "/..."
name: "volume-3"

New: on the same Kubernetes cloud I've multiple Pod Templates
* a Template "jnlp", with *1* Container Template "jnlp" having Run in privileged mode = {*}true{*}.
* a Template "jnlp-child" inheriting from pod template "jnlp", with *2* Containers Templates: 1 having Run in privileged mode = *true* and 1 having Run in privileged mode = *false*

I run a pipeline on label of "jnlp-child"

I check the containers in K8s: both have privileged mode = *true*

It seems the root cause is that the templates are inheriting the privileged mode from the parent, and overriding the flag on the child

IMHO, a Boolean flag shall not be inherited from parent, since when unchecking it in the child cannot override it if it is checked in the parent

N.B.: if I set privileged mode = *false* in the parent, and run the same scenario, I get 1 child container with privileged mode = *true* and 1 with privileged mode = *false* - +as expected+

Michael made changes - 2023-04-20 09:12

Summary

Original: Kubernetes plugin doesn't respect privileged=false in jnlp container if it is defined differently in multiple Pod Templates

New: Kubernetes plugin doesn't respect privileged=false in jnlp container if it is defined differently in multiple Pod Templates and inheriting from a parent with privileged=true

Assignee:: Michael

Reporter:: Michael

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Created:: 2022-12-15 11:42

Updated:: 2023-05-01 10:14

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Vincent Latombe added a comment - 2023-01-28 10:23

Expand comment: Vincent Latombe added a comment - 2023-01-28 10:23

People

Dates