-
Bug
-
Resolution: Unresolved
-
Major
-
None
Jenkins Master: 2.440.3
kubernetes-plugin: 4246.v5a_12b_1fe120e
Kubernetes Client API: 6.10.0-240.v57880ce8b_0b_2
RKE2 version: v1.25.4+rke2r1
Jnlp container: `jenkins/inbound-agent:latest-jdk17`
Current setup
- Jenkins Master deployed in rke2 cluster in statefulset mode
- jnlp agents are running in the same cluster as the controller , same namespace
- RKE2 based on ubuntu 22.04
- We're using nginx ingress controller
- All infra runs on vmware
- This is the list of plugins:
Jenkins: 2.440.3 OS: Linux - 5.15.0-105-generic Java: 17.0.10 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- PrioritySorter:5.0.0 ace-editor:1.1 active-directory:2.31 analysis-model-api:11.6.0 ansicolor:1.0.3 ant:497.v94e7d9fffa_b_9 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 apache-httpcomponents-client-5-api:5.2.1-1.0 artifactory:3.18.9 asm-api:9.6-3.v2e1fa_b_338cd7 async-http-client:1.9.40.0 authentication-tokens:1.53.v1c90fd9191a_b_ authorize-project:1.7.1 aws-java-sdk:1.12.529-406.vdeff15e5817d aws-java-sdk-cloudformation:1.12.529-406.vdeff15e5817d aws-java-sdk-codebuild:1.12.529-406.vdeff15e5817d aws-java-sdk-ec2:1.12.529-406.vdeff15e5817d aws-java-sdk-ecr:1.12.529-406.vdeff15e5817d aws-java-sdk-ecs:1.12.529-406.vdeff15e5817d aws-java-sdk-efs:1.12.529-406.vdeff15e5817d aws-java-sdk-elasticbeanstalk:1.12.529-406.vdeff15e5817d aws-java-sdk-iam:1.12.529-406.vdeff15e5817d aws-java-sdk-kinesis:1.12.529-406.vdeff15e5817d aws-java-sdk-logs:1.12.529-406.vdeff15e5817d aws-java-sdk-minimal:1.12.529-406.vdeff15e5817d aws-java-sdk-secretsmanager:1.12.529-406.vdeff15e5817d aws-java-sdk-sns:1.12.529-406.vdeff15e5817d aws-java-sdk-sqs:1.12.529-406.vdeff15e5817d aws-java-sdk-ssm:1.12.529-406.vdeff15e5817d badge:1.9.1 basic-branch-build-strategies:81.v05e333931c7d bitbucket:223.vd12f2bca5430 blueocean:1.27.6 blueocean-autofavorite:1.2.5 blueocean-bitbucket-pipeline:1.27.6 blueocean-commons:1.27.6 blueocean-config:1.27.6 blueocean-core-js:1.27.6 blueocean-dashboard:1.27.6 blueocean-display-url:2.4.2 blueocean-events:1.27.6 blueocean-executor-info:1.27.6 blueocean-git-pipeline:1.27.6 blueocean-github-pipeline:1.27.6 blueocean-i18n:1.27.6 blueocean-jira:1.27.6 blueocean-jwt:1.27.6 blueocean-personalization:1.27.6 blueocean-pipeline-api-impl:1.27.6 blueocean-pipeline-editor:1.27.6 blueocean-pipeline-scm-api:1.27.6 blueocean-rest:1.27.6 blueocean-rest-impl:1.27.6 blueocean-web:1.27.6 bootstrap4-api:4.6.0-6 bootstrap5-api:5.3.2-2 bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9 branch-api:2.1122.v09cb_8ea_8a_724 build-failure-analyzer:2.4.1 build-pipeline-plugin:1.5.8 build-timeout:1.31 build-user-vars-plugin:1.9 built-on-column:1.4 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.2 claim:516.v36293563731d cloud-stats:316.vd6d6b_292238d cloudbees-bitbucket-branch-source:825.va_6a_dc46a_f97d cloudbees-disk-usage-simple:182.v62ca_0c992a_f3 cloudbees-folder:6.858.v898218f3609d cobertura:1.17 code-coverage-api:4.7.0 command-launcher:107.v773860566e2e commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.10.0-78.v3e7b_ea_d5a_fe1 conditional-buildstep:1.4.3 config-file-provider:953.v0432a_802e4d2 configuration-as-code:1670.v564dc8b_982d0 configurationslicing:548.ve92d48e66b_f8 copyartifact:714.v28a_34f8c563f credentials:1319.v7eb_51b_3a_c97b_ credentials-binding:657.v2b_19db_7d6e6d dashboard-view:2.508.va_74654f026d1 data-tables-api:1.13.5-1 dependency-track:4.3.1 display-url-api:2.200.vb_9327d658781 docker-commons:439.va_3cb_0a_6a_fb_29 docker-java-api:3.3.1-79.v20b_53427e041 docker-plugin:1.4 docker-workflow:572.v950f58993843 downstream-build-cache:1.7 dtkit-api:3.0.2 durable-task:550.v0930093c4b_a_6 echarts-api:5.4.0-7 email-ext:2.100 envinject:2.908.v66a_774b_31d93 envinject-api:1.199.v3ce31253ed13 export-params:1.9 external-monitor-job:207.v98a_a_37a_85525 favorite:2.4.3 file-operations:214.v2e7dc7f25757 flexible-publish:0.16.1 font-awesome-api:6.4.2-1 forensics-api:2.3.0 fortify:22.2.39 gerrit-trigger:2.39.1 git:5.2.0 git-client:4.4.0 git-server:99.va_0826a_b_cdfa_d github:1.37.3 github-api:1.314-431.v78d72a_3fe4c3 github-branch-source:1732.v3f1889a_c475b_ github-organization-folder:1.6 gitlab-plugin:1.7.15 google-oauth-plugin:1.0.9 gradle:2.8.2 groovy:453.vcdb_a_c5c99890 handlebars:3.0.8 handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 hashicorp-vault-plugin:368.v48134f694db_f htmlpublisher:1.32 icon-shim:3.0.0 instance-identity:173.va_37c494ec4e5 ionicons-api:56.v1b_1c8c49374e ivy:2.5 jackson2-api:2.16.1-373.ve709c6871598 jacoco:3.3.6 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javadoc:243.vb_b_503b_b_45537 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.9-1 jdk-tool:73.vddf737284550 jenkins-design-language:1.27.6 jenkins-multijob-plugin:623.v03401733c9a_9 jersey2-api:2.40-1 jira:3.10 jjwt-api:0.11.5-77.v646c772fddb_0 job-dsl:1.87 job-restrictions:0.8 jobConfigHistory:1227.v7a_79fc4dc01f joda-time-api:2.12.5-5.v5495a_235fedf jquery:1.12.4-1 jquery-ui:1.0.2 jquery3-api:3.7.1-1 jsch:0.2.8-65.v052c39de79b_2 junit:1252.vfc2e5efa_294f jython:1.9 kubernetes:4246.v5a_12b_1fe120e kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2 kubernetes-credentials:0.11 ldap:694.vc02a_69c9787f locale:314.v22ce953dfe9e lockable-resources:1185.v0c528656ce04 logfilesizechecker:1.5 logstash:2.5.0205.vd05825ed46bd mailer:463.vedf8358e006b_ managed-scripts:1.5.6 mapdb-api:1.0.9-28.vf251ce40855d mask-passwords:150.vf80d33113e80 matrix-auth:3.1.10 matrix-project:822.v01b_8c85d16d2 maven-plugin:3.23 mercurial:1260.vdfb_723cdcc81 metrics:4.2.21-449.v6960d7c54c69 mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_ mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_ momentjs:1.1.1 mq-notifier:1.3.1 mstest:1.0.1 multiple-scms:0.8 muuri-api:0.9.5-3 naginator:1.19.2 nant:248.vcc8a_3eec8db_a node-iterator-api:49.v58a_8b_35f8363 nodejs:1.6.1 oauth-credentials:0.645.ve666a_c332668 okhttp-api:4.11.0-157.v6852a_a_fa_ec11 p4:1.14.2 pam-auth:1.10 parameterized-trigger:2.46 pipeline-aggregator-view:1.15 pipeline-build-step:505.v5f0844d8d126 pipeline-github-lib:42.v0739460cda_c4 pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-groovy-lib:671.v07c339c842e8 pipeline-input-step:477.v339683a_8d55e pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2175.v76a_fff0a_2618 pipeline-model-declarative-agent:1.1.1 pipeline-model-definition:2.2144.v077a_d1928a_40 pipeline-model-extensions:2.2175.v76a_fff0a_2618 pipeline-rest-api:2.33 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40 pipeline-stage-view:2.33 pipeline-utility-steps:2.16.0 plain-credentials:143.v1b_df8b_d3b_e48 plugin-usage-plugin:4.1 plugin-util-api:3.6.0 popper-api:1.16.1-3 popper2-api:2.11.6-2 powershell:2.0 prism-api:1.29.0-7 project-inheritance:21.04.03 prometheus:2.2.3 promoted-builds:892.vd6219fc0a_efb pubsub-light:1.17 pull-request-monitoring:1.7.8 python:1.3 rabbitmq-consumer:2.8 rebuild:320.v5a_0933a_e7d61 resource-disposer:0.23 robot:3.4.0 rocketchatnotifier:1.5.2 role-strategy:680.v3a_6a_1698b_864 run-condition:1.6 scm-api:676.v886669a_199a_a_ script-security:1313.v7a_6067dc7087 select2-api:4.0.13-8 simple-theme-plugin:160.vb_76454b_67900 slack:684.v833089650554 snakeyaml-api:2.2-111.vc6598e30cc65 sonar:2.15 sse-gateway:1.26 ssh-agent:333.v878b_53c89511 ssh-credentials:308.ve4497b_ccd8f4 ssh-slaves:2.916.vd17b_43357ce4 sshd:3.312.v1c601b_c83b_0e structs:337.v1b_04ea_4df7c8 subversion:2.17.3 support-core:1371.v709cef748b_d0 theme-manager:211.vef2a_42c645a_b_ thinBackup:1.18 timestamper:1.26 token-macro:384.vf35b_f26814ec trilead-api:2.84.v72119de229b_7 variant:60.v7290fc0eb_b_cd vsphere-cloud:2.27 warnings-ng:10.4.0 windows-slaves:1.8.1 workflow-aggregator:596.v8c21c963d92d workflow-api:1291.v51fd2a_625da_7 workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3880.vb_ef4b_5cfd270 workflow-cps-global-lib:609.vd95673f149b_b workflow-durable-task-step:1331.vc8c2fed35334 workflow-job:1385.vb_58b_86ea_fff1 workflow-multibranch:756.v891d88f2cd46 workflow-scm-step:415.v434365564324 workflow-step-api:657.v03b_e8115821b_ workflow-support:865.v43e78cc44e0d ws-cleanup:0.45 xunit:3.1.3
Problem Statement
Jenkins agent pod containing 9 to 11 containers occasionally run into the following error :
io.fabric8.kubernetes.client.KubernetesClientException: Received 401 on websocket. Failure executing: GET at: https://kubernetes.default/api/v1/namespaces/jenkins/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dxxxxxxxxxxxxxxxxxx-feature-2fdevops-develop-demo-8-dbt8g--9p398&resourceVersion=70196475&timeoutSeconds=600&watch=true. Message: Unauthorized.
=>the issue occured after an upgrade from 2.375.3 LTS to 2.440.3 LTS