Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-73351

io.fabric8.kubernetes.client.KubernetesClientException: Received 401 on websocket . Message: Unauthorized.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • kubernetes-plugin
    • None

      Jenkins Master: 2.440.3

      kubernetes-plugin: 4246.v5a_12b_1fe120e

      Kubernetes Client API: 6.10.0-240.v57880ce8b_0b_2

      RKE2 version: v1.25.4+rke2r1    

      Jnlp container: `jenkins/inbound-agent:latest-jdk17`

       

      Current setup

      • Jenkins Master deployed in rke2 cluster  in statefulset mode 
      • jnlp agents are running in the same cluster as the controller , same namespace
      • RKE2 based on ubuntu 22.04 
      • We're using nginx ingress controller
      • All infra runs on vmware
      • This is the list of plugins:
      Jenkins: 2.440.3
      OS: Linux - 5.15.0-105-generic
      Java: 17.0.10 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
      ---
      PrioritySorter:5.0.0
      ace-editor:1.1
      active-directory:2.31
      analysis-model-api:11.6.0
      ansicolor:1.0.3
      ant:497.v94e7d9fffa_b_9
      antisamy-markup-formatter:162.v0e6ec0fcfcf6
      apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
      apache-httpcomponents-client-5-api:5.2.1-1.0
      artifactory:3.18.9
      asm-api:9.6-3.v2e1fa_b_338cd7
      async-http-client:1.9.40.0
      authentication-tokens:1.53.v1c90fd9191a_b_
      authorize-project:1.7.1
      aws-java-sdk:1.12.529-406.vdeff15e5817d
      aws-java-sdk-cloudformation:1.12.529-406.vdeff15e5817d
      aws-java-sdk-codebuild:1.12.529-406.vdeff15e5817d
      aws-java-sdk-ec2:1.12.529-406.vdeff15e5817d
      aws-java-sdk-ecr:1.12.529-406.vdeff15e5817d
      aws-java-sdk-ecs:1.12.529-406.vdeff15e5817d
      aws-java-sdk-efs:1.12.529-406.vdeff15e5817d
      aws-java-sdk-elasticbeanstalk:1.12.529-406.vdeff15e5817d
      aws-java-sdk-iam:1.12.529-406.vdeff15e5817d
      aws-java-sdk-kinesis:1.12.529-406.vdeff15e5817d
      aws-java-sdk-logs:1.12.529-406.vdeff15e5817d
      aws-java-sdk-minimal:1.12.529-406.vdeff15e5817d
      aws-java-sdk-secretsmanager:1.12.529-406.vdeff15e5817d
      aws-java-sdk-sns:1.12.529-406.vdeff15e5817d
      aws-java-sdk-sqs:1.12.529-406.vdeff15e5817d
      aws-java-sdk-ssm:1.12.529-406.vdeff15e5817d
      badge:1.9.1
      basic-branch-build-strategies:81.v05e333931c7d
      bitbucket:223.vd12f2bca5430
      blueocean:1.27.6
      blueocean-autofavorite:1.2.5
      blueocean-bitbucket-pipeline:1.27.6
      blueocean-commons:1.27.6
      blueocean-config:1.27.6
      blueocean-core-js:1.27.6
      blueocean-dashboard:1.27.6
      blueocean-display-url:2.4.2
      blueocean-events:1.27.6
      blueocean-executor-info:1.27.6
      blueocean-git-pipeline:1.27.6
      blueocean-github-pipeline:1.27.6
      blueocean-i18n:1.27.6
      blueocean-jira:1.27.6
      blueocean-jwt:1.27.6
      blueocean-personalization:1.27.6
      blueocean-pipeline-api-impl:1.27.6
      blueocean-pipeline-editor:1.27.6
      blueocean-pipeline-scm-api:1.27.6
      blueocean-rest:1.27.6
      blueocean-rest-impl:1.27.6
      blueocean-web:1.27.6
      bootstrap4-api:4.6.0-6
      bootstrap5-api:5.3.2-2
      bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
      branch-api:2.1122.v09cb_8ea_8a_724
      build-failure-analyzer:2.4.1
      build-pipeline-plugin:1.5.8
      build-timeout:1.31
      build-user-vars-plugin:1.9
      built-on-column:1.4
      caffeine-api:3.1.8-133.v17b_1ff2e0599
      checks-api:2.0.2
      claim:516.v36293563731d
      cloud-stats:316.vd6d6b_292238d
      cloudbees-bitbucket-branch-source:825.va_6a_dc46a_f97d
      cloudbees-disk-usage-simple:182.v62ca_0c992a_f3
      cloudbees-folder:6.858.v898218f3609d
      cobertura:1.17
      code-coverage-api:4.7.0
      command-launcher:107.v773860566e2e
      commons-lang3-api:3.13.0-62.v7d18e55f51e2
      commons-text-api:1.10.0-78.v3e7b_ea_d5a_fe1
      conditional-buildstep:1.4.3
      config-file-provider:953.v0432a_802e4d2
      configuration-as-code:1670.v564dc8b_982d0
      configurationslicing:548.ve92d48e66b_f8
      copyartifact:714.v28a_34f8c563f
      credentials:1319.v7eb_51b_3a_c97b_
      credentials-binding:657.v2b_19db_7d6e6d
      dashboard-view:2.508.va_74654f026d1
      data-tables-api:1.13.5-1
      dependency-track:4.3.1
      display-url-api:2.200.vb_9327d658781
      docker-commons:439.va_3cb_0a_6a_fb_29
      docker-java-api:3.3.1-79.v20b_53427e041
      docker-plugin:1.4
      docker-workflow:572.v950f58993843
      downstream-build-cache:1.7
      dtkit-api:3.0.2
      durable-task:550.v0930093c4b_a_6
      echarts-api:5.4.0-7
      email-ext:2.100
      envinject:2.908.v66a_774b_31d93
      envinject-api:1.199.v3ce31253ed13
      export-params:1.9
      external-monitor-job:207.v98a_a_37a_85525
      favorite:2.4.3
      file-operations:214.v2e7dc7f25757
      flexible-publish:0.16.1
      font-awesome-api:6.4.2-1
      forensics-api:2.3.0
      fortify:22.2.39
      gerrit-trigger:2.39.1
      git:5.2.0
      git-client:4.4.0
      git-server:99.va_0826a_b_cdfa_d
      github:1.37.3
      github-api:1.314-431.v78d72a_3fe4c3
      github-branch-source:1732.v3f1889a_c475b_
      github-organization-folder:1.6
      gitlab-plugin:1.7.15
      google-oauth-plugin:1.0.9
      gradle:2.8.2
      groovy:453.vcdb_a_c5c99890
      handlebars:3.0.8
      handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
      hashicorp-vault-plugin:368.v48134f694db_f
      htmlpublisher:1.32
      icon-shim:3.0.0
      instance-identity:173.va_37c494ec4e5
      ionicons-api:56.v1b_1c8c49374e
      ivy:2.5
      jackson2-api:2.16.1-373.ve709c6871598
      jacoco:3.3.6
      jakarta-activation-api:2.0.1-3
      jakarta-mail-api:2.0.1-3
      javadoc:243.vb_b_503b_b_45537
      javax-activation-api:1.2.0-6
      javax-mail-api:1.6.2-9
      jaxb:2.3.9-1
      jdk-tool:73.vddf737284550
      jenkins-design-language:1.27.6
      jenkins-multijob-plugin:623.v03401733c9a_9
      jersey2-api:2.40-1
      jira:3.10
      jjwt-api:0.11.5-77.v646c772fddb_0
      job-dsl:1.87
      job-restrictions:0.8
      jobConfigHistory:1227.v7a_79fc4dc01f
      joda-time-api:2.12.5-5.v5495a_235fedf
      jquery:1.12.4-1
      jquery-ui:1.0.2
      jquery3-api:3.7.1-1
      jsch:0.2.8-65.v052c39de79b_2
      junit:1252.vfc2e5efa_294f
      jython:1.9
      kubernetes:4246.v5a_12b_1fe120e
      kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2
      kubernetes-credentials:0.11
      ldap:694.vc02a_69c9787f
      locale:314.v22ce953dfe9e
      lockable-resources:1185.v0c528656ce04
      logfilesizechecker:1.5
      logstash:2.5.0205.vd05825ed46bd
      mailer:463.vedf8358e006b_
      managed-scripts:1.5.6
      mapdb-api:1.0.9-28.vf251ce40855d
      mask-passwords:150.vf80d33113e80
      matrix-auth:3.1.10
      matrix-project:822.v01b_8c85d16d2
      maven-plugin:3.23
      mercurial:1260.vdfb_723cdcc81
      metrics:4.2.21-449.v6960d7c54c69
      mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_
      mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_
      momentjs:1.1.1
      mq-notifier:1.3.1
      mstest:1.0.1
      multiple-scms:0.8
      muuri-api:0.9.5-3
      naginator:1.19.2
      nant:248.vcc8a_3eec8db_a
      node-iterator-api:49.v58a_8b_35f8363
      nodejs:1.6.1
      oauth-credentials:0.645.ve666a_c332668
      okhttp-api:4.11.0-157.v6852a_a_fa_ec11
      p4:1.14.2
      pam-auth:1.10
      parameterized-trigger:2.46
      pipeline-aggregator-view:1.15
      pipeline-build-step:505.v5f0844d8d126
      pipeline-github-lib:42.v0739460cda_c4
      pipeline-graph-analysis:202.va_d268e64deb_3
      pipeline-groovy-lib:671.v07c339c842e8
      pipeline-input-step:477.v339683a_8d55e
      pipeline-milestone-step:111.v449306f708b_7
      pipeline-model-api:2.2175.v76a_fff0a_2618
      pipeline-model-declarative-agent:1.1.1
      pipeline-model-definition:2.2144.v077a_d1928a_40
      pipeline-model-extensions:2.2175.v76a_fff0a_2618
      pipeline-rest-api:2.33
      pipeline-stage-step:305.ve96d0205c1c6
      pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40
      pipeline-stage-view:2.33
      pipeline-utility-steps:2.16.0
      plain-credentials:143.v1b_df8b_d3b_e48
      plugin-usage-plugin:4.1
      plugin-util-api:3.6.0
      popper-api:1.16.1-3
      popper2-api:2.11.6-2
      powershell:2.0
      prism-api:1.29.0-7
      project-inheritance:21.04.03
      prometheus:2.2.3
      promoted-builds:892.vd6219fc0a_efb
      pubsub-light:1.17
      pull-request-monitoring:1.7.8
      python:1.3
      rabbitmq-consumer:2.8
      rebuild:320.v5a_0933a_e7d61
      resource-disposer:0.23
      robot:3.4.0
      rocketchatnotifier:1.5.2
      role-strategy:680.v3a_6a_1698b_864
      run-condition:1.6
      scm-api:676.v886669a_199a_a_
      script-security:1313.v7a_6067dc7087
      select2-api:4.0.13-8
      simple-theme-plugin:160.vb_76454b_67900
      slack:684.v833089650554
      snakeyaml-api:2.2-111.vc6598e30cc65
      sonar:2.15
      sse-gateway:1.26
      ssh-agent:333.v878b_53c89511
      ssh-credentials:308.ve4497b_ccd8f4
      ssh-slaves:2.916.vd17b_43357ce4
      sshd:3.312.v1c601b_c83b_0e
      structs:337.v1b_04ea_4df7c8
      subversion:2.17.3
      support-core:1371.v709cef748b_d0
      theme-manager:211.vef2a_42c645a_b_
      thinBackup:1.18
      timestamper:1.26
      token-macro:384.vf35b_f26814ec
      trilead-api:2.84.v72119de229b_7
      variant:60.v7290fc0eb_b_cd
      vsphere-cloud:2.27
      warnings-ng:10.4.0
      windows-slaves:1.8.1
      workflow-aggregator:596.v8c21c963d92d
      workflow-api:1291.v51fd2a_625da_7
      workflow-basic-steps:1042.ve7b_140c4a_e0c
      workflow-cps:3880.vb_ef4b_5cfd270
      workflow-cps-global-lib:609.vd95673f149b_b
      workflow-durable-task-step:1331.vc8c2fed35334
      workflow-job:1385.vb_58b_86ea_fff1
      workflow-multibranch:756.v891d88f2cd46
      workflow-scm-step:415.v434365564324
      workflow-step-api:657.v03b_e8115821b_
      workflow-support:865.v43e78cc44e0d
      ws-cleanup:0.45
      xunit:3.1.3 

       

      Problem Statement

      Jenkins agent pod containing 9 to 11 containers occasionally run into the following error :

      io.fabric8.kubernetes.client.KubernetesClientException: Received 401 on websocket. Failure executing: GET at: https://kubernetes.default/api/v1/namespaces/jenkins/pods?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dxxxxxxxxxxxxxxxxxx-feature-2fdevops-develop-demo-8-dbt8g--9p398&resourceVersion=70196475&timeoutSeconds=600&watch=true. Message: Unauthorized. 

      =>the issue occured after an upgrade from 2.375.3 LTS to 2.440.3 LTS

            Unassigned Unassigned
            elyess_mez Mohamed ilyess
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: