[JENKINS-65885] Deadlock in CpsFlowExecution

Mirko created issue - 2021-06-14 01:48

Mirko made changes - 2021-06-14 02:08

Description

Original: We are experiencing dead locks in Jenkins. Recently upgraded to Jenkins 2.289.1 and since then we had two dead locks. Server does only stop when using kill -9.
{code:java}
Found one Java-level deadlock:
=============================
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/ecommerceapps%252Fecommerce/branches/ from 127.0.0.1 : Jetty (winstone)-263503":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]"
"Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]":
  waiting to lock monitor 0x00007f0b9c01ed18 (object 0x000000046cb7de38, a org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView),
  which is held by "Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660"
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]" {code}

New: We are experiencing dead locks in Jenkins. Recently upgraded to Jenkins 2.289.1 and since then we had two dead locks. Server does only stop when using kill -9.
{code:java}
Found one Java-level deadlock:
=============================
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/ecommerceapps%252Fecommerce/branches/ from 127.0.0.1 : Jetty (winstone)-263503":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]"
"Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]":
  waiting to lock monitor 0x00007f0b9c01ed18 (object 0x000000046cb7de38, a org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView),
  which is held by "Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660"
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]" {code}

{code:java}
Java stack information for the threads listed above:
===================================================
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/ecommerceapps%252Fecommerce/branches/ from 127.0.0.1 : Jetty (winstone)-263503":
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads(CpsFlowExecution.java:981)
        - waiting to lock <0x000000046cb7ddc8> (a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution)
        at org.jenkinsci.plugins.workflow.flow.FlowExecution.isComplete(FlowExecution.java:208)
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.isComplete(CpsFlowExecution.java:1238) {code}

Mirko made changes - 2021-06-14 02:09

Description

Original: We are experiencing dead locks in Jenkins. Recently upgraded to Jenkins 2.289.1 and since then we had two dead locks. Server does only stop when using kill -9.
{code:java}
Found one Java-level deadlock:
=============================
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/ecommerceapps%252Fecommerce/branches/ from 127.0.0.1 : Jetty (winstone)-263503":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]"
"Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]":
  waiting to lock monitor 0x00007f0b9c01ed18 (object 0x000000046cb7de38, a org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView),
  which is held by "Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660"
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]" {code}

{code:java}
Java stack information for the threads listed above:
===================================================
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/ecommerceapps%252Fecommerce/branches/ from 127.0.0.1 : Jetty (winstone)-263503":
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads(CpsFlowExecution.java:981)
        - waiting to lock <0x000000046cb7ddc8> (a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution)
        at org.jenkinsci.plugins.workflow.flow.FlowExecution.isComplete(FlowExecution.java:208)
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.isComplete(CpsFlowExecution.java:1238) {code}

New: We are experiencing dead locks in Jenkins. I am not sure if its related but we recently upgraded to Jenkins 2.289.1. Since then we had two dead locks about 3 days apart. Jenkins does not shut down gracefully and does only stop when using kill -9.
{code:java}
Found one Java-level deadlock:
=============================
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/ecommerceapps%252Fecommerce/branches/ from 127.0.0.1 : Jetty (winstone)-263503":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]"
"Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]":
  waiting to lock monitor 0x00007f0b9c01ed18 (object 0x000000046cb7de38, a org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView),
  which is held by "Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660"
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/pipelines/ecommerceapps%2Fecommerce/branches/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/runs/17/ from 10.120.1.90 : Jetty (winstone)-72660":
  waiting to lock monitor 0x00007f0bc0186788 (object 0x000000046cb7ddc8, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
  which is held by "Running CpsFlowExecution[Owner[acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test/17:acme-gitlab/ecommerceapps%2Fecommerce/PLAT-475_Set_up_S3_bucket_for_xyz_replacement_on_test #17]]" {code}

{code:java}
Java stack information for the threads listed above:
===================================================
"Handling GET /blue/rest/organizations/jenkins/pipelines/acme-gitlab/ecommerceapps%252Fecommerce/branches/ from 127.0.0.1 : Jetty (winstone)-263503":
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads(CpsFlowExecution.java:981)
        - waiting to lock <0x000000046cb7ddc8> (a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution)
        at org.jenkinsci.plugins.workflow.flow.FlowExecution.isComplete(FlowExecution.java:208)
        at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.isComplete(CpsFlowExecution.java:1238) {code}

Matt Russell added a comment - 2021-06-15 11:37

On Jenkins 2.291 we've been getting deadlocks that look like they might potentially be similar to this:

Found one Java-level deadlock:
 =============================
 "Handling GET /job/project-name/job/fullbuild-ci-monitor/ from 82.8.18.53 : Jetty (winstone)-11928 Job/index.jelly WorkflowJob/sidepanel.jelly HistoryWidget/index.jelly HistoryPageFilter/entries.jelly HistoryPageFilter/entries.jelly HistoryPageFilter/entry.jelly":
 waiting to lock monitor 0x00007fa69c084ec8 (object 0x000000073c1e3fa0, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
 which is held by "Running CpsFlowExecutionOwner[project-name/fullbuild-ci-monitor/31:project-name/fullbuild-ci-monitor #31]"
 "Running CpsFlowExecutionOwner[project-name/fullbuild-ci-monitor/31:project-name/fullbuild-ci-monitor #31]":
 waiting to lock monitor 0x00007fa7501d1758 (object 0x000000073c2f8f40, a org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView),
 which is held by "Handling GET /blue/rest/organizations/jenkins/pipelines/project-name/branches/fullbuild-ci-monitor/runs/31/ from 152.37.122.188 : Jetty (winstone)-8305"
 "Handling GET /blue/rest/organizations/jenkins/pipelines/project-name/branches/fullbuild-ci-monitor/runs/31/ from 152.37.122.188 : Jetty (winstone)-8305":
 waiting to lock monitor 0x00007fa69c084ec8 (object 0x000000073c1e3fa0, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution),
 which is held by "Running CpsFlowExecutionOwner[project-name/fullbuild-ci-monitor/31:project-name/fullbuild-ci-monitor #31]"

Matt Russell added a comment - 2021-06-15 11:37 On Jenkins 2.291 we've been getting deadlocks that look like they might potentially be similar to this: Found one Java-level deadlock: ============================= "Handling GET /job/project-name/job/fullbuild-ci-monitor/ from 82.8.18.53 : Jetty (winstone)-11928 Job/index.jelly WorkflowJob/sidepanel.jelly HistoryWidget/index.jelly HistoryPageFilter/entries.jelly HistoryPageFilter/entries.jelly HistoryPageFilter/entry.jelly": waiting to lock monitor 0x00007fa69c084ec8 (object 0x000000073c1e3fa0, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution), which is held by "Running CpsFlowExecutionOwner[project-name/fullbuild-ci-monitor/31:project-name/fullbuild-ci-monitor #31]" "Running CpsFlowExecutionOwner[project-name/fullbuild-ci-monitor/31:project-name/fullbuild-ci-monitor #31]": waiting to lock monitor 0x00007fa7501d1758 (object 0x000000073c2f8f40, a org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView), which is held by "Handling GET /blue/rest/organizations/jenkins/pipelines/project-name/branches/fullbuild-ci-monitor/runs/31/ from 152.37.122.188 : Jetty (winstone)-8305" "Handling GET /blue/rest/organizations/jenkins/pipelines/project-name/branches/fullbuild-ci-monitor/runs/31/ from 152.37.122.188 : Jetty (winstone)-8305": waiting to lock monitor 0x00007fa69c084ec8 (object 0x000000073c1e3fa0, a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution), which is held by "Running CpsFlowExecutionOwner[project-name/fullbuild-ci-monitor/31:project-name/fullbuild-ci-monitor #31]"

Mirko added a comment - 2021-06-15 17:37

matthew_russell Thanks for your comment. Is this also something that started recently?

Mirko added a comment - 2021-06-15 17:37 matthew_russell Thanks for your comment. Is this also something that started recently?

Liam added a comment - 2021-06-15 21:24

We upgraded from 2.222.1 to 2.289.1 yesterday and have experienced 2 deadlock situations since then. The monitor shows

Warning, the following threads are deadlocked : Handling GET /blue/rest/organizations/jenkins/pipelines/<job-name>/pipelines/rocblas-performance/pipelines/rocBLAS-internal/branches/PR-712/runs/12/nodes/26/steps/ from 172.17.0.1 : Jetty (winstone)-90128, Handling GET /job/<job-name>/job/rocblas-performance/job/rocBLAS-internal/job/PR-712/ from 172.17.0.1 : Jetty (winstone)-112137 Job/index.jelly WorkflowJob/sidepanel.jelly HistoryWidget/index.jelly, Running CpsFlowExecutionOwner[<job-name>/rocblas-performance/rocBLAS-internal/PR-712/12:<job-name>/rocblas-performance/rocBLAS-internal/PR-712 #12]]

The mentioned threads are executing

org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$Timing.close(CpsFlowExecution.java:426)

and

org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads(CpsFlowExecution.java:981)

There are many additional threads running org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads as well, the monitor warning only warns about one of them. None of these threads can be killed through the monitor, and only shut down when Jenkins is killed.

Liam added a comment - 2021-06-15 21:24 We upgraded from 2.222.1 to 2.289.1 yesterday and have experienced 2 deadlock situations since then. The monitor shows Warning, the following threads are deadlocked : Handling GET /blue/rest/organizations/jenkins/pipelines/<job-name>/pipelines/rocblas-performance/pipelines/rocBLAS-internal/branches/PR-712/runs/12/nodes/26/steps/ from 172.17.0.1 : Jetty (winstone)-90128, Handling GET /job/<job-name>/job/rocblas-performance/job/rocBLAS-internal/job/PR-712/ from 172.17.0.1 : Jetty (winstone)-112137 Job/index.jelly WorkflowJob/sidepanel.jelly HistoryWidget/index.jelly, Running CpsFlowExecution Owner[<job-name>/rocblas-performance/rocBLAS-internal/PR-712/12:<job-name>/rocblas-performance/rocBLAS-internal/PR-712 #12] ] The mentioned threads are executing org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$Timing.close(CpsFlowExecution.java:426) and org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads(CpsFlowExecution.java:981) There are many additional threads running org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getCurrentHeads as well, the monitor warning only warns about one of them. None of these threads can be killed through the monitor, and only shut down when Jenkins is killed.

Kyle added a comment - 2021-06-15 23:58 - edited

Hi all,

Running in to this hourly (at the latest) on our build system. We updated to Jenkins LTS v2.277.3 -> v2.289.1 late last week and updated all our Jenkins plugins as well as the same time.

Symptoms are that the builds hang, Blue Ocean UI is un-responsive on a handful of pages (sometimes all?), and the threads deadlock. It's impossible to cleanly shutdown Jenkins and I am forced to kill it with SIGKILL. Some jobs will succeed, but ultimately the entire server will become unusable.

Reviewing the thread dump shows the following calls are waiting on each other:

Handling GET /jenkins/blue/rest/organizations/jenkins/pipelines/sw/pipelines/client/branches/PR-2126/runs/5/nodes/ from 10.12.34.56 : Jetty (winstone)-32
"Handling GET /jenkins/blue/rest/organizations/jenkins/pipelines/sw/pipelines/client/branches/PR-2126/runs/5/nodes/ from 10.12.34.56 : Jetty (winstone)-32" Id=32 Group=main BLOCKED on org.jenkinsci.plugins.workflow.cps.CpsFlowExecution@c3afc42 owned by "Running CpsFlowExecution[Owner[sw/client/PR-2126/5:sw/client/PR-2126 #5]]" Id=2656
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$Timing.close(CpsFlowExecution.java:426)
	-  blocked on org.jenkinsci.plugins.workflow.cps.CpsFlowExecution@c3afc42

Running CpsFlowExecution[Owner[sw/client/PR-2126/5:sw/client/PR-2126 #5]]
"Running CpsFlowExecution[Owner[sw/client/PR-2126/5:sw/client/PR-2126 #5]]" Id=2656 Group=main BLOCKED on org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView@44395be7 owned by "Handling GET /jenkins/blue/rest/organizations/jenkins/pipelines/sw/pipelines/client/branches/PR-2126/runs/5/nodes/ from 10.12.34.56 : Jetty (winstone)-32" Id=32
	at org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView.onNewHead(StandardGraphLookupView.java:41)
	-  blocked on org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView@44395be7

Plugin list:

{
  "Apache HttpComponents Client 4.x API Plugin": "4.5.13-1.0",
  "Authentication Tokens API Plugin": "1.4",
  "Authorize Project": "1.4.0",
  "Autofavorite for Blue Ocean": "1.2.4",
  "Bitbucket Branch Source Plugin": "2.9.9",
  "Bitbucket Pipeline for Blue Ocean": "1.24.7",
  "Bitbucket Plugin": "1.1.29",
  "Blue Ocean": "1.24.7",
  "Blue Ocean Core JS": "1.24.7",
  "Blue Ocean Pipeline Editor": "1.24.7",
  "Bootstrap 4 API Plugin": "4.6.0-3",
  "Bootstrap 5 API Plugin": "5.0.1-2",
  "Branch API Plugin": "2.6.4",
  "Build Timeout": "1.20",
  "Caffeine API Plugin": "2.9.1-23.v51c4e2c879c8",
  "Checks API plugin": "1.7.0",
  "Command Agent Launcher Plugin": "1.6",
  "Common API for Blue Ocean": "1.24.7",
  "Conditional BuildStep": "1.4.1",
  "Config API for Blue Ocean": "1.24.7",
  "Copy Artifact Plugin": "1.46.1",
  "Credentials Binding Plugin": "1.25",
  "Credentials Plugin": "2.5",
  "Dashboard for Blue Ocean": "1.24.7",
  "Design Language": "1.24.7",
  "Display URL API": "2.3.5",
  "Display URL for Blue Ocean": "2.4.1",
  "Docker Commons Plugin": "1.17",
  "Docker Pipeline": "1.26",
  "Durable Task Plugin": "1.37",
  "ECharts API Plugin": "5.1.2-2",
  "Email Extension Plugin": "2.83",
  "Events API for Blue Ocean": "1.24.7",
  "External Monitor Job Type Plugin": "1.7",
  "Favorite": "2.3.3",
  "Folders Plugin": "6.15",
  "Font Awesome API Plugin": "5.15.3-3",
  "GIT server Plugin": "1.9",
  "Git Pipeline for Blue Ocean": "1.24.7",
  "Git client plugin": "3.7.2",
  "Git plugin": "4.7.2",
  "GitHub API Plugin": "1.123",
  "GitHub Branch Source Plugin": "2.11.1",
  "GitHub Pipeline for Blue Ocean": "1.24.7",
  "GitHub plugin": "1.33.1",
  "GitLab Branch Source Plugin": "1.5.7",
  "GitLab Plugin": "1.5.20",
  "Gitlab API Plugin": "1.0.6",
  "Google Login Plugin": "1.6",
  "Gradle Plugin": "1.36",
  "HTML Publisher plugin": "1.25",
  "HTTP Request Plugin": "1.8.27",
  "Handy Uri Templates 2.x API Plugin": "2.1.8-1.0",
  "JIRA Integration for Blue Ocean": "1.24.7",
  "JQuery3 API Plugin": "3.6.0-1",
  "JSch dependency plugin": "0.1.55.2",
  "JUnit Plugin": "1.50",
  "JWT for Blue Ocean": "1.24.7",
  "Jackson 2 API Plugin": "2.12.3",
  "Java JSON Web Token (JJWT) Plugin": "0.11.2-9.c8b45b8bb173",
  "JavaScript GUI Lib: ACE Editor bundle plugin": "1.1",
  "JavaScript GUI Lib: Handlebars bundle plugin": "3.0.8",
  "JavaScript GUI Lib: Moment.js bundle plugin": "1.1.1",
  "JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin": "1.2.1",
  "Javadoc Plugin": "1.6",
  "Jira plugin": "3.3",
  "Job DSL": "1.77",
  "LDAP Plugin": "2.7",
  "Lockable Resources plugin": "2.11",
  "Mailer Plugin": "1.34",
  "MapDB API Plugin": "1.0.9.0",
  "Matrix Authorization Strategy Plugin": "2.6.7",
  "Matrix Project Plugin": "1.19",
  "Maven Integration plugin": "3.11",
  "Mercurial plugin": "2.15",
  "Metrics Plugin": "4.0.2.8",
  "OWASP Markup Formatter Plugin": "2.1",
  "OkHttp Plugin": "3.14.9",
  "Oracle Java SE Development Kit Installer Plugin": "1.5",
  "PAM Authentication plugin": "1.6",
  "Personalization for Blue Ocean": "1.24.7",
  "Pipeline": "2.6",
  "Pipeline Graph Analysis Plugin": "1.11",
  "Pipeline SCM API for Blue Ocean": "1.24.7",
  "Pipeline Utility Steps": "2.8.0",
  "Pipeline implementation for Blue Ocean": "1.24.7",
  "Pipeline: API": "2.44",
  "Pipeline: Basic Steps": "2.23",
  "Pipeline: Build Step": "2.13",
  "Pipeline: Declarative": "1.8.5",
  "Pipeline: Declarative Extension Points API": "1.8.5",
  "Pipeline: GitHub Groovy Libraries": "1.0",
  "Pipeline: Groovy": "2.92",
  "Pipeline: Input Step": "2.12",
  "Pipeline: Job": "2.41",
  "Pipeline: Milestone Step": "1.3.2",
  "Pipeline: Model API": "1.8.5",
  "Pipeline: Multibranch": "2.24",
  "Pipeline: Nodes and Processes": "2.39",
  "Pipeline: REST API Plugin": "2.19",
  "Pipeline: SCM Step": "2.12",
  "Pipeline: Shared Groovy Libraries": "2.19",
  "Pipeline: Stage Step": "2.5",
  "Pipeline: Stage Tags Metadata": "1.8.5",
  "Pipeline: Stage View Plugin": "2.19",
  "Pipeline: Step API": "2.23",
  "Pipeline: Supporting APIs": "3.8",
  "Plain Credentials Plugin": "1.7",
  "Plugin Utilities API Plugin": "2.3.0",
  "Popper.js 2 API Plugin": "2.5.4-2",
  "Popper.js API Plugin": "1.16.1-2",
  "Pub-Sub \"light\" Bus": "1.15",
  "REST API for Blue Ocean": "1.24.7",
  "REST Implementation for Blue Ocean": "1.24.7",
  "Resource Disposer Plugin": "0.16",
  "Run Condition Plugin": "1.5",
  "SCM API Plugin": "2.6.4",
  "SSH Agent Plugin": "1.23",
  "SSH Build Agents plugin": "1.32.0",
  "SSH Credentials Plugin": "1.19",
  "SSH server": "3.0.3",
  "Script Security Plugin": "1.77",
  "Server Sent Events (SSE) Gateway Plugin": "1.24",
  "Skip Notifications Trait plugin": "1.0.5",
  "Slack Notification Plugin": "2.48",
  "Snakeyaml API Plugin": "1.29.1",
  "Structs Plugin": "1.23",
  "Subversion Plug-in": "2.14.3",
  "Timestamper": "1.13",
  "Token Macro Plugin": "2.15",
  "Trilead API Plugin": "1.0.13",
  "Variant Plugin": "1.4",
  "WMI Windows Agents Plugin": "1.8",
  "Web for Blue Ocean": "1.24.7",
  "Workspace Cleanup Plugin": "0.39",
  "bouncycastle API Plugin": "2.20",
  "i18n for Blue Ocean": "1.24.7"
}

Running under OpenJDK 8 JRE 8u292-b10-ubuntu~18.04

Kyle added a comment - 2021-06-15 23:58 - edited Hi all, Running in to this hourly (at the latest) on our build system. We updated to Jenkins LTS v2.277.3 -> v2.289.1 late last week and updated all our Jenkins plugins as well as the same time. Symptoms are that the builds hang, Blue Ocean UI is un-responsive on a handful of pages (sometimes all?), and the threads deadlock. It's impossible to cleanly shutdown Jenkins and I am forced to kill it with SIGKILL. Some jobs will succeed, but ultimately the entire server will become unusable. Reviewing the thread dump shows the following calls are waiting on each other: Handling GET /jenkins/blue/ rest /organizations/jenkins/pipelines/sw/pipelines/client/branches/PR-2126/runs/5/nodes/ from 10.12.34.56 : Jetty (winstone)-32 "Handling GET /jenkins/blue/ rest /organizations/jenkins/pipelines/sw/pipelines/client/branches/PR-2126/runs/5/nodes/ from 10.12.34.56 : Jetty (winstone)-32" Id=32 Group=main BLOCKED on org.jenkinsci.plugins.workflow.cps.CpsFlowExecution@c3afc42 owned by "Running CpsFlowExecution[Owner[sw/client/PR-2126/5:sw/client/PR-2126 #5]]" Id=2656 at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$Timing.close(CpsFlowExecution.java:426) - blocked on org.jenkinsci.plugins.workflow.cps.CpsFlowExecution@c3afc42 Running CpsFlowExecution[Owner[sw/client/PR-2126/5:sw/client/PR-2126 #5]] "Running CpsFlowExecution[Owner[sw/client/PR-2126/5:sw/client/PR-2126 #5]]" Id=2656 Group=main BLOCKED on org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView@44395be7 owned by "Handling GET /jenkins/blue/ rest /organizations/jenkins/pipelines/sw/pipelines/client/branches/PR-2126/runs/5/nodes/ from 10.12.34.56 : Jetty (winstone)-32" Id=32 at org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView.onNewHead(StandardGraphLookupView.java:41) - blocked on org.jenkinsci.plugins.workflow.graph.StandardGraphLookupView@44395be7 Plugin list: { "Apache HttpComponents Client 4.x API Plugin" : "4.5.13-1.0" , "Authentication Tokens API Plugin" : "1.4" , "Authorize Project" : "1.4.0" , "Autofavorite for Blue Ocean" : "1.2.4" , "Bitbucket Branch Source Plugin" : "2.9.9" , "Bitbucket Pipeline for Blue Ocean" : "1.24.7" , "Bitbucket Plugin" : "1.1.29" , "Blue Ocean" : "1.24.7" , "Blue Ocean Core JS" : "1.24.7" , "Blue Ocean Pipeline Editor" : "1.24.7" , "Bootstrap 4 API Plugin" : "4.6.0-3" , "Bootstrap 5 API Plugin" : "5.0.1-2" , "Branch API Plugin" : "2.6.4" , "Build Timeout" : "1.20" , "Caffeine API Plugin" : "2.9.1-23.v51c4e2c879c8" , "Checks API plugin" : "1.7.0" , "Command Agent Launcher Plugin" : "1.6" , "Common API for Blue Ocean" : "1.24.7" , "Conditional BuildStep" : "1.4.1" , "Config API for Blue Ocean" : "1.24.7" , "Copy Artifact Plugin" : "1.46.1" , "Credentials Binding Plugin" : "1.25" , "Credentials Plugin" : "2.5" , "Dashboard for Blue Ocean" : "1.24.7" , "Design Language" : "1.24.7" , "Display URL API" : "2.3.5" , "Display URL for Blue Ocean" : "2.4.1" , "Docker Commons Plugin" : "1.17" , "Docker Pipeline" : "1.26" , "Durable Task Plugin" : "1.37" , "ECharts API Plugin" : "5.1.2-2" , "Email Extension Plugin" : "2.83" , "Events API for Blue Ocean" : "1.24.7" , "External Monitor Job Type Plugin" : "1.7" , "Favorite" : "2.3.3" , "Folders Plugin" : "6.15" , "Font Awesome API Plugin" : "5.15.3-3" , "GIT server Plugin" : "1.9" , "Git Pipeline for Blue Ocean" : "1.24.7" , "Git client plugin" : "3.7.2" , "Git plugin" : "4.7.2" , "GitHub API Plugin" : "1.123" , "GitHub Branch Source Plugin" : "2.11.1" , "GitHub Pipeline for Blue Ocean" : "1.24.7" , "GitHub plugin" : "1.33.1" , "GitLab Branch Source Plugin" : "1.5.7" , "GitLab Plugin" : "1.5.20" , "Gitlab API Plugin" : "1.0.6" , "Google Login Plugin" : "1.6" , "Gradle Plugin" : "1.36" , "HTML Publisher plugin" : "1.25" , "HTTP Request Plugin" : "1.8.27" , "Handy Uri Templates 2.x API Plugin" : "2.1.8-1.0" , "JIRA Integration for Blue Ocean" : "1.24.7" , "JQuery3 API Plugin" : "3.6.0-1" , "JSch dependency plugin" : "0.1.55.2" , "JUnit Plugin" : "1.50" , "JWT for Blue Ocean" : "1.24.7" , "Jackson 2 API Plugin" : "2.12.3" , "Java JSON Web Token (JJWT) Plugin" : "0.11.2-9.c8b45b8bb173" , "JavaScript GUI Lib: ACE Editor bundle plugin" : "1.1" , "JavaScript GUI Lib: Handlebars bundle plugin" : "3.0.8" , "JavaScript GUI Lib: Moment.js bundle plugin" : "1.1.1" , "JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin" : "1.2.1" , "Javadoc Plugin" : "1.6" , "Jira plugin" : "3.3" , "Job DSL" : "1.77" , "LDAP Plugin" : "2.7" , "Lockable Resources plugin" : "2.11" , "Mailer Plugin" : "1.34" , "MapDB API Plugin" : "1.0.9.0" , "Matrix Authorization Strategy Plugin" : "2.6.7" , "Matrix Project Plugin" : "1.19" , "Maven Integration plugin" : "3.11" , "Mercurial plugin" : "2.15" , "Metrics Plugin" : "4.0.2.8" , "OWASP Markup Formatter Plugin" : "2.1" , "OkHttp Plugin" : "3.14.9" , "Oracle Java SE Development Kit Installer Plugin" : "1.5" , "PAM Authentication plugin" : "1.6" , "Personalization for Blue Ocean" : "1.24.7" , "Pipeline" : "2.6" , "Pipeline Graph Analysis Plugin" : "1.11" , "Pipeline SCM API for Blue Ocean" : "1.24.7" , "Pipeline Utility Steps" : "2.8.0" , "Pipeline implementation for Blue Ocean" : "1.24.7" , "Pipeline: API" : "2.44" , "Pipeline: Basic Steps" : "2.23" , "Pipeline: Build Step" : "2.13" , "Pipeline: Declarative" : "1.8.5" , "Pipeline: Declarative Extension Points API" : "1.8.5" , "Pipeline: GitHub Groovy Libraries" : "1.0" , "Pipeline: Groovy" : "2.92" , "Pipeline: Input Step" : "2.12" , "Pipeline: Job" : "2.41" , "Pipeline: Milestone Step" : "1.3.2" , "Pipeline: Model API" : "1.8.5" , "Pipeline: Multibranch" : "2.24" , "Pipeline: Nodes and Processes" : "2.39" , "Pipeline: REST API Plugin" : "2.19" , "Pipeline: SCM Step" : "2.12" , "Pipeline: Shared Groovy Libraries" : "2.19" , "Pipeline: Stage Step" : "2.5" , "Pipeline: Stage Tags Metadata" : "1.8.5" , "Pipeline: Stage View Plugin" : "2.19" , "Pipeline: Step API" : "2.23" , "Pipeline: Supporting APIs" : "3.8" , "Plain Credentials Plugin" : "1.7" , "Plugin Utilities API Plugin" : "2.3.0" , "Popper.js 2 API Plugin" : "2.5.4-2" , "Popper.js API Plugin" : "1.16.1-2" , "Pub-Sub \" light\ " Bus" : "1.15" , "REST API for Blue Ocean" : "1.24.7" , "REST Implementation for Blue Ocean" : "1.24.7" , "Resource Disposer Plugin" : "0.16" , "Run Condition Plugin" : "1.5" , "SCM API Plugin" : "2.6.4" , "SSH Agent Plugin" : "1.23" , "SSH Build Agents plugin" : "1.32.0" , "SSH Credentials Plugin" : "1.19" , "SSH server" : "3.0.3" , "Script Security Plugin" : "1.77" , "Server Sent Events (SSE) Gateway Plugin" : "1.24" , "Skip Notifications Trait plugin" : "1.0.5" , "Slack Notification Plugin" : "2.48" , "Snakeyaml API Plugin" : "1.29.1" , "Structs Plugin" : "1.23" , "Subversion Plug- in " : "2.14.3" , "Timestamper" : "1.13" , "Token Macro Plugin" : "2.15" , "Trilead API Plugin" : "1.0.13" , "Variant Plugin" : "1.4" , "WMI Windows Agents Plugin" : "1.8" , "Web for Blue Ocean" : "1.24.7" , "Workspace Cleanup Plugin" : "0.39" , "bouncycastle API Plugin" : "2.20" , "i18n for Blue Ocean" : "1.24.7" } Running under OpenJDK 8 JRE 8u292-b10-ubuntu~18.04

Kyle added a comment - 2021-06-16 00:55

Trying to find a workaround without luck:

1. Downgrading to v2.277.4 fails and loads a white page on the web UI and the logs print a stack trace as a result of what I assume are incompatible plugins:

Jun 16 00:29:26 build java[117888]: 2021-06-16 00:29:26.717+0000 [id=56]        WARNING        o.e.jetty.server.HttpChannel#handleException: /jenkins/
Jun 16 00:29:26 build java[117888]: org.apache.commons.jelly.JellyTagException: jar:file:/var/cache/jenkins/war/WEB-INF/lib/jenkins-core-2.277.4.jar!/lib/layout/layout.jelly:97:72: <st:include> Could not initialize class org.codehaus.groovy.runtime.InvokerHelper
Jun 16 00:29:26 build java[117888]:         at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:745)

2. Attempted to downgrade the "Pipeline: Groovy" plugin from v2.92 -> v 2.90 which it was but that blew up with many mismatch dependencies.

Questions:

Can anyone recommend a (good/any?) way to get back to a working plugin configuration?
Has anyone tested the weekly releases with up to date plugins or will that create a bigger dependency issue?

Kyle added a comment - 2021-06-16 00:55 Trying to find a workaround without luck: 1. Downgrading to v2.277.4 fails and loads a white page on the web UI and the logs print a stack trace as a result of what I assume are incompatible plugins: Jun 16 00:29:26 build java[117888]: 2021-06-16 00:29:26.717+0000 [id=56] WARNING o.e.jetty.server.HttpChannel#handleException: /jenkins/ Jun 16 00:29:26 build java[117888]: org.apache.commons.jelly.JellyTagException: jar:file:/ var /cache/jenkins/war/WEB-INF/lib/jenkins-core-2.277.4.jar!/lib/layout/layout.jelly:97:72: <st:include> Could not initialize class org.codehaus.groovy.runtime.InvokerHelper Jun 16 00:29:26 build java[117888]: at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:745) 2. Attempted to downgrade the "Pipeline: Groovy" plugin from v2.92 -> v 2.90 which it was but that blew up with many mismatch dependencies. Questions: Can anyone recommend a (good/any?) way to get back to a working plugin configuration? Has anyone tested the weekly releases with up to date plugins or will that create a bigger dependency issue?

Kyle added a comment - 2021-06-16 02:21

Was able to workaround the issue by downgrading a number of packages necessary to roll back the Groovy plugin v2.92 -> v2.90 which is where I suspect the issue is.

Here's the working package diff still on Jenkins LTS v2.289.1

--- plugins-2021-06-14.json	2021-06-15 17:01:57.537752150 -0700
+++ plugins-2021-06-14-fix.json	2021-06-15 19:15:19.422134736 -0700
@@ -4,11 +4,11 @@
   "Authorize Project": "1.4.0",
   "Autofavorite for Blue Ocean": "1.2.4",
   "Bitbucket Branch Source Plugin": "2.9.9",
-  "Bitbucket Pipeline for Blue Ocean": "1.24.7",
+  "Bitbucket Pipeline for Blue Ocean": "1.24.6",
   "Bitbucket Plugin": "1.1.29",
-  "Blue Ocean": "1.24.7",
+  "Blue Ocean": "1.24.6",
   "Blue Ocean Core JS": "1.24.7",
-  "Blue Ocean Pipeline Editor": "1.24.7",
+  "Blue Ocean Pipeline Editor": "1.24.6",
   "Bootstrap 4 API Plugin": "4.6.0-3",
   "Bootstrap 5 API Plugin": "5.0.1-2",
   "Branch API Plugin": "2.6.4",
@@ -31,18 +31,18 @@
   "Durable Task Plugin": "1.37",
   "ECharts API Plugin": "5.1.2-2",
   "Email Extension Plugin": "2.83",
-  "Events API for Blue Ocean": "1.24.7",
+  "Events API for Blue Ocean": "1.24.6",
   "External Monitor Job Type Plugin": "1.7",
   "Favorite": "2.3.3",
   "Folders Plugin": "6.15",
   "Font Awesome API Plugin": "5.15.3-3",
   "GIT server Plugin": "1.9",
-  "Git Pipeline for Blue Ocean": "1.24.7",
+  "Git Pipeline for Blue Ocean": "1.24.6",
   "Git client plugin": "3.7.2",
   "Git plugin": "4.7.2",
   "GitHub API Plugin": "1.123",
   "GitHub Branch Source Plugin": "2.11.1",
-  "GitHub Pipeline for Blue Ocean": "1.24.7",
+  "GitHub Pipeline for Blue Ocean": "1.24.6",
   "GitHub plugin": "1.33.1",
   "GitLab Branch Source Plugin": "1.5.7",
   "GitLab Plugin": "1.5.20",
@@ -50,7 +50,7 @@
   "Google Login Plugin": "1.6",
   "Gradle Plugin": "1.36",
   "HTML Publisher plugin": "1.25",
-  "HTTP Request Plugin": "1.8.27",
+  "HTTP Request Plugin": "1.9.0",
   "Handy Uri Templates 2.x API Plugin": "2.1.8-1.0",
   "JIRA Integration for Blue Ocean": "1.24.7",
   "JQuery3 API Plugin": "3.6.0-1",
@@ -80,30 +80,30 @@
   "Oracle Java SE Development Kit Installer Plugin": "1.5",
   "PAM Authentication plugin": "1.6",
   "Personalization for Blue Ocean": "1.24.7",
-  "Pipeline": "2.6",
-  "Pipeline Graph Analysis Plugin": "1.11",
+  "Pipeline": "2.5",
+  "Pipeline Graph Analysis Plugin": "1.10",
   "Pipeline SCM API for Blue Ocean": "1.24.7",
   "Pipeline Utility Steps": "2.8.0",
-  "Pipeline implementation for Blue Ocean": "1.24.7",
+  "Pipeline implementation for Blue Ocean": "1.24.6",
   "Pipeline: API": "2.44",
   "Pipeline: Basic Steps": "2.23",
   "Pipeline: Build Step": "2.13",
   "Pipeline: Declarative": "1.8.5",
   "Pipeline: Declarative Extension Points API": "1.8.5",
   "Pipeline: GitHub Groovy Libraries": "1.0",
-  "Pipeline: Groovy": "2.92",
+  "Pipeline: Groovy": "2.90",
   "Pipeline: Input Step": "2.12",
   "Pipeline: Job": "2.41",
   "Pipeline: Milestone Step": "1.3.2",
   "Pipeline: Model API": "1.8.5",
   "Pipeline: Multibranch": "2.24",
   "Pipeline: Nodes and Processes": "2.39",
-  "Pipeline: REST API Plugin": "2.19",
+  "Pipeline: REST API Plugin": "2.17",
   "Pipeline: SCM Step": "2.12",
   "Pipeline: Shared Groovy Libraries": "2.19",
   "Pipeline: Stage Step": "2.5",
   "Pipeline: Stage Tags Metadata": "1.8.5",
-  "Pipeline: Stage View Plugin": "2.19",
+  "Pipeline: Stage View Plugin": "2.17",
   "Pipeline: Step API": "2.23",
   "Pipeline: Supporting APIs": "3.8",
   "Plain Credentials Plugin": "1.7",
@@ -126,7 +126,7 @@
   "Slack Notification Plugin": "2.48",
   "Snakeyaml API Plugin": "1.29.1",
   "Structs Plugin": "1.23",
-  "Subversion Plug-in": "2.14.3",
+  "Subversion Plug-in": "2.14.4",
   "Timestamper": "1.13",
   "Token Macro Plugin": "2.15",
   "Trilead API Plugin": "1.0.13",

The HTTP Request Plugin and Subversion Plug-in are likely extraneous changes from earlier debugging and unapplied updates.

Not familiar enough with Jenkins and plugins to offer much assistance in patching it, but hopefully this helps narrow it down and resolve issues for others encountering the same issue.

Kyle added a comment - 2021-06-16 02:21 Was able to workaround the issue by downgrading a number of packages necessary to roll back the Groovy plugin v2.92 -> v2.90 which is where I suspect the issue is. Here's the working package diff still on Jenkins LTS v2.289.1 --- plugins-2021-06-14.json 2021-06-15 17:01:57.537752150 -0700 +++ plugins-2021-06-14-fix.json 2021-06-15 19:15:19.422134736 -0700 @@ -4,11 +4,11 @@ "Authorize Project" : "1.4.0" , "Autofavorite for Blue Ocean" : "1.2.4" , "Bitbucket Branch Source Plugin" : "2.9.9" , - "Bitbucket Pipeline for Blue Ocean" : "1.24.7" , + "Bitbucket Pipeline for Blue Ocean" : "1.24.6" , "Bitbucket Plugin" : "1.1.29" , - "Blue Ocean" : "1.24.7" , + "Blue Ocean" : "1.24.6" , "Blue Ocean Core JS" : "1.24.7" , - "Blue Ocean Pipeline Editor" : "1.24.7" , + "Blue Ocean Pipeline Editor" : "1.24.6" , "Bootstrap 4 API Plugin" : "4.6.0-3" , "Bootstrap 5 API Plugin" : "5.0.1-2" , "Branch API Plugin" : "2.6.4" , @@ -31,18 +31,18 @@ "Durable Task Plugin" : "1.37" , "ECharts API Plugin" : "5.1.2-2" , "Email Extension Plugin" : "2.83" , - "Events API for Blue Ocean" : "1.24.7" , + "Events API for Blue Ocean" : "1.24.6" , "External Monitor Job Type Plugin" : "1.7" , "Favorite" : "2.3.3" , "Folders Plugin" : "6.15" , "Font Awesome API Plugin" : "5.15.3-3" , "GIT server Plugin" : "1.9" , - "Git Pipeline for Blue Ocean" : "1.24.7" , + "Git Pipeline for Blue Ocean" : "1.24.6" , "Git client plugin" : "3.7.2" , "Git plugin" : "4.7.2" , "GitHub API Plugin" : "1.123" , "GitHub Branch Source Plugin" : "2.11.1" , - "GitHub Pipeline for Blue Ocean" : "1.24.7" , + "GitHub Pipeline for Blue Ocean" : "1.24.6" , "GitHub plugin" : "1.33.1" , "GitLab Branch Source Plugin" : "1.5.7" , "GitLab Plugin" : "1.5.20" , @@ -50,7 +50,7 @@ "Google Login Plugin" : "1.6" , "Gradle Plugin" : "1.36" , "HTML Publisher plugin" : "1.25" , - "HTTP Request Plugin" : "1.8.27" , + "HTTP Request Plugin" : "1.9.0" , "Handy Uri Templates 2.x API Plugin" : "2.1.8-1.0" , "JIRA Integration for Blue Ocean" : "1.24.7" , "JQuery3 API Plugin" : "3.6.0-1" , @@ -80,30 +80,30 @@ "Oracle Java SE Development Kit Installer Plugin" : "1.5" , "PAM Authentication plugin" : "1.6" , "Personalization for Blue Ocean" : "1.24.7" , - "Pipeline" : "2.6" , - "Pipeline Graph Analysis Plugin" : "1.11" , + "Pipeline" : "2.5" , + "Pipeline Graph Analysis Plugin" : "1.10" , "Pipeline SCM API for Blue Ocean" : "1.24.7" , "Pipeline Utility Steps" : "2.8.0" , - "Pipeline implementation for Blue Ocean" : "1.24.7" , + "Pipeline implementation for Blue Ocean" : "1.24.6" , "Pipeline: API" : "2.44" , "Pipeline: Basic Steps" : "2.23" , "Pipeline: Build Step" : "2.13" , "Pipeline: Declarative" : "1.8.5" , "Pipeline: Declarative Extension Points API" : "1.8.5" , "Pipeline: GitHub Groovy Libraries" : "1.0" , - "Pipeline: Groovy" : "2.92" , + "Pipeline: Groovy" : "2.90" , "Pipeline: Input Step" : "2.12" , "Pipeline: Job" : "2.41" , "Pipeline: Milestone Step" : "1.3.2" , "Pipeline: Model API" : "1.8.5" , "Pipeline: Multibranch" : "2.24" , "Pipeline: Nodes and Processes" : "2.39" , - "Pipeline: REST API Plugin" : "2.19" , + "Pipeline: REST API Plugin" : "2.17" , "Pipeline: SCM Step" : "2.12" , "Pipeline: Shared Groovy Libraries" : "2.19" , "Pipeline: Stage Step" : "2.5" , "Pipeline: Stage Tags Metadata" : "1.8.5" , - "Pipeline: Stage View Plugin" : "2.19" , + "Pipeline: Stage View Plugin" : "2.17" , "Pipeline: Step API" : "2.23" , "Pipeline: Supporting APIs" : "3.8" , "Plain Credentials Plugin" : "1.7" , @@ -126,7 +126,7 @@ "Slack Notification Plugin" : "2.48" , "Snakeyaml API Plugin" : "1.29.1" , "Structs Plugin" : "1.23" , - "Subversion Plug-in" : "2.14.3" , + "Subversion Plug-in" : "2.14.4" , "Timestamper" : "1.13" , "Token Macro Plugin" : "2.15" , "Trilead API Plugin" : "1.0.13" , The HTTP Request Plugin and Subversion Plug-in are likely extraneous changes from earlier debugging and unapplied updates. Not familiar enough with Jenkins and plugins to offer much assistance in patching it, but hopefully this helps narrow it down and resolve issues for others encountering the same issue.

Mirko made changes - 2021-06-16 03:01

Component/s

New: groovy-plugin [ 15549 ]

Jenkins

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Matt Russell added a comment - 2021-06-15 11:37

Expand comment: Matt Russell added a comment - 2021-06-15 11:37

Collapse comment: Mirko added a comment - 2021-06-15 17:37

Expand comment: Mirko added a comment - 2021-06-15 17:37

Collapse comment: Liam added a comment - 2021-06-15 21:24

Expand comment: Liam added a comment - 2021-06-15 21:24

Collapse comment: Kyle added a comment - 2021-06-15 23:58, Edited by Kyle - 2021-06-16 00:12

Expand comment: Kyle added a comment - 2021-06-15 23:58, Edited by Kyle - 2021-06-16 00:12

Collapse comment: Kyle added a comment - 2021-06-16 00:55

Expand comment: Kyle added a comment - 2021-06-16 00:55

Collapse comment: Kyle added a comment - 2021-06-16 02:21

Expand comment: Kyle added a comment - 2021-06-16 02:21

People

Dates