Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47173

Offer a high-performance storage engine for pipeline at some cost to resumability

    • Pipeline - October, Pipeline - December

      As a user of pipeline, I WANNA GO FAST.  

      To do that, I'm okay with sacrificing some of the ability to resume a pipeline after a hard Jenkins crash, as long as it means pipelines run fast.  It would be nice if I can still resume the pipeline after a clean/safe restart of the master, though!

      This differs from https://issues.jenkins-ci.org/browse/JENKINS-30896 because for this we're sacrificing some of the guarantees pipeline normally provides.

      Concrete needs:

      • Implement storage engine (workflow-support)
      • Implement options to select that storage engine (workflow-job)
      • Implement ability to RUN that storage engine (workflow-cps)

          [JENKINS-47173] Offer a high-performance storage engine for pipeline at some cost to resumability

          Sam Van Oort added a comment -

          Based on a discussion I had today, it may be helpful to have a Jenkins system property (or option) to control what the default storage engine and Durability Mode for pipeline are.

          Sam Van Oort added a comment - Based on a discussion I had today, it may be helpful to have a Jenkins system property (or option) to control what the default storage engine and Durability Mode for pipeline are.

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/workflow/support/DefaultStepContext.java
          src/main/java/org/jenkinsci/plugins/workflow/support/PipelineIOUtils.java
          src/main/java/org/jenkinsci/plugins/workflow/support/actions/LogActionImpl.java
          src/main/java/org/jenkinsci/plugins/workflow/support/storage/BulkFlowNodeStorage.java
          src/main/java/org/jenkinsci/plugins/workflow/support/storage/FlowNodeStorage.java
          src/main/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamFlowNodeStorage.java
          src/test/java/org/jenkinsci/plugins/workflow/ScalabilityTest.java
          src/test/java/org/jenkinsci/plugins/workflow/support/storage/AbstractStorageTest.java
          src/test/java/org/jenkinsci/plugins/workflow/support/storage/BulkStorageTest.java
          src/test/java/org/jenkinsci/plugins/workflow/support/storage/MockFlowExecution.java
          src/test/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamStorageTest.java
          src/test/java/org/jenkinsci/plugins/workflow/support/storage/StorageTestUtils.java
          http://jenkins-ci.org/commit/workflow-support-plugin/5f8c0433d1b0d8d5f5062eedba0b791ffad449ec
          Log:
          Merge pull request #47 from svanoort/defer-action-write-mk2

          Granular flownode persistence and new node storage engine JENKINS-47172 JENKINS-47173

          Compare: https://github.com/jenkinsci/workflow-support-plugin/compare/c0c4bbef3e36...5f8c0433d1b0

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/support/DefaultStepContext.java src/main/java/org/jenkinsci/plugins/workflow/support/PipelineIOUtils.java src/main/java/org/jenkinsci/plugins/workflow/support/actions/LogActionImpl.java src/main/java/org/jenkinsci/plugins/workflow/support/storage/BulkFlowNodeStorage.java src/main/java/org/jenkinsci/plugins/workflow/support/storage/FlowNodeStorage.java src/main/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamFlowNodeStorage.java src/test/java/org/jenkinsci/plugins/workflow/ScalabilityTest.java src/test/java/org/jenkinsci/plugins/workflow/support/storage/AbstractStorageTest.java src/test/java/org/jenkinsci/plugins/workflow/support/storage/BulkStorageTest.java src/test/java/org/jenkinsci/plugins/workflow/support/storage/MockFlowExecution.java src/test/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamStorageTest.java src/test/java/org/jenkinsci/plugins/workflow/support/storage/StorageTestUtils.java http://jenkins-ci.org/commit/workflow-support-plugin/5f8c0433d1b0d8d5f5062eedba0b791ffad449ec Log: Merge pull request #47 from svanoort/defer-action-write-mk2 Granular flownode persistence and new node storage engine JENKINS-47172 JENKINS-47173 Compare: https://github.com/jenkinsci/workflow-support-plugin/compare/c0c4bbef3e36...5f8c0433d1b0

          Sam Van Oort added a comment -

          Released with workflow-support 2.17

          Sam Van Oort added a comment - Released with workflow-support 2.17

          Basil Crow added a comment -

          I'm afraid I have to report a possible regression. After the following plugin upgrade this weekend to pull in Durablity Settings, a significant number of my pipeline jobs failed to re-hydrate cleanly:

          @@ -72,14 +72,14 @@ timestamper:1.8.9
           token-macro:2.3
           windows-slaves:1.3.1
           workflow-aggregator:2.5
          -workflow-api:2.24
          +workflow-api:2.25
           workflow-basic-steps:2.6
           workflow-cps-global-lib:2.9
          -workflow-cps:2.42
          -workflow-durable-task-step:2.17
          -workflow-job:2.16
          -workflow-multibranch:2.16
          +workflow-cps:2.43
          +workflow-durable-task-step:2.18
          +workflow-job:2.17
          +workflow-multibranch:2.17
           workflow-scm-step:2.6
           workflow-step-api:2.14
          -workflow-support:2.16
          +workflow-support:2.17
          

          Here is the stack trace:

          Resuming build at Sat Jan 27 13:35:48 PST 2018 after Jenkins restart
          Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: ???
          Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: ???
          Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting for next available executor on machine1
          Waiting for next available executor on machine2
          Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting for next available executor on machine2
          [Pipeline] End of Pipeline
          Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting for next available executor on machine2
          java.lang.IllegalStateException: JENKINS-37121: something already locked /path/to/regression-run-workspace/jenkins-regression/machine1/workspace@20
          	at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:75)
          	at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:51)
          	at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:92)
          	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          Caused: java.io.IOException: Failed to load build state
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:814)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:812)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:861)
          	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
          	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          Finished: FAILURE
          

          I can already see one problem:

          Waiting for next available executor on machine1
          Waiting for next available executor on machine2
          

          This is not only different from what I used to see, but it doesn't make logical sense. When I've done restarts like this in the past, before I upgraded the plugins to get Durability Settings, this used to say:

          Waiting for next available executor on machine1
          

          Here is my theory about what is going on. The job has this structure:

          node('machine2') {
              [...]
          }
          node('machine1') {
              node('machine2') {
                  [...]
              }
              [...] <- long-running operation that was running before the restart
          }
          

          The long-running operation was running before I restarted Jenkins. Note that I did not change any durability settings, I just upgraded the plugins to the latest version. My jobs are running at MAX_SURVIVABILITY. When Jenkins came up and re-hydrated, this previously would just wait for machine1 to be back online, then the job would resume. Apparently Jenkins now wants to wait for both machine1 and machine2 to have free executors before the job will resume, even though we are no longer running on machine2 (because the inner node block has gone out of scope and we have disconnected from machine2). In this case, machine2 happens to have a very small # of executors, so the job can't immediately claim an executor slot on machine2. Instead, it has to wait a long time for one to free up. By this time, a brand new (and unrelated) job has started and claimed the workspace, so we hit the exception.

          If my theory is correct, there is a regression in the recent Durability Settings work. We should not be trying to grab an executor slot for a node that we have already disconnected from when rehydrating. This wasn't the case before the plugin upgrade, but is the case after the plugin upgrade. This took down a large percentage of my regression test runs this weekend.

          Another (possibly separate) issue is that we continued to try to grab an executor on machine2 AFTER the "End of Pipeline" message, then threw an IllegalStateException.

          Basil Crow added a comment - I'm afraid I have to report a possible regression. After the following plugin upgrade this weekend to pull in Durablity Settings, a significant number of my pipeline jobs failed to re-hydrate cleanly: @@ -72,14 +72,14 @@ timestamper:1.8.9 token-macro:2.3 windows-slaves:1.3.1 workflow-aggregator:2.5 -workflow-api:2.24 +workflow-api:2.25 workflow-basic-steps:2.6 workflow-cps-global-lib:2.9 -workflow-cps:2.42 -workflow-durable-task-step:2.17 -workflow-job:2.16 -workflow-multibranch:2.16 +workflow-cps:2.43 +workflow-durable-task-step:2.18 +workflow-job:2.17 +workflow-multibranch:2.17 workflow-scm-step:2.6 workflow-step-api:2.14 -workflow-support:2.16 +workflow-support:2.17 Here is the stack trace: Resuming build at Sat Jan 27 13:35:48 PST 2018 after Jenkins restart Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: ??? Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: ??? Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting for next available executor on machine1 Waiting for next available executor on machine2 Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting for next available executor on machine2 [Pipeline] End of Pipeline Waiting to resume part of sybase-ase_full » SYBASE_RHEL69_16.0 #26: Waiting for next available executor on machine2 java.lang.IllegalStateException: JENKINS-37121: something already locked /path/to/regression-run-workspace/jenkins-regression/machine1/workspace@20 at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:75) at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:51) at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:92) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) Caused: java.io.IOException: Failed to load build state at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:814) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:812) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:861) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Finished: FAILURE I can already see one problem: Waiting for next available executor on machine1 Waiting for next available executor on machine2 This is not only different from what I used to see, but it doesn't make logical sense. When I've done restarts like this in the past, before I upgraded the plugins to get Durability Settings, this used to say: Waiting for next available executor on machine1 Here is my theory about what is going on. The job has this structure: node('machine2') { [...] } node('machine1') { node('machine2') { [...] } [...] <- long-running operation that was running before the restart } The long-running operation was running before I restarted Jenkins. Note that I did not change any durability settings, I just upgraded the plugins to the latest version. My jobs are running at MAX_SURVIVABILITY. When Jenkins came up and re-hydrated, this previously would just wait for machine1 to be back online, then the job would resume. Apparently Jenkins now wants to wait for both machine1 and machine2 to have free executors before the job will resume, even though we are no longer running on machine2 (because the inner node block has gone out of scope and we have disconnected from machine2). In this case, machine2 happens to have a very small # of executors, so the job can't immediately claim an executor slot on machine2. Instead, it has to wait a long time for one to free up. By this time, a brand new (and unrelated) job has started and claimed the workspace, so we hit the exception. If my theory is correct, there is a regression in the recent Durability Settings work. We should not be trying to grab an executor slot for a node that we have already disconnected from when rehydrating. This wasn't the case before the plugin upgrade, but is the case after the plugin upgrade. This took down a large percentage of my regression test runs this weekend. Another (possibly separate) issue is that we continued to try to grab an executor on machine2 AFTER the "End of Pipeline" message, then threw an IllegalStateException.

          Sam Van Oort added a comment -

          basil Thank you for reporting your issues, and I'm sorry to hear they bit you rather hard.  I've been doing some investigation on this – it sounds like you may be hitting a pre-existing bug that we saw reported couldn't reproduce (very similar symptoms around use of nesting blocks and executors).  However whether it's pre-existing or a regression from this, if a nested pair of nodes enables us to reproduce (and thus solve it), then you can look forward to a fix soonishly. 

          Sam Van Oort added a comment - basil Thank you for reporting your issues, and I'm sorry to hear they bit you rather hard.  I've been doing some investigation on this – it sounds like you may be hitting a pre-existing bug that we saw reported couldn't reproduce (very similar symptoms around use of nesting blocks and executors).  However whether it's pre-existing or a regression from this, if a nested pair of nodes enables us to reproduce (and thus solve it), then you can look forward to a fix soonishly. 

          Sam Van Oort added a comment -

          basil I do not appear to be able to reproduce what you describe with restarts using the new version – after digging deeper, my suspicion is that it might be an issue with the program data persisted before shutdown (using older plugin versions), or as you say it may be a race condition where a job somehow claims a workspace it is not entitled to (since it should be owned by another job) – though I've not seen such before. 

          The WorkspaceListLease un/pickling logic itself doesn't have any alterations, and we did a fair bit of testing around upgrade/downgrade scenarios.  But of course, this logic is quite subtle, complex, and in some cases prior bugs fixed in the course of this could only be observed if actions occurred in specific time windows.

           

          Have you observed / are you able to reproduce this behavior during restarts with the new plugin versions in place?

          Sam Van Oort added a comment - basil I do not appear to be able to reproduce what you describe with restarts using the new version – after digging deeper, my suspicion is that it might be an issue with the program data persisted before shutdown (using older plugin versions), or as you say it may be a race condition where a job somehow claims a workspace it is not entitled to (since it should be owned by another job) – though I've not seen such before.  The WorkspaceListLease un/pickling logic itself doesn't have any alterations, and we did a fair bit of testing around upgrade/downgrade scenarios.  But of course, this logic is quite subtle, complex, and in some cases prior bugs fixed in the course of this could only be observed if actions occurred in specific time windows.   Have you observed / are you able to reproduce this behavior during restarts with the new plugin versions in place?

          George C added a comment -

          I'm also seeing something similar with a pipeline job. This happened after I did two "Reload Configuration From Disk" operations about 5 minutes apart. We see that the job's queue only shows builds up to #6, but builds newer than 6 exist. This is using Jenkins 2.46.3

           

           

          [Pipeline] End of Pipeline
          java.lang.IllegalStateException: JENKINS-37121: something already locked /Users/hudson/build/workspace/ci-ios-bento-sample-apps-apple
                  at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:75)
                  at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:51)
                  at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:92)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
                  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          Caused: java.io.IOException: Failed to load build state
                  at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:697)
                  at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:695)
                  at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:744)
                  at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                  at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
                  at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                  at java.lang.Thread.run(Thread.java:745)
          Finished: FAILURE

           

          George C added a comment - I'm also seeing something similar with a pipeline job. This happened after I did two "Reload Configuration From Disk" operations about 5 minutes apart. We see that the job's queue only shows builds up to #6, but builds newer than 6 exist. This is using Jenkins 2.46.3     [Pipeline] End of Pipeline java.lang.IllegalStateException: JENKINS-37121: something already locked /Users/hudson/build/workspace/ci-ios-bento-sample-apps-apple         at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:75)         at org.jenkinsci.plugins.workflow.support.pickles.WorkspaceListLeasePickle$1.tryResolve(WorkspaceListLeasePickle.java:51)         at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:92)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) Caused: java.io.IOException: Failed to load build state         at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:697)         at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:695)         at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:744)         at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)         at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang. Thread .run( Thread .java:745) Finished: FAILURE  

          Basil Crow added a comment -

          svanoort Thanks for your reply and for looking into this.

          Have you observed / are you able to reproduce this behavior during restarts with the new plugin versions in place?

          No, I haven't observed this again and I haven't been able to reproduce it. I have also started doing clean shutdowns with the /exit API to ensure that the build queue gets properly flushed to disk before doing Jenkins upgrades.

          Thanks again!

          Basil Crow added a comment - svanoort Thanks for your reply and for looking into this. Have you observed / are you able to reproduce this behavior during restarts with the new plugin versions in place? No, I haven't observed this again and I haven't been able to reproduce it. I have also started doing clean shutdowns with the /exit API to ensure that the build queue gets properly flushed to disk before doing Jenkins upgrades. Thanks again!

          Sam Van Oort added a comment -

          basil georgecnj I just released workflow-cps 2.47 and workflow-job 2.18 which include a ton of fixes to related logic and likely fix the cause(s) of this – if you see this issue recur, could you please open a new JIRA issue and assign to me. Thanks!

          Sam Van Oort added a comment - basil georgecnj I just released workflow-cps 2.47 and workflow-job 2.18 which include a ton of fixes to related logic and likely fix the cause(s) of this – if you see this issue recur, could you please open a new JIRA issue and assign to me. Thanks!

            svanoort Sam Van Oort
            svanoort Sam Van Oort
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: