Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33761

Ability to disable Pipeline durability and "resume" build.

    XMLWordPrintable

    Details

    • Similar Issues:
    • Epic Link:
    • Sprint:
      Blue Ocean 1.4 - beta 2, Pipeline - December

      Description

      Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing down. 

      It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

      Implementation notes:

      • Requires a new OptionalJobProperty on the job, optionally a new BranchProperty in workflow-multibranch-plugin that echoes that same property
      • Needs some way to signal to storage (workflow-support) and execution (workflow-cps) that the pipeline is running with resume OFF to hint that they can use faster nondurable execution.

        Attachments

          Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-49961 NPE from CpsFlowExecution.saveOwner

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-37475 Add a configurable switch on job that causes CPS to silently not continue on non-serializable code

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-28183 Hard killed job's stage blocks stage in following jobs

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-45917 [Jenkins v2.63] Build queue deadlocks

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-47173 Offer a high-performance storage engine for pipeline at some cost to resumability

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-47390 Allow durable task step to, uh, not be. Durable, that is.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-36013 Automatically abort ExecutorPickle rehydration from an ephemeral node

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-49079 Copy job properties for new branch project from base/default branch

          • Major - Major loss of function.
          • Open
          links to

          Web Link CloudBees Internal CD-298

          Web Link CloudBees Internal CD-390

            Activity

            Ascending order - Click to sort in descending order
            jtilander Jim Tilander created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Epic Link JENKINS-35399 [ 171192 ]
            larsmeynberg Lars Meynberg made changes -
            Link This issue is related to JENKINS-28183 [ JENKINS-28183 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 169740 ] JNJira + In-Review [ 183632 ]
            abayer Andrew Bayer made changes -
            Component/s pipeline-general [ 21692 ]
            abayer Andrew Bayer made changes -
            Component/s workflow-plugin [ 18820 ]
            jglick Jesse Glick made changes -
            Component/s workflow-job-plugin [ 21716 ]
            Component/s pipeline [ 21692 ]
            jglick Jesse Glick made changes -
            Component/s workflow-cps-plugin [ 21713 ]
            jglick Jesse Glick made changes -
            Issue Type Improvement [ 4 ] New Feature [ 2 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-37475 [ JENKINS-37475 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-36013 [ JENKINS-36013 ]
            jayv Jo Voordeckers made changes -
            Attachment Screen Shot 2017-07-19 at 11.14.33 AM.png [ 38963 ]
            jayv Jo Voordeckers made changes -
            Attachment Screen Shot 2017-07-19 at 11.14.33 AM.png [ 38963 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-45917 [ JENKINS-45917 ]
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ]
            michaelneale Michael Neale made changes -
            Assignee Sam Van Oort [ svanoort ]
            svanoort Sam Van Oort made changes -
            Summary Ability to disable "resume" build. Ability to disable Pipeline durability and "resume" build.
            svanoort Sam Van Oort made changes -
            Description Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes. I've also never ever seen it work correctly, each time a job tries to resume it hangs infinitely and I always have to go in and kill the job manually.

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.             		Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.
            michaelneale Michael Neale made changes -
            Component/s blueocean-plugin [ 21481 ]
            michaelneale Michael Neale made changes -
            Sprint Blue Ocean 1.3 - candidates [ 326 ]
            svanoort Sam Van Oort made changes -
            Epic Link JENKINS-35399 [ 171192 ] JENKINS-47170 [ 185575 ]
            svanoort Sam Van Oort made changes -
            Component/s blueocean-plugin [ 21481 ]
            svanoort Sam Van Oort made changes -
            Link This issue is related to JENKINS-47173 [ JENKINS-47173 ]
            svanoort Sam Van Oort made changes -
            Description Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.             		Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

            Implementation notes:

            * Requires a new OptionalJobProperty on the job, plus a new BranchProperty in branch-api-plugin that echoes that
            svanoort Sam Van Oort made changes -
            Description Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

            Implementation notes:

            * Requires a new OptionalJobProperty on the job, plus a new BranchProperty in branch-api-plugin that echoes that             		Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

            Implementation notes:
             * Requires a new OptionalJobProperty on the job, plus a new BranchProperty in branch-api-plugin that echoes that
             * Needs some way to signal to storage (workflow-support) and execution (workflow-cps) that the pipeline is running with resume OFF to hint that they can use faster nondurable execution.
            svanoort Sam Van Oort made changes -
            Description Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

            Implementation notes:
             * Requires a new OptionalJobProperty on the job, plus a new BranchProperty in branch-api-plugin that echoes that
             * Needs some way to signal to storage (workflow-support) and execution (workflow-cps) that the pipeline is running with resume OFF to hint that they can use faster nondurable execution.             		Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

            Implementation notes:
             * Requires a new OptionalJobProperty on the job, optionally a new BranchProperty in workflow-multibranch-plugin that echoes that same property
             * Needs some way to signal to storage (workflow-support) and execution (workflow-cps) that the pipeline is running with resume OFF to hint that they can use faster nondurable execution.
            svanoort Sam Van Oort made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            svanoort Sam Van Oort made changes -
            Component/s workflow-api-plugin [ 21711 ]
            svanoort Sam Van Oort made changes -
            Description Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing done. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

            Implementation notes:
             * Requires a new OptionalJobProperty on the job, optionally a new BranchProperty in workflow-multibranch-plugin that echoes that same property
             * Needs some way to signal to storage (workflow-support) and execution (workflow-cps) that the pipeline is running with resume OFF to hint that they can use faster nondurable execution.             		Having some state being generated at the each node during execution, resuming builds after jenkins restarts or nodes reboots are just not feasible sometimes and can result in infinite hangs in some cases.  Also, providing durability results in extensive writes to disk that can bring performance crashing down. 

            It would be great to be able to specify that jobs don't resume upon interruptions, but rather just fail. This would increase the robustness of the system ideally, since upon nodes restarting, they quickly pick up jobs that tries to resume and hangs exhausting all available executors quickly.

            Implementation notes:
             * Requires a new OptionalJobProperty on the job, optionally a new BranchProperty in workflow-multibranch-plugin that echoes that same property
             * Needs some way to signal to storage (workflow-support) and execution (workflow-cps) that the pipeline is running with resume OFF to hint that they can use faster nondurable execution.
            jglick Jesse Glick made changes -
            Component/s workflow-durable-task-step-plugin [ 21715 ]
            svanoort Sam Van Oort made changes -
            Link This issue is related to JENKINS-47390 [ JENKINS-47390 ]
            svanoort Sam Van Oort made changes -
            Component/s workflow-durable-task-step-plugin [ 21715 ]
            jamesdumay James Dumay made changes -
            Sprint Blue Ocean 1.4 - beta 2 [ 326 ] Blue Ocean 1.4 - beta 2, Pipeline - December [ 326, 446 ]
            jamesdumay James Dumay made changes -
            Rank Ranked lower
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal CD-298 (Web Link)" [ 19061 ]
            svanoort Sam Van Oort made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal CD-390 (Web Link)" [ 19741 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-49079 [ JENKINS-49079 ]
            svanoort Sam Van Oort made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            svanoort Sam Van Oort made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            svanoort Sam Van Oort made changes -
            Labels project-cheetah
            jglick Jesse Glick made changes -
            Link This issue is blocked by JENKINS-49961 [ JENKINS-49961 ]
            rg Russell Gallop made changes -
            Attachment JENKINS-33671_thread_dump.txt [ 44546 ]

              People

              Assignee:
              svanoort Sam Van Oort
              Reporter:
              jtilander Jim Tilander
              Votes:
              47 Vote for this issue
              Watchers:
              52 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: