• Pipeline - April 2018

      Use cases

      • Restarting a successful stage so to rollback or redeploy code
      • When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

      In scope

      • Declarative pipeline only
      • Restart the stage with the same parameters/scm/actions as the previous run
      • No management of workspace
      • Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
        • Implication is we do not have to copy flow nodes to the new run
      • Add a when condition that will evaluate as true only when on a restarted 

      Out of scope

      • Automatically stash/unstash the workspace
      • Scripted pipeline (technical limitation)

      Open questions

      • Should we be keeping stash data for every run? What is the option of least surprise?
        • JD thinks that keeping stashes around by default is the least surprising option - that way they do not have to turn anything on to make the feature work.
        • AB says this might blow out the storage requirements

      Notes
      There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

      Technical notes
      Split off from JENKINS-33846 since abayer removed the component from that one, and I do not want this lost:

      For Declarative Pipelines, in principle you could skip some early stage's and just start running on a later one, using FlowCopier.

      This would be instead of using a checkpoint step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it could be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

      (Conversely, you could also offer the ability to run a build skipping some of the later stages, somewhat analogously to running ant some-non-default-target.)

          [JENKINS-45455] Restarting stages

          Frank Genois added a comment -

          Regarding restarting from sub-stages, I created the following issue : https://issues.jenkins-ci.org/browse/JENKINS-52391

          Please upvote if you are also looking forward to this feature.

           

          Frank Genois added a comment - Regarding restarting from sub-stages, I created the following issue : https://issues.jenkins-ci.org/browse/JENKINS-52391 Please upvote if you are also looking forward to this feature.  

          Blake Devcich added a comment - - edited

          Is there a way to always run a stage when restarting stages? Our pipeline requires a few things to be obtained dynamically in the beginning and when restarting stages we don't get that functionality.

          I think something like a pre section (like post) would be really helpful here.

          Example:

          Stage A -> Stage B -> Stage C -> Stage D

          I want to restart Stage C or Stage D but I want to ensure that Stage A always runs first.

           

          Blake Devcich added a comment - - edited Is there a way to always run a stage when restarting stages? Our pipeline requires a few things to be obtained dynamically in the beginning and when restarting stages we don't get that functionality. I think something like a pre section (like post) would be really helpful here. Example: Stage A -> Stage B -> Stage C -> Stage D I want to restart Stage C or Stage D but I want to ensure that Stage A always runs first.  

          Jan Hoppe added a comment -

          I might be wrong, but I don't get how to switch off the restart stage option.

          My pipelines currently don't work correctly after restarting a stage (I check out git only at first stage...)

          Jan Hoppe added a comment - I might be wrong, but I don't get how to switch off the restart stage option. My pipelines currently don't work correctly after restarting a stage (I check out git only at first stage...)

          Andrew Bayer added a comment -

          There currently isn't a way to disable stage restart, but that's a good idea for us to add - could you create a new ticket for that? Thanks!

          Andrew Bayer added a comment - There currently isn't a way to disable stage restart, but that's a good idea for us to add - could you create a new ticket for that? Thanks!

          Subbu K added a comment -

          bdevcich I am looking for exactly what you asked. Have you figured out a way to achieve this ? Thanks!

          Subbu K added a comment - bdevcich I am looking for exactly what you asked. Have you figured out a way to achieve this ? Thanks!

          Jay Spang added a comment -

          hoppej abayer please update this thread if you created a ticket to disable stage restart. We desperately need this.

          This feature does not work at all with my pipelines (it does not run on the same node as the original run), so it's generating false positives and wreaking havoc with my customers. I'm considering implementing some custom CSS to hide the silly thing before it causes us any more grief!

          Jay Spang added a comment - hoppej abayer please update this thread if you created a ticket to disable stage restart. We desperately need this. This feature does not work at all with my pipelines (it does not run on the same node as the original run), so it's generating false positives and wreaking havoc with my customers. I'm considering implementing some custom CSS to hide the silly thing before it causes us any more grief!

          Blake Devcich added a comment -

          ben5556 - No, we haven't found a way around this, so we're not taking advantage of stage restarting. I have use boolean parameters which can be unchecked to skip stages - which works as a sort of poor mans stage restarting.

          To expand on my previous example, I have check boxes as Job parameters for Stage B, C, D. I always want Stage A to run so there's no check box for that. If I want to restart Stage D, I uncheck the boxes for Stages B, C. Each stage in the pipeline checks the true/false value before running the stage.

          It's not ideal, and it's not really taking advantage of this feature, but it works pretty well.

          I also did create https://issues.jenkins-ci.org/browse/JENKINS-53167, so you might want to watch that.

          Blake Devcich added a comment - ben5556 - No, we haven't found a way around this, so we're not taking advantage of stage restarting. I have use boolean parameters which can be unchecked to skip stages - which works as a sort of poor mans stage restarting. To expand on my previous example, I have check boxes as Job parameters for Stage B, C, D. I always want Stage A to run so there's no check box for that. If I want to restart Stage D, I uncheck the boxes for Stages B, C. Each stage in the pipeline checks the true/false value before running the stage. It's not ideal, and it's not really taking advantage of this feature, but it works pretty well. I also did create https://issues.jenkins-ci.org/browse/JENKINS-53167 , so you might want to watch that.

          Garbageyard added a comment - - edited

          bdevcich: I have also voted for the ticket JENKINS-53167. For the workaround that you have in place, have you tied the job to a particular slave and use the same workspace each time the job runs? I am asking this because if a job ran the build and test phases successfully and then failed during packaging phase, how will you restart job and ensure that it picks up the already built binary for packaging phase.

           

          I am also looking for running a particular phase each time (or may be having the option to run it whenever i want) irrespective of whichever phase i restart my job from. I have included my use case in your ticket JENKINS-53167

          Garbageyard added a comment - - edited bdevcich : I have also voted for the ticket  JENKINS-53167 . For the workaround that you have in place, have you tied the job to a particular slave and use the same workspace each time the job runs? I am asking this because if a job ran the build and test phases successfully and then failed during packaging phase, how will you restart job and ensure that it picks up the already built binary for packaging phase.   I am also looking for running a particular phase each time (or may be having the option to run it whenever i want) irrespective of whichever phase i restart my job from. I have included my use case in your ticket  JENKINS-53167

          Blake Devcich added a comment -

          gc: Yes, in this particular case, we have it tied to a slave for hardware dependency reasons.

           

          Blake Devcich added a comment - gc : Yes, in this particular case, we have it tied to a slave for hardware dependency reasons.  

          Liam Newman added a comment -

          Bulk closing resolved issues.

          Liam Newman added a comment - Bulk closing resolved issues.

            abayer Andrew Bayer
            jglick Jesse Glick
            Votes:
            63 Vote for this issue
            Watchers:
            110 Start watching this issue

              Created:
              Updated:
              Resolved: