Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45455

Restarting stages

    XMLWordPrintable

    Details

    • Similar Issues:
    • Sprint:
      Pipeline - April 2018

      Description

      Use cases

      • Restarting a successful stage so to rollback or redeploy code
      • When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

      In scope

      • Declarative pipeline only
      • Restart the stage with the same parameters/scm/actions as the previous run
      • No management of workspace
      • Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
        • Implication is we do not have to copy flow nodes to the new run
      • Add a when condition that will evaluate as true only when on a restarted 

      Out of scope

      • Automatically stash/unstash the workspace
      • Scripted pipeline (technical limitation)

      Open questions

      • Should we be keeping stash data for every run? What is the option of least surprise?
        • JD thinks that keeping stashes around by default is the least surprising option - that way they do not have to turn anything on to make the feature work.
        • AB says this might blow out the storage requirements

      Notes
      There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

      Technical notes
      Split off from JENKINS-33846 since Andrew Bayer removed the component from that one, and I do not want this lost:

      For Declarative Pipelines, in principle you could skip some early stage's and just start running on a later one, using FlowCopier.

      This would be instead of using a checkpoint step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it could be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

      (Conversely, you could also offer the ability to run a build skipping some of the later stages, somewhat analogously to running ant some-non-default-target.)

        Attachments

          Issue Links

            Activity

            jglick Jesse Glick created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Link This issue relates to JENKINS-33846 [ JENKINS-33846 ]
            jglick Jesse Glick made changes -
            Description Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            Hide
            abayer Andrew Bayer added a comment -

            So this would entail stashing things behind the scenes, right?

            Show
            abayer Andrew Bayer added a comment - So this would entail stashing things behind the scenes, right?
            Hide
            jglick Jesse Glick added a comment -

            If you have a separate node per stage then there should be nothing special needed. This would not work if you were reusing one node across stages—but then again, neither does checkpoint.

            Show
            jglick Jesse Glick added a comment - If you have a separate node  per stage then there should be nothing special needed. This would not work if you were reusing one node across stages—but then again, neither does checkpoint .
            jamesdumay James Dumay made changes -
            Description Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use case*
            When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Original description*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Description *Use case*
            When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Original description*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use case*
            When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Link This issue is duplicated by JENKINS-46043 [ JENKINS-46043 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: James William Dumay
            Path:
            content/projects/blueocean/roadmap/data.json
            http://jenkins-ci.org/commit/jenkins.io/3203ce31f3ba009573c8b6886b28f9d589390c6b
            Log:
            Add JENKINS-45455 to declarative roadmap

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: James William Dumay Path: content/projects/blueocean/roadmap/data.json http://jenkins-ci.org/commit/jenkins.io/3203ce31f3ba009573c8b6886b28f9d589390c6b Log: Add JENKINS-45455 to declarative roadmap
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: R. Tyler Croy
            Path:
            content/projects/blueocean/roadmap/data.json
            http://jenkins-ci.org/commit/jenkins.io/c2ca1d3b21aae0971d42aefdf4b8b4442204d39a
            Log:
            Merge pull request #1064 from i386/patch-6

            Add JENKINS-45455 to declarative roadmap

            Compare: https://github.com/jenkins-infra/jenkins.io/compare/920bf01f2450...c2ca1d3b21aa

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: content/projects/blueocean/roadmap/data.json http://jenkins-ci.org/commit/jenkins.io/c2ca1d3b21aae0971d42aefdf4b8b4442204d39a Log: Merge pull request #1064 from i386/patch-6 Add JENKINS-45455 to declarative roadmap Compare: https://github.com/jenkins-infra/jenkins.io/compare/920bf01f2450...c2ca1d3b21aa
            Hide
            hummerstudio Tang Ming added a comment -

            Come from JENKINS-33846. Does anybody work on this?

            Show
            hummerstudio Tang Ming added a comment - Come from JENKINS-33846 . Does anybody work on this?
            Hide
            stevenfoster Steven Foster added a comment -

            Would this be suitable for revisiting an ignored input step later, e.g. input at the end of the build?

            Use case: input step guarding an entirely optional and discretionary step that should be available to invoke at any time but not block a build from completing successfully.

            Show
            stevenfoster Steven Foster added a comment - Would this be suitable for revisiting an ignored input step later, e.g. input at the end of the build? Use case: input step guarding an entirely optional and discretionary step that should be available to invoke at any time but not block a build from completing successfully.
            Hide
            linyuxiang yuxiang lin added a comment -

            Any update? Will integrate this feature with open source version? 

            Show
            linyuxiang yuxiang lin added a comment - Any update? Will integrate this feature with open source version? 
            Hide
            ccaraivan Costin Caraivan added a comment -

            I'd add a comment related to this: there is a very common use case which can be covered easily (I hope ) by extending this feature a bit.

            The use case is deployments or test executions triggered manually, especially by changing parameters.

             

            Right now pipelines are meant to be fully automatic but there are situations where for some reason this is not desirable or even possible. Example: small single point of failure demo environment used by clients. In this case you would not want to trigger a deployment for each commit since the environment would go down constantly.

             

            By changing this feature to allow the restart of any stage, instead of just failed stages, this use case would be achieved.

             

            QA people could just select the stage in the job UI and trigger it, parameterize it and profit! Otherwise we have to fall back to the old and ugly jobs...

            Show
            ccaraivan Costin Caraivan added a comment - I'd add a comment related to this: there is a very common use case which can be covered easily (I hope ) by extending this feature a bit. The use case is deployments or test executions triggered manually, especially by changing parameters.   Right now pipelines are meant to be fully automatic but there are situations where for some reason this is not desirable or even possible. Example: small single point of failure demo environment used by clients. In this case you would not want to trigger a deployment for each commit since the environment would go down constantly.   By changing this feature to allow the restart of any stage, instead of just failed stages, this use case would be achieved.   QA people could just select the stage in the job UI and trigger it, parameterize it and profit! Otherwise we have to fall back to the old and ugly jobs...
            Hide
            jglick Jesse Glick added a comment -

            Costin Caraivan perhaps better covered by JENKINS-36089 or whatever it morphs into.

            Show
            jglick Jesse Glick added a comment - Costin Caraivan perhaps better covered by  JENKINS-36089 or whatever it morphs into.
            Hide
            iplaman Idan Bidani added a comment -

            I agree with Costin Caraivan this use case could be useful functionality to have.

            Show
            iplaman Idan Bidani added a comment - I agree with  Costin Caraivan  this use case could be useful functionality to have.
            Hide
            zindello Josh Mesilane added a comment -

            I'll add that this is critical functionality for the company that I work for. We're looking at reverting to the older Deployment or Build Pipeline view plugins, and if this isn't far away I'd much rather wait for this to be implemented.

            Show
            zindello Josh Mesilane added a comment - I'll add that this is critical functionality for the company that I work for. We're looking at reverting to the older Deployment or Build Pipeline view plugins, and if this isn't far away I'd much rather wait for this to be implemented.
            Hide
            seanwcom Sean Williams added a comment -

            Adding my agreement with Costin and others... being able to restart from any stage is crucial.

            Show
            seanwcom Sean Williams added a comment - Adding my agreement with Costin and others... being able to restart from any stage is crucial.
            Hide
            ayub_thg ayub malik added a comment - - edited

            I am trying to migrate projects to Jenkins and the pipeline really needs this feature.

            Our stages are defined as:

            Build -> Integration Tests -> Deploy to QA -> Deploy To Live

            The 'Deploy to QA' stage has an input() step with a timeout to make it a manual step,

            We need to re-run this and subsequent stages when approved by QAs. This is an essential feature that other CI servers provide.

            Show
            ayub_thg ayub malik added a comment - - edited I am trying to migrate projects to Jenkins and the pipeline really needs this feature. Our stages are defined as: Build -> Integration Tests -> Deploy to QA -> Deploy To Live The 'Deploy to QA' stage has an input() step with a timeout to make it a manual step, We need to re-run this and subsequent stages when approved by QAs. This is an essential feature that other CI servers provide.
            jamesdumay James Dumay made changes -
            Summary Restart Declarative stage after failure Restart declarative stage
            jamesdumay James Dumay made changes -
            Description *Use case*
            When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Epic Link JENKINS-48356 [ 186951 ]
            jamesdumay James Dumay made changes -
            Sprint Pipeline - Candidates [ 381 ]
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal CD-367 (Web Link)" [ 18941 ]
            Hide
            abayer Andrew Bayer added a comment -

            Starting to think this through. Do we want to add an option for "preserve this Pipeline for resuming after the last successful top-level stage (because we're not going to mess around with nested parallel/sequential stages) including workspace etc"? Because we definitely don't want to do all that automatically for every top-level stage of every build - that's a big performance hit to stash everything after every stage, etc. Or do we want to go simpler and allow any Declarative Pipeline to be "resumed" from a given top-level stage, with the Pipeline author responsible for making sure that the workspace is stashed and unstashed, etc? The latter seems more reasonable to me, albeit not perfect.

            Show
            abayer Andrew Bayer added a comment - Starting to think this through. Do we want to add an option for "preserve this Pipeline for resuming after the last successful top-level stage (because we're not going to mess around with nested parallel/sequential stages) including workspace etc"? Because we definitely don't want to do all that automatically for every top-level stage of every build - that's a big performance hit to stash everything after every stage, etc. Or do we want to go simpler and allow any Declarative Pipeline to be "resumed" from a given top-level stage, with the Pipeline author responsible for making sure that the workspace is stashed and unstashed, etc? The latter seems more reasonable to me, albeit not perfect.
            jamesdumay James Dumay made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History
            ** Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.

            *Out of scope*
            * Automatically stash/unstash the workspace

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History
            ** Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.

            *Out of scope*
            * Automatically stash/unstash the workspace

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History
            ** Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Alternatively: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History
            ** Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Alternatively: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History - investigate and touch base with James on findings:
            ** Option 1: Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Option 2: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Sprint Pipeline - Candidates [ 381 ] Pipeline - December [ 446 ]
            jamesdumay James Dumay made changes -
            Summary Restart declarative stage Restarting stages
            jamesdumay James Dumay made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History - investigate and touch base with James on findings:
            ** Option 1: Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Option 2: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History - investigate and touch base with James on findings:
            ** Option 1: Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Option 2: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History - investigate and touch base with James on findings:
            ** Option 1: Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Option 2: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History - investigate and touch base with James on findings:
            ** Option 1: Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Option 2: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            Hide
            iplaman Idan Bidani added a comment -

            I think preserving(like pinning) the pipeline is cool idea, I think the expected simple behavior is based on "number of last builds to keep" function or maybe "keep last number of pipelines visible in stage view"

            Show
            iplaman Idan Bidani added a comment - I think preserving(like pinning) the pipeline is cool idea, I think the expected simple behavior is based on "number of last builds to keep" function or maybe "keep last number of pipelines visible in stage view"
            Hide
            lesinge Yoh Ann added a comment -

            We need this feature : Retry a stage! 

            Show
            lesinge Yoh Ann added a comment - We need this feature : Retry a stage! 
            Hide
            abayer Andrew Bayer added a comment -

            Hrm, I misremembered how stashes work. Unless we explicitly override the normal behavior, all stashes for a build get wiped out when the build finishes. I think what we'll have to do to support reusing stashes from the original build is probably add an option to preserve stashes for a Declarative Pipeline, defaulting to off.

            Show
            abayer Andrew Bayer added a comment - Hrm, I misremembered how stashes work. Unless we explicitly override the normal behavior, all stashes for a build get wiped out when the build finishes. I think what we'll have to do to support reusing stashes from the original build is probably add an option to preserve stashes for a Declarative Pipeline, defaulting to off.
            Hide
            jglick Jesse Glick added a comment -

            Andrew Bayer right, something like that. Currently checkpoint skirts the issue by just suppressing stash cleanup unconditionally (and there is no way to instruct Jenkins that a given build has been definitively closed for resumption and storage can be reclaimed, other than deleting the whole build).

            For purposes of this feature, if there is no explicit marker that the Pipeline is ready for stage restarting—and perhaps there ought to be?—then you would need to be careful to ensure that most stashes are by default cleaned up most of the time. Perhaps you could clean up all but the last build (which gets tricky if concurrent builds are enabled), or clean up successful builds, etc.

            Show
            jglick Jesse Glick added a comment - Andrew Bayer right, something like that. Currently checkpoint skirts the issue by just suppressing stash cleanup unconditionally (and there is no way to instruct Jenkins that a given build has been definitively closed for resumption and storage can be reclaimed, other than deleting the whole build). For purposes of this feature, if there is no explicit marker that the Pipeline is ready for stage restarting—and perhaps there ought to be?—then you would need to be careful to ensure that most stashes are by default cleaned up most of the time. Perhaps you could clean up all but the last build (which gets tricky if concurrent builds are enabled), or clean up successful builds, etc.
            abayer Andrew Bayer made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            llibicpep Dee Kryvenko added a comment -

            What are the complications with retrying a stage for scripted pipeline?

            I have implemented a shared library that allows users to declaratively explain what's inside a repository with some platform specific parameters, so my library then based on that input produces a scripted pipeline, so that we have unified pipelines within the company that covers supported platforms. I think to switch this library to be producing declarative-style DSL would be kind of costly, and I'm not really sure all the things I've been using would be available there.

            Is there anything else to read except https://jenkins.io/doc/book/pipeline/syntax/? This is not clear from there what are the benefits and limitations for each approach, i.e. what would be not available for me should I go with Declarative style vs Scripted one. I hope Jenkins team are not trying to put users community into a position of choosing what feature is more important where feature A would be available Scripted only and feature B Declarative only and both are needed. This causes so many confusion and frustration.

            Show
            llibicpep Dee Kryvenko added a comment - What are the complications with retrying a stage for scripted pipeline? I have implemented a shared library that allows users to declaratively explain what's inside a repository with some platform specific parameters, so my library then based on that input produces a scripted pipeline, so that we have unified pipelines within the company that covers supported platforms. I think to switch this library to be producing declarative-style DSL would be kind of costly, and I'm not really sure all the things I've been using would be available there. Is there anything else to read except https://jenkins.io/doc/book/pipeline/syntax/?  This is not clear from there what are the benefits and limitations for each approach, i.e. what would be not available for me should I go with Declarative style vs Scripted one. I hope Jenkins team are not trying to put users community into a position of choosing what feature is more important where feature A would be available Scripted only and feature B Declarative only and both are needed. This causes so many confusion and frustration.
            Hide
            abayer Andrew Bayer added a comment -

            James Dumay - so re: reusing the old build etc. We definitely can't use the same build number - that's just not possible. So BUILD_ID will be different for the restarted build than for the original. But we can make the visualization treat it as one build - by recording that a build is a restarted build of the original build number, and marking top-level stages on the restarted build as skipped for restart until we get to the one we're restarting from, it should be possible for Blue Ocean to "replace" the visualization for the original build, using the original build's visualization for every top-level stage (and their children) that's skipped for restart, and the new build's visualization for everything after (and replacing the original build's links for tests, artifacts, etc with the restarted build's equivalent). How's that sound?

            Show
            abayer Andrew Bayer added a comment - James Dumay - so re: reusing the old build etc. We definitely can't use the same build number - that's just not possible. So BUILD_ID will be different for the restarted build than for the original. But we can make the visualization treat it as one build - by recording that a build is a restarted build of the original build number, and marking top-level stages on the restarted build as skipped for restart until we get to the one we're restarting from, it should be possible for Blue Ocean to "replace" the visualization for the original build, using the original build's visualization for every top-level stage (and their children) that's skipped for restart, and the new build's visualization for everything after (and replacing the original build's links for tests, artifacts, etc with the restarted build's equivalent). How's that sound?
            Hide
            abayer Andrew Bayer added a comment -

            Dee Kryvenko - with Declarative, we can guarantee consistency in behavior. Everything but the actual steps will be executed for every single Declarative Pipeline in the same order and way. This makes it much easier to say "Re-run build #X, with the same parameters, SCM revision, checkouts, etc, but don't run any stages until the one we've specified", because we know, for example, how agents, environment variables, etc are handled outside of the actual stages themselves. That means we can treat each top-level stage in a Declarative Pipeline as an atomic entity to some extent, at least enough to know that we can pick back up where we left off without having to do weird Groovy hacks to ensure we're at exactly the right point in a loop or a conditional etc.

            And as a general rule, everything in Scripted Pipeline is available in Declarative (though outside of script blocks in Declarative, you can't do things like set/declare variables, use if statements, for loops, etc, but Declarative itself is built on top of Scripted, so you're not going to find steps that work in Scripted but not Declarative, etc), but not necessarily vice versa. That's the whole reason for Declarative - to be able to implement functionality and syntactic sugar that we can't do easily for Scripted because, well, it's a full-blown programming language and covering every possible use case scenario is just not viable.

            Show
            abayer Andrew Bayer added a comment - Dee Kryvenko - with Declarative, we can guarantee consistency in behavior. Everything but the actual steps will be executed for every single Declarative Pipeline in the same order and way. This makes it much easier to say "Re-run build #X, with the same parameters, SCM revision, checkouts, etc, but don't run any stages until the one we've specified", because we know, for example, how agents, environment variables, etc are handled outside of the actual stages themselves. That means we can treat each top-level stage in a Declarative Pipeline as an atomic entity to some extent, at least enough to know that we can pick back up where we left off without having to do weird Groovy hacks to ensure we're at exactly the right point in a loop or a conditional etc. And as a general rule, everything in Scripted Pipeline is available in Declarative (though outside of script blocks in Declarative, you can't do things like set/declare variables, use if statements, for loops, etc, but Declarative itself is built on top of Scripted, so you're not going to find steps that work in Scripted but not Declarative, etc), but not necessarily vice versa. That's the whole reason for Declarative - to be able to implement functionality and syntactic sugar that we can't do easily for Scripted because, well, it's a full-blown programming language and covering every possible use case scenario is just not viable.
            abayer Andrew Bayer made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            jamesdumay James Dumay made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * History - investigate and touch base with James on findings:
            ** Option 1: Need to investigate if we can reuse the Pipeline Run. For example, if we have a Pipeline "Build -> Test -> Deploy" and we restarted the "Test" stage, the same run is reused and Deploy stage would be set to "not built". This would lose the history of proceeding stages.
            ** Option 2: can we graft the information from the previous run to the new run?

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
            ** Implication is we do not have to copy flow nodes to the new run

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            jamesdumay James Dumay made changes -
            Link This issue blocks JENKINS-48643 [ JENKINS-48643 ]
            jamesdumay James Dumay made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
            ** Implication is we do not have to copy flow nodes to the new run

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
            ** Implication is we do not have to copy flow nodes to the new run

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Open questions*
            * Should we be keeping stash data for every run? What is the option of least surprise?
            ** JD thinks that keeping stashes around by default is the least surprising option - that way they do not have to turn anything on to make the feature work.
            ** AB says this might blow out the storage requirements

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            Hide
            jamesdumay James Dumay added a comment -

            For those who are watching this ticket, we have JENKINS-48643 to cover the UX for this in Blue Ocean and a few mockups you can view

            Show
            jamesdumay James Dumay added a comment - For those who are watching this ticket, we have JENKINS-48643 to cover the UX for this in Blue Ocean and a few mockups you can view
            Hide
            ljohnston Lance Johnston added a comment -

            After reading through this issue, reading up on blue ocean, the checkpoint plugin, etc., I'm still not clear on something...

            Will this feature allow for the creation of manual pipeline steps? For example, the pipeline runs up to the production deploy step, but does not invoked it automatically, instead waiting for someone to manually execute that step (without doing the silly "prompt" thing that just causes the pipeline to run forever)?

            Show
            ljohnston Lance Johnston added a comment - After reading through this issue, reading up on blue ocean, the checkpoint plugin, etc., I'm still not clear on something... Will this feature allow for the creation of manual pipeline steps? For example, the pipeline runs up to the production deploy step, but does not invoked it automatically, instead waiting for someone to manually execute that step (without doing the silly "prompt" thing that just causes the pipeline to run forever)?
            Hide
            stevenfoster Steven Foster added a comment -

            >Will this feature allow for the creation of manual pipeline steps?

            I'm interested in this too, I guess it's possible if the pipeline can be aware of whether it's a fresh run or a restarted run. If there's some kind of condition that works with when blocks, then your pipeline can skip certain stages (probably at the end) and complete as normal.

            I've also found the input step isn't a great solution for this pattern.

            Show
            stevenfoster Steven Foster added a comment - >Will this feature allow for the creation of manual pipeline steps? I'm interested in this too, I guess it's possible if the pipeline can be aware of whether it's a fresh run or a restarted run. If there's some kind of condition that works with when blocks, then your pipeline can skip certain stages (probably at the end) and complete as normal. I've also found the input step isn't a great solution for this pattern.
            Hide
            abayer Andrew Bayer added a comment -

            Something like that could be possible, yeah.

            Show
            abayer Andrew Bayer added a comment - Something like that could be possible, yeah.
            Hide
            jamesdumay James Dumay added a comment -

            Steven Foster mind creating a new issue that lays your use case out with examples? a ideal Jenkinsfile would be cool too

            Show
            jamesdumay James Dumay added a comment - Steven Foster mind creating a new issue that lays your use case out with examples? a ideal Jenkinsfile would be cool too
            Hide
            abayer Andrew Bayer added a comment -

            James Dumay Steven Foster - don't even bother, I just added a new when condition, isRestartedRun(), to the WIP PR for this that does exactly what it says on the tin. =)

            Show
            abayer Andrew Bayer added a comment - James Dumay Steven Foster - don't even bother, I just added a new when condition, isRestartedRun() , to the WIP PR for this that does exactly what it says on the tin. =)
            abayer Andrew Bayer made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
            ** Implication is we do not have to copy flow nodes to the new run

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Open questions*
            * Should we be keeping stash data for every run? What is the option of least surprise?
            ** JD thinks that keeping stashes around by default is the least surprising option - that way they do not have to turn anything on to make the feature work.
            ** AB says this might blow out the storage requirements

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
            ** Implication is we do not have to copy flow nodes to the new run
            * Add a {{when}} condition that will evaluate as true only when on a restarted run.

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Open questions*
            * Should we be keeping stash data for every run? What is the option of least surprise?
            ** JD thinks that keeping stashes around by default is the least surprising option - that way they do not have to turn anything on to make the feature work.
            ** AB says this might blow out the storage requirements

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            Hide
            jamesdumay James Dumay added a comment -

            Andrew Bayer nice. Lets make sure that it gets doc'd too

            Show
            jamesdumay James Dumay added a comment - Andrew Bayer nice. Lets make sure that it gets doc'd too
            Hide
            abayer Andrew Bayer added a comment -

            Yeah, the whole thing is gonna need docs. But I've added it to the scope in the ticket description to make sure we don't forget.

            Show
            abayer Andrew Bayer added a comment - Yeah, the whole thing is gonna need docs. But I've added it to the scope in the ticket description to make sure we don't forget.
            Hide
            stevenfoster Steven Foster added a comment -

            Andrew Bayer ah whoops, should have checked before submitting

            Show
            stevenfoster Steven Foster added a comment - Andrew Bayer ah whoops, should have checked before submitting
            Hide
            llibicpep Dee Kryvenko added a comment - - edited

            As to the manual pipeline step I feel like it's some misunderstanding happening there. The input step done outside of `node {}` is not really running, it's just the UI renders it as so. Initially I was, like every one else human being, feeling uncomfortable about the change (comparing to the old pipelines). But changing the status quo is not always a bad thing. Given that nothing really running in terms of consuming your resources, and it does not blocks your queue or nodes, and also thinking about your pipeline as a bit of wider process than just a piece of software - it is something really 'in progress' for the whole lifecycle up to prod or till overran by the newest progress (in combination with `milestones` plugin). This is literally represents WIP state and nature of the given version of the artifact - given that DoD is correctly set in your team (availability on prod). It's being built but not in prod yet - let's say it's in QA and some testing happening there - so the team are working on this version of artifact, so the WIP. Then all the interim results (older pipelines that never reached prod because of being aborted by `milestone`) - marked as failed because they are literally failed in terms of that DoD - they never made it to prod. So the new render makes a perfect sense to me.

            Show
            llibicpep Dee Kryvenko added a comment - - edited As to the manual pipeline step I feel like it's some misunderstanding happening there. The input step done outside of `node {}` is not really running, it's just the UI renders it as so. Initially I was, like every one else human being, feeling uncomfortable about the change (comparing to the old pipelines). But changing the status quo is not always a bad thing. Given that nothing really running in terms of consuming your resources, and it does not blocks your queue or nodes, and also thinking about your pipeline as a bit of wider process than just a piece of software - it is something really 'in progress' for the whole lifecycle up to prod or till overran by the newest progress (in combination with `milestones` plugin). This is literally represents WIP state and nature of the given version of the artifact - given that DoD is correctly set in your team (availability on prod). It's being built but not in prod yet - let's say it's in QA and some testing happening there - so the team are working on this version of artifact, so the WIP. Then all the interim results (older pipelines that never reached prod because of being aborted by `milestone`) - marked as failed because they are literally failed in terms of that DoD - they never made it to prod. So the new render makes a perfect sense to me.
            Hide
            akrapfl Adam Krapfl added a comment - - edited

            To extend Steven Foster's comment, would it be possible to create a new step that explicitly exits allowing for restart?  I'm horrible with names, but what if there was a:

            hibernate() //intentionally stop the build, allowing it to be resumed later

            I know it's just a variation on the normal error() syntax, but it makes more sense to read the code, and the UX wouldn't have to display as an error if it's a deliberate stop.

            Show
            akrapfl Adam Krapfl added a comment - - edited To extend Steven Foster's comment, would it be possible to create a new step that explicitly exits allowing for restart?  I'm horrible with names, but what if there was a: hibernate() //intentionally stop the build, allowing it to be resumed later I know it's just a variation on the normal error() syntax, but it makes more sense to read the code, and the UX wouldn't have to display as an error if it's a deliberate stop.
            Hide
            jamesdumay James Dumay added a comment -

            Adam Krapfl thats what you should use input 'Continue?' for. We are going to allow a stage to define an input without having consume an agent as part of JENKINS-48379.

            Show
            jamesdumay James Dumay added a comment - Adam Krapfl thats what you should use input 'Continue?' for. We are going to allow a stage to define an input without having consume an agent as part of JENKINS-48379 .
            Hide
            jamesdumay James Dumay added a comment -

            Dee Kryvenko you might be interested in reading JENKINS-48379. It will allow you to define input on a stage without consuming an agent.

            Show
            jamesdumay James Dumay added a comment - Dee Kryvenko you might be interested in reading JENKINS-48379 . It will allow you to define input on a stage without consuming an agent.
            Hide
            abayer Andrew Bayer added a comment -
            Show
            abayer Andrew Bayer added a comment - Forgot to link the work in progress PR - https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/228
            abayer Andrew Bayer made changes -
            Remote Link This issue links to "PR #228 (Web Link)" [ 20121 ]
            vivek Vivek Pandey made changes -
            Sprint Pipeline - December [ 446 ] Pipeline - March 2018 [ 506 ]
            Hide
            pgarbe Philipp Garbe added a comment -

            Are there any updates?

            Show
            pgarbe Philipp Garbe added a comment - Are there any updates?
            Hide
            deniskrizanovic denis krizanovic added a comment -

            This is a big feature for me. Considering the code is almost done, when might it hit the plugin? 

            Show
            deniskrizanovic denis krizanovic added a comment - This is a big feature for me. Considering the code is almost done, when might it hit the plugin? 
            Hide
            jbriden Jenn Briden added a comment -

            denis krizanovic, we are working through the user experience design now to be sure that we take various use cases into account. We have an offsite next week with the Blue Ocean and Pipeline teams, and one of our sessions is to define that UX. We'll then need to do that work. I cannot commit to a specific timeline for when it will be merged to the main Blue Ocean plugin, but we are working on it. I will be sure to update this ticket after our offsite, as well as the corresponding ticket for the UX.

            Show
            jbriden Jenn Briden added a comment - denis krizanovic , we are working through the user experience design now to be sure that we take various use cases into account. We have an offsite next week with the Blue Ocean and Pipeline teams, and one of our sessions is to define that UX. We'll then need to do that work. I cannot commit to a specific timeline for when it will be merged to the main Blue Ocean plugin, but we are working on it. I will be sure to update this ticket after our offsite, as well as the corresponding ticket for the UX.
            Hide
            wsaxon Will Saxon added a comment -

            So is this a Blue Ocean feature, or a Pipeline feature? I know I would be pleased to have restart capability even if it was outside Blue Ocean, even if it was just an API call.

            Show
            wsaxon Will Saxon added a comment - So is this a Blue Ocean feature, or a Pipeline feature? I know I would be pleased to have restart capability even if it was outside Blue Ocean, even if it was just an API call.
            Hide
            seanwcom Sean Williams added a comment -

            Thanks for the update Jenn. Will asks a good question. I hope this is a pipeline feature - we are using declarative, but not blue ocean. While Blue Ocean is very pretty and looks very promising, we can't switch until it's all encompassing. But we do need the restart functionality. Or perhaps I'm just confused and the pipeline rendering itself is part of Blue Ocean without even having BO enabled...

            Show
            seanwcom Sean Williams added a comment - Thanks for the update Jenn. Will asks a good question. I hope this is a pipeline feature - we are using declarative, but not blue ocean. While Blue Ocean is very pretty and looks very promising, we can't switch until it's all encompassing. But we do need the restart functionality. Or perhaps I'm just confused and the pipeline rendering itself is part of Blue Ocean without even having BO enabled...
            Hide
            abayer Andrew Bayer added a comment -

            Sean Williams Will Saxon - this will not be Blue Ocean-exclusive - there'll be a classic UI for it, you can interact with it via the REST API, etc. We're just holding up merging/releasing it because we want to make sure we've got the Blue Ocean user experience for stage restarting nailed down and on its way to completion beforehand, so that we know the Blue Ocean side is coming soon and we know that the Declarative side doesn't need to be changed post-release to work with the right approach in the Blue Ocean UX. Coordination is fun. =)

            (also we've got to get JENKINS-46809 merged before we can merge this, and that little bugger requires, at a minimum, updates to the Blue Ocean Pipeline editor to handle changes to the intermediate representation of a Declarative Pipeline or it just blows up horribly)

            Show
            abayer Andrew Bayer added a comment - Sean Williams Will Saxon - this will not be Blue Ocean-exclusive - there'll be a classic UI for it, you can interact with it via the REST API, etc. We're just holding up merging/releasing it because we want to make sure we've got the Blue Ocean user experience for stage restarting nailed down and on its way to completion beforehand, so that we know the Blue Ocean side is coming soon and we know that the Declarative side doesn't need to be changed post-release to work with the right approach in the Blue Ocean UX. Coordination is fun. =) (also we've got to get JENKINS-46809 merged before we can merge this, and that little bugger requires, at a minimum, updates to the Blue Ocean Pipeline editor to handle changes to the intermediate representation of a Declarative Pipeline or it just blows up horribly)
            Hide
            vaivars Klavs Dzelzs-Vaivars added a comment -

            Just to add - this really is a feature that we are looking forward to. Are there any potential estimates of how long 46809 might take to push through? 

            Show
            vaivars Klavs Dzelzs-Vaivars added a comment - Just to add - this really is a feature that we are looking forward to. Are there any potential estimates of how long 46809 might take to push through? 
            Hide
            jbriden Jenn Briden added a comment -

            Klavs Dzelzs-Vaivars, I cannot commit us to a specific date, except to say that it's our top priority right now.

            Show
            jbriden Jenn Briden added a comment - Klavs Dzelzs-Vaivars , I cannot commit us to a specific date, except to say that it's our top priority right now.
            Hide
            jbriden Jenn Briden added a comment -

            Sean Williams, will you please email me at jbriden@gmail.com so that we can discuss your feedback about the gaps that you see in the pipeline visualization UX that you see in Blue Ocean? I can also connect via the Jenkins IRC or gitter, if those are better for you.

            Show
            jbriden Jenn Briden added a comment - Sean Williams , will you please email me at jbriden@gmail.com  so that we can discuss your feedback about the gaps that you see in the pipeline visualization UX that you see in Blue Ocean? I can also connect via the Jenkins IRC or gitter, if those are better for you.
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal UX-733 (Web Link)" [ 20813 ]
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal UX-729 (Web Link)" [ 20814 ]
            Hide
            kutzi kutzi added a comment -

            Great feature!

            When it says 'Declarative pipeline only' - does it mean strictly only declarative pipeline feature or are script blocks in declarative pipelines supported as well?

            Show
            kutzi kutzi added a comment - Great feature! When it says 'Declarative pipeline only' - does it mean strictly only declarative pipeline feature or are script blocks in declarative pipelines supported as well?
            Hide
            jglick Jesse Glick added a comment -

            Should this be Resolved/Fixed?

            Show
            jglick Jesse Glick added a comment - Should this be  Resolved/Fixed ?
            ranjith_kumar ranjith kumar made changes -
            Description *Use cases*
            * Restarting a successful stage so to rollback or redeploy code
            * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
            * Declarative pipeline only
            * Restart the stage with the same parameters/scm/actions as the previous run
            * No management of workspace
            * Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
            ** Implication is we do not have to copy flow nodes to the new run
            * Add a {{when}} condition that will evaluate as true only when on a restarted run.

            *Out of scope*
            * Automatically stash/unstash the workspace
            * Scripted pipeline (technical limitation)

            *Open questions*
            * Should we be keeping stash data for every run? What is the option of least surprise?
            ** JD thinks that keeping stashes around by default is the least surprising option - that way they do not have to turn anything on to make the feature work.
            ** AB says this might blow out the storage requirements

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            *Use cases*
             * Restarting a successful stage so to rollback or redeploy code
             * When long running Pipelines fail intermittently for environmental purposes the developer must be able to restart the execution of the stage that failed within the Pipeline. This allows the developer to recoup time lost running the pipeline to the point of failure.

            *In scope*
             * Declarative pipeline only
             * Restart the stage with the same parameters/scm/actions as the previous run
             * No management of workspace
             * Need to be able to union the previous run and the current runs flownodes to produce a visualisation for Blue Ocean
             ** Implication is we do not have to copy flow nodes to the new run
             * Add a {{when}} condition that will evaluate as true only when on a restarted 

            *Out of scope*
             * Automatically stash/unstash the workspace
             * Scripted pipeline (technical limitation)

            *Open questions*
             * Should we be keeping stash data for every run? What is the option of least surprise?
             ** JD thinks that keeping stashes around by default is the least surprising option - that way they do not have to turn anything on to make the feature work.
             ** AB says this might blow out the storage requirements

            *Notes*
            There is less complexity and edge cases needed to be handled by making this functionality a Declarative Pipeline feature only.

            *Technical notes*
            Split off from JENKINS-33846 since [~abayer] removed the component from that one, and I do not want this lost:
            {quote}For Declarative Pipelines, in principle you could skip some early {{stage}}'s and just start running on a later one, using {{FlowCopier}}.
            {quote}
            This would be instead of using a {{checkpoint}} step, and would not be dependent on Groovy internals. While it is less flexible in some respects, it is no less flexible than it _could_ be given the restrictions Declarative already imposes. This behavior would more closely match what you get from, say, the Build Pipeline plugin, but with the benefits of Jenkins Pipeline: stashes, exact SCM revisions, etc.

            (Conversely, you could also offer the ability to run a build skipping some of the _later_ stages, somewhat analogously to running {{ant some-non-default-target}}.)
            Hide
            abayer Andrew Bayer added a comment -

            Whoops, forgot to close this. Released in Declarative 1.3.

            Show
            abayer Andrew Bayer added a comment - Whoops, forgot to close this. Released in Declarative 1.3.
            abayer Andrew Bayer made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            Hide
            jbriden Jenn Briden added a comment -

            kutzi, this applies to the top-level stage, so it's irrelevant whether there are script blocks within a stage. If the format is syntactically correct declarative, then this will work for you.

            We've added support in the declarative plugin. Blue Ocean support is now in development.

            Show
            jbriden Jenn Briden added a comment - kutzi , this applies to the top-level stage, so it's irrelevant whether there are script blocks within a stage. If the format is syntactically correct declarative, then this will work for you. We've added support in the declarative plugin. Blue Ocean support is now in development.
            Hide
            franknarf8_ni Frank Genois added a comment -

            Hello, is there a plan to support restarting from sub-stages?

            Let's say we have the following stages to build a multi-platform desktop application : 

             

            Build-Win64 -> Test-Win64 -| 
            Build-Win32 -> Test-Win32 -|-> Deploy
            Build-MacOS -> Test-MacOS -|
            Build-Linux -> Test-Linux -|

             

            Where all the 'Build-XXX' are running in parallel within a global 'Build' stage. (The Build&Test stages could take quite a long time)

            If let's say 'Build-Win32' fails for some unexpected environmental reasons, you do not want to rebuild the three other one. It would use unnecessary build server time. Same goes if 'Test-Win32' fails, you do not want to rebuild all the tests from the other platforms.

            I'm sorry if there is already a way to do so, and if it is the case, could you provide me some guidance on how to achieve such a behavior?

            Show
            franknarf8_ni Frank Genois added a comment - Hello, is there a plan to support restarting from sub-stages? Let's say we have the following stages to build a multi-platform desktop application :    Build-Win64 -> Test-Win64 -| Build-Win32 -> Test-Win32 -|-> Deploy Build-MacOS -> Test-MacOS -| Build-Linux -> Test-Linux -|   Where all the 'Build-XXX' are running in parallel within a global 'Build' stage. (The Build&Test stages could take quite a long time) If let's say 'Build-Win32' fails for some unexpected environmental reasons, you do not want to rebuild the three other one. It would use unnecessary build server time. Same goes if 'Test-Win32' fails, you do not want to rebuild all the tests from the other platforms. I'm sorry if there is already a way to do so, and if it is the case, could you provide me some guidance on how to achieve such a behavior?
            Hide
            shantur Shantur Rathore added a comment -

            Support for nested stages for parallel builds would be really cool. 

            Most of the projects I work on do the heavy lifting in parallel stages and restart the failed part from parallel stage would be perfect.

            Show
            shantur Shantur Rathore added a comment - Support for nested stages for parallel builds would be really cool.  Most of the projects I work on do the heavy lifting in parallel stages and restart the failed part from parallel stage would be perfect.
            Hide
            afolgado Andres Folgado added a comment - - edited

            Hi all,

            In my project I use a Singleton-annotated class to store some variables across all stages. This object is not preserved when restarting a stage.

            So, my question is: how can I persist some data during the execution so that I can get it later when restarting from a stage?

            Thanks in advance.

            Show
            afolgado Andres Folgado added a comment - - edited Hi all, In my project I use a Singleton-annotated class to store some variables across all stages. This object is not preserved when restarting a stage. So, my question is: how can I persist some data during the execution so that I can get it later when restarting from a stage? Thanks in advance.
            Hide
            webrat Andreas Sieferlinger added a comment -

            When restarting a stage, in the console the follwoing output is shown

            Restarted from build #4, stage foobar

            Is there any way to access this information within the restarted build? Eg via ENV variables or globals in the Jenkinsfile?

            Show
            webrat Andreas Sieferlinger added a comment - When restarting a stage, in the console the follwoing output is shown Restarted from build #4, stage foobar Is there any way to access this information within the restarted build? Eg via ENV variables or globals in the Jenkinsfile?
            Hide
            jbriden Jenn Briden added a comment -

            Frank Genois, the current design restarts at the top-level stage. Please open a separate ticket to request the ability to restart at a parallel stage. 

            Show
            jbriden Jenn Briden added a comment - Frank Genois , the current design restarts at the top-level stage. Please open a separate ticket to request the ability to restart at a parallel stage. 
            Hide
            abayer Andrew Bayer added a comment -

            Andres Folgado - no, sorry - we don't preserve any Groovy state across restarts. If you need to preserve some data, you'd probably want to write it to a file, stash it, and then unstash it in the later stage. Make sure you have preserveStashes() (for the default of preserving the most recent build's stashes - pass a higher number as the argument if you need to preserve more) in the options block of your pipeline block or the stash will be cleared.

            Show
            abayer Andrew Bayer added a comment - Andres Folgado - no, sorry - we don't preserve any Groovy state across restarts. If you need to preserve some data, you'd probably want to write it to a file, stash it, and then unstash it in the later stage. Make sure you have preserveStashes() (for the default of preserving the most recent build's stashes - pass a higher number as the argument if you need to preserve more) in the options block of your pipeline block or the stash will be cleared.
            Hide
            franknarf8_ni Frank Genois added a comment -

            Regarding restarting from sub-stages, I created the following issue : https://issues.jenkins-ci.org/browse/JENKINS-52391

            Please upvote if you are also looking forward to this feature.

             

            Show
            franknarf8_ni Frank Genois added a comment - Regarding restarting from sub-stages, I created the following issue : https://issues.jenkins-ci.org/browse/JENKINS-52391 Please upvote if you are also looking forward to this feature.  
            franknarf8_ni Frank Genois made changes -
            Link This issue is related to JENKINS-52391 [ JENKINS-52391 ]
            Hide
            bdevcich Blake Devcich added a comment - - edited

            Is there a way to always run a stage when restarting stages? Our pipeline requires a few things to be obtained dynamically in the beginning and when restarting stages we don't get that functionality.

            I think something like a pre section (like post) would be really helpful here.

            Example:

            Stage A -> Stage B -> Stage C -> Stage D

            I want to restart Stage C or Stage D but I want to ensure that Stage A always runs first.

             

            Show
            bdevcich Blake Devcich added a comment - - edited Is there a way to always run a stage when restarting stages? Our pipeline requires a few things to be obtained dynamically in the beginning and when restarting stages we don't get that functionality. I think something like a pre section (like post) would be really helpful here. Example: Stage A -> Stage B -> Stage C -> Stage D I want to restart Stage C or Stage D but I want to ensure that Stage A always runs first.  
            victorbjelkholm Victor Bjelkholm made changes -
            Link This issue is related to JENKINS-53730 [ JENKINS-53730 ]
            Hide
            hoppej Jan Hoppe added a comment -

            I might be wrong, but I don't get how to switch off the restart stage option.

            My pipelines currently don't work correctly after restarting a stage (I check out git only at first stage...)

            Show
            hoppej Jan Hoppe added a comment - I might be wrong, but I don't get how to switch off the restart stage option. My pipelines currently don't work correctly after restarting a stage (I check out git only at first stage...)
            Hide
            abayer Andrew Bayer added a comment -

            There currently isn't a way to disable stage restart, but that's a good idea for us to add - could you create a new ticket for that? Thanks!

            Show
            abayer Andrew Bayer added a comment - There currently isn't a way to disable stage restart, but that's a good idea for us to add - could you create a new ticket for that? Thanks!
            hoppej Jan Hoppe made changes -
            Link This issue is related to JENKINS-54250 [ JENKINS-54250 ]
            Hide
            ben5556 Subbu K added a comment -

            Blake Devcich I am looking for exactly what you asked. Have you figured out a way to achieve this ? Thanks!

            Show
            ben5556 Subbu K added a comment - Blake Devcich I am looking for exactly what you asked. Have you figured out a way to achieve this ? Thanks!
            Hide
            jazzyjayx Jay Spang added a comment -

            Jan Hoppe Andrew Bayer please update this thread if you created a ticket to disable stage restart. We desperately need this.

            This feature does not work at all with my pipelines (it does not run on the same node as the original run), so it's generating false positives and wreaking havoc with my customers. I'm considering implementing some custom CSS to hide the silly thing before it causes us any more grief!

            Show
            jazzyjayx Jay Spang added a comment - Jan Hoppe Andrew Bayer please update this thread if you created a ticket to disable stage restart. We desperately need this. This feature does not work at all with my pipelines (it does not run on the same node as the original run), so it's generating false positives and wreaking havoc with my customers. I'm considering implementing some custom CSS to hide the silly thing before it causes us any more grief!
            Hide
            bdevcich Blake Devcich added a comment -

            Subbu K - No, we haven't found a way around this, so we're not taking advantage of stage restarting. I have use boolean parameters which can be unchecked to skip stages - which works as a sort of poor mans stage restarting.

            To expand on my previous example, I have check boxes as Job parameters for Stage B, C, D. I always want Stage A to run so there's no check box for that. If I want to restart Stage D, I uncheck the boxes for Stages B, C. Each stage in the pipeline checks the true/false value before running the stage.

            It's not ideal, and it's not really taking advantage of this feature, but it works pretty well.

            I also did create https://issues.jenkins-ci.org/browse/JENKINS-53167, so you might want to watch that.

            Show
            bdevcich Blake Devcich added a comment - Subbu K - No, we haven't found a way around this, so we're not taking advantage of stage restarting. I have use boolean parameters which can be unchecked to skip stages - which works as a sort of poor mans stage restarting. To expand on my previous example, I have check boxes as Job parameters for Stage B, C, D. I always want Stage A to run so there's no check box for that. If I want to restart Stage D, I uncheck the boxes for Stages B, C. Each stage in the pipeline checks the true/false value before running the stage. It's not ideal, and it's not really taking advantage of this feature, but it works pretty well. I also did create https://issues.jenkins-ci.org/browse/JENKINS-53167 , so you might want to watch that.
            Hide
            gc Garbageyard added a comment - - edited

            Blake Devcich: I have also voted for the ticket JENKINS-53167. For the workaround that you have in place, have you tied the job to a particular slave and use the same workspace each time the job runs? I am asking this because if a job ran the build and test phases successfully and then failed during packaging phase, how will you restart job and ensure that it picks up the already built binary for packaging phase.

             

            I am also looking for running a particular phase each time (or may be having the option to run it whenever i want) irrespective of whichever phase i restart my job from. I have included my use case in your ticket JENKINS-53167

            Show
            gc Garbageyard added a comment - - edited Blake Devcich : I have also voted for the ticket  JENKINS-53167 . For the workaround that you have in place, have you tied the job to a particular slave and use the same workspace each time the job runs? I am asking this because if a job ran the build and test phases successfully and then failed during packaging phase, how will you restart job and ensure that it picks up the already built binary for packaging phase.   I am also looking for running a particular phase each time (or may be having the option to run it whenever i want) irrespective of whichever phase i restart my job from. I have included my use case in your ticket  JENKINS-53167
            Hide
            bdevcich Blake Devcich added a comment -

            Garbageyard: Yes, in this particular case, we have it tied to a slave for hardware dependency reasons.

             

            Show
            bdevcich Blake Devcich added a comment - Garbageyard : Yes, in this particular case, we have it tied to a slave for hardware dependency reasons.  
            gc Garbageyard made changes -
            Comment [ Thanks for the response [~bdevcich]. Hardware dependency can definitely be one reason but will tying a job to a slave also _ensure_ that subsequent builds will use the same workspace which was used by last failed build? ]
            Hide
            bitwiseman Liam Newman added a comment -

            Bulk closing resolved issues.

            Show
            bitwiseman Liam Newman added a comment - Bulk closing resolved issues.
            bitwiseman Liam Newman made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

              People

              Assignee:
              abayer Andrew Bayer
              Reporter:
              jglick Jesse Glick
              Votes:
              63 Vote for this issue
              Watchers:
              110 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: