Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52391

Restarting Parallel Stages

    XMLWordPrintable

    Details

    • Similar Issues:
    • Sprint:
      Declarative backlog

      Description

      This issue is based on JENKINS-45455 which solves the issue of restarting a Jenkins Pipeline from Top-Level stages.

      Description
      When building on multiple platforms, the builds are usually ran in parallel. We should be able to replay the build of a single platform. (e.g. if the Win64 build fails, we do not want to retrigger the already successful Macos and Linux builds)
      In a multiplatform build pipeline, the steps taking the most time are the "build" steps themselves and they are running in parallel to get fast build times. Restarting from a top level in that context boils down to re-running the whole job.

      Use cases

      • Restarting a Jenkins Pipeline from a sub-stage or parallel stage
      • Given the following graph, we should be able to replay from any stage (e.g. Build-Win64, Test-MacOs, Deploy) :
        Build-Win64 -> Test-Win64 -| 
        Build-Win32 -> Test-Win32 -|-> Deploy
        Build-MacOS -> Test-MacOS -|
        Build-Linux -> Test-Linux -|

         

        Attachments

          Issue Links

            Activity

            Hide
            macdrega Joerg Schwaerzler added a comment -

            Gabriel Herisanu Can you confirm that the preserveStashes option will be honored by any stash/unstash called from within the scripted block? Currently it seems like those stashes are not preserved on our Jenkins instance when calling stash from within the script block. But maybe we just need an update of Jenkins or some plugin...

            Show
            macdrega Joerg Schwaerzler added a comment - Gabriel Herisanu Can you confirm that the preserveStashes option will be honored by any stash/unstash called from within the scripted block? Currently it seems like those stashes are not preserved on our Jenkins instance when calling stash from within the script block. But maybe we just need an update of Jenkins or some plugin...
            Hide
            gabih Gabriel Herisanu added a comment -

            Joerg Schwaerzler preserveStashes(/buildCount: 5/) works if the pipeline is started form the same Jenkins instance

            Show
            gabih Gabriel Herisanu added a comment - Joerg Schwaerzler  preserveStashes(/ buildCount: 5 /) works if the pipeline is started form the same Jenkins instance
            Hide
            macdrega Joerg Schwaerzler added a comment -

            What I did not realize is that preserverStashes will only work when restarting from some stage... Looks better now - thanks.

            Show
            macdrega Joerg Schwaerzler added a comment - What I did not realize is that preserverStashes will only work when restarting from some stage... Looks better now - thanks.
            Hide
            zaktaccardi Zak Taccardi added a comment -

            Is the matrix feature blocked by a lack of support for restarting here?

            Show
            zaktaccardi Zak Taccardi added a comment - Is the matrix feature blocked by a lack of support for restarting here?
            Hide
            fipciu1996 Mateusz Filipek added a comment -

            My version of workaround built on solution from Gabriel Herisanu. Because stash can preserve to maximum of 50 builds I've prepared it on Archive artifacts and additionally it doesn't keep a stash in memory. It also requires Copy Artifact plugin - https://plugins.jenkins.io/copyartifact/ 

             def withCheck(String blockName, Closure closure) {
                script {
                    def buildStage = true
                    catchError(message: 'check previous build status', stageResult:'SUCCESS', buildResult: 'SUCCESS') {
                        copyArtifactsFromRestartedBuild(context.env.JOB_NAME, context.env.BUILD_NUMBER, "stageStates/${blockName}")
                        buildStage = false
                    }
            
                    if (buildStage) {
                        closure.call()
                        writeFile file: "${blockName}", text: "1"
                        archiveArtifacts artifacts: "stageStates/${blockName}"
                    }
                }
            }
            
            def getBuildActions(String jobName, int buildNumber) {
               return Jenkins.instance.getItemByFullName(jobName).getBuildByNumber(buildNumber).properties.actions
            }
            
            def filterBuildAction(def buildActions, String filter = "RestartFlowFactoryAction") {
               return buildActions.findAll { it.getClass().toString().contains(filter) }
            }
            
            String getRestartedBuildData(String jobName, int buildNumber) {
               def buildActions = getBuildActions(jobName, buildNumber)
               def filteredAction = filterBuildAction(buildActions, "RestartFlowFactoryAction")
               if (filteredAction.size() == 1) {
                  return filteredAction[0].getOriginRunId()
               } else {
                  return null
               }
            }
            
            void copyArtifactsFromRestartedBuild(String jobName, String buildNumber, String fileFilter) {
               String restartedBuildData = getRestartedBuildData(jobName, buildNumber as Integer)
               String restartedJobName = restartedBuildData.split("#")[0]
               String restartedBuildNumber = restartedBuildData.split("#")[-1]
               copyArtifacts(filter: fileFilter, projectName: restartedJobName, selector: context.specific("${restartedBuildNumber}"))
            }

             

            Show
            fipciu1996 Mateusz Filipek added a comment - My version of workaround built on solution from Gabriel Herisanu . Because stash can preserve to maximum of 50 builds I've prepared it on Archive artifacts and additionally it doesn't keep a stash in memory. It also requires Copy Artifact plugin - https://plugins.jenkins.io/copyartifact/   def withCheck( String blockName, Closure closure) { script { def buildStage = true catchError(message: 'check previous build status' , stageResult: 'SUCCESS' , buildResult: 'SUCCESS' ) { copyArtifactsFromRestartedBuild(context.env.JOB_NAME, context.env.BUILD_NUMBER, "stageStates/${blockName}" ) buildStage = false } if (buildStage) { closure.call() writeFile file: "${blockName}" , text: "1" archiveArtifacts artifacts: "stageStates/${blockName}" } } } def getBuildActions( String jobName, int buildNumber) { return Jenkins.instance.getItemByFullName(jobName).getBuildByNumber(buildNumber).properties.actions } def filterBuildAction(def buildActions, String filter = "RestartFlowFactoryAction" ) { return buildActions.findAll { it.getClass().toString().contains(filter) } } String getRestartedBuildData( String jobName, int buildNumber) { def buildActions = getBuildActions(jobName, buildNumber) def filteredAction = filterBuildAction(buildActions, "RestartFlowFactoryAction" ) if (filteredAction.size() == 1) { return filteredAction[0].getOriginRunId() } else { return null } } void copyArtifactsFromRestartedBuild( String jobName, String buildNumber, String fileFilter) { String restartedBuildData = getRestartedBuildData(jobName, buildNumber as Integer ) String restartedJobName = restartedBuildData.split( "#" )[0] String restartedBuildNumber = restartedBuildData.split( "#" )[-1] copyArtifacts(filter: fileFilter, projectName: restartedJobName, selector: context.specific( "${restartedBuildNumber}" )) }  

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              franknarf8_ni Frank Genois
              Votes:
              48 Vote for this issue
              Watchers:
              33 Start watching this issue

                Dates

                Created:
                Updated: