Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52391

Restarting Parallel Stages

    XMLWordPrintable

Details

    • Declarative backlog

    Description

      This issue is based on JENKINS-45455 which solves the issue of restarting a Jenkins Pipeline from Top-Level stages.

      Description
      When building on multiple platforms, the builds are usually ran in parallel. We should be able to replay the build of a single platform. (e.g. if the Win64 build fails, we do not want to retrigger the already successful Macos and Linux builds)
      In a multiplatform build pipeline, the steps taking the most time are the "build" steps themselves and they are running in parallel to get fast build times. Restarting from a top level in that context boils down to re-running the whole job.

      Use cases

      • Restarting a Jenkins Pipeline from a sub-stage or parallel stage
      • Given the following graph, we should be able to replay from any stage (e.g. Build-Win64, Test-MacOs, Deploy) :
        Build-Win64 -> Test-Win64 -| 
        Build-Win32 -> Test-Win32 -|-> Deploy
        Build-MacOS -> Test-MacOS -|
        Build-Linux -> Test-Linux -|

         

      Attachments

        Issue Links

          Activity

            gabih Can you confirm that the preserveStashes option will be honored by any stash/unstash called from within the scripted block? Currently it seems like those stashes are not preserved on our Jenkins instance when calling stash from within the script block. But maybe we just need an update of Jenkins or some plugin...

            macdrega Joerg Schwaerzler added a comment - gabih Can you confirm that the preserveStashes option will be honored by any stash/unstash called from within the scripted block? Currently it seems like those stashes are not preserved on our Jenkins instance when calling stash from within the script block. But maybe we just need an update of Jenkins or some plugin...

            macdrega preserveStashes(/buildCount: 5/) works if the pipeline is started form the same Jenkins instance

            gabih Gabriel Herisanu added a comment - macdrega  preserveStashes(/ buildCount: 5 /) works if the pipeline is started form the same Jenkins instance

            What I did not realize is that preserverStashes will only work when restarting from some stage... Looks better now - thanks.

            macdrega Joerg Schwaerzler added a comment - What I did not realize is that preserverStashes will only work when restarting from some stage... Looks better now - thanks.
            zaktaccardi Zak Taccardi added a comment -

            Is the matrix feature blocked by a lack of support for restarting here?

            zaktaccardi Zak Taccardi added a comment - Is the matrix feature blocked by a lack of support for restarting here?

            My version of workaround built on solution from gabih. Because stash can preserve to maximum of 50 builds I've prepared it on Archive artifacts and additionally it doesn't keep a stash in memory. It also requires Copy Artifact plugin - https://plugins.jenkins.io/copyartifact/ 

             def withCheck(String blockName, Closure closure) {
                script {
                    def buildStage = true
                    catchError(message: 'check previous build status', stageResult:'SUCCESS', buildResult: 'SUCCESS') {
                        copyArtifactsFromRestartedBuild(context.env.JOB_NAME, context.env.BUILD_NUMBER, "stageStates/${blockName}")
                        buildStage = false
                    }
            
                    if (buildStage) {
                        closure.call()
                        writeFile file: "${blockName}", text: "1"
                        archiveArtifacts artifacts: "stageStates/${blockName}"
                    }
                }
            }
            
            def getBuildActions(String jobName, int buildNumber) {
               return Jenkins.instance.getItemByFullName(jobName).getBuildByNumber(buildNumber).properties.actions
            }
            
            def filterBuildAction(def buildActions, String filter = "RestartFlowFactoryAction") {
               return buildActions.findAll { it.getClass().toString().contains(filter) }
            }
            
            String getRestartedBuildData(String jobName, int buildNumber) {
               def buildActions = getBuildActions(jobName, buildNumber)
               def filteredAction = filterBuildAction(buildActions, "RestartFlowFactoryAction")
               if (filteredAction.size() == 1) {
                  return filteredAction[0].getOriginRunId()
               } else {
                  return null
               }
            }
            
            void copyArtifactsFromRestartedBuild(String jobName, String buildNumber, String fileFilter) {
               String restartedBuildData = getRestartedBuildData(jobName, buildNumber as Integer)
               String restartedJobName = restartedBuildData.split("#")[0]
               String restartedBuildNumber = restartedBuildData.split("#")[-1]
               copyArtifacts(filter: fileFilter, projectName: restartedJobName, selector: context.specific("${restartedBuildNumber}"))
            }

             

            fipciu1996 Mateusz Filipek added a comment - My version of workaround built on solution from gabih . Because stash can preserve to maximum of 50 builds I've prepared it on Archive artifacts and additionally it doesn't keep a stash in memory. It also requires Copy Artifact plugin - https://plugins.jenkins.io/copyartifact/   def withCheck( String blockName, Closure closure) { script { def buildStage = true catchError(message: 'check previous build status' , stageResult: 'SUCCESS' , buildResult: 'SUCCESS' ) { copyArtifactsFromRestartedBuild(context.env.JOB_NAME, context.env.BUILD_NUMBER, "stageStates/${blockName}" ) buildStage = false } if (buildStage) { closure.call() writeFile file: "${blockName}" , text: "1" archiveArtifacts artifacts: "stageStates/${blockName}" } } } def getBuildActions( String jobName, int buildNumber) { return Jenkins.instance.getItemByFullName(jobName).getBuildByNumber(buildNumber).properties.actions } def filterBuildAction(def buildActions, String filter = "RestartFlowFactoryAction" ) { return buildActions.findAll { it.getClass().toString().contains(filter) } } String getRestartedBuildData( String jobName, int buildNumber) { def buildActions = getBuildActions(jobName, buildNumber) def filteredAction = filterBuildAction(buildActions, "RestartFlowFactoryAction" ) if (filteredAction.size() == 1) { return filteredAction[0].getOriginRunId() } else { return null } } void copyArtifactsFromRestartedBuild( String jobName, String buildNumber, String fileFilter) { String restartedBuildData = getRestartedBuildData(jobName, buildNumber as Integer ) String restartedJobName = restartedBuildData.split( "#" )[0] String restartedBuildNumber = restartedBuildData.split( "#" )[-1] copyArtifacts(filter: fileFilter, projectName: restartedJobName, selector: context.specific( "${restartedBuildNumber}" )) }  

            People

              Unassigned Unassigned
              franknarf8_ni Frank Genois
              Votes:
              52 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated: