jguigui this is related to a bunch of problems around sophisticated Pipelines.

I am in the process of drafting a RFC to rationalise parallel usage.

For example, in Declarative 1.2 we've introduced the following syntax called "Parallel Stages"

stage('foo') {
    parallel {
        stage('Firefox') {
            steps {
                echo "First branch"
            }
        }
        stage('Chrome') {
            steps {
                echo "Second branch"
            }
        }
    }
}

The advantage here is that we now have different syntaxes for parallel execution and parallel stages.

For reference the current syntax looks like:

stage('foo') {
    parallel(
		'copy-1': {
			... copy some files ...
		},
		'copy-2': {
			... copy some files ...
		},
		'some other process': {
			 parallel(
				'branch-1': {
					... run process 1 ...
				},
				'branch-2': {
					... run process 2 ...
				}
			)
		}
	)
}

With the syntax above its possible to next parallels infinitely and this is something Blue Ocean isn't likely to support.

What I would like to do is:

• Make the new Parallel Stage syntax the only way to visualise parallel
• Parallel executions get folded into the nearest "stage" and their steps appear in the step listing according to their execution order.

This is still some time away but it would cover the case you have outlined here.

Let me know what you think

Hi James,

I think there are two orthogonal concepts here:

• the parallel branches used to run in parallel (on different cpu for instance) several tasks
• the stage used for display purpose

To be flexible and cover all the use cases, we need to have these two concept independent.
Typically we may want:

1. in a same branch have sequential stages
2. in a same branch have nested stages
3. having branch without stage (aka no display)

Looks like your RFC is covering cases 1. and 2. but not 3.

Today limitation with current implementation are:

• the parallel branch is creating a stage with this branch name as the stage name
• branches with no stages are displayed (linked to first point here above)
• nested stages are not displayed

Having the branch name (used in the log) different from the stage name (used in Blue Ocean display) is useful too.

Regarding the representation of nested stages I like your suggestion. There is similar discussion on JENKINS-38442.

Last point I can think about, is how to represent stage with same name in a parallel step. Typically when lot of branches are generated by a for loop. I'm not sure of the best option:

• either aggregate them in one display (with eventually a badge indicating the number of stages with same name)
• either display each of them

Jean-Paul

jguigui

having branch without stage (aka no display)

These will still need to be in a stage but you would not see the "parallels" in the visualisation. You need the enclosing stage for us to display the steps within the parallel.

In your example, if we made these changes, your visualisation would look like:

It is also highly unlikely we will be ever able to visualise infinitely nested parallels. This proposal would not attempt to address that use case.

Thanks for the response

Hi James,

If I understood properly, you mean that the following pipeline:

stage("begin") { println "begin" }

stage('stage_parallel') {
    parallel {
        stage('stage_p1') {
            steps {
                echo "First branch"
            }
        }
        stage(''stage_p2') {
            steps {
                echo "Second branch"
            }
        }
    }
}

stage("end") { println "end" }

will not display the nested stage_p1 and stage_p2 and generate this:

jguigui that is correct

jguigui actually, I spoke too soon. This pipeline:

stage("begin") { println "begin" }

stage('stage_parallel') {
    parallel {
        'stage_p1': {
                echo "First branch"
        }
         'stage_p2': {
                echo "Second branch"
        }
    }
}

stage("end") { println "end" }

Would produce:

And the following pipeline with some branches not inside any stage:

stage("begin") { println "begin" }

stage('stage_parallel') {
    parallel {
        stage('stage_p1') {
            steps {
                echo "First branch"
            }
        }
        stage('stage_p2') {
            steps {
                echo "Second branch"
            }
        }
        'invisible_branch_p3': {
                echo "Third branch"
        }
        'invisible_branch_p4': {
                echo "Fourth branch"
        }
    }
}

stage("end") { println "end" }

would produce with the proposed RFC:

If it's the case, I've my use cases covered.

Agree with you, multiple nested case will be difficult to display.

Thanks a lot,

Jean-Paul

jguigui you wouldn't be able to put both parallel syntaxes within the same stage with the proposal as it stands

I hope you may reconsider the RFC. Definitely by splitting the execution (branch) and the display (stage) concepts, users will be given a much better flexibility. 

jguigui its still in the planning stages so I will consider when driving a decision on this one

Great ! Thank you very much 

Jean-Paul

Merging this within JENKINS-47799 as it will likely be the solution we are heading towards.