Here's a use case that might help with design.
Consider a typical pipeline that build an artifact[s] and deploys it to sequential environments (let's say Dev and QA for the sake of example).
Consider that you might want to deploy to multiple locations, for instance - to aws, azure and gcp. These are parallel independent worlds, so environments progression would be happening in parallel there.
Consider that within each location, you also want to deploy into multiple regions. Regions are also isolated independent worlds, so they would have to be deployed in parallel within it's location (aka aws, gcp, azure etc).
That creates at least two levels of nesting, but generally you can extrapolate this idea, for instance to deploy to each AZ separately, or maybe - deploy to multiple groups of locations or something like that, I do not see why there should be a limit of just two.
Anyway, here is a scripted pipeline representation of that use case (just two levels for the sake of simplicity):
stage('Realms') {
parallel 'aws': {
stage('Regions (aws)') {
parallel 'us-east-1 (aws)': {
stage('Dev (aws us-east-1)') {
echo 'aws us-east-1 dev'
}
stage('QA (aws us-east-1)') {
input message: 'Approve aws us-east-1 qa?', ok: 'Approve aws us-east-1 qa!'
echo 'aws us-east-1 qa'
}
}, 'us-west-2 (aws)': {
stage('Dev (aws us-west-2)') {
echo 'aws us-west-2 dev'
}
stage('QA (aws us-west-2)') {
input message: 'Approve aws us-west-2 qa?', ok: 'Approve aws us-west-2 qa!'
echo 'aws us-west-2 qa'
}
}, failFast: false
}
}, 'gcp': {
stage('Regions (gcp)') {
parallel 'us-east-1 (gcp)': {
stage('Dev (gcp us-east4)') {
echo 'gcp us-east4 dev'
}
stage('QA (gcp us-east4)') {
input message: 'Approve gcp us-east4 qa?', ok: 'Approve gcp us-east4 qa!'
echo 'gcp us-east4 qa'
}
}, 'us-west-2 (gcp)': {
stage('Dev (gcp us-west4)') {
echo 'gcp us-west4 dev'
}
stage('QA (gcp us-west4)') {
input message: 'Approve gcp us-west4 qa?', ok: 'Approve gcp us-west4 qa!'
echo 'gcp us-west4 qa'
}
}, failFast: false
}
}, failFast: false
}
Also consider a deployment stage itself (currently represented with just one simple echo step) - it will most likely have a lot of low level steps such as sh and so on. From the pipeline user perspective - it doesn't makes any sense to see all that implementation details. The user will want to see aggregation of steps at some level of abstractions, for instance if deployment is implemented with Terraform - the user will want to see things like plan, display diff and apply. The user might want to dig into any of these abstractions to the level of actual steps, of course, so that should be an expandable option.
I think implementing more than two levels of nesting using a use case from above will also solve that second use case.
Having multiple levels of parallel stages would optimize the use of my executors and significantly reduce my pipeline times.
This would be very helpful to me for stages that create multiple platform specific artifacts. When I do this now, I have to have one matrix stage that builds Windows and Linux packages (using creative use of Dockerfile agents, I'm able to get these to run in parallel), and one stage dedicated to building Mac packages. Because matrix stages seem to use up a parallel stage under the hood, I can only run these stages one after another, even though the physical servers could easily run these steps independently .