mbadran thanks for reporting this. Do you have a concrete use case you could share with us for nested stages?

Well currently I'm using it to be able to be able to have a like a container for the shell script in order to know what this shell script is doing since i'm not able to add a description to the shell script that will show up next to Shell Script in blueocean, as well as pipeline visualization. My ideal solution in here is to have availability to add description next to shell script so if i'm running multiple shell scripts in one stage i'd be able to easily know what is that shell script doing. Currently there are no way to do it as i understand so I've tried using nested stages to be able to separate the shell scripts and know what they do. Usage of nested stages like this is a workaround for lack of ability to add description for the shell script.

Regarding a concrete use case of using nested stages would be being able to relate multiple stages to one stage, for example if I've a major step which prepares the environment, and that preparation consists of fewer stages for example 1- cleaning old data, 2- copy new data, 3- prepare area to run builds. My ideal look for that would be a parent stage which is called Prepare environment, and then when pressing on that node it would fork 3 other nodes (which are the 3 stages I've mentioned), and when you select any of the inner stages you'll be able to get the shell scripts and any other pipeline commands you were doing in such stage.

mbadran so for now it shows up as a log prefix which helps you a bit - but ideally it would show up as a labelled step (or group of scripts, that are labelled as one thing you can "expand" into?)

Perhaps we need to draw out what the options could be for this.

But to me the main thing is you need to label a step, or group together a bunch of steps under a label... I think? Which implies that you really want stages to be the top level things, and when you drill down into them, you see groupings of labelled steps - woudl that be fair to say?

michaelneale You got it correctly Michael that i need to have label to a step (if possible we can make that label applicable on any kind of step not just a shell script), or group together bunch of steps under a label. This implies what you've mentioned as well that i need to have a top level stage that i can drill down into it to see grouping of labeled steps, or i see in the log a step next to it a label in order to know what is that step doing.

In my real use case I could have for example like 16 shell scripts running under one stage in parallel to build variant platforms, looking at this in blueocean, or pipeline view would show me 16 shell scripts and that's it which makes it pretty hard for developers to know which build actually failed, unless they go to pipeline steps to know the stage name and they figure out which build actually failed, or they've to do a full log view in order to find out the command that was used for a build.

mbadran well, some slight good news: there is a proposal that more information about scripts is returned so instead of saying "shell script" it can show a snippet of the script itself (just a little bit) or the output if run (that isn't being done as part of blue ocean project though, as deeper changes are needed) - so that helps a bit.

But to clarify, this is clearer. So you are happy with a stage graph at a high level, up the top, but need ways to group multiple steps into one, and label them (down the bottom). Maybe nested stage is useful for that, but not sure. To me nested stage implies a nesting of the visual graph, but we will see...

jamesdumay perhaps this is a case for labelled steps of some kind?

michaelneale The stage graph i think should be enhanced as well to give any indication that the the node actually is formed from multi-stages for example you can have a node with double circles around it to give a visual indicator that something is different in this stage.

For example in the code I've given earlier if you actually uncomment //stage 'start' it will show in the graph something like this if node "a" and node "b" have double rings around them then this could mean to the user that something is different in that stage. I think to support nested stages maybe when clicking on a node that is formed of multistage you can remove the original high level graph and just zoom in the small graph only so in my case when i press on node "a" that would remove the original graph, and show new graph of "a" as start then "ab", and "xyz" as children for it, and if we've further down stages you can always clean the visualization and show just all the previous nodes as a line formed having multi nodes in there, and only the deepest stage you should start to expand it. Once the user finishes there could be a reset button to reset the graphs to the parent, or they just press on first node. (This is just an proposal, but the simplest would be just a different indicator for a node that is formed of multi stages).

Regarding the bottom part I'd like in the attached image to see the first shell script under a/ab, and the second shell script under a/xyz so it could be something like this

steps a/ab
shell script

steps a/xyz
shell script

These are all thoughts to try to help in how to visualize things but definitely there could be other ways. To me for example i'd want something like
shell script : (This is platform 1)
shell script: (This is platform 2)

So yes i need something either in the visualization part, or in the lower level that would show that nesting in someway and show the grouping, and preferably both parts. But i just need to give me any means of feedback of what's happening.

One way we could make this work is:

Where we have a Pipeline like

  stage ("Build") {
    stage ("Inner 1") {
       ....
    }
    stage ("Inner 2") {
        ....
    }
  }

Then again I've heard use cases where these become boxes around stages in the pipeline graph or lines connecting stages together within stages.... need to go back to the Pipeline developers and get guidance around what the intended use case was here.

jamesdumay and michaelneale

Actually that last comment image looks good, and it actually follows one of my suggestions as well so that will be good enhancement. One point as you mentioned in the upper graph to add like any visualization indicator that this is a nested stage as a box, or as anything else.

Im still hesitant because we have no idea what pipelines like this should look like

stage ("outer") {
    stage ("inner") {
       stage ("most inner") {
          stage ("turtles all the way down") {
             ...
          }
       }
    }
  }

Right - are there nested stages in a visual stage graph, or are they grouping of steps? if the latter, then there is an assumption that nested stages are represented that way that isn't obvious from the config.

I don't think anyone has clearly asked for nested stages (given no one can explain what they are meant to be). What has been asked for is a way to label a step, or group steps under a label. nested stages does not achieve this clearly.

michaelneale

Nested stages doesn't do the the label part but it is a workaround, however nested stages could be needed in cases as I've mentioned earlier. Visual graph(I'm talking about Cloudbees one not blueocean upper graph) doesn't understand any nesting each stage will always show each stage as a separate block which is really ugly since you never know that this step is related to anything.

jamesdumay

Having nested stages could still be expanded in a flat way. This could happen the same way you did it previously but add like a path to it so that would look like:

Steps Outer
Inner Stage inner
Inner Stage inner/most inner

I believe by this way users will always be able to see things clearly

I think the reason many are wanting to even have nested stages is to help solve this problem:

We would like to give these shell scripts a more descriptive name but are unable to.

kdawgwilk exactly - that is exactly what I was thinking. Do you want to label those steps explicitly, or if they showed a more helpful description (say, the first N chars of the script) would that help?

I would like to have a syntax in the Jenkinfile to be able to say:

sh([cmd: 'script.sh', name: 'Deploy script']) 

or maybe something more generic that would work for all build steps?

step('Deploy script') {
    sh 'script.sh'
}

Even the name of the script being called would help eg.

sh 'script.sh'

if the step was displayed as "script.sh" it would be much better than it is right now

kdawgwilk good to know. Yes showing something meaningful certainly should happen no matter what. Labelling is also very nice.

jdumay if we get labelled steps (via any means) probably can close this.

IMO you should stop showing individual steps at all (display the complete output per block), and just let stage be used to show substructure in the natural way.

alanwu mentioned on the mailing list he would like to represent a Pipeline like the following:

Says if I now have the parallel block within a stage called "Build and test" and add in a "deploy" stage after it,
here is my two cents on how I imagine my use case should look. 

                         / Slave1 -- configure -- build -- test \
  Build and test - Slave2 -- configure -- build -- test - deploy
                         \ Slave3 -- configure -- build -- test /

jdumay so would that could be something like:

stage('build and test') {
   parallel('Slave1' : {
       stage('configure') {
       }
       stage('build') {
       }
       stage('test') {
       }
   }
}

stage('deploy') {

}

Given that everything needs to happen under a parent stage. The nested stages in parallel are the different groups of steps for configure etc.

michaelneale yes I'd say that syntax looks about right for what alanwu is proposing. Though it gets ugly with multiple parallel branches as you have to repeat the stage structure. Perhaps we need to come up with a specialised syntax to support this structure? Worth exploring IMO cc abayer

Hi
I am very concerned by the same issue:
for me in BlueOcean, only stages, but all stages, should be visible with Green balls.
branch names (SlaveX) should be a kind of label above the arrow but not a green ball by itself.
Therefore, if a branch has no label, then you would see just a line (with its label).
If a branch has a serie of N stages, then you would see those stages in that line (with its label), as Alan Wu proposed, I guess.

Furthermore, if a stage within a branch consists of a new parallel block or a serie of parallel blocks, then all these sub-stages/branches should be seen as well in their relative branch in the same way.
I know there is a question regarding how deep to display such a kind of nested graph, maybe could you add a 'depth' parameter in the page (default: 2?) and display the graph accordingly?
Thanks
Regards
Patrick

Regarding how deep the nested graph should show, I think making each of the dot displaying stage collapsible will be quite helpful. Clicking on a parent stage will open up the next level of children stages.

alanwu we may not allow for infinite nesting in the way that you have described (comes with a lot of design challenges). It's more likely that declarative pipeline will be updated to support a single "sub-stage" under a top level stage.

jamesdumay Sure, I can imagine the difficulties of dealing with layers upon layers of stages.

I also agree with you that a specialised syntax to allow multiple parallel brenches to be executed in the same manner without having to copy and paste blocks of code will be very useful.

alanwu thats great. FWIW, we want to build the best tool for the job rather than a bunch of round shape things and then fitting them to square holes. Just takes a but longer but I guarantee will be worth the wait

predivo unfortunately it isn't that simple. The visualization isn't a generic graph rendering that can just render things like the way you described without serious changes. Without making any promises a nesting feature is on our radar after the release of Blue Ocean 1.0 early next year.

I thought this was the reason that stage was made into a block step?

teilo yes but we failed to figure out how it should be visualized and if it can be visualized. There seems to be different ways users expect the visualization to be displayed and the clarity of the visualization decreases the deeper the stages are nested. As described in the description of this ticket we need to solve both of these issues before we can make it work. I am sure we could implement something but without rigor applied I am not certain it will be good.

Any resolution dates on this ? I have the same issue with the nested stages not being displayed at all.

kaulk as the issue description says we currently do not have a resolution for this just yet. Make sure you watch this ticket for updates.

got it, thanks.

Proposal

This proposal takes the position that the vast majority of uses cases can be adequately represented in the graph by only 2 levels, with deeper nesting merely indicated on the graph. Nesting beyond the 2nd level defaults the list view with indentation indicating nesting depth.

Representing more than 2 levels in the graph would lead to complex hierarchies that would detract from the role of the graph as an overview.

1. New 'parallel_stage' command (I'm not proposing this exact naming, just the concept)

• Acts like the existing 'stage' command, however can be executed at the same time as other 'parallel_stage' regardless of nesting level
• In this design, all 18 'parallel_stage' of the "Test" stage containing a 'node' are added to the build queue simultaneously. You can think of this a bit like a matrix, however without fixed axes.

2. 'stage' and 'parallel_stage' can be nested arbitrarily

• The graph only shows the 1st and 2nd levels
• If all stages at the 2nd level do not themselves contain further stages, they are presented as a vertical linear chain (e.g Build stage in design)
• If any of the stages at the 2nd level contain nested stages, then they are represented as siblings (e.g Test stage in design)
• Stages at the 3rd level are indicated in the graph by the number of immediate descendants on the parent node (e.g "Functional (iPhone 7 10.2)" in the Build stage).
• Stages at the 3rd level and beyond are presented in the list view, using indentation to indicate nesting level
• Neither can be called within a 'node' block
• Not sure what desired behavior should be if a 'stage' is nested within a 'parallel_stage', perhaps only 'parallel_stage' can be nested within another 'parallel_stage'?

3. The existing 'parallel' command is superseded by multiple `parallel_stage` and can be deprecated

stage("Lint") { node { sh "lint" }  }

stage("Build") {
  parallel_stage("Shared") { node { sh "build" } }
  parallel_stage("App 1") { node { sh "build" } }
  parallel_stage("App 2") { node { sh "build" } }
}

stage("Test") {
  parallel_stage("Shared") {
    parallel_stage("Unit (9.3)") { node { sh "test" } }
    parallel_stage("Unit (10.2)") { node { sh "test" } }
    parallel_stage("Snapshot (iPhone 7 10.2)") { node { sh "test" } }
  }

  parallel_stage("App 1") {
    parallel_stage("Unit (9.3)") { node { sh "test" } }
    parallel_stage("Unit (10.2)") { node { sh "test" } }
    parallel_stage("Snapshot (iPhone 7 10.2)") { node { sh "test" } }
    parallel_stage("Functional (iPhone 7 10.2)") {
      parallel_stage("Batch 1") { node { sh "test" } }
      parallel_stage("Batch 2" { node { sh "test" } }
      parallel_stage("Batch 3") { node { sh "test" } }
    }
  }

  parallel_stage("App 2") {
    parallel_stage("Unit (9.3)") { node { sh "test" } }
    parallel_stage("Unit (10.2)") { node { sh "test" } }
    parallel_stage("Snapshot (iPhone 7 10.2)") { node { sh "test" } }
    parallel_stage("Functional (iPhone 7 10.2)") {
      parallel_stage("Batch 1") { node { sh "test" } }
      parallel_stage("Batch 2" { node { sh "test" } }
      parallel_stage("Batch 3") { node { sh "test" } }
    }
    parallel_stage("Functional (iPad Air 2 10.2)") { node { sh "test" } }
    parallel_stage("Functional (iPad Pro 10.2)") { node { sh "test" } }
  }
}

stage("Collect") { node { sh "collect" }  }

ileitch Thanks a lot for the update, can you please comment on how the following case can be represent with your proposal?

stage('build and test') {
parallel('Slave1' : {
stage('configure') {
}
stage('build') {
}
stage('test') {
}
}
parallel('Slave2' : {
stage('configure') {
}
stage('build') {
}
stage('test') {
}
}
}

If stages nested within a parallel stage is not supported, does that mean stages in slave1 and slave2 cannot be shown as desired?

If stages nested within a parallel stage is not supported, does that mean stages in slave1 and slave2 cannot be shown as desired?

alanwu yes, that is correct - we can't display those right now.

alanwu jamesdumay I'd still love for some kind of `named_step`, which has nothing to do without how the graph is shown, but for naming/grouping the individual steps taken within each stage.

ileitch That's great, it will be very useful to have named_step. Thank you.

FWIW here is an example of a build where information about different substages has been omitted. It looks worse during the build, when some of the nested stages are temporarily displayed, but in a confused order.

jglick yes it should be flattened out sensibly (there are a few things going on that should eventually fix that) so it is less confusing even without showing it as explicitly nested

There has been a fair bit of off-list discussion on this, the vast majority seem to be satisfied by: 

• One level of stages shown in parallel streams of execution
• some first class support for matrix of combinations (in paralllel)

I haven't heard a strong case for nested parallel "circles" inside a non parallel stage (in that case people want to label a step or a group of steps, which is different to having a different circle/node during a parallel stream of execution). 

Here's a real world example of why one might have parallel stages and with some of those stages needing parallel "circles". We have an application that has these basic stages:

1. Clone SCM
2. Build
3. Build documentation on Windows slave (slow)
4. Custom static analysis tool (slow)
5. Multiple Test suites
6. Archive build artifacts and test results

Steps 1-3 can all run in parallel and since 2 and 3 both take a long time, it's actually desirable for them to be running while we're doing all our various suites in parallel. Here's what the ideal Jenkinsfile would look like (IMHO):

stage('Clone') {
    checkout scm
}

testSuites = [ 'suite1', 'suite2', 'suite3' ]

parallel([
    buildAndTest: {
        stage('Build') {
            node('linux') {
                sh 'make'
                stash includes: 'pkg/**/*', name: 'build-artifacts'
            }
        }
        stage('Test') {
            testSteps = [:]
            for (suite in testSuites) {
                testSteps[suite] = {
                    node('linux') {
                        unstash 'build-artifacts'
                        sh "run_tests $suite"
                        stash includes: 'test_results/**/*.xml', name: "test-$suite"
                    }
                }
            }
            parallel(testSteps)
        }
    },
    docs: {
        stage('Documentation') {
            node('windows') {
                bat 'build_docs.exe'
            }
        }
    },
    staticAnalysis: {
        stage('Static Analysis') {
            node('linux') {
                sh 'run_static_analysis'
            }
        }
    }
])

stage('Archive') {
    unstash 'build-artifacts'
    for (suite in testSuites) {
        unstash "test-$suite"
    }
    archive 'pkg/**/*'
    junit 'test_results/**/*.xml'
}

I'm not exactly sure about where that checkout scm should go with all the parallel stuff, I haven't actually run this particular thing yet. Also while writing this I wasn't really sure what to name the parallel sections at the top level, since I really want them to get their names from the nested stages.

As for how this would look in the UI (only really thinking of Blue Ocean), I would want it to look how this graphviz DAG would render:

digraph {
  graph [rankdir=LR]

  Clone -> Build
  Clone -> Documentation
  Clone -> "Static Analysis"

  Documentation -> Archive
  "Static Analysis" -> Archive

  Build -> "Test suite1"
  Build -> "Test suite2"
  Build -> "Test suite3"

  "Test suite1" -> Archive
  "Test suite2" -> Archive
  "Test suite3" -> Archive
}

Here's what I was thinking it might look like in Blue Ocean

The workaround I have planned for this particular build is to have the docs and static analysis happen in other builds that I then have to wait for them to finish. Something like this:

stage('Clone') {
    checkout scm
}

stage('Build') {
    parallel([
        build: {
            node('linux') {
                sh 'make'
                stash includes: 'pkg/**/*', name: 'build'
            }
        },
        docs: {
            build job: 'docs', wait: false
        },
        staticAnalysis: {
            build job: 'staticAnalysis', wait: false
        }
    ])
}

testSuites = [ 'suite1', 'suite2', 'suite3' ]

stage('Test') {
    testSteps = [:]
    for (suite in testSuites) {
        testSteps[suite] = {
            node('linux') {
                unstash 'build'
                sh "run_tests $suite"
                stash includes: 'test_results/**/*.xml', name: "test-$suite"
            }
        }
    }
    parallel(testSteps)
}

// method to lookup the docs and staticAnalysis jobs we started and see if they're done
@NonCPS
def findSomeBuild(jobName)
    // do Jenkins groovy stuff
    return correctBuildNumber || null
end

stage('Archive') {
    unstash 'build'

    def docsBuildNumber = null    
    waitUntil {
        docsBuildNumber = findSomeBuild('docs')
    }
    
    def staticAnalysisBuildNumber = null
    waitUntil {
        staticAnalysisBuildNumber = findSomeBuild('staticAnalysis')
    }
    
    step([$class: 'CopyArtifact', filter: 'pkg/**/*', projectName: 'docs', selector: [$class: 'SpecificBuildSelector', buildNumber: docsBuildNumber]])
    step([$class: 'CopyArtifact', filter: 'pkg/**/*', projectName: 'staticAnalysis', selector: [$class: 'SpecificBuildSelector', buildNumber: staticAnalysisBuildNumber]])

    unstash 'documentation'
    unstash 'static_analysis'
    for (suite in testSuites) {
        unstash "test-$suite"
    }
    archive 'pkg/**/*'
    junit 'test_results/**/*.xml'
}

@b_dean: I totally agree with your proposition!

We have a similar need. Currently, we are using FreestyleJob and DeliveryPipeline plugin to handle it. I'm looking forward to having such feature implemented to switch to JenkinsFile and BlueOcean.

May I suggest the following as a means to represent nested stages visually?

Display the top-most stages and consider the result of the children stages as their success/failure. Additionally, to view the nested stages, you could then click on a stage in the display, which would "drill-down" and show the sub-stages within that stage, like a zoom of sorts.

This seems it would allow for an infinite level of nesting without creating a drastic overhead in display logic.

I know it doesn't give a 100% overhead of all tasks simultaneously, but it would at least be some kind of a middle-road in my opinion.

ikari7789 could you draw a picture and upload it here please?

Sorry for my lack of artistic ability, but I've tried to portray what I'm thinking in my mind.

So at the top layer, you'd be presented with the normal staged output for the pipeline, if any stages have nested stages, it would display a zoom-in button or something (this is obviously not final).

When you click the zoom button, it will then display the stage(s) within that stage (in a single level increment):

This process could basically allow for an infinite expansion inwards.

The red/green status would be determined by the final result of all children stages.

Please look into this issue as this will give neater and intuitive UI for debugging purpose where we can see separate logs stage wise.

In my opinion, syntax given in Example2 should be the correct syntax to visualize stages in parallel and also the word "Parallel" can be removed from parallel step and may be we can show Labels there instead over the parallel steps with stage names for each steps.

Lable foo
---O--O--O-----------

---O--O--O-
s1 s2 s3

Above illustration is not looking good in visual mode, please ignore the table structure that got created. Stages for each label are in parallel with label name over stages of a particular line of execution.

Please refer to "Proposed Parallel Stages.JPG" attached here.

I like Andrew's idea, perhaps simply represent the sub-pipeline as a pipeline in itself? Continuing to re-use existing resources

As he mentioned, this should provide a UI framework to support additional nesting.

Here's a sketch that might help Andrew feel better about his artwork  Note the continuation from & to the rest of the build, perhaps add the previous & next step names and/or a better location for zoom out icons.

I agree, "zooming" into a nested stage seems like the most sensible and scalable solution, with the best possible user experience for all potential situations.

It also allows for re-usability of existing designs/assets, as this would be an addition on top of the existing way of working, instead of a re-thinking of the current way of working.