Is there any possibility that this issue will be picked up? We're stuck in the old pipeline world and would love to use the new 2.0 pipeline, but people here are very fond of the visual representation of the parallel jobs that the older pipeline plugins offer.

In addition, since the old pipeline doesn't work in Jenkins 2, that blocks upgrades to newer Jenkins releases

You can try the new Jenkins Pipeline GUI: https://jenkins.io/projects/blueocean/. It has already support for the parallel steps visualization.

The Project Blue Ocean is still in alpha, but is under active development: https://jenkins.io/blog/2016/07/19/blue-ocean-update/. It is available in the experimental update center: http://updates.jenkins-ci.org/experimental/update-center.json.

GitHub: https://github.com/jenkinsci/blueocean-plugin

Wow, looks slick. I'm not sure it's stable enough for us to use for our main CI just yet, but that definitely is more along the lines of what we're looking for. Thanks for the update!

johnament Wait, what's the issue you're experiencing with Jenkins 2.0? This is supposed to be 100% compatible, and I haven't seen any JIRAs opened about an issue. If there is one, it will be a very high priority fix.

barryoneill I've updated status to reflect that we are indeed working towards this. It's not forgotten, not abandoned, just far more work to implement than anyone would expect without seeing the code or edge cases. I think I've currently invested ~1-2 months.

The challenge is that we did not have a common set of APIs for decomposing the directed acyclic graphs from a pipeline execution, and have hit multiple problems like this one that are painful without them. So, I've had to write those components to provide the functionality we need. And they need to handle a lot of nasty, complex edge cases to work correctly – for example: nested parallel blocks, where both parallel blocks are incomplete.

The current plan is that the BlueOcean UI linked above will also consume the same APIs to provide additional functionality.

gitt the Jenkins Pipeline GUI is great, but it still doesn't show steps inside each parallel branch, even though the syntax is accepted. Do you know if that is planned?

I see 2 issues with that plugin in latest version:

1. Step time is invalid, like it stop counting when all paraell tasks are started, it looks like step finish but in real job is still waiting for finish of parallel taks until move to next step.
2. No option to scroll to see older builds, it shows only last ~20, but cannot see older one. Would be nice to see same info when entering a specific build page.

Except that, I like it..

clockman Do you mean the Blue-Ocean plugin? Stage View? Pipeline view plugin? Build Pipeline Plugin?

These are all different things.

What would be the current expect way to define a job with multiple parallel tasks if we cannot use stage inside it? How we are going to distinguish between "parallel-build-foo" and "parallel-build-bar"?

I imagine that since the parallel step takes a map, the keys can act as identifiers for the parallel tasks.

ssbarnea The answer from antonjenkinsjira is correct. The block-scoped version of the stage step, i.e.

stage ('myStageName') {
  // do stuff here
}

can also be included in parallels. However there is not a UI visualization for that approach yet.

The name of the parallel "branches" do indeed look like stages, and provide and identifier (this shows up int he log as a prefix, and also in blue ocean visualisation as branches of execution).

svanoort I wouldn't recomment putting stages inside of things until we decide how we want to visualise it.

Indeed – the branch name is the identifier in this case, it's up in the air if stages inside parallels will be supported in the UI or not. The syntax allows it, but just because you can doesn't mean you should!

To recap visualization needs I'd suggest swimlines (it is simpler that rectangles in rectangles..) :
1. branches of a stage
Stage can fork into parallel branches (as described above) - these need parallel visualization by a) names described in parallel statement and b) their times.
Stage time shall be actual time incl. branches.
Visualization should make swimline for each branch below stage they fork.
2. branch stages
Each branch may have stages - time of branch shall include sum of it's steps and stages.
Visualization should make swimline for each stage below branch they are included in.
3. sub-stages
Each stage (e.g. A) can have sub-stages (e.g. A1 and A2) - Visualization of substages (A1, A2) as one swimline for all substages one-after-another below main stage (A) so to show that substage times are included in stage time.

Do we have a timeline for this fix?

This missing feature might be one of main reasons why Jenkins is on HOLD in one of the most popular tech radars:
https://www.thoughtworks.com/radar/tools

elgalu take a look at Blue Ocean - we're building an entirely new user experience for CD pipelines, including a visual editor for Pipeline.

Here's a screenshot that includes parallel visualisation:

elgalu Please remember that ThoughtWorks has its own CI/CD solutions so I would count their tech radar a biased one. Still, I have to confess that I partially support your view, I would love to be able to migrate OpenStack build jobs to pipelines but I doubt it would be possible in the next 1-2 years, and that's because they are not mature enough and because there were some design decisions regarding not supporting old jobs, making progressive transitions hardly possible.

Shortly, future is bright for small companies with simple workflows as they would be able to benefit from Jenkins pipelines. Others are kinda screwed as I don't see any migration scenario working at this moment.

Yes ssbarnea I also thought about them being biased, I assisted their tech radar presentation and the topic came, the presenter assured they are not biased and they make a big effort to be objective and explained all the reasons behind it.

jamesdumay Blue Ocean looked promising, I installed it. Why a visual editor? shouldn't the view be generated out of the Jenkinsfile? I thought we want pipeline as code.
So I tried to write `parallel` Jenkinsfiles and the generated view looks weird, both examples:

Left example

node() {
  stage('Build') {
    println 'I prepare the build for the parallel steps'
  }

  parallel (
   "chrome" : { println 'running tests in chrome' },
   "firefox" : { println 'running tests in firefox' },
   "edge" : { println 'running tests in edge' }
  )
}

Right example

node() {
  stage('Build') {
    println 'I prepare the build for the parallel steps'
  }

  parallel (
   "chrome" : { stage('chrome') { println 'running tests in chrome' } },
   "firefox" : { stage('firefox') { println 'running tests in firefox' } },
   "edge" : { stage('edge') { println 'running tests in edge' } }
  )
}

Can you try this instead:

node() {
  stage('Build') {
    println 'I prepare the build for the parallel steps'
  }

  stage('Test') {
   parallel (
   "chrome" : { println 'running tests in chrome' },
   "firefox" : { println 'running tests in firefox' },
   "edge" : { println 'running tests in edge' }
  )
  }
}

Yay! thanks crummy (my bad)

I guess it is better to read a bit how it works before elgalu

Guys one more question, as far as I understood from jglick post here https://issues.jenkins-ci.org/browse/JENKINS-28293?focusedCommentId=277255&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-277255 the parallel steps visualisation should be also supported if stages are nested under the parallel step since stage accepts block parameter, correct?

Does this mean that the Blue Ocean plugin has to be still updated in order the visualisation displays it correctly?

Does/will the visualization in BlueOcean support sub-stages in parallel?

I think we have a very common use-case, but it's not really supported in the current BlueOcean gui:

parallel (
 "win7-vs2012" : {node("vs2012") {stage("checkout") {...}; stage("build") {...}; stage("test") {...};} },
 "win10-vs2015" : {node ("vs2015") {stage("checkout") {...}; stage("build") {...}; stage("test") {...};} },
 "linux-gcc5" : {node ("gcc5") {stage("checkout") {...}; stage("build") {...}; stage("test") {...};} }
)
stage("email notifications") {...}

pjaytycy, I just tested the following code:

node() {
  stage('Build') {
    println 'I prepare the build for the parallel steps'
  }

  stage('Test') {
   parallel (
 "win7-vs2012" : { stage("checkout") { }; stage("build") { }; stage("test") { } },
 "win10-vs2015" : { stage("checkout") { }; stage("build") { }; stage("test") { }},
 "linux-gcc5" : { stage("checkout") { }; stage("build") { }; stage("test") { } }
)
  }
}

I didn't see the stages in the parallel steps. I don't think that's supported in blue ocean right now:

According to the following tickets, parallel steps in Blue Ocean are not supported very well:

• https://issues.jenkins-ci.org/browse/JENKINS-39847
• https://issues.jenkins-ci.org/browse/JENKINS-39464
• https://issues.jenkins-ci.org/browse/JENKINS-39158

Ping jamesdumay and vivek for questions/requests about Blue Ocean and parallels/nesting.

crummy pjaytycy nested stages (anywhere where there is a outer and inner stage) as you've suggested can't be visualised in Blue Ocean as we haven't been able to design a visualisation that meets all users requirements and expectations. There have been many solutions suggested but none that fit. You can follow and vote for it at JENKINS-38442. For the time being, we ignore the inner stages but show the steps within.

jhack the only ticket you have quoted that is unscheduled is JENKINS-39847. Aside, I think we do support parellel very well - Blue Ocean is the only place where parellel visualisation is supported so far

As you can see, its a hard problem to solve visually when there are so many variations in the way that parallel can be used within Pipeline Script. The challenge with creating a visualization for Pipeline is to create a visualisation for turing complete programming language. That isn't practically possible under all circumstances.

crummy Malcolm - could you sketch out what you would hope that the test stage above would look like? could be on a metaphorical back of a napkin (or literal napkin, although they can be hard to draw on).

My guess is something like:

I understand completely it's hard to try and arrange a display for the results of a turing-complete programming language, but perhaps the compromise is allowing the programmer define where things should go, instead of having the language try to divine out the positioning post-facto. For instance (pseudo code, please be gentle)

stage("checkout",0,0)

stage("build",1,0)

stage("build",1,1)

stage("deploy",2,0)

stage("deploy",2,1)

Since the person writing the code knows (ostensibly) where things should go, you could just trust them to make the right grid-like display of each node/step

perilousapricot being able to deliver a consistent visualization is one of the drivers of the new Declarative Pipeline syntax.

Declarative Pipeline is structured. For example, you can only nest a single parellel block within a stage. There is no nesting of stages but parallel branches can have names that do what a lot of people on the comments of this ticket have done by defining a stage within a parellel.

To use elgalu's example, in Pipeline Script:

node() {
  stage('Build') {
    println 'I prepare the build for the parallel steps'
  }

  parallel (
   "chrome" : { stage('chrome') { println 'running tests in chrome' } },
   "firefox" : { stage('firefox') { println 'running tests in firefox' } },
   "edge" : { stage('edge') { println 'running tests in edge' } }
  )
}

Would be the following in Declarative:

pipeline {
  agent label:''" // same as node { ... }
  stages {
    stage('build') {
      steps {
        println 'I prepare the build for the parallel steps'
      }
    }
    stage('browser testing') {
      parallel (
        'chrome': {
          println 'running tests in chrome'
        },
        'firefox': {
          println 'running tests in firefox'
        },
        'edge': {
          println 'running tests in edge'
        }
      )
    }
  }
}

Now the more awesome part is that we can figure out how the Pipeline is displayed by reading the model vs executing it. That allows us to describe all the stages, steps and parallel branches before execution (making a linter feasible), build your own tooling to modify or construct Jenkinsfiles and even provides a foundation for the Jenkins project to build a Pipeline Editor.

It may not be as flexible as the scripting variant that you've come to know but it will give much more consistent results across many of the pain points discussed here (however, you can use Script from a shared library in Declarative as an escape hatch).

All this does not mean that Blue Ocean won't be trying to visualize your Jenkinsfiles written in Pipeline Script. It's important to us that it works for the majority case. However, we recognise that due to the nature of the beast there are going to be some things that just won't be possible to visualize when using Script.

I like it! My only hitch would be trying to describe the dependency that wasn't a series of stages (forgive my pseudo-code) where things would block unnecessarily:

stage("checkout")

node("linux-ubuntu") { stage("compile")

}
node("linux-centos") { stage ("compile")

; stage ("test")

}

It's a contrived example, but I currently do something like:
state("checkout") {
git checkout XXX
}
stage("compile") {
parallel {
node ("linux-ubuntu")

node ("linux-centos")

}
stage ("test") {
node ("linux-centos")

}

which means every part of "compile" has to finish before the "test" phase happens, even if there are available executors. If there's a rewrite in a declarative language, I'd hope that the "test" parts could at least start before all the "compile" stages start. I know it's not what's exposed in the current or blue ocean implementations, but it would be nice to fire off additional nodes/stages without waiting on unnecessary parallel branches.

I think the visualization is a second-order optimization, but at least firing off the subordinate nodes would speed things up a lot. And ,as a script writer, if I could hint to you who goes where visually, that would simplify your job

perilousapricot BTW you can wrap code in JIRA in {code} some code {code} to preserve formatting

Sorry, James Durmay – I'm not super familiar with JIRA quoting. I can rewrite/edit if it makes it clearer.

perilousapricot I'm glad you like it. We've got a more formal beta coming towards the end of the year for Declarative but if you want to try it out now please do!

If you've got any specific use cases in mind, like the one above, you can file an issue on JIRA against the pipeline-model-definition component and hrmpw will be in touch

Yessir, Java's not one of my proficiencies, and I failed at even trying to build Jenkins on my local (OSX) machine, but I'm willing to try again to help scratch my itch. I'll write a JIRA against the above component and give it another shot to try and help. Cheers!

Can someone point me to some sample Jenkinsfile that runs two scripts in parallel while having this visible in the UI? I find extremely confusing to see lots and lots of nice screenshots of parallel pipelines but not even one code example for achieving it.

ssbarnea I think this might be what you are looking for:

node {
  stage('Build') {
    println 'I prepare the build for the parallel steps'
  }

  stage('Browser Tests') {
    parallel (
       "chrome" : { println 'running tests in chrome' },
       "firefox" : { println 'running tests in firefox' },
       "edge" : { println 'running tests in edge' }
     )
  }
}

Would love to see something like johnwynne proposed here as an intermediate solution for the current pipeline stage view:
https://issues.jenkins-ci.org/browse/JENKINS-33185?focusedCommentId=258129&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-258129

Blue Ocean is a huge update (30 plugins?) and probably not everyone can (or wants) to adopt it and has to stay with the classical jenkins UI for quite a while.

Is there any ongoing work in PRs to implement johnwynne's proposal? Would it even have a chance to be accepted as an intermediate solution?

We have a use case to drive parallel builds over multiple stages, since we code C++ for multiple architectures and environments.
Would this be possible with the Pipeline Stage, or even Pipeline at its core?

In the example you see

• Multiple design targets of the code lead to parallel streams of execution,
• The same stages occur in every stream.

I expect to

• Name a Stage not only on top level, but also within a node step.
• Be able to *reuse* the Stage by name -> collect certain activity in the same Stage column.
• visualize multiple steps under the same Stage title.
• Within a Node(eg "B") step, switch stage -> split output from same node to two Stages.

The Build and Test Stages would be active at the same time, depending on design stream.

Output from this pipeline should be separated into exactly three Stages and activity within each stage.

• Setup.Default.console
• Build.web.console
• Build.embedded.console
• Test.web.console
• Test.embedded.test-lowend.console
• Test.embedded.test-highend.console

node('A') {
    stage('Setup') {
        println 'Preparing multiple builds'
    }
}
parallel (
    "web" : {
        node('B') {
            stage('Build') {
                println 'Build web application'
            }
            stage('Test') {
                println 'Selenium Tests'
            }
        }
    },
    "embedded" : {
        node('C') {
            stage('Build') {
                println 'Build embedded solution'
            }
        }
        stage('Test') {
            parallel (
                "test-lowend" : {
                    node('D1') {
                        println 'Test lowend embedded'
                    }
                }, 
                "test-highend" : {
                    node('D2') {
                        println 'Build highend embedded'
                    }
                }
            )
        }
    }
)

Is there any ongoing work in PRs to implement John Wynne's proposal?

Not that I am aware of.

Would it even have a chance to be accepted as an intermediate solution?

If you would like to take over ownership of this plugin.

Put a bit less bluntly, what I believe jglick is trying to say is that (as per James Dumay's earlier comment) this one will probably need a community PR to implement it. Right now CloudBees is putting most of our front-end development effort behind Blue Ocean since it promises a better & easier path to this functionality.

Stage view is maintained but right now bandwidth is limited due to other projects. Doing this properly would take more time than I have available to do it as a personal side project – and I'm not going to knock together a poor quality implementation and bring the overall plugin quality down.

svanoort jglick thanks for the feedback!

Good to hear that stage view is still maintained. Understood it needs a community PR. Would love to see it, but currently too short on time too :-/

Is Blue Ocean going to be in a usable state any time soon? At the moment, it just shows a progress bar which never finishes, so it's not really a viable workaround, let alone a good one. (On investigating this, it turns out a ticket was raised, but then resolved as "Duplicate" without actually fixing anything. Excellent!)

As far as visualising the parallel stages, couldn't it be displayed serially like the existing serial stages already are? That would satisfy that desire to keep the table view.

Although, I am a little in doubt over any supposed "benefits" to the table view. All I see is a gigantic table which forces me to scroll horizontally to see all the information. Even if I do scroll right to try and see more information, now the row headers roll off the left, so I have no idea which builds are red and which builds are green anyway. Showing it vertically might improve things, but I'm not sure it's a good visualisation in the first place, so I'm a little baffled as to why anyone would want to keep it. (I think perhaps as someone who actually has to use the software, I might have a different view of its prettiness to someone involved with developing it?)

trejkaz "progress bar which never finishes" means the app installation is broken (would be errors in the browser console) - can you link the ticket to it? 

I didn't think to check the browser console, since the page itself didn't indicate any kind of error. (Is there a ticket for that already?)

Error: Must call ExtensionStore.init(\{ extensionData: array, typeInfoProvider: (type, cb) => ... }) first
at ExtensionStore._initExtensionPointList (blueocean.js:49604)
at ExtensionStore.init (blueocean.js:49449)
at Object.init (blueocean.js:50102)
at Object.execute (blueocean.js:143579)
at APromise.<anonymous> (blueocean.js:144211)
at Object.exports.make (blueocean.js:51393)
at ___$$$___exec (blueocean.js:144208)
at ___$$$___doBundleInit (blueocean.js:144296)
at Object.1720.../../src/main/js/init (blueocean.js:144311)
at s (blueocean.js:1)

Searching for that error, the only occurrence I see in the tracker is JENKINS-44752 but it says that was "Fixed" too.

trejkaz can you please open a new ticket for that error and attach a HAR file and support bundle?

usually that means some plugins have been removed/need to be updated (all blue ocean plugins).

trejkaz ok that is good to hear - although I don't think other plugins yet depend on blue ocean parts, so not quite sure how it ended up doing that. There are other tickets on better generic error reporting that may make this error more obvious (although like with all error cases, edge cases mean it sometimes just wont work,but hopefully will at least give a hint).

I'm not sure this view is really much better. I have to move down multiple pages of "pipelines" which aren't even pipelines to get to the only actual pipeline we have configured so far ... then when we do get to the one we want and look at some of the builds, the visualisation on each page doesn't even seem to show all the stages, just the one it's currently working on and then "End".

Oh well.

I was wondering whatever happened to this? We have an architecture like the pseudo-code below, where the parallel is a stage, but the stages in the parallel aren't broken out. We need to do this to get different nodes within a stage. If there is a better way to do this so the visualization works, we'd love to know, but it is currently an ugly visual representation with only the first stage and the parallel stage, basically.

def jobMap (
      some['thing'] = {node (label) {
         stage1
         stage2
      } }
)
 
node {
   def jobs = jobMap()
   stage1
   stage2 {
      parallel jobs
   }
}

Blue Ocean is unusable to me for its page load time alone.

I can see how the tabular view can be useful to people.

I have on many occasions seen the table data disappear completely, because one stage was renamed or moved. If the tabular overview is regarded this highly I would expect that to be stable and merging as well; fill previously or afterwards non-existing columns with empty cells. But it does not. Instead it throws away all of it except the last build information.

As it is not stable in this case, the table stability is regarded too highly and inconsistently in the reasoning in this ticket (ticket conclusion). At least for me definitely so.

Especially so as I can still see steps with multiple substeps in the tabular format be comparable and parseable very well. Not having the parallel information is a much bigger hit on the overview.

If a cell contains multiple sub-steps I don't see how it is much worse for tabular overview? It will still be tabular, in columns. And it can still prepare and fill cells as if it were just one level.

With parallel the order in which the steps report their existence is not deterministic. Hence, the linear steps order is not stable. This results in the entire table history being dropped. So even by the argument of keeping the overview in a tabular format, this is not consistently possible, but only randomly!? Seems like a very weak argument.

I just saw the table disappear on me (presumably) because of this.

/edit: Okay, it looks like it appeared again? Very confusing. Is there a reordering-according-to-config-declaration at the end??

I would like to add this item into the Jenkins roadmap: https://jenkins.io/project/roadmap/

Would be everyone fine if I reword it to "Support Blue Ocean - alike browsing of Pipelines inside Jenkins web UI"? I would also remove the component, because it is likely to become a separate plugin if it gets implemented.

Support Blue Ocean - alike browsing of Pipelines inside Jenkins web UI

Makes no sense. B.O. already supports this.

I would also remove the component, because it is likely to become a separate plugin if it gets implemented.

Also makes no sense. The limitation is specifically in the stage view plugin.

I am kind of lost here. I just learned about this https://groups.google.com/g/jenkinsci-users/c/xngZrSsXIjc/m/btasuPpYCgAJ and it was discussed a bit in the https://issues.jenkins.io/browse/JENKINS-54010. That leaves us in a weird state where B.O. is half baked and still missing pieces of functionality (like aforementioned ticket) and get very little support, but the classic UI has no support for parallel stages and steps whatsoever. Running stuff in parallel is kind of basic fundamental necessity that any other CI system has support for. What is the current direction with this ticket? Obviously current description encouraging to use B.O. is outdated.

I mentioned pipeline-graph-view-plugin in my comment on https://issues.jenkins.io/browse/JENKINS-54010. That plugin brings visualization of parallel steps into the classic UI.