Details
-
Improvement
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
jenkins 2.138.1, pipeline-model-definition-plugin 1.3.2
Description
Yesterday, I tried using sequential stages. I have a pipeline consisting of 2 distinct parts, let's say JAVA PART and DOCKER PART, each with a different agent and several stages. Thus, I grouped them into two stages, assigned an agent and well, it worked as expected. Things went south when I wanted to use parallel stages inside JAVA PART:
pipeline {
agent none
stages {
stage("JAVA PART") {
agent A
stages {
stage("Java one") {
steps {
echo "Java one"
}
}
stage("Java two") {
parallel {
stage("Java parallel one") {
steps {
echo "Java parallel one"
}
}
stage("Java parallel two") {
steps {
echo "Java parallel two"
}
}
}
}
stage("Java three") {
steps {
echo "Java three"
}
}
}
}
stage("DOCKER PART") {
agent B
steps {
echo "Docker one"
}
}
}
}
I ended up with
Parallel stages or branches can only be included in a top-level stage.
I wonder what the rationale behind this restriction is, because when I switch from parallel to stages and make it serial, everything is fine.
pipeline {
agent none
stages {
stage("JAVA PART") {
agent A
stages {
stage("Java one") {
steps {
echo "Java one"
}
}
stage("Java two") {
stages {
stage("Java parallel one") {
steps {
echo "Java parallel one"
}
}
stage("Java parallel two") {
steps {
echo "Java parallel two"
}
}
}
}
stage("Java three") {
steps {
echo "Java three"
}
}
}
}
stage("DOCKER PART") {
agent B
steps {
echo "Docker one"
}
}
}
}
Attachments
Issue Links
- is duplicated by
-
JENKINS-53659 support for parallel Stages nested in stages for ressource locking
-
- Closed
-
- links to
Activity
So the motivation was that we don't want to allow arbitrarily deep parallel nesting - nested parallels won't be visualized properly by Blue Ocean, most notably.
That said, https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/292 - this would allow nesting parallel within a stage's stages if and only if the stage isn't itself already within a parallel. I think that it won't visualize particularly nicely in Blue Ocean (i.e., it may just visualize the whole nested stage with a parallel on its own, without visualizing its children independently), but I think we can live with that anyway.
I don't agree that this should not be rendered! Visualization of a pipeline for end user is important! Every other job control software has no problem with such parallelisation's. And for optimization of build time I need a lot of parallelisation's.
So I would request the corresponding changes in blue ocean too after implementation of this change.
In the meantime I run in several issues in Jenkins, where implementing one change in some plugin contradicts other restrictions in another plugin.
Alexey Marin
added a comment - I agree that this will be very useful. I use sequential stages to lock a single resource (DB) for multiple stages (as suggested in JENKINS-43336)
For example, a pipeline like this one is currently impossible:
pipeline {
agent any
stages {
stage ('Non-production') {
options {
lock 'DB1'
}
stages {
stage ('Deploy') {
steps {
echo 'Deploy'
}
}
stage ('Test') {
parallel {
stage ('Unit testing') {
steps {
echo 'Unit'
}
}
stage ('E2E testing') {
steps {
echo 'E2E'
}
}
}
}
}
}
stage ('Production') {
stages {
stage ('Deploy') {
steps {
echo 'Deploy'
}
}
}
}
}
}
Alexey Marin
added a comment - I agree that this will be very useful. I use sequential stages to lock a single resource (DB) for multiple stages (as suggested in JENKINS-43336 )
For example, a pipeline like this one is currently impossible:
pipeline {
agent any
stages {
stage ( 'Non-production' ) {
options {
lock 'DB1'
}
stages {
stage ( 'Deploy' ) {
steps {
echo 'Deploy'
}
}
stage ( 'Test' ) {
parallel {
stage ( 'Unit testing' ) {
steps {
echo 'Unit'
}
}
stage ( 'E2E testing' ) {
steps {
echo 'E2E'
}
}
}
}
}
}
stage ( 'Production' ) {
stages {
stage ( 'Deploy' ) {
steps {
echo 'Deploy'
}
}
}
}
}
}
Found a workaround via scripted pipelines, but it is kinda ugly:
pipeline { agent none stages { stage("JAVA PART") { agent A stages { stage("Java one") { steps { echo "Java one" } } stage("Java two") { steps { script { def parallelStepsMap = [:] parallelStepsMap["Java parallel one"] = { echo "Java parallel one" } parallelStepsMap["Java parallel two"] = { echo "Java parallel two" } parallel parallelStepsMap } } } stage("Java three") { steps { echo "Java three" } } } } stage("DOCKER PART") { agent B steps { echo "Docker one" } } } }I believe declarative pipelines should isolate me from having to do such things.