Minor When using pipeline combined with Docker, physical agents and the 'parallel' syntax the result produces unwanted behaviour.
Some background for this piece of code:
- linux_fleet are a group of instances a high limit of instances that are spun up when Jenkins requests their specific label
- they all have 1 executor
- they are able to scale up without issue in non-parallel builds
Intended:
Test1 and Test2 build at the same time on different physical agents within docker containers.
Actual:
It picks a random agent for any, then attempts to build & run docker containers on that top level agent. It does not spin up anything except the initial agent.
pipeline {
agent any
stages {
stage('Build Tests') {
parallel {
stage('Build Test1') {
agent {
docker {
image 'debian:latest'
label 'linux_fleet'
alwaysPull true
reuseNode true
}
}
steps {
sh """
touch test1.test
"""
}
}
stage('Build Test2') {
agent {
docker {
image 'debian:latest'
label 'linux_fleet'
alwaysPull true
reuseNode true
}
}
steps {
sh """
touch test2.test
"""
}
}
}
}
}
}
If I change this code to instead define only labels of physical agents and not use the Docker syntax and instead use 'withDockerContainer' to run the code, it results in more physical agents being spun up for each parallel build, but deadlocking issues actually starting the builds.
agent any stage('Build Test') { parallel { stage('Build Test1') { agent {label 'linux_fleet'} steps { withDockerContainer(url: "dockerurl") { sh """ touch test1.test """ } } }
A very related issue:
https://issues.jenkins.io/browse/JENKINS-46831
