-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Jenkins 2.504.1 LTS JDK21 installed in Kubernetes
Installed 5.8.49 jenkins helm chart
Plugins installed via jenkins-plugin-cli in base container extending jenkins 2.504.1-lts-jdk21 container image
Attached an old list of plugins (old-plugins-to-install.yaml) and a new list of plugins (new-plugins-to-install.yaml) set of files. Both can be installed into 2.504.1 LTS jenkins, but the new list of plugins runs significantly slower.
The performance problem is most easily demonstrated demonstrating running with a simple pipeline:
pipeline { agent any stages { stage('Stage1') { steps { echo 'Hello World' } } stage('Stage2') { steps { echo 'Hello World' } } stage('Stage3') { steps { echo 'Hello World' } } } }
With a newer list of plugins (see attachment), a Jenkins job defined with the code directly in the job takes roughly 10 seconds to execute, where the pipeline view shows each stage taking 3-4 seconds to run.
With the older list of plugins (see attachment), a Jenkins job setup the same way takes well under 1 second to execute, where the pipeline view shows roughly 100 ms per stage to execute.
—
First noticed this issue while trying to standard pipeline code from SCM using a shared global library. After much troubleshooting and disabling many extraneous plugins in a fresh install, was able to come up with the list above. After digging and bumping log levels and comparing release notes on various items, I was unable to come up with the root cause of this problem.
In environments where others are using it, I will be holding off on upgrade until we can get a fix because the performance impact at scale is impactful. At this time, have not truly isolated the specific plugins impacted.
Some other notes:
- When using pipeline from SCM (used in tandem with cloudbees-bitbucket-branch-source and an organization job) , the first build for a branch is significantly faster than subsequent builds
- Example: First build can be 17 seconds whereas 2nd, 3rd, 4th, etc. builds take closer to 2 minutes to complete
- Freestyle jobs continue to execute just fine
- Imperative jobs execute much faster than declarative, but still likely slower than ideal
- Saw roughly 1 second per stage instead of 3-4 seconds per stage, but much higher than the ~100ms per stage given above example code
- Changing pipeline durability (performance or otherwise) does not have a major impact the overall performance of the job