-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Jenkins: 2.365
OS: Linux - 5.10.0-8-cloud-amd64
---
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61
bootstrap4-api:4.6.0-5
bootstrap5-api:5.2.0-2
bouncycastle-api:2.26
build-timeout:1.21
caffeine-api:2.9.3-65.v6a_47d0f4d1fe
checks-api:1.7.5
command-launcher:84.v4a_97f2027398
copyartifact:1.47
credentials:1139.veb_9579fca_33b_
credentials-binding:523.vd859a_4b_122e6
dashboard-view:2.447.vda_124dd35f11
display-url-api:2.3.6
durable-task:500.v8927d9fd99d8
echarts-api:5.3.3-1
font-awesome-api:6.1.2-1
git:4.11.5
git-client:3.11.2
google-compute-engine:4.3.11
google-oauth-plugin:1.0.6
instance-identity:116.vf8f487400980
jackson2-api:2.13.3-285.vc03c0256d517
jakarta-activation-api:2.0.1-1
jakarta-mail-api:2.0.1-1
javax-activation-api:1.2.0-4
javax-mail-api:1.6.2-7
jaxb:2.3.6-1
jdk-tool:55.v1b_32b_6ca_f9ca
jquery3-api:3.6.0-4
jsch:0.1.55.61.va_e9ee26616e7
junit:1119.1121.vc43d0fc45561
mailer:438.v02c7f0a_12fa_4
matrix-auth:3.1.5
matrix-project:785.v06b_7f47b_c631
mina-sshd-api-common:2.8.0-36.v8e25ce90d4b_1
mina-sshd-api-core:2.8.0-36.v8e25ce90d4b_1
oauth-credentials:0.5
plain-credentials:139.ved2b_9cf7587b
plugin-util-api:2.17.0
popper-api:1.16.1-3
popper2-api:2.11.6-1
project-build-times:1.2.1
scm-api:621.vda_a_b_055e58f7
script-security:1175.v4b_d517d6db_f0
snakeyaml-api:1.30.2-76.vc104f7ce9870
ssh-credentials:295.vced876c18eb_4
sshd:3.249.v2dc2ea_416e33
structs:324.va_f5d6774f3a_d
token-macro:308.v4f2b_ed62b_b_16
trilead-api:1.71.v9e7860a_67a_df
windows-slaves:1.8.1
workflow-api:1192.v2d0deb_19d212
workflow-scm-step:400.v6b_89a_1317c9a_
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:838.va_3a_087b_4055b
Jenkins: 2.365 OS: Linux - 5.10.0-8-cloud-amd64 --- antisamy-markup-formatter:2.7 apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61 bootstrap4-api:4.6.0-5 bootstrap5-api:5.2.0-2 bouncycastle-api:2.26 build-timeout:1.21 caffeine-api:2.9.3-65.v6a_47d0f4d1fe checks-api:1.7.5 command-launcher:84.v4a_97f2027398 copyartifact:1.47 credentials:1139.veb_9579fca_33b_ credentials-binding:523.vd859a_4b_122e6 dashboard-view:2.447.vda_124dd35f11 display-url-api:2.3.6 durable-task:500.v8927d9fd99d8 echarts-api:5.3.3-1 font-awesome-api:6.1.2-1 git:4.11.5 git-client:3.11.2 google-compute-engine:4.3.11 google-oauth-plugin:1.0.6 instance-identity:116.vf8f487400980 jackson2-api:2.13.3-285.vc03c0256d517 jakarta-activation-api:2.0.1-1 jakarta-mail-api:2.0.1-1 javax-activation-api:1.2.0-4 javax-mail-api:1.6.2-7 jaxb:2.3.6-1 jdk-tool:55.v1b_32b_6ca_f9ca jquery3-api:3.6.0-4 jsch:0.1.55.61.va_e9ee26616e7 junit:1119.1121.vc43d0fc45561 mailer:438.v02c7f0a_12fa_4 matrix-auth:3.1.5 matrix-project:785.v06b_7f47b_c631 mina-sshd-api-common:2.8.0-36.v8e25ce90d4b_1 mina-sshd-api-core:2.8.0-36.v8e25ce90d4b_1 oauth-credentials:0.5 plain-credentials:139.ved2b_9cf7587b plugin-util-api:2.17.0 popper-api:1.16.1-3 popper2-api:2.11.6-1 project-build-times:1.2.1 scm-api:621.vda_a_b_055e58f7 script-security:1175.v4b_d517d6db_f0 snakeyaml-api:1.30.2-76.vc104f7ce9870 ssh-credentials:295.vced876c18eb_4 sshd:3.249.v2dc2ea_416e33 structs:324.va_f5d6774f3a_d token-macro:308.v4f2b_ed62b_b_16 trilead-api:1.71.v9e7860a_67a_df windows-slaves:1.8.1 workflow-api:1192.v2d0deb_19d212 workflow-scm-step:400.v6b_89a_1317c9a_ workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:838.va_3a_087b_4055b
-
-
2.375
Hello
Jenkins cannot start any job on startup. It generates an infinite loop in some Java part of the code:
Aug 23 20:23:51 jenkins-master jenkins[1015358]: java.lang.StackOverflowError
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.Collections$ReverseComparator.compare(Collections.java:5212)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.Collections$ReverseComparator.compare(Collections.java:5203)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.getEntryUsingComparator(TreeMap.java:374)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.getEntry(TreeMap.java:343)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.containsKey(TreeMap.java:231)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:512)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:495)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:428)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:244)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:944)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:138)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Job.isLogUpdated(Job.java:314)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1077)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
(the daemon.log quickly grows to hundred of mb)
Reverting to jenkins 2.363 fixes the issue
Jenkins is configured following this tutorial:
https://cloud.google.com/architecture/using-jenkins-for-distributed-builds-on-compute-engine
- is duplicated by
-
-
- Closed
-
[JENKINS-69419] infinite loop on node startup - Caught exception evaluating: h.escape(item.why) in /ajaxBuildQueue.
Sylvestre Ledru
created issue -
Sylvestre Ledru
made changes -
| Description |
Original:
Hello
Jenkins cannot start any job on startup. It generates an infinite loop in some Java part of the code: ``` Aug 23 20:23:51 jenkins-master jenkins[1015358]: java.lang.StackOverflowError Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.Collections$ReverseComparator.compare(Collections.java:5212) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.Collections$ReverseComparator.compare(Collections.java:5203) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.getEntryUsingComparator(TreeMap.java:374) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.getEntry(TreeMap.java:343) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.containsKey(TreeMap.java:231) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:512) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:495) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:428) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:244) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:944) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:138) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Job.isLogUpdated(Job.java:314) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1077) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115) ``` (the daemon.log quickly grows to hundred of mb) Reverting to jenkins 2.363 fixes the issue Jenkins is configured following this tutorial: https://cloud.google.com/architecture/using-jenkins-for-distributed-builds-on-compute-engine |
New:
Hello
Jenkins cannot start any job on startup. It generates an infinite loop in some Java part of the code: {quote} Aug 23 20:23:51 jenkins-master jenkins[1015358]: java.lang.StackOverflowError Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.Collections$ReverseComparator.compare(Collections.java:5212) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.Collections$ReverseComparator.compare(Collections.java:5203) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.getEntryUsingComparator(TreeMap.java:374) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.getEntry(TreeMap.java:343) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at java.base/java.util.TreeMap.containsKey(TreeMap.java:231) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:512) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:495) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:428) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:244) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:944) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:138) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Job.isLogUpdated(Job.java:314) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1077) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630) Aug 23 20:23:51 jenkins-master jenkins[1015358]: #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115) {quote} (the daemon.log quickly grows to hundred of mb) Reverting to jenkins 2.363 fixes the issue Jenkins is configured following this tutorial: [https://cloud.google.com/architecture/using-jenkins-for-distributed-builds-on-compute-engine] |
Sylvestre Ledru
made changes -
| Environment |
Original:
```text
Jenkins: 2.365 OS: Linux - 5.10.0-8-cloud-amd64 --- antisamy-markup-formatter:2.7 apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61 bootstrap4-api:4.6.0-5 bootstrap5-api:5.2.0-2 bouncycastle-api:2.26 build-timeout:1.21 caffeine-api:2.9.3-65.v6a_47d0f4d1fe checks-api:1.7.5 command-launcher:84.v4a_97f2027398 copyartifact:1.47 credentials:1139.veb_9579fca_33b_ credentials-binding:523.vd859a_4b_122e6 dashboard-view:2.447.vda_124dd35f11 display-url-api:2.3.6 durable-task:500.v8927d9fd99d8 echarts-api:5.3.3-1 font-awesome-api:6.1.2-1 git:4.11.5 git-client:3.11.2 google-compute-engine:4.3.11 google-oauth-plugin:1.0.6 instance-identity:116.vf8f487400980 jackson2-api:2.13.3-285.vc03c0256d517 jakarta-activation-api:2.0.1-1 jakarta-mail-api:2.0.1-1 javax-activation-api:1.2.0-4 javax-mail-api:1.6.2-7 jaxb:2.3.6-1 jdk-tool:55.v1b_32b_6ca_f9ca jquery3-api:3.6.0-4 jsch:0.1.55.61.va_e9ee26616e7 junit:1119.1121.vc43d0fc45561 mailer:438.v02c7f0a_12fa_4 matrix-auth:3.1.5 matrix-project:785.v06b_7f47b_c631 mina-sshd-api-common:2.8.0-36.v8e25ce90d4b_1 mina-sshd-api-core:2.8.0-36.v8e25ce90d4b_1 oauth-credentials:0.5 plain-credentials:139.ved2b_9cf7587b plugin-util-api:2.17.0 popper-api:1.16.1-3 popper2-api:2.11.6-1 project-build-times:1.2.1 scm-api:621.vda_a_b_055e58f7 script-security:1175.v4b_d517d6db_f0 snakeyaml-api:1.30.2-76.vc104f7ce9870 ssh-credentials:295.vced876c18eb_4 sshd:3.249.v2dc2ea_416e33 structs:324.va_f5d6774f3a_d token-macro:308.v4f2b_ed62b_b_16 trilead-api:1.71.v9e7860a_67a_df windows-slaves:1.8.1 workflow-api:1192.v2d0deb_19d212 workflow-scm-step:400.v6b_89a_1317c9a_ workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:838.va_3a_087b_4055b ``` |
New:
Jenkins: 2.365
OS: Linux - 5.10.0-8-cloud-amd64 --- antisamy-markup-formatter:2.7 apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61 bootstrap4-api:4.6.0-5 bootstrap5-api:5.2.0-2 bouncycastle-api:2.26 build-timeout:1.21 caffeine-api:2.9.3-65.v6a_47d0f4d1fe checks-api:1.7.5 command-launcher:84.v4a_97f2027398 copyartifact:1.47 credentials:1139.veb_9579fca_33b_ credentials-binding:523.vd859a_4b_122e6 dashboard-view:2.447.vda_124dd35f11 display-url-api:2.3.6 durable-task:500.v8927d9fd99d8 echarts-api:5.3.3-1 font-awesome-api:6.1.2-1 git:4.11.5 git-client:3.11.2 google-compute-engine:4.3.11 google-oauth-plugin:1.0.6 instance-identity:116.vf8f487400980 jackson2-api:2.13.3-285.vc03c0256d517 jakarta-activation-api:2.0.1-1 jakarta-mail-api:2.0.1-1 javax-activation-api:1.2.0-4 javax-mail-api:1.6.2-7 jaxb:2.3.6-1 jdk-tool:55.v1b_32b_6ca_f9ca jquery3-api:3.6.0-4 jsch:0.1.55.61.va_e9ee26616e7 junit:1119.1121.vc43d0fc45561 mailer:438.v02c7f0a_12fa_4 matrix-auth:3.1.5 matrix-project:785.v06b_7f47b_c631 mina-sshd-api-common:2.8.0-36.v8e25ce90d4b_1 mina-sshd-api-core:2.8.0-36.v8e25ce90d4b_1 oauth-credentials:0.5 plain-credentials:139.ved2b_9cf7587b plugin-util-api:2.17.0 popper-api:1.16.1-3 popper2-api:2.11.6-1 project-build-times:1.2.1 scm-api:621.vda_a_b_055e58f7 script-security:1175.v4b_d517d6db_f0 snakeyaml-api:1.30.2-76.vc104f7ce9870 ssh-credentials:295.vced876c18eb_4 sshd:3.249.v2dc2ea_416e33 structs:324.va_f5d6774f3a_d token-macro:308.v4f2b_ed62b_b_16 trilead-api:1.71.v9e7860a_67a_df windows-slaves:1.8.1 workflow-api:1192.v2d0deb_19d212 workflow-scm-step:400.v6b_89a_1317c9a_ workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:838.va_3a_087b_4055b |
Sylvestre Ledru
added a comment - Thanks Basil.
Same issue with 2.365.
Java version is Debian's:
$ javac --version
javac 11.0.16
Unfortunately, this is the best part of the stack 
Sylvestre Ledru
added a comment - Thanks Basil.
Same issue with 2.365.
Java version is Debian's:
$ javac --version
javac 11.0.16
Unfortunately, this is the best part of the stack
Basil Crow
added a comment - Java version is Debian's
So that Java version is working with 2.363? If so, we can rule out Java as the source of the problem.
Unfortunately, this is the best part of the stack
What do you mean by "best"? A StackOverflowError caused by infinite recursion should have repeating portions, which would mean that at least one line would be seen repeating. Is that not the case for you? Can you give me at least a few hundred or thousand lines instead of a few dozen?
And can you reply to the following portion of my last message:
The only significant change to Java code in 2.364 was jenkinsci/jenkins#6907. I have no idea how that could cause a stack overflow, but if you are able to test with that change (and its subsequent follow-up, #7014 - which was released in 2.365) reverted and verify that the problem was caused by that PR, that would be very helpful toward finding a solution. Obviously, any way to reproduce the problem from scratch would be helpful as well.
Basil Crow
added a comment - Java version is Debian's
So that Java version is working with 2.363? If so, we can rule out Java as the source of the problem.
Unfortunately, this is the best part of the stack
What do you mean by "best"? A StackOverflowError caused by infinite recursion should have repeating portions, which would mean that at least one line would be seen repeating. Is that not the case for you? Can you give me at least a few hundred or thousand lines instead of a few dozen?
And can you reply to the following portion of my last message:
The only significant change to Java code in 2.364 was jenkinsci/jenkins#6907. I have no idea how that could cause a stack overflow, but if you are able to test with that change (and its subsequent follow-up, #7014 - which was released in 2.365) reverted and verify that the problem was caused by that PR, that would be very helpful toward finding a solution. Obviously, any way to reproduce the problem from scratch would be helpful as well. Sylvestre Ledru
made changes -
| Summary | Original: infinite loop on startup with 2.364 | New: infinite loop on node startup - Caught exception evaluating: h.escape(item.why) in /ajaxBuildQueue. |
Sylvestre Ledru
added a comment - Basil, sorry for the latency.
So, I tried to upgrade 2.365 (which was still failing) and downgrade to 2.363 (which is now failing too )
As you said, it is probably not jenkins itself but more a extension.
Everything starts correctly, jobs pending but not started in the nodes started by the gcp extension (https://cloud.google.com/architecture/using-jenkins-for-distributed-builds-on-compute-engine)
Here is what I see in the startup log:
Aug 29 08:26:06 jenkins-master jenkins[174325]: 2022-08-29 08:26:06.610+0000 [id=16]#011WARNING#011h.ExpressionFactory2$JexlExpression#evaluate: Caught exception evaluating: h.escape(item.why) in /. Reason: java.lang.reflect.InvocationTargetException
Aug 29 08:26:06 jenkins-master jenkins[174325]: java.lang.StackOverflowError
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:79)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:37)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at java.base/java.util.AbstractList$ListItr.previous(AbstractList.java:416)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:491)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:428)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:244)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:944)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:138)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Job.isLogUpdated(Job.java:314)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1077)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
Aug 29 08:26:06 jenkins-master jenkins[174325]: #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)OR
OR:
Aug 29, 2022 2:14:25 PM INFO hudson.WebAppMain contextInitialized
Jenkins home directory: /var/lib/jenkins found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Aug 29, 2022 2:14:25 PM INFO org.eclipse.jetty.server.handler.ContextHandler doStart
Started w.@5dda6f9{Jenkins v2.365,/,file:///var/cache/jenkins/war/,AVAILABLE}
Unknown macro: {/var/cache/jenkins/war}Aug 29, 2022 2:14:25 PM INFO org.eclipse.jetty.server.AbstractConnector doStart
Started ServerConnector@7181ae3f{HTTP/1.1, (http/1.1)}
Unknown macro: {0.0.0.0}Aug 29, 2022 2:14:25 PM INFO org.eclipse.jetty.server.Server doStart
Started Server@dd3b207{STARTING}[10.0.11,sto=0] @6004ms
Aug 29, 2022 2:14:26 PM INFO winstone.Logger logInternal
Winstone Servlet Engine running: controlPort=disabled
Aug 29, 2022 2:14:26 PM INFO jenkins.InitReactorRunner$1 onAttained
Started initialization
Aug 29, 2022 2:14:26 PM INFO hudson.ClassicPluginStrategy createPluginWrapper
Plugin project-build-times.jpi is disabled
Aug 29, 2022 2:14:26 PM INFO jenkins.InitReactorRunner$1 onAttained
Listed all plugins
Aug 29, 2022 2:14:31 PM INFO jenkins.InitReactorRunner$1 onAttained
Prepared all plugins
Aug 29, 2022 2:14:31 PM INFO jenkins.InitReactorRunner$1 onAttained
Started all plugins
Aug 29, 2022 2:14:31 PM INFO jenkins.InitReactorRunner$1 onAttained
Augmented all extensions
Aug 29, 2022 2:14:32 PM INFO hudson.plugins.build_timeout.global.GlobalTimeOutConfiguration load
global timeout not set
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect arm64-build-3
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect arm64-build-4
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-1
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-2
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-3
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-4
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781852513-5e761bd0364be-ddaeee4f-07f03d62 to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781857194-5e761bd4ad0f5-f5c28e43-05e7971e to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781856154-5e761bd3af30f-9fdd6cb1-14fb7d16 to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781853700-5e761bd1580d8-6bf2b015-da71c591 to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781855056-5e761bd2a31d1-f85e1695-fb339557 to complete...
Aug 29, 2022 2:14:33 PM INFO jenkins.InitReactorRunner$1 onAttained
System config loaded
Aug 29, 2022 2:14:40 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-1
Aug 29, 2022 2:14:40 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-2
Aug 29, 2022 2:14:40 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-3
Aug 29, 2022 2:14:42 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-4
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-7atrj8
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-b99pph
Aug 29, 2022 2:14:42 PM INFO hudson.slaves.CommandLauncher launch
agent launched for arm64-build-4
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-39ijgs
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO hudson.slaves.CommandLauncher launch
agent launched for arm64-build-3
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-zw8bj6
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-gucyvb
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.24 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.13 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.41 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.16 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.26 on port 22, with timeout 10000.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO jenkins.InitReactorRunner$1 onAttained
System config adapted
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO jenkins.InitReactorRunner$1 onAttained
Loaded all jobs
Aug 29, 2022 2:14:45 PM INFO jenkins.InitReactorRunner$1 onAttained
Configuration for all jobs updated
Aug 29, 2022 2:14:45 PM INFO hudson.model.AsyncPeriodicWork lambda$doRun$0
Started Download metadata
Aug 29, 2022 2:14:45 PM INFO hudson.model.AsyncPeriodicWork lambda$doRun$0
Finished Download metadata. 14 ms
Aug 29, 2022 2:14:45 PM SEVERE hudson.triggers.SafeTimerTask run
Timer task hudson.model.Queue$MaintainTask@42373f86 failed
java.lang.StackOverflowError
at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:78)
at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:37)
at java.base/java.util.AbstractList$ListItr.previous(AbstractList.java:416)
at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:491)
at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:428)
at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:244)
at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:944)
at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:138)
at hudson.model.Job.isLogUpdated(Job.java:314)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1077)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
Sylvestre Ledru
added a comment - Basil, sorry for the latency.
So, I tried to upgrade 2.365 (which was still failing) and downgrade to 2.363 (which is now failing too )
As you said, it is probably not jenkins itself but more a extension.
Everything starts correctly, jobs pending but not started in the nodes started by the gcp extension ( https://cloud.google.com/architecture/using-jenkins-for-distributed-builds-on-compute-engine )
Here is what I see in the startup log:
Aug 29 08:26:06 jenkins-master jenkins [174325] : 2022-08-29 08:26:06.610+0000 [id=16] #011WARNING#011h.ExpressionFactory2$JexlExpression#evaluate: Caught exception evaluating: h.escape(item.why) in /. Reason: java.lang.reflect.InvocationTargetException
Aug 29 08:26:06 jenkins-master jenkins [174325] : java.lang.StackOverflowError
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:79)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:37)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at java.base/java.util.AbstractList$ListItr.previous(AbstractList.java:416)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:491)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:428)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:244)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:944)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:138)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Job.isLogUpdated(Job.java:314)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1077)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
Aug 29 08:26:06 jenkins-master jenkins [174325] : #011at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)OR
OR:
Aug 29, 2022 2:14:25 PM INFO hudson.WebAppMain contextInitialized
Jenkins home directory: /var/lib/jenkins found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Aug 29, 2022 2:14:25 PM INFO org.eclipse.jetty.server.handler.ContextHandler doStart
Started w.@5dda6f9{Jenkins v2.365,/, file:///var/cache/jenkins/war/,AVAILABLE }
Unknown macro: {/var/cache/jenkins/war}
Aug 29, 2022 2:14:25 PM INFO org.eclipse.jetty.server.AbstractConnector doStart
Started ServerConnector@7181ae3f{HTTP/1.1, (http/1.1)}
Unknown macro: {0.0.0.0}
Aug 29, 2022 2:14:25 PM INFO org.eclipse.jetty.server.Server doStart
Started Server@dd3b207{STARTING} [10.0.11,sto=0] @6004ms
Aug 29, 2022 2:14:26 PM INFO winstone.Logger logInternal
Winstone Servlet Engine running: controlPort=disabled
Aug 29, 2022 2:14:26 PM INFO jenkins.InitReactorRunner$1 onAttained
Started initialization
Aug 29, 2022 2:14:26 PM INFO hudson.ClassicPluginStrategy createPluginWrapper
Plugin project-build-times.jpi is disabled
Aug 29, 2022 2:14:26 PM INFO jenkins.InitReactorRunner$1 onAttained
Listed all plugins
Aug 29, 2022 2:14:31 PM INFO jenkins.InitReactorRunner$1 onAttained
Prepared all plugins
Aug 29, 2022 2:14:31 PM INFO jenkins.InitReactorRunner$1 onAttained
Started all plugins
Aug 29, 2022 2:14:31 PM INFO jenkins.InitReactorRunner$1 onAttained
Augmented all extensions
Aug 29, 2022 2:14:32 PM INFO hudson.plugins.build_timeout.global.GlobalTimeOutConfiguration load
global timeout not set
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect arm64-build-3
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect arm64-build-4
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-1
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-2
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-3
Aug 29, 2022 2:14:33 PM INFO hudson.slaves.SlaveComputer tryReconnect
Attempting to reconnect s390x-build-4
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781852513-5e761bd0364be-ddaeee4f-07f03d62 to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781857194-5e761bd4ad0f5-f5c28e43-05e7971e to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781856154-5e761bd3af30f-9fdd6cb1-14fb7d16 to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781853700-5e761bd1580d8-6bf2b015-da71c591 to complete...
Aug 29, 2022 2:14:33 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher launch
Launch will wait 300000 for operation operation-1661781855056-5e761bd2a31d1-f85e1695-fb339557 to complete...
Aug 29, 2022 2:14:33 PM INFO jenkins.InitReactorRunner$1 onAttained
System config loaded
Aug 29, 2022 2:14:40 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-1
Aug 29, 2022 2:14:40 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-2
Aug 29, 2022 2:14:40 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-3
Aug 29, 2022 2:14:42 PM INFO hudson.slaves.CommandLauncher launch
agent launched for s390x-build-4
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-7atrj8
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-b99pph
Aug 29, 2022 2:14:42 PM INFO hudson.slaves.CommandLauncher launch
agent launched for arm64-build-4
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-39ijgs
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO hudson.slaves.CommandLauncher launch
agent launched for arm64-build-3
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-zw8bj6
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching instance: deb-gucyvb
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
bootstrap
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Getting keypair...
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Using autogenerated ssh keypair
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Authenticating as jenkins
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.24 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.13 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.41 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.16 on port 22, with timeout 10000.
Aug 29, 2022 2:14:42 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connecting to 10.132.0.26 on port 22, with timeout 10000.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:43 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Connected via SSH.
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Verifying: java -fullversion
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Copying agent.jar to: /tmp
Aug 29, 2022 2:14:44 PM INFO jenkins.InitReactorRunner$1 onAttained
System config adapted
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO com.google.jenkins.plugins.computeengine.ComputeEngineCloud log
Launching Jenkins agent via plugin SSH: java -jar /tmp/agent.jar
Aug 29, 2022 2:14:45 PM INFO jenkins.InitReactorRunner$1 onAttained
Loaded all jobs
Aug 29, 2022 2:14:45 PM INFO jenkins.InitReactorRunner$1 onAttained
Configuration for all jobs updated
Aug 29, 2022 2:14:45 PM INFO hudson.model.AsyncPeriodicWork lambda$doRun$0
Started Download metadata
Aug 29, 2022 2:14:45 PM INFO hudson.model.AsyncPeriodicWork lambda$doRun$0
Finished Download metadata. 14 ms
Aug 29, 2022 2:14:45 PM SEVERE hudson.triggers.SafeTimerTask run
Timer task hudson.model.Queue$MaintainTask@42373f86 failed
java.lang.StackOverflowError
at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:78)
at jenkins.model.lazy.SortedIntList.get(SortedIntList.java:37)
at java.base/java.util.AbstractList$ListItr.previous(AbstractList.java:416)
at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:491)
at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:428)
at jenkins.model.lazy.LazyBuildMixIn.getLastBuild(LazyBuildMixIn.java:244)
at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:944)
at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:138)
at hudson.model.Job.isLogUpdated(Job.java:314)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1077)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089)
at hudson.model.Queue.getCauseOfBlockageForTask(Queue.java:1240)
at hudson.model.Queue.getCauseOfBlockageForItem(Queue.java:1197)
at hudson.model.Queue$BlockedItem.getCauseOfBlockage(Queue.java:2630)
at hudson.model.AbstractProject.getBuildingDownstream(AbstractProject.java:1115)
at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1089) Basil Crow
added a comment - I have no idea what is wrong with your installation based on the information you've provided. You'll have to provide more information or steps to reproduce from scratch if you need more help.
Basil Crow
added a comment - I have no idea what is wrong with your installation based on the information you've provided. You'll have to provide more information or steps to reproduce from scratch if you need more help. Basil Crow
made changes -
| Link |
New:
This issue is duplicated by |
Hi sylvestre_, sorry to hear you are having this problem on 2.364. Are you also having it on 2.365? And can you share some information about the version of Java you are using in both the working and failing scenario? For example, are you using Java from the official Jenkins Docker image, or your own version? And can you share more of the log file? I don't need the full (hundreds of MiB) log, but something more detailed than what you pasted above would be helpful, especially if it illustrates the repeating pattern or the recursive portion of the stack. What I need to see is the entire portion of the stack trace that repeats over and over again resulting in the infinite loop.
The only significant change to Java code in 2.364 was jenkinsci/jenkins#6907. I have no idea how that could cause a stack overflow, but if you are able to test with that change (and its subsequent follow-up, #7014 - which was released in 2.365) reverted and verify that the problem was caused by that PR, that would be very helpful toward finding a solution. Obviously, any way to reproduce the problem from scratch would be helpful as well.