Details
-
Type:
Bug
-
Status: Resolved (View Workflow)
-
Priority:
Critical
-
Resolution: Fixed
-
Component/s: core
-
Labels:
-
Similar Issues:
-
Released As:Jenkins 2.238
Description
After launching an agent it keeps hanging with the following text :
[05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent
This happens since a couple of weeks
jstack reports a deadlock in attachment :
Found one Java-level deadlock:
=============================
"pool-1-thread-4 for channel id=17236":
waiting to lock monitor 0x00007fbd8c015bc8 (object 0x000000076e658428, a hudson.slaves.SlaveComputer$SlaveInitializer$1),
which is held by "Channel reader thread: channel"
"Channel reader thread: channel":
waiting to lock monitor 0x00007fbd9401aed8 (object 0x000000076ded1cc8, a hudson.remoting.RemoteClassLoader),
which is held by "pool-1-thread-4 for channel id=17236"
On april 3 2020 in commit https://github.com/jenkinsci/jenkins/commit/3262d8b7cfbdd1a2e026ad34644e0f9607f5724d
a new synchronized method was added that is partly the cause of this deadlock .See line 1026 in main/java/hudson/slaves/SlaveComputer.java
Attachments
Issue Links
- is duplicated by
-
-
- Resolved
-
- is related to
-
-
- Resolved
-
- relates to
-
-
- Resolved
-
- links to
Activity
Bart Heupers
created issue -
Bart Heupers
made changes -
| Field | Original Value | New Value |
|---|---|---|
| Description |
After launching an agent it keeps hangin with the follwing text : [05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent jstack reports a deadlock in attachment : Found one Java-level deadlock: ============================= "pool-1-thread-4 for channel id=17236": waiting to lock monitor 0x00007fbd8c015bc8 (object 0x000000076e658428, a hudson.slaves.SlaveComputer$SlaveInitializer$1), which is held by "Channel reader thread: channel" "Channel reader thread: channel": waiting to lock monitor 0x00007fbd9401aed8 (object 0x000000076ded1cc8, a hudson.remoting.RemoteClassLoader), which is held by "pool-1-thread-4 for channel id=17236" On april 3 2020 in commit [https://github.com/jenkinsci/jenkins/commit/3262d8b7cfbdd1a2e026ad34644e0f9607f5724d] a new synchronized method was added that is partly the cause of this deadlock .See line 1026 in main/java/hudson/slaves/SlaveComputer.java |
After launching an agent it keeps hanging with the following text : [05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent jstack reports a deadlock in attachment : Found one Java-level deadlock: ============================= "pool-1-thread-4 for channel id=17236": waiting to lock monitor 0x00007fbd8c015bc8 (object 0x000000076e658428, a hudson.slaves.SlaveComputer$SlaveInitializer$1), which is held by "Channel reader thread: channel" "Channel reader thread: channel": waiting to lock monitor 0x00007fbd9401aed8 (object 0x000000076ded1cc8, a hudson.remoting.RemoteClassLoader), which is held by "pool-1-thread-4 for channel id=17236" On april 3 2020 in commit [https://github.com/jenkinsci/jenkins/commit/3262d8b7cfbdd1a2e026ad34644e0f9607f5724d] a new synchronized method was added that is partly the cause of this deadlock .See line 1026 in main/java/hudson/slaves/SlaveComputer.java |
Bart Heupers
made changes -
| Description |
After launching an agent it keeps hanging with the following text : [05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent jstack reports a deadlock in attachment : Found one Java-level deadlock: ============================= "pool-1-thread-4 for channel id=17236": waiting to lock monitor 0x00007fbd8c015bc8 (object 0x000000076e658428, a hudson.slaves.SlaveComputer$SlaveInitializer$1), which is held by "Channel reader thread: channel" "Channel reader thread: channel": waiting to lock monitor 0x00007fbd9401aed8 (object 0x000000076ded1cc8, a hudson.remoting.RemoteClassLoader), which is held by "pool-1-thread-4 for channel id=17236" On april 3 2020 in commit [https://github.com/jenkinsci/jenkins/commit/3262d8b7cfbdd1a2e026ad34644e0f9607f5724d] a new synchronized method was added that is partly the cause of this deadlock .See line 1026 in main/java/hudson/slaves/SlaveComputer.java |
After launching an agent it keeps hanging with the following text : [05/06/20 20:34:24] [SSH] Starting agent process: cd "/var/lib/jenkins" && java -jar remoting.jar -workDir /var/lib/jenkins May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir INFO: Using /var/lib/jenkins/remoting as a remoting work directory May 06, 2020 8:34:25 PM org.jenkinsci.remoting.engine.WorkDirManager setupLogging INFO: Both error and output logs will be printed to /var/lib/jenkins/remoting <===[JENKINS REMOTING CAPACITY]===>channel started Remoting version: 4.3 This is a Unix agent This happens since a couple of weeks jstack reports a deadlock in attachment : Found one Java-level deadlock: ============================= "pool-1-thread-4 for channel id=17236": waiting to lock monitor 0x00007fbd8c015bc8 (object 0x000000076e658428, a hudson.slaves.SlaveComputer$SlaveInitializer$1), which is held by "Channel reader thread: channel" "Channel reader thread: channel": waiting to lock monitor 0x00007fbd9401aed8 (object 0x000000076ded1cc8, a hudson.remoting.RemoteClassLoader), which is held by "pool-1-thread-4 for channel id=17236" On april 3 2020 in commit [https://github.com/jenkinsci/jenkins/commit/3262d8b7cfbdd1a2e026ad34644e0f9607f5724d] a new synchronized method was added that is partly the cause of this deadlock .See line 1026 in main/java/hudson/slaves/SlaveComputer.java |
| Labels | deadlock | deadlock regression |
| Component/s | core [ 15593 ] | |
| Component/s | remoting [ 15489 ] |
| Assignee | Jeff Thompson [ jthompson ] | Jesse Glick [ jglick ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Labels | deadlock regression | deadlock lts-candidate regression |
| Remote Link | This issue links to "jenkins #4714 (Web Link)" [ 24902 ] |
| Status | In Progress [ 3 ] | In Review [ 10005 ] |
| Link |
This issue is duplicated by |
| Released As | Jenkins 2.238 | |
| Resolution | Fixed [ 1 ] | |
| Status | In Review [ 10005 ] | Resolved [ 5 ] |
| Labels | deadlock lts-candidate regression | 2.235.1-fixed deadlock regression |
| Link |
This issue is related to |
| Link |
This issue relates to |