Details
-
Bug
-
Status: Resolved (View Workflow)
-
Blocker
-
Resolution: Duplicate
-
None
Description
Our jenkins pipeline becomes very slow between step and step.
For example:
07:57:35 [manifest Install ESXI 6.0] Timeout set to expire in 1 hr 0 min 07:58:51 [Pipeline] [manifest Install ESXI 6.0] sh 08:05:35 [Pipeline] [manifest Install ESXI 6.0] retry 08:10:24 [Pipeline] [manifest Install ESXI 6.0] {
java: 1.8
Jenkins :2.32.1
pipeline: 2.5
pipeline Utility steps:1.3.0
Pipeline: Basic Steps : 2.4
Pipeline Build Step : 2.5
Pipeline Groovy: 2.30
Attachments
Issue Links
- duplicates
-
JENKINS-38381 [JEP-210] Optimize log handling in Pipeline and Durable Task
-
- Resolved
-
Activity
Scheduler-285377351 "Scheduler-285377351" Id=74 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6f9dc755 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6f9dc755 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Scheduler-5592464 "Scheduler-5592464" Id=692 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3122ca6d at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3122ca6d at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) SocketListener(fe80:0:0:0:20c:29ff:fe2f:e63c%eth0.local.) "SocketListener(fe80:0:0:0:20c:29ff:fe2f:e63c%eth0.local.)" Id=673 Group=main RUNNABLE (in native) at java.net.PlainDatagramSocketImpl.receive0(Native Method) - locked java.net.PlainDatagramSocketImpl@5e7a73b7 at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:143) - locked java.net.PlainDatagramSocketImpl@5e7a73b7 at java.net.DatagramSocket.receive(DatagramSocket.java:812) - locked java.net.DatagramPacket@792cabb7 - locked java.net.MulticastSocket@31d5b7bb at javax.jmdns.impl.SocketListener.run(SocketListener.java:41) TCP agent listener port=0 "TCP agent listener port=0" Id=669 Group=main RUNNABLE (in native) at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) - locked java.lang.Object@3d035f24 at hudson.TcpSlaveAgentListener.run(TcpSlaveAgentListener.java:145) Thread-13 "Thread-13" Id=337 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-14 "Thread-14" Id=338 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-15 "Thread-15" Id=339 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-16 "Thread-16" Id=343 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-17 "Thread-17" Id=342 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-18 "Thread-18" Id=341 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-19 "Thread-19" Id=340 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-20 "Thread-20" Id=347 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-21 "Thread-21" Id=346 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-22 "Thread-22" Id=345 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-23 "Thread-23" Id=344 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-24 "Thread-24" Id=348 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-25 "Thread-25" Id=353 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-26 "Thread-26" Id=352 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-27 "Thread-27" Id=351 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-28 "Thread-28" Id=350 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-29 "Thread-29" Id=356 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-30 "Thread-30" Id=349 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-31 "Thread-31" Id=354 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-32 "Thread-32" Id=355 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-33 "Thread-33" Id=358 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-34 "Thread-34" Id=357 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-35 "Thread-35" Id=361 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-36 "Thread-36" Id=360 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-37 "Thread-37" Id=359 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748)
Thread-38 "Thread-38" Id=362 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-39 "Thread-39" Id=507 Group=main RUNNABLE (in native) at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) at com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) at com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) at com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) at com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:693) at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:489) at java.lang.Thread.run(Thread.java:748) Thread-44 "Thread-44" Id=667 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@d5e6e74 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@d5e6e74 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.jenkinsci.plugins.ssegateway.SubscriptionConfigQueue$1.run(SubscriptionConfigQueue.java:78) Thread-7 "Thread-7" Id=87 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1efaae10 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1efaae10 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.jenkinsci.plugins.pubsub.listeners.SyncQueueListener$1.run(SyncQueueListener.java:93) Timer-1 "Timer-1" Id=665 Group=main TIMED_WAITING on java.util.TaskQueue@61664af7 at java.lang.Object.wait(Native Method) - waiting on java.util.TaskQueue@61664af7 at java.util.TimerThread.mainLoop(Timer.java:552) at java.util.TimerThread.run(Timer.java:505) WorkflowRun.copyLogs [#1] (on-http #214) "WorkflowRun.copyLogs [#1] (on-http #214)" Id=812 Group=main TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@51570942 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@51570942 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) WorkflowRun.copyLogs [#2] (ContinuousFunctionTest #2057) "WorkflowRun.copyLogs [#2] (ContinuousFunctionTest #2057)" Id=814 Group=main RUNNABLE at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at org.kohsuke.stapler.framework.io.LargeText$GzipAwareSession.isGzipStream(LargeText.java:460) at org.kohsuke.stapler.framework.io.LargeText.<init>(LargeText.java:104) at hudson.console.AnnotatedLargeText.<init>(AnnotatedLargeText.java:80) at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl.getLogText(LogActionImpl.java:132) at org.jenkinsci.plugins.workflow.job.WorkflowRun.copyLogs(WorkflowRun.java:455) at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$600(WorkflowRun.java:127) at org.jenkinsci.plugins.workflow.job.WorkflowRun$3.run(WorkflowRun.java:319) - locked java.util.concurrent.atomic.AtomicBoolean@3ee6c982 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@44f77a74 WorkflowRun.copyLogs [#3] (on-core #65) "WorkflowRun.copyLogs [#3] (on-core #65)" Id=828 Group=main RUNNABLE at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at org.kohsuke.stapler.framework.io.LargeText$GzipAwareSession.isGzipStream(LargeText.java:460) at org.kohsuke.stapler.framework.io.LargeText.<init>(LargeText.java:104) at hudson.console.AnnotatedLargeText.<init>(AnnotatedLargeText.java:80) at org.jenkinsci.plugins.workflow.support.actions.LogActionImpl.getLogText(LogActionImpl.java:132) at org.jenkinsci.plugins.workflow.job.WorkflowRun.copyLogs(WorkflowRun.java:455) at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$600(WorkflowRun.java:127) at org.jenkinsci.plugins.workflow.job.WorkflowRun$3.run(WorkflowRun.java:319) - locked java.util.concurrent.atomic.AtomicBoolean@60cf9f85 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@4a308c9a WorkflowRun.copyLogs [#4] (MasterCI #212) "WorkflowRun.copyLogs [#4] (MasterCI #212)" Id=832 Group=main RUNNABLE at java.io.RandomAccessFile.open0(Native Method) at java.io.RandomAccessFile.open(RandomAccessFile.java:316) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) at org.kohsuke.stapler.framework.io.LargeText$FileSession.<init>(LargeText.java:399) at org.kohsuke.stapler.framework.io.LargeText$2.open(LargeText.java:121) at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:212) at hudson.console.AnnotatedLargeText.writeRawLogTo(AnnotatedLargeText.java:162) at org.jenkinsci.plugins.workflow.job.WorkflowRun.writeRawLogTo(WorkflowRun.java:508) at org.jenkinsci.plugins.workflow.job.WorkflowRun.copyLogs(WorkflowRun.java:468) at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$600(WorkflowRun.java:127) at org.jenkinsci.plugins.workflow.job.WorkflowRun$3.run(WorkflowRun.java:319) - locked java.util.concurrent.atomic.AtomicBoolean@e0a099f at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@6cf67d4a WorkflowRun.copyLogs [#5] (on-core #66) "WorkflowRun.copyLogs [#5] (on-core #66)" Id=833 Group=main RUNNABLE at java.io.RandomAccessFile.open0(Native Method) at java.io.RandomAccessFile.open(RandomAccessFile.java:316) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) at org.kohsuke.stapler.framework.io.LargeText$FileSession.<init>(LargeText.java:399) at org.kohsuke.stapler.framework.io.LargeText$2.open(LargeText.java:121) at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:212) at hudson.console.AnnotatedLargeText.writeRawLogTo(AnnotatedLargeText.java:162) at org.jenkinsci.plugins.workflow.job.WorkflowRun.writeRawLogTo(WorkflowRun.java:508) at org.jenkinsci.plugins.workflow.job.WorkflowRun.copyLogs(WorkflowRun.java:468) at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$600(WorkflowRun.java:127) at org.jenkinsci.plugins.workflow.job.WorkflowRun$3.run(WorkflowRun.java:319) - locked java.util.concurrent.atomic.AtomicBoolean@f4fafc5 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@4e0f1104 Finalizer "Finalizer" Id=3 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock@6c9e6e74 at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.ReferenceQueue$Lock@6c9e6e74 at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) Java2D Disposer "Java2D Disposer" Id=21 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock@794e1ddb at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.ReferenceQueue$Lock@794e1ddb at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164) at sun.java2d.Disposer.run(Disposer.java:148) at java.lang.Thread.run(Thread.java:748) Reference Handler "Reference Handler" Id=2 Group=system WAITING on java.lang.ref.Reference$Lock@48a108ff at java.lang.Object.wait(Native Method) - waiting on java.lang.ref.Reference$Lock@48a108ff at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference.tryHandlePending(Reference.java:191) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153) Signal Dispatcher "Signal Dispatcher" Id=4 Group=system RUNNABLE
There are several threads writing Pipeline log messages blocking other Pipeline activity. Perhaps disk and/or network are too slow to handle Pipeline log output?
Have a fix prepared but am not currently merging it due to lack of testers.