-
Type:
Bug
-
Resolution: Not A Defect
-
Priority:
Major
-
Component/s: remoting
-
Environment:Operating System: Arch Linux
Java: OpenJDK 8
Jenkins: 2.151
Plugins:
Git client plugin
Folders Plugin
Docker Commons Plugin
GitHub API Plugin
JSch dependency plugin
Token Macro Plugin
Branch API Plugin
Windows Slaves Plugin
Slave SetupPlugin
Pipeline: Declarative
MapDB API Plugin
Pipeline: Multibranch
Icon Shim Plugin
Gradle Plugin
Git plugin
Matrix Project Plugin
Email Extension Plugin
GIT server Plugin
Plain Credentials Plugin
Pipeline: Basic Steps
Subversion Plug-in
Build Authorization Token Root Plugin
LDAP Plugin
Mercurial plugin
Pipeline: Shared Groovy Libraries
Mailer Plugin
Active Directory plugin
HTML Publisher plugin
Resource Disposer Plugin
JavaScript GUI Lib: Handlebars bundle plugin
Phabricator Differential Plugin
Pipeline: Stage Tags Metadata
Pipeline
Build Timeout
Pipeline Graph Analysis Plugin
SCM API Plugin
Pipeline: Supporting APIs
PAM Authentication plugin
Pipeline: Declarative Agent API
OWASP Markup Formatter Plugin
Durable Task Plugin
HTTP Request Plugin
Pipeline: API
Apache HttpComponents Client 4.x API Plugin
Pipeline: Model API
Timestamper
Jackson 2 API Plugin
Pipeline: Declarative Extension Points API
Authentication Tokens API Plugin
Doxygen Plug-in
Pipeline: Nodes and Processes
Pipeline: Groovy
Script Security Plugin
Pipeline: Job
Simple Theme Plugin
Pipeline: Milestone Step
Display URL API
Matrix Authorization Strategy Plugin
Permissive Script Security Plugin
GitHub Branch Source Plugin
Pipeline: Input Step
Structs Plugin
JavaScript GUI Lib: Moment.js bundle plugin
Docker Pipeline
Credentials Plugin
build-name-setter
GitHub plugin
SSH Credentials Plugin
Command Agent Launcher Plugin
JavaScript GUI Lib: ACE Editor bundle plugin
Test Results Analyzer Plugin
JUnit Plugin
Pipeline: Step API
Credentials Binding Plugin
JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin
Pipeline: REST API Plugin
Pipeline: Stage Step
Pipeline: Stage View Plugin
Javadoc Plugin
Pipeline: Build Step
External Monitor Job Type Plugin
File Operations Plugin
Workspace Cleanup Plugin
Pipeline: SCM Step
JDK Tool Plugin
Lockable Resources plugin
bouncycastle API Plugin
SSH Slaves plugin
Ant Plugin
Pipeline: GitHub Groovy Libraries
How installed: Via pacman
Browser: Firefox 60.3.0esrOperating System: Arch Linux Java: OpenJDK 8 Jenkins: 2.151 Plugins: Git client plugin Folders Plugin Docker Commons Plugin GitHub API Plugin JSch dependency plugin Token Macro Plugin Branch API Plugin Windows Slaves Plugin Slave SetupPlugin Pipeline: Declarative MapDB API Plugin Pipeline: Multibranch Icon Shim Plugin Gradle Plugin Git plugin Matrix Project Plugin Email Extension Plugin GIT server Plugin Plain Credentials Plugin Pipeline: Basic Steps Subversion Plug-in Build Authorization Token Root Plugin LDAP Plugin Mercurial plugin Pipeline: Shared Groovy Libraries Mailer Plugin Active Directory plugin HTML Publisher plugin Resource Disposer Plugin JavaScript GUI Lib: Handlebars bundle plugin Phabricator Differential Plugin Pipeline: Stage Tags Metadata Pipeline Build Timeout Pipeline Graph Analysis Plugin SCM API Plugin Pipeline: Supporting APIs PAM Authentication plugin Pipeline: Declarative Agent API OWASP Markup Formatter Plugin Durable Task Plugin HTTP Request Plugin Pipeline: API Apache HttpComponents Client 4.x API Plugin Pipeline: Model API Timestamper Jackson 2 API Plugin Pipeline: Declarative Extension Points API Authentication Tokens API Plugin Doxygen Plug-in Pipeline: Nodes and Processes Pipeline: Groovy Script Security Plugin Pipeline: Job Simple Theme Plugin Pipeline: Milestone Step Display URL API Matrix Authorization Strategy Plugin Permissive Script Security Plugin GitHub Branch Source Plugin Pipeline: Input Step Structs Plugin JavaScript GUI Lib: Moment.js bundle plugin Docker Pipeline Credentials Plugin build-name-setter GitHub plugin SSH Credentials Plugin Command Agent Launcher Plugin JavaScript GUI Lib: ACE Editor bundle plugin Test Results Analyzer Plugin JUnit Plugin Pipeline: Step API Credentials Binding Plugin JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin Pipeline: REST API Plugin Pipeline: Stage Step Pipeline: Stage View Plugin Javadoc Plugin Pipeline: Build Step External Monitor Job Type Plugin File Operations Plugin Workspace Cleanup Plugin Pipeline: SCM Step JDK Tool Plugin Lockable Resources plugin bouncycastle API Plugin SSH Slaves plugin Ant Plugin Pipeline: GitHub Groovy Libraries How installed: Via pacman Browser: Firefox 60.3.0esr
Some of our jobs rely on an external slave. This has been working for a while without any issues. Recently, the number of jobs that run daily has been increased. This is when our problems started. After a while Jenkins has consumed all the memory available to the VM, and it locks up as a result.
The log is full of this exception:
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: SEVERE: This command is created here
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: Nov 12, 2018 5:03:53 PM hudson.remoting.Channel$1 handle
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: SEVERE: Failed to execute command Pipe.Flush(-1) (channel PLTSTSRV001)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: java.util.concurrent.ExecutionException: Invalid object ID -1 iota=1723
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:478)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ExportTable.get(ExportTable.java:397)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.Channel.getExportedObject(Channel.java:780)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ProxyOutputStream$Flush.execute(ProxyOutputStream.java:307)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.Channel$1.handle(Channel.java:565)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:85)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: Caused by: java.lang.Exception: Object appears to be deallocated at lease before Mon Nov 12 16:39:51 CET 2018
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: at hudson.remoting.ExportTable.diagnoseInvalidObjectId(ExportTable.java:474)
Nov 12 17:03:53 mfa-sp.microflown.loc jenkins[46451]: ... 5 more
I think this is what's causing Jenkins to leak.
- relates to
-
-
- Resolved
-
