Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44330

LogRecorderManager.init failed due to StackOverflowError

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • core
    • Jenkins 2.61
      OpenJDK 1.8.0_131

      On startup, this happens and Jenkins is inaccessible:

      Running from: /usr/share/jenkins/jenkins.war
      May 17, 2017 4:19:01 PM Main deleteWinstoneTempContents
      WARNING: Failed to delete the temporary Winstone file /tmp/winstone/jenkins.war
      May 17, 2017 4:19:01 PM org.eclipse.jetty.util.log.Log initialized
      INFO: Logging initialized @305ms to org.eclipse.jetty.util.log.JavaUtilLog
      May 17, 2017 4:19:01 PM winstone.Logger logInternal
      INFO: Beginning extraction from war file
      May 17, 2017 4:19:01 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath
      WARNING: Empty contextPath
      May 17, 2017 4:19:01 PM org.eclipse.jetty.server.Server doStart
      INFO: jetty-9.4.z-SNAPSHOT
      May 17, 2017 4:19:01 PM org.eclipse.jetty.webapp.StandardDescriptorProcessor visitServlet
      INFO: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
      May 17, 2017 4:19:01 PM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
      INFO: DefaultSessionIdManager workerName=node0
      May 17, 2017 4:19:01 PM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
      INFO: No SessionScavenger set, using defaults
      May 17, 2017 4:19:01 PM org.eclipse.jetty.server.session.HouseKeeper startScavenging
      INFO: Scavenging every 660000ms
      Jenkins home directory: /var/lib/jenkins found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
      May 17, 2017 4:19:02 PM org.eclipse.jetty.server.handler.ContextHandler doStart
      INFO: Started w.@3e08ff24{/,file:///var/cache/jenkins/war/,AVAILABLE} {/var/cache/jenkins/war}May 17, 2017 4:19:02 PM org.eclipse.jetty.server.AbstractConnector doStart
      INFO: Started ServerConnector@4c163e3{HTTP/1.1,[http/1.1]}{::1:8080}
      May 17, 2017 4:19:02 PM org.eclipse.jetty.server.Server doStart
      INFO: Started @1223ms
      May 17, 2017 4:19:02 PM winstone.Logger logInternal
      INFO: Winstone Servlet Engine v4.0 running: controlPort=disabled
      May 17, 2017 4:19:02 PM jenkins.InitReactorRunner$1 onAttained
      INFO: Started initialization
      May 17, 2017 4:19:03 PM jenkins.InitReactorRunner$1 onTaskFailed
      SEVERE: Failed LogRecorderManager.init
      java.lang.Error: java.lang.reflect.InvocationTargetException
      at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
      at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
      at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
      at jenkins.model.Jenkins$7.runTask(Jenkins.java:1088)
      at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
      ... 8 more
      Caused by: java.lang.StackOverflowError
      at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304)
      at java.lang.StringCoding.encode(StringCoding.java:344)
      at java.lang.String.getBytes(String.java:918)
      at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
      at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242)
      at java.io.File.exists(File.java:819)
      at sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1282)
      at sun.misc.URLClassPath$FileLoader.findResource(URLClassPath.java:1249)
      at sun.misc.URLClassPath.findResource(URLClassPath.java:215)
      at java.net.URLClassLoader$2.run(URLClassLoader.java:569)
      at java.net.URLClassLoader$2.run(URLClassLoader.java:567)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findResource(URLClassLoader.java:566)
      at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:546)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at hudson.logging.LogRecorder.orderedTargets(LogRecorder.java:82)
      at hudson.logging.LogRecorder$2.publish(LogRecorder.java:108)
      at hudson.logging.WeakLogHandler.publish(WeakLogHandler.java:53)
      at java.util.logging.Logger.log(Logger.java:738)
      at org.eclipse.jetty.util.log.JavaUtilLog.log(JavaUtilLog.java:167)
      at org.eclipse.jetty.util.log.JavaUtilLog.debug(JavaUtilLog.java:227)
      at org.eclipse.jetty.webapp.WebAppContext.isSystemResource(WebAppContext.java:857)
      at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:548)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at hudson.logging.LogRecorder.orderedTargets(LogRecorder.java:82)
      at hudson.logging.LogRecorder$2.publish(LogRecorder.java:108)
      at hudson.logging.WeakLogHandler.publish(WeakLogHandler.java:53)
      at java.util.logging.Logger.log(Logger.java:738)
      at org.eclipse.jetty.util.log.JavaUtilLog.log(JavaUtilLog.java:167)
      at org.eclipse.jetty.util.log.JavaUtilLog.debug(JavaUtilLog.java:227)
      at org.eclipse.jetty.webapp.WebAppContext.isSystemResource(WebAppContext.java:857)
      at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:548)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at hudson.logging.LogRecorder.orderedTargets(LogRecorder.java:82)
      at hudson.logging.LogRecorder$2.publish(LogRecorder.java:108)
      at hudson.logging.WeakLogHandler.publish(WeakLogHandler.java:53)
      at java.util.logging.Logger.log(Logger.java:738)
      at org.eclipse.jetty.util.log.JavaUtilLog.log(JavaUtilLog.java:167)
      at org.eclipse.jetty.util.log.JavaUtilLog.debug(JavaUtilLog.java:227)

      (snipped many repetitions of the same)

      at org.eclipse.jetty.webapp.WebAppContext.isSystemResource(WebAppContext.java:857)
      at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:548)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at hudson.logging.LogRecorder.orderedTargets(LogRecorder.java:82)
      at hudson.logging.LogRecorder$2.publish(LogRecorder.java:108)
      at hudson.logging.WeakLogHandler.publish(WeakLogHandler.java:53)
      at java.util.logging.Logger.log(Logger.java:738)
      at org.eclipse.jetty.util.log.JavaUtilLog.log(JavaUtilLog.java:167)
      at org.eclipse.jetty.util.log.JavaUtilLog.debug(JavaUtilLog.java:227)
      at org.eclipse.jetty.webapp.WebAppContext.isSystemResource(WebAppContext.java:857)
      at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:548)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at hudson.logging.LogRecorder.orderedTargets(LogRecorder.java:82)
      at hudson.logging.LogRecorder$2.publish(LogRecorder.java:108)
      at hudson.logging.WeakLogHandler.publish(WeakLogHandler.java:53)
      at java.util.logging.Logger.log(Logger.java:738)
      at org.eclipse.jetty.util.log.JavaUtilLog.log(JavaUtilLog.java:167)
      at org.eclipse.jetty.util.log.JavaUtilLog.debug(JavaUtilLog.java:227)
      at org.eclipse.jetty.webapp.WebAppContext.isSystemResource(WebAppContext.java:857)
      at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:548)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at hudson.logging.LogRecorder.orderedTargets(LogRecorder.java:82)
      at hudson.logging.LogRecorder$2.publish(LogRecorder.java:108)
      at hudson.logging.WeakLogHandler.publish(WeakLogHandler.java:53)
      at java.util.logging.Logger.log(Logger.java:738)
      at org.eclipse.jetty.util.log.JavaUtilLog.log(JavaUtilLog.java:167)
      at org.eclipse.jetty.util.log.JavaUtilLog.debug(JavaUtilLog.java:227)
      at org.eclipse.jetty.webapp.WebAppContext.isSystemResource(WebAppContext.java:857)
      at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:548)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at hudson.logging.LogRecorder.orderedTargets(LogRecorder.java:82)

          [JENKINS-44330] LogRecorderManager.init failed due to StackOverflowError

          Oleg Nenashev added a comment -

           CC olamy. I'd guess it is related to the Jetty update

          Oleg Nenashev added a comment -  CC olamy . I'd guess it is related to the Jetty update

          Oleg Nenashev added a comment -

          So it will happen if and only if the Debug logging for Jetty is enabled: https://github.com/eclipse/jetty.project/blob/jetty-9.4.x/jetty-webapp/src/main/java/org/eclipse/jetty/webapp/WebAppContext.java#L857

          It has been added by this commit: https://github.com/eclipse/jetty.project/commit/afb750515e3f5bd2f3ebb7a84b5dc34f7cdcef6a . Will raise an issue in the Jetty bugtracker

          Oleg Nenashev added a comment - So it will happen if and only if the Debug logging for Jetty is enabled: https://github.com/eclipse/jetty.project/blob/jetty-9.4.x/jetty-webapp/src/main/java/org/eclipse/jetty/webapp/WebAppContext.java#L857 It has been added by this commit: https://github.com/eclipse/jetty.project/commit/afb750515e3f5bd2f3ebb7a84b5dc34f7cdcef6a . Will raise an issue in the Jetty bugtracker

          Oleg Nenashev added a comment -

          Oleg Nenashev added a comment - created https://github.com/eclipse/jetty.project/issues/1563. Working n the hotfix

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/src/main/java/hudson/logging/LogRecorder.java
          http://jenkins-ci.org/commit/jenkins/e4c4b810793aa4d118702608c16c1d680edbab89
          Log:
          [FIXED JENKINS-44330] - Prevent classloading of Target comparator in LogRecorder#orderedTargets()

          It is just a hotfix, there may be other LogRecorders affected. Ideally we need a response from Jetty maintainers to https://github.com/eclipse/jetty.project/issues/1563. No tests since I see no way to trigger such classloading + no actual need in it.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/logging/LogRecorder.java http://jenkins-ci.org/commit/jenkins/e4c4b810793aa4d118702608c16c1d680edbab89 Log: [FIXED JENKINS-44330] - Prevent classloading of Target comparator in LogRecorder#orderedTargets() It is just a hotfix, there may be other LogRecorders affected. Ideally we need a response from Jetty maintainers to https://github.com/eclipse/jetty.project/issues/1563 . No tests since I see no way to trigger such classloading + no actual need in it.

          Daniel Beck added a comment - - edited

          Fixed in 2.62.

          Daniel Beck added a comment - - edited Fixed in 2.62.

            oleg_nenashev Oleg Nenashev
            haw777 Christopher Head
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: