Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55070

"Jenkins.instance is missing" error when restarting Jenkins

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • core
    • Windows Server 2016
      Java 1.8.0_191-b12
      Jenkins 2.150.1
    • Jenkins 2.223

      Sometimes when I try to restart Jenkins (eg. after upgrading the plugins), it just starts returning HTTP 500 errors, and I need to restart the Jenkins service.

      When this happens, I see lots of these errors in the logs:

      Dec 07, 2018 12:03:45 AM org.eclipse.jetty.server.HttpChannel handleException
      WARNING: /
      java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
       at jenkins.model.Jenkins.get(Jenkins.java:758)
       at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
       at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
       at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
       at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
       at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
       at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
       at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
       at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
       at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
       at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
       at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
       at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
       at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
       at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
       at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
       at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
       at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
       at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
       at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
       at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
       at org.eclipse.jetty.server.Server.handle(Server.java:503)
       at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
       at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
       at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
       at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
       at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
       at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
       at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
       at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
       at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
       at java.lang.Thread.run(Unknown Source)

       

          [JENKINS-55070] "Jenkins.instance is missing" error when restarting Jenkins

          Daniel Beck added a comment -

          This happens after other errors. Look for those.

          Daniel Beck added a comment - This happens after other errors. Look for those.

          I saw this again today while restarting after installing a plugin... I didn't see any other errors before it though. Seems like it happens after "INFO: Jenkins stopped" and then it doesn't restart properly?

          I did see "java.lang.IllegalStateException: Error already set" and "java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin" errors in the stack of the second instance of the error though.

          Dec 07, 2018 9:40:48 PM jenkins.model.Jenkins$23 run
          INFO: Restart in 10 seconds
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$23 run
          INFO: Restarting VM as requested by daniel15
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins cleanUp
          INFO: Stopping Jenkins
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained
          INFO: Started termination
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained
          INFO: Completed termination
          Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins _cleanUpDisconnectComputers
          INFO: Starting node disconnection
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpShutdownPluginManager
          INFO: Stopping plugin manager
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpPersistQueue
          INFO: Persisting build queue
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpAwaitDisconnects
          INFO: Waiting for node disconnection completion
          Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins cleanUp
          INFO: Jenkins stopped
          Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException
          WARNING: /
          java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          	at jenkins.model.Jenkins.get(Jenkins.java:758)
          	at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
          	at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
          	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.Server.handle(Server.java:503)
          	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
          	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
          	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
          	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
          	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
          	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
          	at java.lang.Thread.run(Unknown Source)Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException
          WARNING: ERROR dispatch failed
          java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          	at jenkins.model.Jenkins.get(Jenkins.java:758)
          	at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
          	at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
          	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
          	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          	at org.eclipse.jetty.server.Server.handle(Server.java:503)
          	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
          	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
          	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
          	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
          	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
          	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
          	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
          	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
          	at java.lang.Thread.run(Unknown Source)
          	Suppressed: java.lang.IllegalStateException: Error already set
          		at org.eclipse.jetty.server.HttpChannelState.onError(HttpChannelState.java:758)
          		at org.eclipse.jetty.server.HttpChannel.handleException(HttpChannel.java:590)
          		at org.eclipse.jetty.server.HttpChannelOverHttp.handleException(HttpChannelOverHttp.java:489)
          		at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:526)
          		... 12 more
          	Caused by: java.lang.NullPointerException
          		at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126)
          		at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
          		at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
          		at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          		at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          		at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
          		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          		at org.eclipse.jetty.server.Server.handle(Server.java:503)
          		at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
          		... 12 more
          

          Daniel Lo Nigro added a comment - I saw this again today while restarting after installing a plugin... I didn't see any other errors before it though. Seems like it happens after "INFO: Jenkins stopped" and then it doesn't restart properly? I did see "java.lang.IllegalStateException: Error already set" and "java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin" errors in the stack of the second instance of the error though. Dec 07, 2018 9:40:48 PM jenkins.model.Jenkins$23 run INFO: Restart in 10 seconds Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$23 run INFO: Restarting VM as requested by daniel15 Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins cleanUp INFO: Stopping Jenkins Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained INFO: Started termination Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins$18 onAttained INFO: Completed termination Dec 07, 2018 9:40:58 PM jenkins.model.Jenkins _cleanUpDisconnectComputers INFO: Starting node disconnection Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpShutdownPluginManager INFO: Stopping plugin manager Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpPersistQueue INFO: Persisting build queue Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins _cleanUpAwaitDisconnects INFO: Waiting for node disconnection completion Dec 07, 2018 9:40:59 PM jenkins.model.Jenkins cleanUp INFO: Jenkins stopped Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: / java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang. Thread .run(Unknown Source)Dec 07, 2018 9:42:45 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: ERROR dispatch failed java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang. Thread .run(Unknown Source) Suppressed: java.lang.IllegalStateException: Error already set at org.eclipse.jetty.server.HttpChannelState.onError(HttpChannelState.java:758) at org.eclipse.jetty.server.HttpChannel.handleException(HttpChannel.java:590) at org.eclipse.jetty.server.HttpChannelOverHttp.handleException(HttpChannelOverHttp.java:489) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:526) ... 12 more Caused by: java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:503) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) ... 12 more

          Daniel Beck added a comment -

          Yep, Jenkins apparently cannot restart itself, and just stopped.

          Potential problem with winsw / how you run Jenkins. Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed.

          Poking oleg_nenashev in case he's still maintaining winsw.

          Daniel Beck added a comment - Yep, Jenkins apparently cannot restart itself, and just stopped. Potential problem with winsw / how you run Jenkins. Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed. Poking oleg_nenashev in case he's still maintaining winsw .

          Oleg Nenashev added a comment -

          danielbeck yes, I somewhat maintain it. Though I do not see evidence that the issue caused by WinSW:

           

           

          Caused by: java.lang.NullPointerException
          		at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126)
          		at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          		at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
          

          Due to whatever reason the autoLogin() logic is invoked before Jenkins holder class is fully initialized: https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/security/TokenBasedRememberMeServices2.java#L126 . This is a regression caused by the SECURITY-996 fix in this commit by wfollonierhttps://github.com/jenkinsci/jenkins/commit/ef9583a24abc4de157e1570cb32d7a273d327f36 

           

           

           

          Oleg Nenashev added a comment - danielbeck yes, I somewhat maintain it. Though I do not see evidence that the issue caused by WinSW:     Caused by: java.lang.NullPointerException at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:126) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) Due to whatever reason the autoLogin() logic is invoked before Jenkins holder class is fully initialized: https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/security/TokenBasedRememberMeServices2.java#L126 . This is a regression caused by the SECURITY-996 fix in this commit by wfollonier :  https://github.com/jenkinsci/jenkins/commit/ef9583a24abc4de157e1570cb32d7a273d327f36        

          Oleg Nenashev added a comment -

          The root exception is also a bug IMHO, but it existed in the code for a while

          java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          	at jenkins.model.Jenkins.get(Jenkins.java:758)
                  at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
          

          Instead of invoking Jenkins.get(), the code should use the Jenkins instance passed by the initializer

          For me it's hard to say what is a root cause, but we really need to cleanup the code a bit

          Oleg Nenashev added a comment - The root exception is also a bug IMHO, but it existed in the code for a while java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) Instead of invoking Jenkins.get(), the code should use the Jenkins instance passed by the initializer For me it's hard to say what is a root cause, but we really need to cleanup the code a bit

          Daniel Beck added a comment -

          This does not look like it would affect a Jenkins restart. Note this occurred two minutes after Jenkins was stopped as part of a restart.

          Daniel Beck added a comment - This does not look like it would affect a Jenkins restart. Note this occurred two minutes after Jenkins was stopped as part of a restart.

          Daniel Lo Nigro added a comment - - edited

          Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed.

          danielbeck - I originally installed Jenkins 2.138.3 via the Windows installer. The only change I've made is that I've changed the service to run as a separate Jenkins user rather than as SYSTEM for better security (maybe that's causing issues?).

          I do see some errors like this in the Event Viewer:

          SIGINT to 6536 failed - Killing as fallback

          What is the "Jenkins service log"? Is that different to the C:\Program Files (x86)\Jenkins\jenkins.err.log file I've been looking at?

           

          Daniel Lo Nigro added a comment - - edited Check the Windows event log, the Jenkins service log, and provide complete information how and when (what version originally) Jenkins was installed. danielbeck - I originally installed Jenkins 2.138.3 via the Windows installer. The only change I've made is that I've changed the service to run as a separate Jenkins user rather than as SYSTEM for better security (maybe that's causing issues?). I do see some errors like this in the Event Viewer: SIGINT to 6536 failed - Killing as fallback What is the "Jenkins service log"? Is that different to the C:\Program Files (x86)\Jenkins\jenkins.err.log file I've been looking at?  

          Manuel Cardenas added a comment - - edited

          danielbeck , we have seen this issue in a non windows installation, Centos 7.x

           * OpenJDK 64-Bit Server VM 25.161-b14

          The complete context: We are upgrading Jenkins from 2.138.3.1 to 2.138.4.3 using a war file. Once that we use the new war file, the service is not able to start. We can see the trace below.

           

          Dec 11, 2018 3:22:10 PM org.eclipse.jetty.server.HttpChannel handleException
           WARNING: /login
           java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
           at jenkins.model.Jenkins.get(Jenkins.java:758) 
          ...
          
          Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException
           WARNING: /tcpSlaveAgentListener/
           java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
           at jenkins.model.Jenkins.get(Jenkins.java:758)
           at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
           at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
           at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
           at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
           at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
           at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
           at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
           at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
           at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.Server.handle(Server.java:531)
           at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
           at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
           at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
           at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
           at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
           at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
           at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
           at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
           at java.lang.Thread.run(Thread.java:748)
           Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException
           WARNING: ERROR dispatch failed
           java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
           at jenkins.model.Jenkins.get(Jenkins.java:758)
           at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
           at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
           at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
           at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
           at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
           at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
           at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
           at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
           at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.Server.handle(Server.java:531)
           at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
           Stopping Jenkins Daemon...
           jenkins.service: main process exited, code=exited, status=143/n/a
           Stopped Jenkins Daemon.
           Unit jenkins.service entered failed state.
           jenkins.service failed.
          

          Manuel Cardenas added a comment - - edited danielbeck , we have seen this issue in a non windows installation, Centos 7.x  * OpenJDK 64-Bit Server VM 25.161-b14 The complete context: We are upgrading Jenkins from 2.138.3.1 to 2.138.4.3 using a war file. Once that we use the new war file, the service is not able to start. We can see the trace below.   Dec 11, 2018 3:22:10 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: /login java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) ... Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: /tcpSlaveAgentListener/ java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) at java.lang. Thread .run( Thread .java:748) Dec 11, 2018 3:22:42 PM org.eclipse.jetty.server.HttpChannel handleException WARNING: ERROR dispatch failed java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36) at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) Stopping Jenkins Daemon... jenkins.service: main process exited, code=exited, status=143/n/a Stopped Jenkins Daemon. Unit jenkins.service entered failed state. jenkins.service failed.

          mcardenasblanco From my PoV, you are not affected by the same exception (rememberMe) but another introduced in previous release. It seems very likely that the code in rememberMe is not the culprit.

          In your case the full stacktrace shows:

          Caused by: java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          at jenkins.model.Jenkins.get(Jenkins.java:758)
          at jenkins.telemetry.impl.StaplerDispatches$StaplerTrace.record(StaplerDispatches.java:105)
          at org.kohsuke.stapler.EvaluationTrace$ApplicationTracer.trace(EvaluationTrace.java:77)
          at org.kohsuke.stapler.Dispatcher.anonymizedTraceEval(Dispatcher.java:76)
          at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:685)
          at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870)
          at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668)
          at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
          at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
          at com.cloudbees.jenkins.ha.HAHealthCheckFilter.doFilter(HAHealthCheckFilter.java:35)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
          at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
          at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:64)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:169)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
          at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
          at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
          at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
          at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
          at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
          at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
          at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
          at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
          at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
          at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)
          at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
          at org.eclipse.jetty.server.Server.handle(Server.java:531)
          at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
          ... 12 more
          

          So I created 2 PR for this ticket, 3805 to correct the rememberMe but I highly suspect to not correct the problem and 3806 to correct the legacy behavior there.
          From my PoV, the root cause is something else, in the startup mechanism that caused both exceptions.

          Wadeck Follonier added a comment - mcardenasblanco From my PoV, you are not affected by the same exception (rememberMe) but another introduced in previous release. It seems very likely that the code in rememberMe is not the culprit. In your case the full stacktrace shows: Caused by: java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:758) at jenkins.telemetry.impl.StaplerDispatches$StaplerTrace.record(StaplerDispatches.java:105) at org.kohsuke.stapler.EvaluationTrace$ApplicationTracer.trace(EvaluationTrace.java:77) at org.kohsuke.stapler.Dispatcher.anonymizedTraceEval(Dispatcher.java:76) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:685) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) at com.cloudbees.jenkins.ha.HAHealthCheckFilter.doFilter(HAHealthCheckFilter.java:35) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:64) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:169) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) ... 12 more So I created 2 PR for this ticket, 3805 to correct the rememberMe but I highly suspect to not correct the problem and 3806 to correct the legacy behavior there. From my PoV, the root cause is something else, in the startup mechanism that caused both exceptions.

          Thank you for your analysis wfollonier, we will try to dig a little bit more on what is happening as it is preventing us from applying a security upgrade.

          Manuel Cardenas added a comment - Thank you for your analysis wfollonier , we will try to dig a little bit more on what is happening as it is preventing us from applying a security upgrade.

          Devin Nusbaum added a comment -

          I think the problem in mcardenasblanco's case would be fixed by https://github.com/jenkinsci/jenkins/pull/3712 which made it into the 2.150 LTS line but not 2.138. Even so, it's not clear to me why that issue would happen consistently during a restart.

          Devin Nusbaum added a comment - I think the problem in mcardenasblanco 's case would be fixed by  https://github.com/jenkinsci/jenkins/pull/3712  which made it into the 2.150 LTS line but not 2.138. Even so, it's not clear to me why that issue would happen consistently during a restart.

          dnusbaum are you aware of any potential workaround for this?

          Manuel Cardenas added a comment - dnusbaum are you aware of any potential workaround for this?

          Devin Nusbaum added a comment - - edited

          No, but maybe if the problem is inbound HTTP requests hitting the servlet/Stapler layer while Jenkins is shutting down/restarting then blocking all HTTP traffic to Jenkins while the restart is happening might help (only really possible if you have some kind of reverse proxy in front of Jenkins or something, and I am not sure what kind of configuration would be needed).

          Edit: mcardenasblanco thinking about it some more, I think you could also work around the issue by passing -Dhudson.model.UsageStatistics.disabled=true as a system property when starting Jenkins so that this comparison short circuits before it gets to the Jenkins.get() call.

          Devin Nusbaum added a comment - - edited No, but maybe if the problem is inbound HTTP requests hitting the servlet/Stapler layer while Jenkins is shutting down/restarting then blocking all HTTP traffic to Jenkins while the restart is happening might help (only really possible if you have some kind of reverse proxy in front of Jenkins or something, and I am not sure what kind of configuration would be needed). Edit: mcardenasblanco thinking about it some more, I think you could also work around the issue by passing -Dhudson.model.UsageStatistics.disabled=true as a system property when starting Jenkins so that this comparison short circuits before it gets to the Jenkins.get() call.

          Craig Lutgen added a comment -

          This still happens. Every restart from an upgrade or otherwise gets stuck with the Jenkins.instance is missing exception and requires a service restart.

           

          Jenkins v 2.202

          Windows Server 2016

          Java 1.8.0_221-b11

          Craig Lutgen added a comment - This still happens. Every restart from an upgrade or otherwise gets stuck with the Jenkins.instance is missing exception and requires a service restart.   Jenkins v 2.202 Windows Server 2016 Java 1.8.0_221-b11

          Daniel Beck added a comment -

          csclutgen It's almost a year later, and you haven't previously been involved in this issue. It's also not a problem users regularly encounter. It's therefore possible, perhaps likely, that your problem is not the exact same, so please file a new issue and provide as much information about your setup as possible (see guidelines linked from Create Issue form).

          Daniel Beck added a comment - csclutgen It's almost a year later, and you haven't previously been involved in this issue. It's also not a problem users regularly encounter. It's therefore possible, perhaps likely, that your problem is not the exact same, so please file a new issue and provide as much information about your setup as possible (see guidelines linked from Create Issue form).

          Craig Lutgen added a comment -

          It is the exact same symptom and the exactly the same stack trace that was never resolved and forgotten about. So, one should consider it the same bug. And who is to say that other users aren't regularly seeing this problem. All the same, please reference new issue JENKINS-59992. We cannot supply full logs for security reason. But, let me know if there are specific items that could be provied.

          Craig Lutgen added a comment - It is the exact same symptom and the exactly the same stack trace that was never resolved and forgotten about. So, one should consider it the same bug. And who is to say that other users aren't regularly seeing this problem. All the same, please reference new issue JENKINS-59992 . We cannot supply full logs for security reason. But, let me know if there are specific items that could be provied.

          ethorsa added a comment -

          Since my Upgrade from Oracle Java 8 to AdoptOpenJDK 11 I run into this problem too. The Jenkins log fills with following error, eventually crashing the server:

           

          WARNING	o.e.jetty.server.HttpChannel#handleException: ERROR dispatch failed
          Also:   java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
          		at jenkins.model.Jenkins.get(Jenkins.java:773)
          		at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:164)
          		at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104)
          		at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          [...]

          (Full log excerpt of this error attached)

          After first appearance of this error it's no longer possible to start Jenkins. Executing a full reboot of the host brings it up again.

           

          The Windows event log does not contain any useful information (only "exited with 1").

          Jenkins Master:

          • Jenkins v2.190.3
          • AdoptOpenJDK 11.0.4
          • Windows 7

          ethorsa added a comment - Since my Upgrade from Oracle Java 8 to AdoptOpenJDK 11 I run into this problem too. The Jenkins log fills with following error, eventually crashing the server:   WARNING o.e.jetty.server.HttpChannel#handleException: ERROR dispatch failed Also: java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong. at jenkins.model.Jenkins.get(Jenkins.java:773) at hudson.security.TokenBasedRememberMeServices2.autoLogin(TokenBasedRememberMeServices2.java:164) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:104) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) [...] (Full log excerpt of this error attached) After first appearance of this error it's no longer possible to start Jenkins. Executing a full reboot of the host brings it up again.   The Windows event log does not contain any useful information (only "exited with 1" ). Jenkins Master: Jenkins v2.190.3 AdoptOpenJDK 11.0.4 Windows 7

          Wadeck Follonier added a comment - First analysis explained in https://issues.jenkins-ci.org/browse/JENKINS-59992?focusedCommentId=379193&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-379193 about a potential root cause. No plan to work more on this from my side.

          If you have the backup of the Plugins folder before you did the upgrade of the plugin. Restore the backup folder into the Jenkins home directory and restart the instance, it would work

          Satheesh Kumar added a comment - If you have the backup of the Plugins folder before you did the upgrade of the plugin. Restore the backup folder into the Jenkins home directory and restart the instance, it would work

          Wadeck Follonier added a comment - Symptom correction proposal: https://github.com/jenkinsci/jenkins/pull/4525

          Oleg Nenashev added a comment -

          It was released in Jenkins 2.223 in March.  https://www.jenkins.io/changelog/#v2.223

          Oleg Nenashev added a comment - It was released in Jenkins 2.223 in March.   https://www.jenkins.io/changelog/#v2.223

            wfollonier Wadeck Follonier
            daniel15 Daniel Lo Nigro
            Votes:
            4 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: