Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55006

/monitoring/nodes?format=prometheus throws error 500

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Minor Minor
    • monitoring-plugin
    • OS: OS Linux, 2.6.32-696.3.2.el6.x86_64 , amd64/64 (4 cores)
      Java: OpenJDK Runtime Environment, 1.8.0_181-b13
      JVM: OpenJDK 64-Bit Server VM, 25.181-b13, mixed mode
      Jenkins ver. 2.138.3
      Monitoring Plugin v1.74

      Accessing /monitoring/nodes?format=prometheus gives error 500

       

      Error:

      java.lang.IllegalStateException: WRITER
      at org.eclipse.jetty.server.Response.getOutputStream(Response.java:917)
      at javax.servlet.ServletResponseWrapper.getOutputStream(ServletResponseWrapper.java:142)
      at org.kohsuke.stapler.compression.CompressionServletResponse.activate(CompressionServletResponse.java:61)
      at org.kohsuke.stapler.compression.CompressionFilter.activate(CompressionFilter.java:108)
      at org.kohsuke.stapler.ResponseImpl.getCompressedOutputStream(ResponseImpl.java:302)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.createOutputStream(DefaultScriptInvoker.java:88)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.createXMLOutput(DefaultScriptInvoker.java:68)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:51)
      at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:103)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668)
      at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
      at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
      at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.Server.handle(Server.java:531)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
      at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
      at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
      at java.lang.Thread.run(Thread.java:748)

          [JENKINS-55006] /monitoring/nodes?format=prometheus throws error 500

          z Z created issue -
          evernat made changes -
          Description Original: Accessing /monitoring/nodes?format=prometheus gives error 500

           

          Error:

          _java.lang.IllegalStateException: WRITER_
                _at org.eclipse.jetty.server.Response.getOutputStream([Response.java:917|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.Response#917])_
                _at javax.servlet.ServletResponseWrapper.getOutputStream([ServletResponseWrapper.java:142|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=javax.servlet.ServletResponseWrapper#142])_
                _at org.kohsuke.stapler.compression.CompressionServletResponse.activate([CompressionServletResponse.java:61|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.compression.CompressionServletResponse#61])_
                _at org.kohsuke.stapler.compression.CompressionFilter.activate([CompressionFilter.java:108|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.compression.CompressionFilter#108])_
                _at org.kohsuke.stapler.ResponseImpl.getCompressedOutputStream([ResponseImpl.java:302|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.ResponseImpl#302])_
                _at org.kohsuke.stapler.jelly.DefaultScriptInvoker.createOutputStream([DefaultScriptInvoker.java:88|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.jelly.DefaultScriptInvoker#88])_
                _at org.kohsuke.stapler.jelly.DefaultScriptInvoker.createXMLOutput([DefaultScriptInvoker.java:68|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.jelly.DefaultScriptInvoker#68])_
                _at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript([DefaultScriptInvoker.java:51|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.jelly.DefaultScriptInvoker#51])_
                _at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch([JellyFacet.java:103|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.jelly.JellyFacet$1#103])_
                _at org.kohsuke.stapler.Stapler.tryInvoke([Stapler.java:739|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.Stapler#739])_
                _at org.kohsuke.stapler.Stapler.invoke([Stapler.java:870|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.Stapler#870])_
                _at org.kohsuke.stapler.Stapler.invoke([Stapler.java:668|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.Stapler#668])_
                _at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0([InstallUncaughtExceptionHandler.java:36|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=hudson.init.impl.InstallUncaughtExceptionHandler#36])_
                _at org.kohsuke.stapler.compression.CompressionFilter.reportException([CompressionFilter.java:77|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.compression.CompressionFilter#77])_
                _at org.kohsuke.stapler.compression.CompressionFilter.doFilter([CompressionFilter.java:58|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.compression.CompressionFilter#58])_
                _at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter([ServletHandler.java:1642|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.servlet.ServletHandler$CachedChain#1642])_
                _at hudson.util.CharacterEncodingFilter.doFilter([CharacterEncodingFilter.java:82|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=hudson.util.CharacterEncodingFilter#82])_
                _at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter([ServletHandler.java:1642|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.servlet.ServletHandler$CachedChain#1642])_
                _at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter([DiagnosticThreadNameFilter.java:30|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.kohsuke.stapler.DiagnosticThreadNameFilter#30])_
                _at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter([ServletHandler.java:1642|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.servlet.ServletHandler$CachedChain#1642])_
                _at org.eclipse.jetty.servlet.ServletHandler.doHandle([ServletHandler.java:533|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.servlet.ServletHandler#533])_
                _at org.eclipse.jetty.server.handler.ScopedHandler.handle([ScopedHandler.java:146|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ScopedHandler#146])_
                _at org.eclipse.jetty.security.SecurityHandler.handle([SecurityHandler.java:524|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.security.SecurityHandler#524])_
                _at org.eclipse.jetty.server.handler.HandlerWrapper.handle([HandlerWrapper.java:132|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.HandlerWrapper#132])_
                _at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle([ScopedHandler.java:257|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ScopedHandler#257])_
                _at org.eclipse.jetty.server.session.SessionHandler.doHandle([SessionHandler.java:1595|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.session.SessionHandler#1595])_
                _at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle([ScopedHandler.java:255|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ScopedHandler#255])_
                _at org.eclipse.jetty.server.handler.ContextHandler.doHandle([ContextHandler.java:1317|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ContextHandler#1317])_
                _at org.eclipse.jetty.server.handler.ScopedHandler.nextScope([ScopedHandler.java:203|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ScopedHandler#203])_
                _at org.eclipse.jetty.servlet.ServletHandler.doScope([ServletHandler.java:473|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.servlet.ServletHandler#473])_
                _at org.eclipse.jetty.server.session.SessionHandler.doScope([SessionHandler.java:1564|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.session.SessionHandler#1564])_
                _at org.eclipse.jetty.server.handler.ScopedHandler.nextScope([ScopedHandler.java:201|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ScopedHandler#201])_
                _at org.eclipse.jetty.server.handler.ContextHandler.doScope([ContextHandler.java:1219|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ContextHandler#1219])_
                _at org.eclipse.jetty.server.handler.ScopedHandler.handle([ScopedHandler.java:144|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.ScopedHandler#144])_
                _at org.eclipse.jetty.server.handler.HandlerWrapper.handle([HandlerWrapper.java:132|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.handler.HandlerWrapper#132])_
                _at org.eclipse.jetty.server.Server.handle([Server.java:531|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.Server#531])_
                _at org.eclipse.jetty.server.HttpChannel.handle([HttpChannel.java:352|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.HttpChannel#352])_
                _at org.eclipse.jetty.server.HttpConnection.onFillable([HttpConnection.java:260|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.server.HttpConnection#260])_
                _at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded([AbstractConnection.java:281|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.io.AbstractConnection$ReadCallback#281])_
                _at org.eclipse.jetty.io.FillInterest.fillable([FillInterest.java:102|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.io.FillInterest#102])_
                _at org.eclipse.jetty.io.ChannelEndPoint$2.run([ChannelEndPoint.java:118|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.io.ChannelEndPoint$2#118])_
                _at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask([EatWhatYouKill.java:333|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.util.thread.strategy.EatWhatYouKill#333])_
                _at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce([EatWhatYouKill.java:310|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.util.thread.strategy.EatWhatYouKill#310])_
                _at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce([EatWhatYouKill.java:168|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.util.thread.strategy.EatWhatYouKill#168])_
                _at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run([EatWhatYouKill.java:126|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.util.thread.strategy.EatWhatYouKill#126])_
                _at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run([ReservedThreadExecutor.java:366|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread#366])_
                _at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob([QueuedThreadPool.java:762|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.util.thread.QueuedThreadPool#762])_
                _at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run([QueuedThreadPool.java:680|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=org.eclipse.jetty.util.thread.QueuedThreadPool$2#680])_
                _at java.lang.Thread.run([Thread.java:748|http://ci.corp.atmel.com/mcu/monitoring?part=source&class=java.lang.Thread#748])_
          New: Accessing /monitoring/nodes?format=prometheus gives error 500

           

          Error:

          java.lang.IllegalStateException: WRITER
           at org.eclipse.jetty.server.Response.getOutputStream(Response.java:917)
           at javax.servlet.ServletResponseWrapper.getOutputStream(ServletResponseWrapper.java:142)
           at org.kohsuke.stapler.compression.CompressionServletResponse.activate(CompressionServletResponse.java:61)
           at org.kohsuke.stapler.compression.CompressionFilter.activate(CompressionFilter.java:108)
           at org.kohsuke.stapler.ResponseImpl.getCompressedOutputStream(ResponseImpl.java:302)
           at org.kohsuke.stapler.jelly.DefaultScriptInvoker.createOutputStream(DefaultScriptInvoker.java:88)
           at org.kohsuke.stapler.jelly.DefaultScriptInvoker.createXMLOutput(DefaultScriptInvoker.java:68)
           at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:51)
           at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:103)
           at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739)
           at org.kohsuke.stapler.Stapler.invoke(Stapler.java:870)
           at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668)
           at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:36)
           at org.kohsuke.stapler.compression.CompressionFilter.reportException(CompressionFilter.java:77)
           at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:58)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
           at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
           at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
           at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
           at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
           at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
           at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
           at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
           at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
           at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
           at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
           at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
           at org.eclipse.jetty.server.Server.handle(Server.java:531)
           at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
           at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
           at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
           at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
           at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
           at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
           at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
           at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
           at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
           at java.lang.Thread.run(Thread.java:748)

          evernat added a comment - - edited

          I can't reproduce the issue in Jenkins 1.251 with monitoring plugin 1.74.

          There is a root cause exception indeed, but the problem is that Jenkins fails to give you this root cause exception and the stack-trace is of its failure to report the exception.

          evernat added a comment - - edited I can't reproduce the issue in Jenkins 1.251 with monitoring plugin 1.74. There is a root cause exception indeed, but the problem is that Jenkins fails to give you this root cause exception and the stack-trace is of its failure to report the exception.
          evernat made changes -
          Link New: This issue is related to JENKINS-21695 [ JENKINS-21695 ]

          evernat added a comment -

          evernat added a comment - So I have reopened https://issues.jenkins-ci.org/browse/JENKINS-21695

          evernat added a comment -

          JENKINS-21695 was closed, so closing this one.

          evernat added a comment - JENKINS-21695 was closed, so closing this one.
          evernat made changes -
          Resolution New: Cannot Reproduce [ 5 ]
          Status Original: Open [ 1 ] New: Closed [ 6 ]

            evernat evernat
            zauxst z Z
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: