Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38089

Unexpected Exception during accessing json pipeline status

      When we try to access the current pipeline status by executing the following request:
      <pipline job name>/lastBuild/wfapi/

      We sometimes get the following exception when the pipeline is running:

      java.lang.RuntimeException: Unexpected exception while serving JSON
      	at com.cloudbees.workflow.util.ServeJson$Processor.invoke(ServeJson.java:34)
      	at org.kohsuke.stapler.PreInvokeInterceptedFunction.invoke(PreInvokeInterceptedFunction.java:26)
      	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:167)
      	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:100)
      	at org.kohsuke.stapler.IndexDispatcher.dispatch(IndexDispatcher.java:26)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$11.dispatch(MetaClass.java:380)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$3.doDispatch(MetaClass.java:196)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:233)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:233)
      	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
      	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
      	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
      	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135)
      	at com.smartcodeltd.jenkinsci.plugin.assetbundler.filters.LessCSS.doFilter(LessCSS.java:47)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132)
      	at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59)
      	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132)
      	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      	at org.eclipse.jetty.server.Server.handle(Server.java:499)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
      	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
      	at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:722)
      Caused by: java.lang.reflect.InvocationTargetException
      	at sun.reflect.GeneratedMethodAccessor376.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:601)
      	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:324)
      	at com.cloudbees.workflow.util.ServeJson$Processor.invoke(ServeJson.java:30)
      	... 78 more
      Caused by: java.util.NoSuchElementException
      	at java.util.ArrayDeque.removeFirst(ArrayDeque.java:278)
      	at java.util.ArrayDeque.pop(ArrayDeque.java:507)
      	at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.setHeads(ForkScanner.java:390)
      	at org.jenkinsci.plugins.workflow.graphanalysis.AbstractFlowScanner.setup(AbstractFlowScanner.java:140)
      	at org.jenkinsci.plugins.workflow.graphanalysis.AbstractFlowScanner.setup(AbstractFlowScanner.java:151)
      	at org.jenkinsci.plugins.workflow.graphanalysis.ForkScanner.visitSimpleChunks(ForkScanner.java:551)
      	at com.cloudbees.workflow.rest.external.RunExt.createNew(RunExt.java:318)
      	at com.cloudbees.workflow.rest.external.RunExt.create(RunExt.java:306)
      	at com.cloudbees.workflow.rest.endpoints.RunAPI.doDescribe(RunAPI.java:102)
      	at com.cloudbees.workflow.rest.endpoints.RunAPI.doIndex(RunAPI.java:96)
      	... 83 more
      

      Note that it works most of the time. But at some point, during the pipeline build, we get always the error above. After some time everything is ok and we get valid json again.

          [JENKINS-38089] Unexpected Exception during accessing json pipeline status

          I experience the same Exception every day - sometimes it works, sometimes not.

          Running Jenkins 1.651.2 with the following (up-to-date) Pipeline plugins:

          Pipeline Graph Analysis Plugin
          1.1

          Pipeline: API
          2.3

          Pipeline: Basic Steps
          2.1

          Pipeline: Build Step
          2.2

          Pipeline: Groovy
          2.17

          Pipeline: Input Step
          2.1

          Pipeline: Job
          2.6

          Pipeline: Multibranch
          2.8

          Pipeline: Nodes and Processes
          2.4

          Pipeline: REST API Plugin
          2.0

          Pipeline: SCM Step
          2.2

          Pipeline: Shared Groovy Libraries
          2.3

          Pipeline: Stage Step
          2.2

          Pipeline: Stage View Plugin
          2.0

          Pipeline: Step API
          2.3

          Pipeline: Supporting APIs
          2.5

          Alexander Schwartz added a comment - I experience the same Exception every day - sometimes it works, sometimes not. Running Jenkins 1.651.2 with the following (up-to-date) Pipeline plugins: Pipeline Graph Analysis Plugin 1.1 Pipeline: API 2.3 Pipeline: Basic Steps 2.1 Pipeline: Build Step 2.2 Pipeline: Groovy 2.17 Pipeline: Input Step 2.1 Pipeline: Job 2.6 Pipeline: Multibranch 2.8 Pipeline: Nodes and Processes 2.4 Pipeline: REST API Plugin 2.0 Pipeline: SCM Step 2.2 Pipeline: Shared Groovy Libraries 2.3 Pipeline: Stage Step 2.2 Pipeline: Stage View Plugin 2.0 Pipeline: Step API 2.3 Pipeline: Supporting APIs 2.5

          Seems to happen only when showing a pipeline where a parallel step is currently running. I hope this help investigating this.

          Alexander Schwartz added a comment - Seems to happen only when showing a pipeline where a parallel step is currently running. I hope this help investigating this.

          Sam Van Oort added a comment -

          mschwartau Do you have a snippet of pipeline code that will reproduce the issue? It should only occur when a you have parallels in progress (which is why it disappears), so you should be able to reproduce it more consistently by adding sleep steps in place of some longer-running operations.

          Sam Van Oort added a comment - mschwartau Do you have a snippet of pipeline code that will reproduce the issue? It should only occur when a you have parallels in progress (which is why it disappears), so you should be able to reproduce it more consistently by adding sleep steps in place of some longer-running operations.

          svanoort Reposted from https://issues.jenkins-ci.org/browse/JENKINS-39541

          This seems to break 100% of the time. It only breaks if the second task ends before the first

          stage 'test' , {
          parallel([
          unit: {
          node('unit') {
          dir('some_folder')

          { sh "sleep 5" sh "sleep 10" sh "sleep 15" sh "sleep 20" sh "echo 'hello'" }

          }
          },
          otherunit: {
          node('unit')

          { sh "sleep 10" sh "echo 'goodbye'" }

          }
          ])
          }

          Sorry for the bad formatting, it seems I cannot wrap this in preformatted text.

          Matt Kaczorowski added a comment - svanoort Reposted from https://issues.jenkins-ci.org/browse/JENKINS-39541 This seems to break 100% of the time. It only breaks if the second task ends before the first stage 'test' , { parallel([ unit: { node('unit') { dir('some_folder') { sh "sleep 5" sh "sleep 10" sh "sleep 15" sh "sleep 20" sh "echo 'hello'" } } }, otherunit: { node('unit') { sh "sleep 10" sh "echo 'goodbye'" } } ]) } Sorry for the bad formatting, it seems I cannot wrap this in preformatted text.

          Sam Van Oort added a comment -

          Found the cause - it's a complex set of conditions involved with how we find the start of in-progress parallel branches.

          I'm reworking the fix though because I spotted a second issue after installing patched SNAPSHOT version on my host (possibly not related).

          Sam Van Oort added a comment - Found the cause - it's a complex set of conditions involved with how we find the start of in-progress parallel branches. I'm reworking the fix though because I spotted a second issue after installing patched SNAPSHOT version on my host (possibly not related).

          Code changed in jenkins
          User: Sam Van Oort
          Path:
          src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScanner.java
          src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScannerTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/7a3bd901bc30db1220253b3f8f2904ab3e52edc3
          Log:
          Merge pull request #25 from jenkinsci/parallel-errors

          Fix JENKINS-38089 which is caused by one specific edge case with incomplete parallels

          Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/e44f5066e0c1...7a3bd901bc30

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: src/main/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScanner.java src/test/java/org/jenkinsci/plugins/workflow/graphanalysis/ForkScannerTest.java http://jenkins-ci.org/commit/workflow-api-plugin/7a3bd901bc30db1220253b3f8f2904ab3e52edc3 Log: Merge pull request #25 from jenkinsci/parallel-errors Fix JENKINS-38089 which is caused by one specific edge case with incomplete parallels Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/e44f5066e0c1...7a3bd901bc30

          Sam Van Oort added a comment -

          mkaczorowski Install workflow-api plugin 2.7 (now working its way into the update center) and your issue is solved.

          Sam Van Oort added a comment - mkaczorowski Install workflow-api plugin 2.7 (now working its way into the update center) and your issue is solved.

          Sam Van Oort added a comment -

          Fixed with workflow-api 2.7 release

          Sam Van Oort added a comment - Fixed with workflow-api 2.7 release

          Awesome! Thanks.

          Matt Kaczorowski added a comment - Awesome! Thanks.

            svanoort Sam Van Oort
            mschwartau Meinert Schwartau
            Votes:
            9 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: