Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37208

Slowness by too many calls in /wfapi/runs on complex pipelines

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Minor Minor
    • None
    • Stage view 1.3

      The way this plugin works is that through JavaScript and each 5 seconds the stage-view-ui plugin will perform a HTTP GET to wfapi/runs even if there is not any build currently running, which looks very inefficient.

      See:

      Again, in my opinion, once there is not any build running the timeout should not be 5 seconds but an higher value, or even better it should be the backend to tell the frontend to continue performing HTTP GET.

      Also, each time that an user is on the stage-view-ui graph the rest api HTTP GET wfapi/runs will return ALL the builds of the job. I am not sure about how costly is this, but it looks like we should do a cache or something and only get the new builds.

      This problem can be partially reproduced creating a pipeline script with iota close to 700. Something like:

      node('master') {
          for (i = 0; i <350; i++) {
             stage 'stage'
             sh 'Hello world!'
             sh 'Hello world!'
             sh 'Hello world!'
          }
      }
      

      Then, start performing builds and open 40-50 tabs on your browser so each 5 seconds each one is doing HTTP GET wfapi/runs

      The following appears a lot of times on the thread dumps taken.

      13085msec elapsed in Handling GET /jenkins/<ROOT_TO_THE_JOB>/wfapi/runs from <IP> : RequestHandlerThread[#269]
          java.io.FileInputStream.readBytes(Native Method)
          …
          com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)
          hudson.XmlFile.read(XmlFile.java:142)
          org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage.load(SimpleXStreamFlowNodeStorage.java:114)
          …
          org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage.getNode(SimpleXStreamFlowNodeStorage.java:80)
          org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getNode(CpsFlowExecution.java:775)
          org.jenkinsci.plugins.workflow.graph.FlowNode.loadParents(FlowNode.java:128)
          org.jenkinsci.plugins.workflow.graph.FlowNode.getParents(FlowNode.java:119)
          org.jenkinsci.plugins.workflow.graph.FlowGraphWalker$FlowGraphWalkerIterator.getNext(FlowGraphWalker.java:103)
          org.jenkinsci.plugins.workflow.graph.FlowGraphWalker$FlowGraphWalkerIterator.next(FlowGraphWalker.java:86)
          org.jenkinsci.plugins.workflow.graph.FlowGraphWalker$FlowGraphWalkerIterator.next(FlowGraphWalker.java:68)
          com.cloudbees.workflow.flownode.FlowNodeUtil.getIdSortedExecutionNodeList(FlowNodeUtil.java:486)
          com.cloudbees.workflow.rest.external.RunExt.createOld(RunExt.java:336)
          com.cloudbees.workflow.rest.external.RunExt.create(RunExt.java:303)
          com.cloudbees.workflow.rest.external.JobExt.create(JobExt.java:126)
          com.cloudbees.workflow.rest.endpoints.JobAPI.doRuns(JobAPI.java:68)
      

      The current flow graph storage is just not efficient for builds with large numbers of steps. No particular recommendation other than either get faster disk, or turn off the stage view, or try to move complex script logic into external programs rather than doing too much in Groovy.

            svanoort Sam Van Oort
            fbelzunc Félix Belzunce Arcos
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: