Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34722

Unbounded searches with BuildUtil.getDownstreamBuild

XMLWordPrintable

      BuildUtil.getDownstreamBuild does an unbounded search of build records, which is never acceptable due to Jenkins lazy-loading, and particular not in the context of an HTTP handler thread.

      It would be better is to use AbstractBuild.getDownstreamBuilds(AbstractProject), which is efficient, though it only works for project chains which are correctly configured with fingerprints.

      Example:

      "Handling GET /jenkins/org/job/Pipelines/job/org-build/view/Pipelines_View/ from 10.x.x.x : RequestHandlerThread[#2296] View/index.jelly" Id=126293 BLOCKED on hudson.model.RunMap@607d5beb owned by "Handling GET /jenkins/org/job/Pipelines/job/org-build/ from 10.x.x.x : RequestHandlerThread[#2297] View/index.jelly" Id=126471
    at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java)
    -  blocked on hudson.model.RunMap@607d5beb
    at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:448)
    at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:356)
    at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:332)
    at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:74)
    at jenkins.model.lazy.LazyLoadRunMapEntrySet$1.next(LazyLoadRunMapEntrySet.java:63)
    at java.util.AbstractMap$2$1.next(AbstractMap.java:396)
    at au.com.centrumsystems.hudson.plugin.util.BuildUtil.getDownstreamBuild(BuildUtil.java:66)

          [JENKINS-34722] Unbounded searches with BuildUtil.getDownstreamBuild

          Dilip Mahadevappa added a comment -

          Hi,

          Any news or workaround for this please?

          Thanks!
          Dilip Mahadevappa

          Dilip Mahadevappa added a comment - Hi, Any news or workaround for this please? Thanks! Dilip Mahadevappa

          Steven Christou added a comment -

          dilipm79 How many builds are currently in your pipeline view, and how many pipelines are you currently displaying? A large part of the performance issue also revolves around the number of pipeline runs that are being displayed.

          Steven Christou added a comment - dilipm79 How many builds are currently in your pipeline view, and how many pipelines are you currently displaying? A large part of the performance issue also revolves around the number of pipeline runs that are being displayed.

          J F added a comment -

          There are ~280 builds, and 50 pipelines, with varying numbers of build jobs each.

          (Answering on behalf of Dilip)

          J F added a comment - There are ~280 builds, and 50 pipelines, with varying numbers of build jobs each. (Answering on behalf of Dilip)

          Audrey Azra added a comment -

          Steven Christou: Any news or workaround on this issue?

          Audrey Azra added a comment - Steven Christou: Any news or workaround on this issue?

          Andrew Buckles added a comment -

          Hi, a bit more information here with regards to the pipeline we are having issues with.

          Total number of potential jobs for the pipeline is 17... however, each one of those jobs has a range of build history in the thousands (some 8-9K, some 5K and others have 1-2K).

          The pipeline is configured to only show the last 100 builds, however, will Jenkins still attempt to load all of the build history for each of the jobs involved in the pipeline?

          For instance, what we observe with a standard job is that Jenkins will take a significant amount of time to load a job page if there are 8000 builds in the build history versus 100-200. Even though Jenkins won't display all of the build history, it still seems to load all of them into memory.

          Does the pipeline job loading/rendering do a similar style of data fetch for each job in the pipeline?

          Thanks,
          Andy Buckles

          Andrew Buckles added a comment - Hi, a bit more information here with regards to the pipeline we are having issues with. Total number of potential jobs for the pipeline is 17... however, each one of those jobs has a range of build history in the thousands (some 8-9K, some 5K and others have 1-2K). The pipeline is configured to only show the last 100 builds, however, will Jenkins still attempt to load all of the build history for each of the jobs involved in the pipeline? For instance, what we observe with a standard job is that Jenkins will take a significant amount of time to load a job page if there are 8000 builds in the build history versus 100-200. Even though Jenkins won't display all of the build history, it still seems to load all of them into memory. Does the pipeline job loading/rendering do a similar style of data fetch for each job in the pipeline? Thanks, Andy Buckles

          Sean McIntyre added a comment -

          Hi there,

          I wanted to chime in and let everyone know we are having a similar issue with our Jenkins setup using the Pipeline plug-in. Each page load in Jenkins was approaching 11s, which is unacceptable for our CI pipeline and causing friction on our team.

          I found that this was the issue by sampling the Java stack every second during a GET request, here are the results in a gist. I saw the common denominator of each sample was the BuildUtil.getDownstreamBuild, and found this ticket.

          We have 7 pipelines in our installation, each having about 6 jobs, and most of those jobs having 1-3K builds in their histories. I purged the histories of these pipelines, reducing their histories to about 300 each (total 60K -> 17K), and Jenkins pageloads are now 2s. I had to do a Jenkins restart after the purges.

          We are running Build Pipeline Plugin 1.5.4 and Jenkins 2.30.

          Thanks!

          Sean McIntyre added a comment - Hi there, I wanted to chime in and let everyone know we are having a similar issue with our Jenkins setup using the Pipeline plug-in. Each page load in Jenkins was approaching 11s, which is unacceptable for our CI pipeline and causing friction on our team. I found that this was the issue by sampling the Java stack every second during a GET request, here are the results in a gist . I saw the common denominator of each sample was the BuildUtil.getDownstreamBuild , and found this ticket. We have 7 pipelines in our installation, each having about 6 jobs, and most of those jobs having 1-3K builds in their histories. I purged the histories of these pipelines, reducing their histories to about 300 each (total 60K -> 17K), and Jenkins pageloads are now 2s. I had to do a Jenkins restart after the purges. We are running Build Pipeline Plugin 1.5.4 and Jenkins 2.30. Thanks!

          Dan Alvizu added a comment -

          Fixed in 1.5.6

          Dan Alvizu added a comment - Fixed in 1.5.6

          Dan Alvizu added a comment -

          1.5.6 included this fix:
          https://github.com/jenkinsci/build-pipeline-plugin/pull/88

          Dan Alvizu added a comment - 1.5.6 included this fix: https://github.com/jenkinsci/build-pipeline-plugin/pull/88

          Sean McIntyre added a comment -

          Great, dalvizu. Our Jenkins page load times fell from 2s to 20ms with the upgrade to 1.5.6.

          Sean McIntyre added a comment - Great, dalvizu . Our Jenkins page load times fell from 2s to 20ms with the upgrade to 1.5.6.

          Dan Alvizu added a comment -

          good to hear, all credit to jpederzolli

          Dan Alvizu added a comment - good to hear, all credit to jpederzolli

            schristou Steven Christou
            ganthore Mark Austin
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: