Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7535

Rebuilding dependency graph slow on large installations

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • None
    • Apache Software Foundation, Hudson 1.376, Ubuntu server, Sun 1.6 JDK, 4Gb RAM

      In the ASF Hudson installation, rebuilding the dependency graph is quite slow (minutes). As rebuilding seems to happen twice on saving project configuration (AbstractProject.doConfigSubmit()), working with the project configuration is very slow and will frequently time out.

      We have a large number of jobs (>300) and from a quick look at the code, it seems like building the dependency graph is done even if there are no changes to the dependencies. Also, building the graph seems to be done by iterating over all jobs which will in turn iterator over all other jobs. This doesn't seem to scale to our size

      Here's a typical thread dump from saving a project configuration:

      "Handling POST /hudson/job/ftpserver-trunk-jdk1.6-osx/configSubmit : http-8090-27" daemon prio=10 tid=0x000000004340a000 nid=0x4576 runnable [0x00007f2fa8e48000]
      java.lang.Thread.State: RUNNABLE
      at java.lang.String.intern(Native Method)
      at hudson.maven.ModuleDependency.<init>(ModuleDependency.java:48)
      at hudson.maven.ModuleDependency.<init>(ModuleDependency.java:54)
      at hudson.maven.MavenModule.asDependency(MavenModule.java:296)
      at hudson.maven.MavenModule.buildDependencyGraph(MavenModule.java:385)
      at hudson.maven.MavenModuleSet.buildDependencyGraph(MavenModuleSet.java:487)
      at hudson.model.DependencyGraph.<init>(DependencyGraph.java:100)
      at hudson.model.Hudson.rebuildDependencyGraph(Hudson.java:3346)
      at hudson.model.AbstractProject.doConfigSubmit(AbstractProject.java:588)
      at sun.reflect.GeneratedMethodAccessor1225.invoke(Unknown Source)
      ...

          [JENKINS-7535] Rebuilding dependency graph slow on large installations

          Martin Vanek added a comment -

          I have installed the current snapshot (1.406-SNAPSHOT) it seems fine. Start of the server is faster, memory consumption also seems better.

          Martin Vanek added a comment - I have installed the current snapshot (1.406-SNAPSHOT) it seems fine. Start of the server is faster, memory consumption also seems better.

          kutzi added a comment -

          Thanks for testing.
          So I guess, I can mark this as resolved, again.

          kutzi added a comment - Thanks for testing. So I guess, I can mark this as resolved, again.

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/main/java/hudson/model/DependencyGraphCalculator.java
          core/src/main/java/hudson/model/Hudson.java
          http://jenkins-ci.org/commit/jenkins/f10e737b1147ca742f3a50c03b01945d803c6ea2
          Log:
          JENKINS-7535 make sure only one thread at a time rebuilds the dependency graph

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/main/java/hudson/model/DependencyGraphCalculator.java core/src/main/java/hudson/model/Hudson.java http://jenkins-ci.org/commit/jenkins/f10e737b1147ca742f3a50c03b01945d803c6ea2 Log: JENKINS-7535 make sure only one thread at a time rebuilds the dependency graph

          kutzi added a comment -

          See JENKINS-9301 for additional ideas to improve the time and resource consumption of building the dependency graph.

          kutzi added a comment - See JENKINS-9301 for additional ideas to improve the time and resource consumption of building the dependency graph.

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/main/java/hudson/model/DependencyGraph.java
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java
          http://jenkins-ci.org/commit/jenkins/6a5330fd936fb949da74d738b82dcf5333282b61
          Log:
          [FIXED JENKINS-7535] building up the dependency graph for maven jobs
          takes too long

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/main/java/hudson/model/DependencyGraph.java maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSet.java http://jenkins-ci.org/commit/jenkins/6a5330fd936fb949da74d738b82dcf5333282b61 Log: [FIXED JENKINS-7535] building up the dependency graph for maven jobs takes too long

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          changelog.html
          http://jenkins-ci.org/commit/jenkins/d3978478486e4030a058b05ff86a01f132c720a3
          Log:
          record changes for JENKINS-7535

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: changelog.html http://jenkins-ci.org/commit/jenkins/d3978478486e4030a058b05ff86a01f132c720a3 Log: record changes for JENKINS-7535

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          http://jenkins-ci.org/commit/jenkins/cccb301fbe28ff8b33b044f81f1d02fcd7ea02ee
          Log:
          JENKINS-7535 don't remember per-moduleset modules to prevent OOMEs

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java http://jenkins-ci.org/commit/jenkins/cccb301fbe28ff8b33b044f81f1d02fcd7ea02ee Log: JENKINS-7535 don't remember per-moduleset modules to prevent OOMEs

          Code changed in jenkins
          User: Olivier Lamy
          Path:
          src/main/java/hudson/maven/MavenModule.java
          src/main/java/hudson/maven/ModuleDependency.java
          http://jenkins-ci.org/commit/maven-plugin/004eadacbe5bb56011e3134e7b904bb4f08c9f3f
          Log:
          JENKINS-7535 Rebuilding dependency graph slow on large installations
          Submitted by evernat.

          Originally-Committed-As: 7d46b22599c6001eaeff7f47715fa80491b3fdd5

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Olivier Lamy Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/ModuleDependency.java http://jenkins-ci.org/commit/maven-plugin/004eadacbe5bb56011e3134e7b904bb4f08c9f3f Log: JENKINS-7535 Rebuilding dependency graph slow on large installations Submitted by evernat. Originally-Committed-As: 7d46b22599c6001eaeff7f47715fa80491b3fdd5

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          src/main/java/hudson/maven/MavenModule.java
          src/main/java/hudson/maven/MavenModuleSet.java
          http://jenkins-ci.org/commit/maven-plugin/70da285e456a34febb9705199a4aa6287506b38e
          Log:
          [FIXED JENKINS-7535] building up the dependency graph for maven jobs
          takes too long
          Originally-Committed-As: 6a5330fd936fb949da74d738b82dcf5333282b61

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/MavenModuleSet.java http://jenkins-ci.org/commit/maven-plugin/70da285e456a34febb9705199a4aa6287506b38e Log: [FIXED JENKINS-7535] building up the dependency graph for maven jobs takes too long Originally-Committed-As: 6a5330fd936fb949da74d738b82dcf5333282b61

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          src/main/java/hudson/maven/MavenModule.java
          http://jenkins-ci.org/commit/maven-plugin/28e769c1549bea397ce17081e50d1477489735df
          Log:
          JENKINS-7535 don't remember per-moduleset modules to prevent OOMEs
          Originally-Committed-As: cccb301fbe28ff8b33b044f81f1d02fcd7ea02ee

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/maven/MavenModule.java http://jenkins-ci.org/commit/maven-plugin/28e769c1549bea397ce17081e50d1477489735df Log: JENKINS-7535 don't remember per-moduleset modules to prevent OOMEs Originally-Committed-As: cccb301fbe28ff8b33b044f81f1d02fcd7ea02ee

            kutzi kutzi
            protocol7b protocol7b
            Votes:
            14 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: