Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-9301

Improve dependency graph calculation (for maven jobs)

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • core, maven-plugin
    • None

      Further ideas on how to improve dependency graph calculation - beyond JENKINS-7535

      Comment from tdunning:

      "It would be lovely if the dependency computation were to use a more clever algorithm so that it is very fast, but there are some quick wins available:

      a) don't do it if nothing changed

      b) don't do it if another thread is already doing a global recomputation. At the least, just have a single thread that does this dependency computation and have save mark the graph is needing an update. That way nobody is delayed and the worst that happens is one thread does the updates back-to-back forever.

      c) only change as much as is needed by the recently changed dependencies. This is an extension of (a) to make the entire computation incremental. Basically, you can keep an update time on each node and on the graph in general. In doing the dependency computation, you can skip part of the computation as soon as you note that the node under consideration hasn't changed more recently than the last graph update.
      "

          [JENKINS-9301] Improve dependency graph calculation (for maven jobs)

          kutzi created issue -
          kutzi made changes -
          Link New: This issue depends on JENKINS-7535 [ JENKINS-7535 ]

          kutzi added a comment -

          I've already implemented something for b on the JENKINS-7535 branch, but it needs more testing.

          kutzi added a comment - I've already implemented something for b on the JENKINS-7535 branch, but it needs more testing.

          kutzi added a comment -

          d) as a specification of c): don't recalculate the whole moduleset graph for each contained module.
          As seen in JENKINS-7535 that has some memory trade-offs, so use something like SoftReferences to clear the remembered moduleset if the heap is getting too full.

          kutzi added a comment - d) as a specification of c): don't recalculate the whole moduleset graph for each contained module. As seen in JENKINS-7535 that has some memory trade-offs, so use something like SoftReferences to clear the remembered moduleset if the heap is getting too full.

          kutzi added a comment -

          e) remove versionless Maven dependencies as they probably only take away space/time. See https://issues.jenkins-ci.org/browse/JENKINS-7535?focusedCommentId=147673#comment-147673

          kutzi added a comment - e) remove versionless Maven dependencies as they probably only take away space/time. See https://issues.jenkins-ci.org/browse/JENKINS-7535?focusedCommentId=147673#comment-147673

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          core/src/main/java/hudson/model/DependencyGraphCalculator.java
          core/src/main/java/hudson/model/Hudson.java
          http://jenkins-ci.org/commit/jenkins/7fa4e51e807f3393ae000ccca4da230ad8b85e48
          Log:
          JENKINS-9301 add proper synchronisation

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: core/src/main/java/hudson/model/DependencyGraphCalculator.java core/src/main/java/hudson/model/Hudson.java http://jenkins-ci.org/commit/jenkins/7fa4e51e807f3393ae000ccca4da230ad8b85e48 Log: JENKINS-9301 add proper synchronisation
          kutzi made changes -
          Link New: This issue depends on JENKINS-9845 [ JENKINS-9845 ]
          kutzi made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          maven-plugin/src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49
          Log:
          Merge pull request #185 from kutzi/maven-dep-calculation

          JENKINS-9301 don't recalculate dependency graph if modules haven't changed

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49 Log: Merge pull request #185 from kutzi/maven-dep-calculation JENKINS-9301 don't recalculate dependency graph if modules haven't changed

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          maven-plugin/src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49
          Log:
          Merge pull request #185 from kutzi/maven-dep-calculation

          JENKINS-9301 don't recalculate dependency graph if modules haven't changed

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49 Log: Merge pull request #185 from kutzi/maven-dep-calculation JENKINS-9301 don't recalculate dependency graph if modules haven't changed

            kutzi kutzi
            kutzi kutzi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: