Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-9301

Improve dependency graph calculation (for maven jobs)

    XMLWordPrintable

Details

    Description

      Further ideas on how to improve dependency graph calculation - beyond JENKINS-7535

      Comment from tdunning:

      "It would be lovely if the dependency computation were to use a more clever algorithm so that it is very fast, but there are some quick wins available:

      a) don't do it if nothing changed

      b) don't do it if another thread is already doing a global recomputation. At the least, just have a single thread that does this dependency computation and have save mark the graph is needing an update. That way nobody is delayed and the worst that happens is one thread does the updates back-to-back forever.

      c) only change as much as is needed by the recently changed dependencies. This is an extension of (a) to make the entire computation incremental. Basically, you can keep an update time on each node and on the graph in general. In doing the dependency computation, you can skip part of the computation as soon as you note that the node under consideration hasn't changed more recently than the last graph update.
      "

      Attachments

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-7535 Rebuilding dependency graph slow on large installations

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-9845 Improve dependency graph calculation for maven projects: only use dependencies with 'unknown' version if needed

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            Ascending order - Click to sort in descending order
            View 11 older comments
            kutzi kutzi added a comment -

            Markus, if you have any question, please address them to the users mailing list. Thanks

            kutzi kutzi added a comment - Markus, if you have any question, please address them to the users mailing list. Thanks
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Christoph Kutzinski
            Path:
            maven-plugin/src/main/java/hudson/maven/MavenModule.java
            maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
            maven-plugin/src/main/java/hudson/maven/PomInfo.java
            http://jenkins-ci.org/commit/jenkins/65678ad31dbc9a8199d0076a9edd124040db7883
            Log:
            JENKINS-9301 don't recalculate dependency graph in modules haven't changed

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/65678ad31dbc9a8199d0076a9edd124040db7883 Log: JENKINS-9301 don't recalculate dependency graph in modules haven't changed
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Christoph Kutzinski
            Path:
            maven-plugin/src/main/java/hudson/maven/MavenModule.java
            maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
            maven-plugin/src/main/java/hudson/maven/PomInfo.java
            http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49
            Log:
            Merge pull request #185 from kutzi/maven-dep-calculation

            JENKINS-9301 don't recalculate dependency graph if modules haven't changed

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49 Log: Merge pull request #185 from kutzi/maven-dep-calculation JENKINS-9301 don't recalculate dependency graph if modules haven't changed
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Christoph Kutzinski
            Path:
            src/main/java/hudson/maven/MavenModule.java
            src/main/java/hudson/maven/MavenModuleSetBuild.java
            src/main/java/hudson/maven/PomInfo.java
            http://jenkins-ci.org/commit/maven-plugin/9ba4f7aa84787ebd3daf2df86f144f56414edad5
            Log:
            JENKINS-9301 don't recalculate dependency graph in modules haven't changed
            Originally-Committed-As: 65678ad31dbc9a8199d0076a9edd124040db7883

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/MavenModuleSetBuild.java src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/maven-plugin/9ba4f7aa84787ebd3daf2df86f144f56414edad5 Log: JENKINS-9301 don't recalculate dependency graph in modules haven't changed Originally-Committed-As: 65678ad31dbc9a8199d0076a9edd124040db7883
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Christoph Kutzinski
            Path:
            src/main/java/hudson/maven/MavenModule.java
            src/main/java/hudson/maven/MavenModuleSetBuild.java
            src/main/java/hudson/maven/PomInfo.java
            http://jenkins-ci.org/commit/maven-plugin/dff94c2fb016b948e49e943a887382dd80cafc6b
            Log:
            Merge pull request #185 from kutzi/maven-dep-calculation

            JENKINS-9301 don't recalculate dependency graph if modules haven't changed
            Originally-Committed-As: 05131ef1b311f625bcfa836372479b4a7f7a2a49

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/MavenModuleSetBuild.java src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/maven-plugin/dff94c2fb016b948e49e943a887382dd80cafc6b Log: Merge pull request #185 from kutzi/maven-dep-calculation JENKINS-9301 don't recalculate dependency graph if modules haven't changed Originally-Committed-As: 05131ef1b311f625bcfa836372479b4a7f7a2a49

            People

              kutzi kutzi
              kutzi kutzi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: