Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-9301

Improve dependency graph calculation (for maven jobs)

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • core, (1)
      maven-plugin
    • None

      Further ideas on how to improve dependency graph calculation - beyond JENKINS-7535

      Comment from tdunning:

      "It would be lovely if the dependency computation were to use a more clever algorithm so that it is very fast, but there are some quick wins available:

      a) don't do it if nothing changed

      b) don't do it if another thread is already doing a global recomputation. At the least, just have a single thread that does this dependency computation and have save mark the graph is needing an update. That way nobody is delayed and the worst that happens is one thread does the updates back-to-back forever.

      c) only change as much as is needed by the recently changed dependencies. This is an extension of (a) to make the entire computation incremental. Basically, you can keep an update time on each node and on the graph in general. In doing the dependency computation, you can skip part of the computation as soon as you note that the node under consideration hasn't changed more recently than the last graph update.
      "

          [JENKINS-9301] Improve dependency graph calculation (for maven jobs)

          kutzi added a comment -

          I've now implemented a solution for 'a'.
          Although there is room for more improvement (e.g. I've already implement 'b' partially on the JENKINS-7535 branch), I'd like to close this issue now, as I don't think we will get much more improvement.
          And the danger to introduce new bugs (and unnecessary complexity) is quite high.

          kutzi added a comment - I've now implemented a solution for 'a'. Although there is room for more improvement (e.g. I've already implement 'b' partially on the JENKINS-7535 branch), I'd like to close this issue now, as I don't think we will get much more improvement. And the danger to introduce new bugs (and unnecessary complexity) is quite high.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #933
          JENKINS-9301 don't recalculate dependency graph in modules haven't changed

          Christoph Kutzinski : 65678ad31dbc9a8199d0076a9edd124040db7883
          Files :

          • maven-plugin/src/main/java/hudson/maven/MavenModule.java
          • maven-plugin/src/main/java/hudson/maven/PomInfo.java
          • maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java

          dogfood added a comment - Integrated in jenkins_main_trunk #933 JENKINS-9301 don't recalculate dependency graph in modules haven't changed Christoph Kutzinski : 65678ad31dbc9a8199d0076a9edd124040db7883 Files : maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/PomInfo.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          maven-plugin/src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/jenkins/65678ad31dbc9a8199d0076a9edd124040db7883
          Log:
          JENKINS-9301 don't recalculate dependency graph in modules haven't changed

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/65678ad31dbc9a8199d0076a9edd124040db7883 Log: JENKINS-9301 don't recalculate dependency graph in modules haven't changed

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          maven-plugin/src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49
          Log:
          Merge pull request #185 from kutzi/maven-dep-calculation

          JENKINS-9301 don't recalculate dependency graph if modules haven't changed

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49 Log: Merge pull request #185 from kutzi/maven-dep-calculation JENKINS-9301 don't recalculate dependency graph if modules haven't changed

          Markus Helbig added a comment -

          we just have upgrade from 1.419 - 1.429 and now all dependencies are lost ... just if we begin new jobs it is correct ... is there any way to force recalculation in any case?

          Markus Helbig added a comment - we just have upgrade from 1.419 - 1.429 and now all dependencies are lost ... just if we begin new jobs it is correct ... is there any way to force recalculation in any case?

          kutzi added a comment -

          Markus, if you have any question, please address them to the users mailing list. Thanks

          kutzi added a comment - Markus, if you have any question, please address them to the users mailing list. Thanks

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          maven-plugin/src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/jenkins/65678ad31dbc9a8199d0076a9edd124040db7883
          Log:
          JENKINS-9301 don't recalculate dependency graph in modules haven't changed

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/65678ad31dbc9a8199d0076a9edd124040db7883 Log: JENKINS-9301 don't recalculate dependency graph in modules haven't changed

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java
          maven-plugin/src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49
          Log:
          Merge pull request #185 from kutzi/maven-dep-calculation

          JENKINS-9301 don't recalculate dependency graph if modules haven't changed

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/MavenModuleSetBuild.java maven-plugin/src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/jenkins/05131ef1b311f625bcfa836372479b4a7f7a2a49 Log: Merge pull request #185 from kutzi/maven-dep-calculation JENKINS-9301 don't recalculate dependency graph if modules haven't changed

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          src/main/java/hudson/maven/MavenModule.java
          src/main/java/hudson/maven/MavenModuleSetBuild.java
          src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/maven-plugin/9ba4f7aa84787ebd3daf2df86f144f56414edad5
          Log:
          JENKINS-9301 don't recalculate dependency graph in modules haven't changed
          Originally-Committed-As: 65678ad31dbc9a8199d0076a9edd124040db7883

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/MavenModuleSetBuild.java src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/maven-plugin/9ba4f7aa84787ebd3daf2df86f144f56414edad5 Log: JENKINS-9301 don't recalculate dependency graph in modules haven't changed Originally-Committed-As: 65678ad31dbc9a8199d0076a9edd124040db7883

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          src/main/java/hudson/maven/MavenModule.java
          src/main/java/hudson/maven/MavenModuleSetBuild.java
          src/main/java/hudson/maven/PomInfo.java
          http://jenkins-ci.org/commit/maven-plugin/dff94c2fb016b948e49e943a887382dd80cafc6b
          Log:
          Merge pull request #185 from kutzi/maven-dep-calculation

          JENKINS-9301 don't recalculate dependency graph if modules haven't changed
          Originally-Committed-As: 05131ef1b311f625bcfa836372479b4a7f7a2a49

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/MavenModuleSetBuild.java src/main/java/hudson/maven/PomInfo.java http://jenkins-ci.org/commit/maven-plugin/dff94c2fb016b948e49e943a887382dd80cafc6b Log: Merge pull request #185 from kutzi/maven-dep-calculation JENKINS-9301 don't recalculate dependency graph if modules haven't changed Originally-Committed-As: 05131ef1b311f625bcfa836372479b4a7f7a2a49

            kutzi kutzi
            kutzi kutzi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: