Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-12735

No upstream or downstream resolution with Maven version ranges

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • maven-plugin
    • None

      This is essentially reopening JENKINS-2787 (https://issues.jenkins-ci.org/browse/JENKINS-2787). We are in an environment where we do have version ranges and ignoring the arguments for or against version ranges, we like to resolve this issue. I have looked at the patch provided in JENKINS-2787 and have integrated this into the latest 1.450 of jenkins and running this in our CI environment.

      What would be required to submit this latest patch to Jenkins and have this promoted?

          [JENKINS-12735] No upstream or downstream resolution with Maven version ranges

          Sergei Ivanov added a comment -

          Please add support for automatic resolution of version ranges in Maven dependencies into upstream and downstream relationships between the correcponding Jenkins jobs.
          We started using version ranges in our project, because we have a very fluid development environment. We manage almost a hundred of modules, which have dependencies between themselves, and which are assembled into applications often with a very short release cycle. Our key requirement is to always keep all module dependencies in sync, essentially by always depending on the latest available version. Without version ranges it becomes a royal pain, because after the release of each module, a newly released module version has to be propagated into all dependencies. If we were to do that, we'd have spent half of our time updating dependencies in the projects.
          If the changes in the patch have a potential to destabilise Jenkins for the existing users, let's make this option configurable from either Jenkins configuration, or individual job configuration.

          Sergei Ivanov added a comment - Please add support for automatic resolution of version ranges in Maven dependencies into upstream and downstream relationships between the correcponding Jenkins jobs. We started using version ranges in our project, because we have a very fluid development environment. We manage almost a hundred of modules, which have dependencies between themselves, and which are assembled into applications often with a very short release cycle. Our key requirement is to always keep all module dependencies in sync, essentially by always depending on the latest available version. Without version ranges it becomes a royal pain, because after the release of each module, a newly released module version has to be propagated into all dependencies. If we were to do that, we'd have spent half of our time updating dependencies in the projects. If the changes in the patch have a potential to destabilise Jenkins for the existing users, let's make this option configurable from either Jenkins configuration, or individual job configuration.

          kutzi added a comment -

          Alex, as said in the previous to last comment on JENKINS-2787: to get this integrated you should convert the patch into a Github pull request and tackle the mentioned problems (performance and missing unit test)

          kutzi added a comment - Alex, as said in the previous to last comment on JENKINS-2787 : to get this integrated you should convert the patch into a Github pull request and tackle the mentioned problems (performance and missing unit test)

          Alex Koon added a comment -

          I have submitted a pull request for this now - #374.

          Alex Koon added a comment - I have submitted a pull request for this now - #374.

          Code changed in jenkins
          User: alexkoon
          Path:
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/ModuleDependency.java
          maven-plugin/src/test/java/hudson/maven/MavenModuleTest.java
          http://jenkins-ci.org/commit/jenkins/65cd6555301535af4f8796a694c4d52664205778
          Log:
          JENKINS-12735 - Maven version range resolution.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: alexkoon Path: maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/ModuleDependency.java maven-plugin/src/test/java/hudson/maven/MavenModuleTest.java http://jenkins-ci.org/commit/jenkins/65cd6555301535af4f8796a694c4d52664205778 Log: JENKINS-12735 - Maven version range resolution.

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          maven-plugin/src/main/java/hudson/maven/MavenModule.java
          maven-plugin/src/main/java/hudson/maven/ModuleDependency.java
          maven-plugin/src/main/java/hudson/maven/ModuleName.java
          maven-plugin/src/test/java/hudson/maven/MavenModuleTest.java
          http://jenkins-ci.org/commit/jenkins/a0c02400313cee84e06a960be29b8f43389ab274
          Log:
          [FIXED JENKINS-12735] Merge branch 'pull-542'

          Compare: https://github.com/jenkinsci/jenkins/compare/7fead6ff7f00...a0c02400313c

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html maven-plugin/src/main/java/hudson/maven/MavenModule.java maven-plugin/src/main/java/hudson/maven/ModuleDependency.java maven-plugin/src/main/java/hudson/maven/ModuleName.java maven-plugin/src/test/java/hudson/maven/MavenModuleTest.java http://jenkins-ci.org/commit/jenkins/a0c02400313cee84e06a960be29b8f43389ab274 Log: [FIXED JENKINS-12735] Merge branch 'pull-542' Compare: https://github.com/jenkinsci/jenkins/compare/7fead6ff7f00...a0c02400313c

          Jay Meyer added a comment - - edited

          in Jenkins 1.482, this is causing Jenkins to kick off downstream builds of un-related versions.

          Example: mygrp-projCore-3.4 is kicking off mygrp-projWeb-3.4 (which is correct) and also the old mygrp-projWeb-3.3. The old projWeb artifact id matches, but the version does not match. So when Jenkins is configured to build both versions in different jobs, Jenkins is kicking off all versions of old downstream jobs.

          Jay Meyer added a comment - - edited in Jenkins 1.482, this is causing Jenkins to kick off downstream builds of un-related versions. Example: mygrp-projCore-3.4 is kicking off mygrp-projWeb-3.4 (which is correct) and also the old mygrp-projWeb-3.3. The old projWeb artifact id matches, but the version does not match. So when Jenkins is configured to build both versions in different jobs, Jenkins is kicking off all versions of old downstream jobs.

          I'm having the same problem as Jay, this change makes this option useless, as we would end up having nearly every project depended on other project.
          Please re-open it.

          Michael Glauche added a comment - I'm having the same problem as Jay, this change makes this option useless, as we would end up having nearly every project depended on other project. Please re-open it.

          Please look on this again, the latest change create huge list of downstream projects to build. (although they are not really affected)

          Michael Glauche added a comment - Please look on this again, the latest change create huge list of downstream projects to build. (although they are not really affected)

          Alex Koon added a comment -

          I was the one that submitted the original patch and Kohsuke kindly integrated and cleaned up the code to provide Maven version range resolution. We have a live production system using this new version and it works very well - practically all module projects (in excess of 100 project modules and Maven reactor projects) are referenced by version ranges and projects up and downstream are initiated correctly when we commit code.

          Your issues with this failing isn't very clear - are you using version ranges and this is now failing (or initiating builds outside the range)? Or is this affecting your projects where the versions are defined by a hard version number? I imagine if the versions are defined (no version ranges) then this would have caused many more people grief.

          If you could clarify the project dependencies you have with real practical examples or better yet a unit test proving the failure that would help in resolving the issue and whether this is the correct JIRA to raise for this issue. I am happy to try and track the issue down subject to time but will need more information (note I am not a committer to the Jenkins project and have contributed this patch as we have a large project base that is very fluid and requires version ranges to function under Maven when we migrated from Ivy + Ant).

          Under maven-plugin/src/test/java/hudson/maven/MavenModuleTest.java are the unit tests for testing the resolution of up and downstream dependencies for reference. If you can supply a patch for this unit test with an example of your project structure and the breakage - this would help reproduce and track your issue down immensely.

          Alex Koon added a comment - I was the one that submitted the original patch and Kohsuke kindly integrated and cleaned up the code to provide Maven version range resolution. We have a live production system using this new version and it works very well - practically all module projects (in excess of 100 project modules and Maven reactor projects) are referenced by version ranges and projects up and downstream are initiated correctly when we commit code. Your issues with this failing isn't very clear - are you using version ranges and this is now failing (or initiating builds outside the range)? Or is this affecting your projects where the versions are defined by a hard version number? I imagine if the versions are defined (no version ranges) then this would have caused many more people grief. If you could clarify the project dependencies you have with real practical examples or better yet a unit test proving the failure that would help in resolving the issue and whether this is the correct JIRA to raise for this issue. I am happy to try and track the issue down subject to time but will need more information (note I am not a committer to the Jenkins project and have contributed this patch as we have a large project base that is very fluid and requires version ranges to function under Maven when we migrated from Ivy + Ant). Under maven-plugin/src/test/java/hudson/maven/MavenModuleTest.java are the unit tests for testing the resolution of up and downstream dependencies for reference. If you can supply a patch for this unit test with an example of your project structure and the breakage - this would help reproduce and track your issue down immensely.

          Jay Meyer added a comment - - edited

          The new bug is not related to Maven version ranges. Instead, its about specific versions. My project has specific version numbers: e.g 1.1 vs. 1.2 in the pom dependencies specified. But when Jenkins reads them, it builds every OLD version of the downstream dependencies. A scenario is spelled out in JENKINS-15367. And also bad, the old downstream versions are NOT built by the old upstream version, because Jenkins has associated the downstream projects with the incorrect upstream projects. So in this state I can no longer build version 1.1 of the system of projects. And the 1.2 version kicks off too many build, because it builds all of the 1.1 builds too.

          Jay Meyer added a comment - - edited The new bug is not related to Maven version ranges. Instead, its about specific versions. My project has specific version numbers: e.g 1.1 vs. 1.2 in the pom dependencies specified. But when Jenkins reads them, it builds every OLD version of the downstream dependencies. A scenario is spelled out in JENKINS-15367 . And also bad, the old downstream versions are NOT built by the old upstream version, because Jenkins has associated the downstream projects with the incorrect upstream projects. So in this state I can no longer build version 1.1 of the system of projects. And the 1.2 version kicks off too many build, because it builds all of the 1.1 builds too.

          I did attach two simple test-scenarios, both with project "A" and "B".

          In the "working" scenario, jenkins did see the relationship correctly between B and A, so that when B was build A will be build later. (makes sense, direct dependency did change)

          In the "failing" scenario, the "B" project did create a release (0.0.1), move the development version to 0.0.2-SNAPSHOT. However the A project decides that they want only the latest stable version, so they switch their dependency to the 0.0.1 release.

          Now, however, when Jenkins (1.487) builds B with 0.0.2-SNAPSHOT it does trigger the A job too! (which makes no sense, because A is using the "stable" build 0.0.1, which should not change, so no re-building ins needed.)
          This did lead to a massive increase of number of projects building (and me having to switch off the option on 100+ jenkins jobs ..)

          I think the behavior was correct at least in 1.474, where only depdencencies with exactly the same dependency version got build.

          Michael Glauche added a comment - I did attach two simple test-scenarios, both with project "A" and "B". In the "working" scenario, jenkins did see the relationship correctly between B and A, so that when B was build A will be build later. (makes sense, direct dependency did change) In the "failing" scenario, the "B" project did create a release (0.0.1), move the development version to 0.0.2-SNAPSHOT. However the A project decides that they want only the latest stable version, so they switch their dependency to the 0.0.1 release. Now, however, when Jenkins (1.487) builds B with 0.0.2-SNAPSHOT it does trigger the A job too! (which makes no sense, because A is using the "stable" build 0.0.1, which should not change, so no re-building ins needed.) This did lead to a massive increase of number of projects building (and me having to switch off the option on 100+ jenkins jobs ..) I think the behavior was correct at least in 1.474, where only depdencencies with exactly the same dependency version got build.

          The behavior was correct up to 1.480, I'm blocked also to this version.

          Arnaud Héritier added a comment - The behavior was correct up to 1.480, I'm blocked also to this version.

          Alex Koon added a comment -

          I am trying to look into this.

          Alex Koon added a comment - I am trying to look into this.

          Alex Koon added a comment -

          I think I have found the bug. I have a unit test to prove the bug and the associated fix. Its related to the way the maven API deals with versions http://maven.apache.org/ref/3.0.3/maven-artifact/apidocs/org/apache/maven/artifact/versioning/VersionRange.html#createFromVersion(java.lang.String)

          Although this is fixed via unit tests, I would like a few days testing on a Jenkins instance to prove it works.

          I will try and submit a patch soon subject to work load.

          Alex Koon added a comment - I think I have found the bug. I have a unit test to prove the bug and the associated fix. Its related to the way the maven API deals with versions http://maven.apache.org/ref/3.0.3/maven-artifact/apidocs/org/apache/maven/artifact/versioning/VersionRange.html#createFromVersion(java.lang.String ) Although this is fixed via unit tests, I would like a few days testing on a Jenkins instance to prove it works. I will try and submit a patch soon subject to work load.

          This could be the reason, I remember there was some breaking change from 3.0.3 to 3.0.4 with SNAPSHOTS in version ranges, but i can't find it anymore.

          But, there's still MNG-3092 , which could be the main reason behind it. This thread on the maven list also goes in detail about the problem.

          Michael Glauche added a comment - This could be the reason, I remember there was some breaking change from 3.0.3 to 3.0.4 with SNAPSHOTS in version ranges, but i can't find it anymore. But, there's still MNG-3092 , which could be the main reason behind it. This thread on the maven list also goes in detail about the problem.

          Alex Koon added a comment -

          Alex Koon added a comment - Commented on this at https://issues.jenkins-ci.org/browse/JENKINS-15367 .

          Alex Koon added a comment -

          Marking as resolved. JENKINS-15367 deals with the bug (now pulled to master) where defined versions are not correctly processes.

          Alex Koon added a comment - Marking as resolved. JENKINS-15367 deals with the bug (now pulled to master) where defined versions are not correctly processes.

          Code changed in jenkins
          User: alexkoon
          Path:
          src/main/java/hudson/maven/MavenModule.java
          src/main/java/hudson/maven/ModuleDependency.java
          src/test/java/hudson/maven/MavenModuleTest.java
          http://jenkins-ci.org/commit/maven-plugin/3d81dd6f9518ce3b1534083da61a05c42c510516
          Log:
          JENKINS-12735 - Maven version range resolution.

          Originally-Committed-As: 65cd6555301535af4f8796a694c4d52664205778

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: alexkoon Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/ModuleDependency.java src/test/java/hudson/maven/MavenModuleTest.java http://jenkins-ci.org/commit/maven-plugin/3d81dd6f9518ce3b1534083da61a05c42c510516 Log: JENKINS-12735 - Maven version range resolution. Originally-Committed-As: 65cd6555301535af4f8796a694c4d52664205778

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          src/main/java/hudson/maven/MavenModule.java
          src/main/java/hudson/maven/ModuleDependency.java
          src/main/java/hudson/maven/ModuleName.java
          src/test/java/hudson/maven/MavenModuleTest.java
          http://jenkins-ci.org/commit/maven-plugin/151a41df0bce7f22d8d129d8bc8dd759e0d9ed95
          Log:
          [FIXED JENKINS-12735] Merge branch 'pull-542'

          Originally-Committed-As: a0c02400313cee84e06a960be29b8f43389ab274

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/maven/MavenModule.java src/main/java/hudson/maven/ModuleDependency.java src/main/java/hudson/maven/ModuleName.java src/test/java/hudson/maven/MavenModuleTest.java http://jenkins-ci.org/commit/maven-plugin/151a41df0bce7f22d8d129d8bc8dd759e0d9ed95 Log: [FIXED JENKINS-12735] Merge branch 'pull-542' Originally-Committed-As: a0c02400313cee84e06a960be29b8f43389ab274

            Unassigned Unassigned
            alexkoon Alex Koon
            Votes:
            1 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: