Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44146

Changeset matching for multiple SCM checkouts broken - missing trigger for incremental builds


    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • ivy-plugin
    • None
    • Observed using Jenkins version 1.651.2 and the Ivy Plugin version 1.27.1

      The matching of SCM changesets with the affected Ivy modules appears to be broken when a job is configured for Build modules as separate jobs for the case when multiple SCM checkout locations are defined. This causes dependant features like Incremental build - only build changed modules and the email notification Send separate e-mails to individuals who broke the build to be broken. Affected/changed modules are not triggered for building, and email notifications are not sent to individuals who have committed changes into the broken modules.

      An Ivy build job configured for Build modules as separate jobs and Incremental build - only build changed modules with multiple SCM checkouts. This can e.g. be several Subversion URLs, each of which is checked out into a separate subfolder (Local module directory) within the Jenkins workspace. When a build is triggered by an SCM changeset, the Ivy plugin is unable to launch module builds based on affected modules (incremental builds) or to dynamically identify email recipients for broken builds based on committers of the changeset.

      The cause of the problem appears to be insufficient representation of two essentially different pieces of information:

      • The top-level folder that defines the scope for discovery of Ivy modules for the Ivy build job. Effectively this should be the root of the Jenkins workspace for the Ivy project.
      • The locations of the checkout roots from the version control system used for matching SCM changesets to identify affected modules. These are the basis for incremental builds using the changeset to identify affected modules to be built and sending emails to the individuals who broke the build (by matching changes with individual Ivy modules and the associated committer of the change).

      Example walk-through of the issue:

      • An Ivy project job configured for 'Build modules as separate jobs' and 'Incremental build - only build changed modules'. The job is set to poll the Subversion server for changes.
      • Two Subversion module checkouts
      • - Repository URL: 'svn://myserver/myrepo/myfirstpath' into Local module directory: 'myfirstpath' with discovered Ivy modules 'myModuleA' and 'myModuleB'
      • - Repository URL: 'svn://myserver/myrepo/mysecondpath' into Local module directory: 'mysecondpath' with discovered Ivy modules 'myModuleC' and 'myModuleD'

      A code change is committed within one of the Ivy modules, say within 'myModuleC'.
      During processing, the isDescendantOf() method of the anonymous ArrayList class defined within the method hudson.ivy.IvyModuleSetBuild.getChangeSetFor() is invoked. At this point a simple String comparison is made to determine if the change falls within a specific module. However, this comparison will fail to detect such cases since the two compared paths do not share a common reference:

      • the change path from ChangeLogSet.Entry.getAffectedPaths() is relative to the value of the Repository URL, say 'myModuleC/myadditionalcodepath/mychangedfile'
      • the module location IvyModule.getRelativePathToModuleRoot() is (as of the fix for JENKINS-13440) relative to the Jenkins workspace root, say 'mysecondpath/myModuleC/myadditionalcodepath/mychangedfile'
        Since the module location for checkout of multiple locations will always cause a prefix to the module path compared to the corresponding change path, matching paths will generally never be found.

      Prior to version 1.26 (prior to the changes related to JENKINS-13440), the changeset matching was partially working (i.e working for the Ivy modules discovered within the first SCM checkout in a multi-checkout scenario). The limitation of modules only being discovered from the first SCM checkout location was essentially the issue reported and fixed with JENKINS-13440. Following the changes in JENKINS-13440 introduced into version 1.26, the changeset matching has effectively been completely broken for the scenario described above.

      The changes in JENKINS-13440 did address the inability to discover Ivy modules beyond the first SCM checkout of a multi-checkout scenario, although the fix constrains the Local module directory (the location of the SCM checkout within the Jenkins workspace) to be only a single folder deep. The fix explicitly assumes that the parent folder of the SCM checkout folder is the Jenkins workspace location. The submitter of the fix for JENKINS-13440 has already submitted a new pull-request (https://github.com/jenkinsci/ivy-plugin/pull/21) that appears to resolve that specific single folder level constraint.

      Proposed solution:
      A pull-request with a suggested fix for this issue has been provided via GitHub with reference to this issue: https://github.com/jenkinsci/ivy-plugin/pull/25
      The patch separates the internal representation of the location of the Jenkins workspace and the SCM checkout root for a given Ivy module.

      The patch adds an additional property to the IvyModuleInfo and IvyModule representation of a discovered Ivy module to provide the location of the Ivy module relative to the SCM checkout root to which it belongs. It then provides additional implementation primarily in IvyModuleBuildSet.IvyXmlParser to be able to populate these added properties of IvyModuleInfo/IvyModule. Finally, the IvyModuleBuildSet.getChangeSetFor() method is reimplemented to match the changeset against the location of the module within the SCM checkout root rather than the location within the Jenkins workspace.

      The patch does modify the signature of the public IvyModuleInfo constructor (adds an additional parameter).
      The patch uses rather verbose naming of properties and methods to compensate some of the clashing nomenclatures within the Ivy plugin implementation (Ivy module vs. SCM Module, Jenkins workspace vs the workspace corresponding to an SCM checkout (a.k.a an SCM Module))

      The patch has been applied and tested with the version 1.27.1 which has the has the original fix for JENKINS-13440 and it has also been tested with version 1.27.1 with the additional application of the not yet accepted pull-request (https://github.com/jenkinsci/ivy-plugin/pull/21) that further modifies/resolves the problem reported in JENKINS-13440.

            tbingaman Timothy Bingaman
            riksen Henrik Andersen
            0 Vote for this issue
            1 Start watching this issue