Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39064

SVN checkout may result in mixed revisions when using externals

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Major Major
    • subversion-plugin
    • None
    • Jenkins ver. 1.651.3.1 (CloudBees Jenkins Enterprise 16.06)
      subversion@2.6

      We are using a rather complex SVN structure for our software projects with many externals. Since checkout of externals is rather slow, it happens regularly to us, that a commit is performed to the repository, while the checkout is still running. Unfortunately this causes the externals to be checked out with different (head) revisions - resulting into a "mixed source" warning of our build script later.

      Is there any way check out a SVN project with a consistent head revision for all externals?
      Is it possible to let the SVN plugin repeat a checkout when a "mixed source" condition is detected?

          [JENKINS-39064] SVN checkout may result in mixed revisions when using externals

          Roland Stahn added a comment - - edited

          This problem is a major issue for our Continuous Integration work flow, since it leads to builds with a inconsistent code base (aka mixed revisions).
          Therefore it would be nice to implement a workaround for this issue into the SVN plugin.

          Is there any way check out a SVN project with a consistent head revision for all externals?

          Here is a possible solution, that is implemented by the TortoiseSVN client (quote from here):

          Updating multiple items is currently not an atomic operation in Subversion. So TortoiseSVN first finds the HEAD revision of the repository, and then updates all items to that particular revision number to avoid creating a mixed revision working copy.
          If only one item is selected for updating or the selected items are not all from the same repository, TortoiseSVN just updates to HEAD.

          So what should be done for checking out a SVN working copy is:

          1. Query the head revision of the repository
          2. Use svn checkout -r <revision> / svn update -r <revision> to checkout / update to a consistent revision even if new commits arrive while checkout is in progress

          Roland Stahn added a comment - - edited This problem is a major issue for our Continuous Integration work flow, since it leads to builds with a inconsistent code base (aka mixed revisions). Therefore it would be nice to implement a workaround for this issue into the SVN plugin. Is there any way check out a SVN project with a consistent head revision for all externals? Here is a possible solution, that is implemented by the TortoiseSVN client (quote from here ): Updating multiple items is currently not an atomic operation in Subversion. So TortoiseSVN first finds the HEAD revision of the repository, and then updates all items to that particular revision number to avoid creating a mixed revision working copy. If only one item is selected for updating or the selected items are not all from the same repository, TortoiseSVN just updates to HEAD. So what should be done for checking out a SVN working copy is: Query the head revision of the repository Use svn checkout -r <revision> / svn update -r <revision> to checkout / update to a consistent revision even if new commits arrive while checkout is in progress

          Roland Stahn added a comment - - edited

          Looking at the source code it seems to me that the only required change would be:

          src/main/java/hudson/scm/subversion/CheckoutUpdater.java
          // UpdateTask()
          [...]
          try {
              SVNRevision r = getRevision(location);
              String revisionName = r.getDate() != null ? fmt.format(r.getDate()) : r.toString();
          
              listener.getLogger().println("Checking out " + location.getSVNURL().toString() + " at revision " +
                      revisionName);
          
              File local = new File(ws, location.getLocalDir());
              SubversionUpdateEventHandler eventHandler = new SubversionUpdateEventHandler(new PrintStream(pos), externals, local, location.getLocalDir());
              svnuc.setEventHandler(eventHandler);
              svnuc.setExternalsHandler(eventHandler);
              svnuc.setIgnoreExternals(location.isIgnoreExternalsOption());
              SVNDepth svnDepth = getSvnDepth(location.getDepthOption());
              SvnCheckout checkout = svnuc.getOperationsFactory().createCheckout();
          -   checkout.setSource(SvnTarget.fromURL(location.getSVNURL(), SVNRevision.HEAD));
          +   checkout.setSource(SvnTarget.fromURL(location.getSVNURL(), r));
              checkout.setSingleTarget(SvnTarget.fromFile(local.getCanonicalFile()));
              checkout.setDepth(svnDepth);
              checkout.setRevision(r);
              checkout.setAllowUnversionedObstructions(true);
              checkout.setIgnoreExternals(location.isIgnoreExternalsOption());
              checkout.setExternalsHandler(SvnCodec.externalsHandler(svnuc.getExternalsHandler()));
          [...]
          

          For the update strategies, the SVN URL needs to be updated with a similar call to setSource() in the following files:

          • UpdateUpdater.java
          • UpdateWithCleanUpdater.java
          • UpdateWithRevertUpdater.java

          Can someone with more insight into the guts of the SVN plugin please confirm this?

          Roland Stahn added a comment - - edited Looking at the source code it seems to me that the only required change would be: src/main/java/hudson/scm/subversion/CheckoutUpdater.java // UpdateTask() [...] try { SVNRevision r = getRevision(location); String revisionName = r.getDate() != null ? fmt.format(r.getDate()) : r.toString(); listener.getLogger().println( "Checking out " + location.getSVNURL().toString() + " at revision " + revisionName); File local = new File(ws, location.getLocalDir()); SubversionUpdateEventHandler eventHandler = new SubversionUpdateEventHandler( new PrintStream(pos), externals, local, location.getLocalDir()); svnuc.setEventHandler(eventHandler); svnuc.setExternalsHandler(eventHandler); svnuc.setIgnoreExternals(location.isIgnoreExternalsOption()); SVNDepth svnDepth = getSvnDepth(location.getDepthOption()); SvnCheckout checkout = svnuc.getOperationsFactory().createCheckout(); - checkout.setSource(SvnTarget.fromURL(location.getSVNURL(), SVNRevision.HEAD)); + checkout.setSource(SvnTarget.fromURL(location.getSVNURL(), r)); checkout.setSingleTarget(SvnTarget.fromFile(local.getCanonicalFile())); checkout.setDepth(svnDepth); checkout.setRevision(r); checkout.setAllowUnversionedObstructions( true ); checkout.setIgnoreExternals(location.isIgnoreExternalsOption()); checkout.setExternalsHandler(SvnCodec.externalsHandler(svnuc.getExternalsHandler())); [...] For the update strategies, the SVN URL needs to be updated with a similar call to setSource() in the following files: UpdateUpdater.java UpdateWithCleanUpdater.java UpdateWithRevertUpdater.java Can someone with more insight into the guts of the SVN plugin please confirm this?

            Unassigned Unassigned
            rstahn Roland Stahn
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: