Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-14551

an update which contains added files merged from a branch results in those files having doubled content

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: subversion-plugin
    • Labels:
      None
    • Environment:
      sles 11, Jenkins 1.470, subversion server is CollabNet Subversion Edge 1.3.1
    • Similar Issues:

      Description

      Using the "Emulate clean checkout by first deleting unversioned/ignored files, then 'svn update'" svn strategy, we recently had a branch merged into
      trunk that added some files. When the Jenkins jobs configured as above
      ran against trunk, those added files had their content duplicated.

      A quick google suggested it has happened before:

      http://grokbase.com/p/lucene/dev/126m8fpw9q/jenkins-lucene-solr-tests-only-4-x-build-137-still-failing

        Attachments

          Issue Links

            Activity

            Hide
            jamesslastfm James Scott added a comment -

            Having the same problem. Jenkins 1.479, Jenkins Subversion Plugin 1.42

            Show
            jamesslastfm James Scott added a comment - Having the same problem. Jenkins 1.479, Jenkins Subversion Plugin 1.42
            Hide
            benxo Ben XO added a comment - - edited

            Seeing this on Jenkins 1.479 with plugin 1.42 as well.

            Worth noting that at Last.fm we merge down branches which add files quite often, but this behaviour only started happening recently when we upgraded. Probably within the last few point versions of the plugin.

            Show
            benxo Ben XO added a comment - - edited Seeing this on Jenkins 1.479 with plugin 1.42 as well. Worth noting that at Last.fm we merge down branches which add files quite often, but this behaviour only started happening recently when we upgraded. Probably within the last few point versions of the plugin.
            Hide
            kutzi kutzi added a comment -

            So this only happens with the mentioned svn strategy?

            Show
            kutzi kutzi added a comment - So this only happens with the mentioned svn strategy?
            Hide
            jamesslastfm James Scott added a comment -

            It happens also when using 'use svn update as much as possible'.

            When the build fails we wipeout the workspace and re-build.

            Show
            jamesslastfm James Scott added a comment - It happens also when using 'use svn update as much as possible'. When the build fails we wipeout the workspace and re-build.
            Hide
            dup2 dup2 added a comment -

            Same issue here, Jenkins 1.485, Jenkins Subversion Plugin 1.43

            Show
            dup2 dup2 added a comment - Same issue here, Jenkins 1.485, Jenkins Subversion Plugin 1.43
            Hide
            jarntang JarnTang added a comment -

            I have same issue,Jenkins 1.459 ,subversion plugin 1.43

            Show
            jarntang JarnTang added a comment - I have same issue,Jenkins 1.459 ,subversion plugin 1.43
            Hide
            jan_z Jan Zuchhold added a comment -
            Show
            jan_z Jan Zuchhold added a comment - As this bug is really annoying i've created a 50$ reward to fix this, expires Nov/09/2012: http://www.freedomsponsors.org/core/issue/61/an-update-which-contains-added-files-merged-from-a-branch-results-in-those-files-having-doubled-content
            Hide
            pst Patrick Streicher added a comment - - edited

            Same issue: Jenkins 1.487, Jenkins Subversion Plug-in 1.43

            It occurs always after a merge from a branch. I'm using the "Use 'svn update' as much as possible" check-out strategy.

            Show
            pst Patrick Streicher added a comment - - edited Same issue: Jenkins 1.487, Jenkins Subversion Plug-in 1.43 It occurs always after a merge from a branch. I'm using the "Use 'svn update' as much as possible" check-out strategy.
            Hide
            mightycaptainkurt Andrea Curtoni added a comment -

            Same issue: Jenkins 1.489, Jenkins Subversion Plug-in 1.43 (it was already bugged in previous versions)

            Show
            mightycaptainkurt Andrea Curtoni added a comment - Same issue: Jenkins 1.489, Jenkins Subversion Plug-in 1.43 (it was already bugged in previous versions)
            Hide
            rocco LuFrija added a comment -

            Same for me. Jenkins 1.424.2 and Subversion plugin 1.43.

            Show
            rocco LuFrija added a comment - Same for me. Jenkins 1.424.2 and Subversion plugin 1.43.
            Hide
            ingmartinlopez ingmartinlopez added a comment -

            Same for me. (It's really annoying!). Jenkins 1.492 and Subversion 1.43

            Show
            ingmartinlopez ingmartinlopez added a comment - Same for me. (It's really annoying!). Jenkins 1.492 and Subversion 1.43
            Hide
            kutzi kutzi added a comment -

            I'm not sure that I understand the issue:
            Who or what is merging the branch into trunk?
            And what if you checkout the trunk with command line svn - is the content duplicated, too?

            Show
            kutzi kutzi added a comment - I'm not sure that I understand the issue: Who or what is merging the branch into trunk? And what if you checkout the trunk with command line svn - is the content duplicated, too?
            Hide
            cjw296 Chris Withers added a comment -

            The problems happens if, in a commit which already committed to the subversion server, a branch has been merged into
            trunk that where some files are added.

            Who or what is merging the branch into trunk?

            • A developer, on their own machine, which is then committed to svn.

            And what if you checkout the trunk with command line svn - is the content duplicated, too?

            • no, if you use the command line tools, the checkout is as expected, with no duplicated content.
            Show
            cjw296 Chris Withers added a comment - The problems happens if, in a commit which already committed to the subversion server, a branch has been merged into trunk that where some files are added. Who or what is merging the branch into trunk? A developer, on their own machine, which is then committed to svn. And what if you checkout the trunk with command line svn - is the content duplicated, too? no, if you use the command line tools, the checkout is as expected, with no duplicated content.
            Hide
            kutzi kutzi added a comment -

            Okay.
            So what would really help to debug the issue would be if you could create a minimal subversion repository which has this characteristics.
            Ideally also a snapshot of the repository BEFORE and AFTER the branch has been merged.

            Show
            kutzi kutzi added a comment - Okay. So what would really help to debug the issue would be if you could create a minimal subversion repository which has this characteristics. Ideally also a snapshot of the repository BEFORE and AFTER the branch has been merged.
            Hide
            cashmann vasyl vuy added a comment -

            I have the same issue, file content is duplicated. update strategy is 'emulate clean...' and I only see this behavior when new files are added after merge.

            Show
            cashmann vasyl vuy added a comment - I have the same issue, file content is duplicated. update strategy is 'emulate clean...' and I only see this behavior when new files are added after merge.
            Hide
            sseidel sseidel added a comment - - edited

            How to reproduce:

            • Do this on the Jenkins host:
            cd /tmp
            mkdir svn ws
            svnadmin create svn
            svn mkdir file:///tmp/svn/trunk -m trunk
            svn mkdir file:///tmp/svn/branch -m branch
            
            cd ws
            svn co file:///tmp/svn/branch
            cd branch
            echo test > singleline
            svn add singleline
            svn commit -m added
            
            • Now create a Jenkins "freestyle" project with SVN URL file:///tmp/svn/trunk and build it once, then continue in the shell:
            cd /tmp/ws
            svn co file:///tmp/svn/trunk
            cd trunk
            svn merge -r 2:3 ../branch/
            svn commit -m merged
            
            • Now build the project again. Check the workspace, the file "singleline" will contain 2 lines!

            Reproducible with: Jenkins 1.502, Subversion Plugin 1.45, Host SVN 1.6.12 (Debian/CollabNet).

            Show
            sseidel sseidel added a comment - - edited How to reproduce: Do this on the Jenkins host: cd /tmp mkdir svn ws svnadmin create svn svn mkdir file:///tmp/svn/trunk -m trunk svn mkdir file:///tmp/svn/branch -m branch cd ws svn co file:///tmp/svn/branch cd branch echo test > singleline svn add singleline svn commit -m added Now create a Jenkins "freestyle" project with SVN URL file:///tmp/svn/trunk and build it once, then continue in the shell: cd /tmp/ws svn co file:///tmp/svn/trunk cd trunk svn merge -r 2:3 ../branch/ svn commit -m merged Now build the project again. Check the workspace, the file "singleline" will contain 2 lines! Reproducible with: Jenkins 1.502, Subversion Plugin 1.45, Host SVN 1.6.12 (Debian/CollabNet).
            Hide
            ricktw Rick Oosterholt added a comment -

            Same issue: Jenkins 1.501, Jenkins Subversion Plug-in 1.45
            Check-out Strategy: Emulate clean checkout by first deleting unversioned/ignored files, then 'svn update'

            Seems not to be file-type related since it occurred on both xml and java files.

            Show
            ricktw Rick Oosterholt added a comment - Same issue: Jenkins 1.501, Jenkins Subversion Plug-in 1.45 Check-out Strategy: Emulate clean checkout by first deleting unversioned/ignored files, then 'svn update' Seems not to be file-type related since it occurred on both xml and java files.
            Hide
            hx_unbanned Linards L added a comment - - edited

            This is Critical! Issues like these can make chain-reaction on ant build script based projects that do not utilize distributed build model or do use any kind of reverting mechanism on failed build.

            Show
            hx_unbanned Linards L added a comment - - edited This is Critical! Issues like these can make chain-reaction on ant build script based projects that do not utilize distributed build model or do use any kind of reverting mechanism on failed build.
            Hide
            alfredocavalcanti Alfredo Cavalcanti added a comment -

            Same problem here.
            Jenkins ver. 1.505
            Jenkins Subversion Plug-in ver. 1.45

            Show
            alfredocavalcanti Alfredo Cavalcanti added a comment - Same problem here. Jenkins ver. 1.505 Jenkins Subversion Plug-in ver. 1.45
            Hide
            kayers Kenny Ayers added a comment - - edited

            We are seeing file contents double when using SVN mv commands as well. This means files that are svn mv'd to a location, then added back to the repo have their contents duplicated.

            Here are the reproduction steps in action:

            $ vim double.txt 
            $ cat double.txt 
            1
            2
            3
            
            $ svn add double.txt 
            A         double.txt
            $ svn ci . -m "Initial commit "
            Adding         double.txt
            Transmitting file data .
            
            Committed revision <revision>.
            
            $ svn mv double.txt double_new.txt
            A         double_new.txt
            D         double.txt
            
            $ svn ci . -m "Renaming double"
            Deleting       double.txt
            Adding         double_new.txt
            
            Committed revision <revision>.
            

            Here is the output for Jenkins when it runs a job. The job updates a checkout which contains file “double_new.txt”, then does a ‘cat’ command on the file. Here’s the output:

            Started by user Kenneth Ayers
            Building remotely on builder in workspace /test
            Updating <repository> at revision <revision>
            A         double_new.txt
            At revision <revision>
            
            [test] $ /bin/sh -xe /tmp/hudson5312350812342753447.sh
            + cat double_new.txt
            1
            2
            3
            
            1
            2
            3
            
            Finished: SUCCESS
            

            As you can see the file contents are now doubled up.

            Versions:
            Jenkins: 1.508
            Jenkins Subversion Plug-in: 1.45

            Show
            kayers Kenny Ayers added a comment - - edited We are seeing file contents double when using SVN mv commands as well. This means files that are svn mv'd to a location, then added back to the repo have their contents duplicated. Here are the reproduction steps in action: $ vim double .txt $ cat double .txt 1 2 3 $ svn add double .txt A double .txt $ svn ci . -m "Initial commit " Adding double .txt Transmitting file data . Committed revision <revision>. $ svn mv double .txt double_new.txt A double_new.txt D double .txt $ svn ci . -m "Renaming double " Deleting double .txt Adding double_new.txt Committed revision <revision>. Here is the output for Jenkins when it runs a job. The job updates a checkout which contains file “double_new.txt”, then does a ‘cat’ command on the file. Here’s the output: Started by user Kenneth Ayers Building remotely on builder in workspace /test Updating <repository> at revision <revision> A double_new.txt At revision <revision> [test] $ /bin/sh -xe /tmp/hudson5312350812342753447.sh + cat double_new.txt 1 2 3 1 2 3 Finished: SUCCESS As you can see the file contents are now doubled up. Versions: Jenkins: 1.508 Jenkins Subversion Plug-in: 1.45
            Hide
            dmigowski dmigowski added a comment -

            We were able to reproduce this very simply:

            • Create a file in SVN
            • Rename this file in SVN (should stay in same folder, content must not change!) (A manual delete/add does not work)
            • Build the project with Jenkins and let him update its project with the "Revert before Update" strategy. I didn't check other strategies.

            Now it is very sure the file has double its content.

            Jenkins: 1.494
            Jenkins Subversion Plugin: 1.45

            Show
            dmigowski dmigowski added a comment - We were able to reproduce this very simply: Create a file in SVN Rename this file in SVN (should stay in same folder, content must not change!) (A manual delete/add does not work) Build the project with Jenkins and let him update its project with the "Revert before Update" strategy. I didn't check other strategies. Now it is very sure the file has double its content. Jenkins: 1.494 Jenkins Subversion Plugin: 1.45
            Hide
            dmigowski dmigowski added a comment -

            PLEASE FIX THIS! Many ways to reproduce the Bug have been shown now. And since this is only a very subtile fail in a critical component which leads to hard to track and reproduce errors (dev environemnt works, production fails), this bug should be fixed immediatly. Because of the bug we had valid, but wrong, datafiles sent to our customers! And this bug ruins our convidence with the whole Jenkins toolchain in my company.

            Show
            dmigowski dmigowski added a comment - PLEASE FIX THIS! Many ways to reproduce the Bug have been shown now. And since this is only a very subtile fail in a critical component which leads to hard to track and reproduce errors (dev environemnt works, production fails), this bug should be fixed immediatly. Because of the bug we had valid, but wrong, datafiles sent to our customers! And this bug ruins our convidence with the whole Jenkins toolchain in my company.
            Hide
            kayers Kenny Ayers added a comment -

            This bug may be in SVNKit, which is the Java library that subversion-plugin uses. I determined this by modifying hudson.scm.subversion.UpdateUpdater by adding a loop to print the contents of each file after the SVN checkout is completed. Immediately after the doUpdate() function which is responsible for updating a checkout is called, I inspect the file contents to find they are corrupted.

            Here's the git patch to hudson.scm.subversion.UpdateUpdater with the debug output. Please note I'm not a Java developer by trade, and this code has cruft from a lot of debugging iterations. I've also modified the parameters being used by doUpdate() as part of the debugging process:

            diff --git a/src/main/java/hudson/scm/subversion/UpdateUpdater.java b/src/main/java/hudson/scm/subversion/UpdateUpdater.java
            index f00ebdb..021e18c 100755
            --- a/src/main/java/hudson/scm/subversion/UpdateUpdater.java
            +++ b/src/main/java/hudson/scm/subversion/UpdateUpdater.java
            @@ -40,11 +40,21 @@ import org.tmatesoft.svn.core.wc.SVNInfo;
             import org.tmatesoft.svn.core.wc.SVNRevision;
             import org.tmatesoft.svn.core.wc.SVNUpdateClient;
             import org.tmatesoft.svn.core.wc.SVNWCClient;
            +import org.tmatesoft.svn.core.wc.SVNClientManager;
            +import org.tmatesoft.svn.core.SVNDepth;
            +import org.tmatesoft.svn.core.wc.SVNRevision;
             
             import java.io.File;
             import java.io.IOException;
             import java.util.ArrayList;
            +import java.util.Arrays;
             import java.util.List;
            +import java.io.InputStream;
            +import java.io.BufferedInputStream;
            +import java.io.FileInputStream;
            +import java.io.DataInputStream;
            +import java.io.InputStreamReader;
            +import java.io.BufferedReader;
             
             /**
              * {@link WorkspaceUpdater} that uses "svn update" as much as possible.
            @@ -154,7 +164,38 @@ public class UpdateUpdater extends WorkspaceUpdater {
                             switch (svnCommand) {
                                 case UPDATE:
                                     listener.getLogger().println("Updating " + location.remote + " at revision " + revisionName);
            -                        svnuc.doUpdate(local.getCanonicalFile(), r, svnDepth, true, true);
            +                        //svnuc.doUpdate(local.getCanonicalFile(), r, svnDepth, true, true);
            +                        SVNClientManager cM = SVNClientManager.newInstance();
            +                        SVNUpdateClient updateClient = cM.getUpdateClient();
            +                        updateClient.doUpdate(local.getCanonicalFile(), SVNRevision.HEAD, SVNDepth.INFINITY, true, true);
            +
            +                        ArrayList<File> files = new ArrayList<File>(Arrays.asList(local.listFiles()));
            +                        for (int i = 0; i < files.size(); i++)
            +                        {
            +                            if (files.get(i).isFile())
            +                            {
            +                                String fname = files.get(i).getName();
            +                                listener.getLogger().println("File: " + fname);
            +                                try{
            +                                  // Open the file that is the first 
            +                                  // command line parameter
            +                                  FileInputStream fstream = new FileInputStream(files.get(i));
            +                                  // Get the object of DataInputStream
            +                                  DataInputStream in = new DataInputStream(fstream);
            +                                  BufferedReader br = new BufferedReader(new InputStreamReader(in));
            +                                  String strLine;
            +                                  //Read File Line By Line
            +                                  while ((strLine = br.readLine()) != null)   {
            +                                  // Print the content on the console
            +                                  listener.getLogger().println(strLine);
            +                                  }
            +                                  //Close the input stream
            +                                  in.close();
            +                                    }catch (Exception e){//Catch exception if any
            +                                  listener.getLogger().println("Error: " + e.getMessage());
            +                                  }
            +                            }
            +                        }
                                     break;
                                 case SWITCH:
                                     listener.getLogger().println("Switching to " + location.remote + " at revision " + revisionName);
            
            Show
            kayers Kenny Ayers added a comment - This bug may be in SVNKit, which is the Java library that subversion-plugin uses. I determined this by modifying hudson.scm.subversion.UpdateUpdater by adding a loop to print the contents of each file after the SVN checkout is completed. Immediately after the doUpdate() function which is responsible for updating a checkout is called, I inspect the file contents to find they are corrupted. Here's the git patch to hudson.scm.subversion.UpdateUpdater with the debug output. Please note I'm not a Java developer by trade, and this code has cruft from a lot of debugging iterations. I've also modified the parameters being used by doUpdate() as part of the debugging process: diff --git a/src/main/java/hudson/scm/subversion/UpdateUpdater.java b/src/main/java/hudson/scm/subversion/UpdateUpdater.java index f00ebdb..021e18c 100755 --- a/src/main/java/hudson/scm/subversion/UpdateUpdater.java +++ b/src/main/java/hudson/scm/subversion/UpdateUpdater.java @@ -40,11 +40,21 @@ import org.tmatesoft.svn.core.wc.SVNInfo; import org.tmatesoft.svn.core.wc.SVNRevision; import org.tmatesoft.svn.core.wc.SVNUpdateClient; import org.tmatesoft.svn.core.wc.SVNWCClient; + import org.tmatesoft.svn.core.wc.SVNClientManager; + import org.tmatesoft.svn.core.SVNDepth; + import org.tmatesoft.svn.core.wc.SVNRevision; import java.io.File; import java.io.IOException; import java.util.ArrayList; + import java.util.Arrays; import java.util.List; + import java.io.InputStream; + import java.io.BufferedInputStream; + import java.io.FileInputStream; + import java.io.DataInputStream; + import java.io.InputStreamReader; + import java.io.BufferedReader; /** * {@link WorkspaceUpdater} that uses "svn update" as much as possible. @@ -154,7 +164,38 @@ public class UpdateUpdater extends WorkspaceUpdater { switch (svnCommand) { case UPDATE: listener.getLogger().println( "Updating " + location.remote + " at revision " + revisionName); - svnuc.doUpdate(local.getCanonicalFile(), r, svnDepth, true , true ); + //svnuc.doUpdate(local.getCanonicalFile(), r, svnDepth, true , true ); + SVNClientManager cM = SVNClientManager.newInstance(); + SVNUpdateClient updateClient = cM.getUpdateClient(); + updateClient.doUpdate(local.getCanonicalFile(), SVNRevision.HEAD, SVNDepth.INFINITY, true , true ); + + ArrayList<File> files = new ArrayList<File>(Arrays.asList(local.listFiles())); + for ( int i = 0; i < files.size(); i++) + { + if (files.get(i).isFile()) + { + String fname = files.get(i).getName(); + listener.getLogger().println( "File: " + fname); + try { + // Open the file that is the first + // command line parameter + FileInputStream fstream = new FileInputStream(files.get(i)); + // Get the object of DataInputStream + DataInputStream in = new DataInputStream(fstream); + BufferedReader br = new BufferedReader( new InputStreamReader(in)); + String strLine; + //Read File Line By Line + while ((strLine = br.readLine()) != null ) { + // Print the content on the console + listener.getLogger().println(strLine); + } + //Close the input stream + in.close(); + } catch (Exception e){ //Catch exception if any + listener.getLogger().println( "Error: " + e.getMessage()); + } + } + } break ; case SWITCH: listener.getLogger().println( "Switching to " + location.remote + " at revision " + revisionName);
            Hide
            kayers Kenny Ayers added a comment -

            I've posted a bug report to the SVNKit issue tracker, hopefully this will get some attention: http://issues.tmatesoft.com/issue/SVNKIT-368

            Show
            kayers Kenny Ayers added a comment - I've posted a bug report to the SVNKit issue tracker, hopefully this will get some attention: http://issues.tmatesoft.com/issue/SVNKIT-368
            Hide
            kayers Kenny Ayers added a comment - - edited

            All,

            I may have found the issue. I'd like the Jenkins devs to take a look, and I've also appended this information to the SVNKit ticket, here: http://issues.tmatesoft.com/issue/SVNKIT-368

            In org.tmatesoft.svn.core.internal.wc.SVNUpdateEditor15.java, in function addFileWithHistory (line 867), there's a code block that calls myFileFetcher.fetchFile() twice. Each time this is called, baseTextOS is written to. Upon the second write, the file contents are duplicated. Here's the code:

            baseTextOS = SVNFileUtil.openFileForWriting(info.copiedBaseText);
            myFileFetcher.fetchFile(copyFromPath, copyFromRevision, baseTextOS, baseProperties);
            SVNChecksumOutputStream checksumBaseTextOS = new SVNChecksumOutputStream(baseTextOS, 
            SVNChecksumOutputStream.MD5_ALGORITHM, true);
            baseTextOS = checksumBaseTextOS;
            myFileFetcher.fetchFile(copyFromPath, copyFromRevision, baseTextOS, baseProperties);
            info.copiedBaseChecksum = checksumBaseTextOS.getDigest();
            

            I was able to find this by stepping through the code using NetBeans IDE 7.3 attached to a remote debugging session on Jenkins. I've compiled and tested this change inside the context of the subversion-plugin and the file contents are no longer duplicated.

            I've forked the svnkit repo used in Jenkins here, and committed this change if anyone would like to download the fix and do some testing:

            https://github.com/theotherwhitemeat/svnkit-1/commit/27decb28216ee4fd15b8fcbdb769bf41d81978eb

            Here's my patch:

            Index: SVNUpdateEditor15.java
            ===================================================================
            --- SVNUpdateEditor15.java	(revision 9722)
            +++ SVNUpdateEditor15.java	(working copy)
            @@ -864,7 +864,6 @@
                         OutputStream baseTextOS = null;
                         try {
                             baseTextOS = SVNFileUtil.openFileForWriting(info.copiedBaseText);
            -                myFileFetcher.fetchFile(copyFromPath, copyFromRevision, baseTextOS, baseProperties);
                             SVNChecksumOutputStream checksumBaseTextOS = new SVNChecksumOutputStream(baseTextOS, 
                                     SVNChecksumOutputStream.MD5_ALGORITHM, true);
                             baseTextOS = checksumBaseTextOS;
            

            Thank you,

            Kenny Ayers

            Show
            kayers Kenny Ayers added a comment - - edited All, I may have found the issue. I'd like the Jenkins devs to take a look, and I've also appended this information to the SVNKit ticket, here: http://issues.tmatesoft.com/issue/SVNKIT-368 In org.tmatesoft.svn.core.internal.wc.SVNUpdateEditor15.java, in function addFileWithHistory (line 867), there's a code block that calls myFileFetcher.fetchFile() twice. Each time this is called, baseTextOS is written to. Upon the second write, the file contents are duplicated. Here's the code: baseTextOS = SVNFileUtil.openFileForWriting(info.copiedBaseText); myFileFetcher.fetchFile(copyFromPath, copyFromRevision, baseTextOS, baseProperties); SVNChecksumOutputStream checksumBaseTextOS = new SVNChecksumOutputStream(baseTextOS, SVNChecksumOutputStream.MD5_ALGORITHM, true ); baseTextOS = checksumBaseTextOS; myFileFetcher.fetchFile(copyFromPath, copyFromRevision, baseTextOS, baseProperties); info.copiedBaseChecksum = checksumBaseTextOS.getDigest(); I was able to find this by stepping through the code using NetBeans IDE 7.3 attached to a remote debugging session on Jenkins. I've compiled and tested this change inside the context of the subversion-plugin and the file contents are no longer duplicated. I've forked the svnkit repo used in Jenkins here, and committed this change if anyone would like to download the fix and do some testing: https://github.com/theotherwhitemeat/svnkit-1/commit/27decb28216ee4fd15b8fcbdb769bf41d81978eb Here's my patch: Index: SVNUpdateEditor15.java =================================================================== --- SVNUpdateEditor15.java (revision 9722) +++ SVNUpdateEditor15.java (working copy) @@ -864,7 +864,6 @@ OutputStream baseTextOS = null ; try { baseTextOS = SVNFileUtil.openFileForWriting(info.copiedBaseText); - myFileFetcher.fetchFile(copyFromPath, copyFromRevision, baseTextOS, baseProperties); SVNChecksumOutputStream checksumBaseTextOS = new SVNChecksumOutputStream(baseTextOS, SVNChecksumOutputStream.MD5_ALGORITHM, true ); baseTextOS = checksumBaseTextOS; Thank you, Kenny Ayers
            Hide
            kayers Kenny Ayers added a comment -

            I went ahead and compiled this change into the subversion plugin, at least I think I did. This is using the latest rev from subversion-plugin along with my svnkit library modification. If you guys would like to do some additional testing to verify my change, the plugin is located here:

            https://github.com/theotherwhitemeat/misc_binaries

            I do not suggest using this for anything but testing / verification.

            -Kenny

            Show
            kayers Kenny Ayers added a comment - I went ahead and compiled this change into the subversion plugin, at least I think I did. This is using the latest rev from subversion-plugin along with my svnkit library modification. If you guys would like to do some additional testing to verify my change, the plugin is located here: https://github.com/theotherwhitemeat/misc_binaries I do not suggest using this for anything but testing / verification. -Kenny
            Hide
            batkinson Brent Atkinson added a comment - - edited

            I just hit this too, version 1.44 on Jenkins 1.513. Since it's fixed in 1.7.10, updating svnkit should fix this. However, the subversion plugin uses a patched version of svnkit so it's not necessarily as easy as updating the dependency. I would love to see the pull request: https://github.com/jenkinsci/svnkit/pull/1 merged and the subversion plugin updated to prevent this from corrupting builds.

            Show
            batkinson Brent Atkinson added a comment - - edited I just hit this too, version 1.44 on Jenkins 1.513. Since it's fixed in 1.7.10, updating svnkit should fix this. However, the subversion plugin uses a patched version of svnkit so it's not necessarily as easy as updating the dependency. I would love to see the pull request: https://github.com/jenkinsci/svnkit/pull/1 merged and the subversion plugin updated to prevent this from corrupting builds.
            Hide
            batkinson Brent Atkinson added a comment -

            I ended up patching the jenkins fork of svnkit locally, building the subversion-plugin 1.45 with the patched version of svnkin. It does indeed appear to fix the behavior.

            Show
            batkinson Brent Atkinson added a comment - I ended up patching the jenkins fork of svnkit locally, building the subversion-plugin 1.45 with the patched version of svnkin. It does indeed appear to fix the behavior.
            Hide
            rocco LuFrija added a comment - - edited

            Kenny, Bret,

            Did you do any modification to the pom files of the fork of jenkinsci-svnkit or to the svnPlugin?

            In fact I'm using the svnkit code patched by Kenny:

            https://github.com/theotherwhitemeat/svnkit-1

            ...and the SVNPlugin 1.45:

            https://github.com/jenkinsci/subversion-plugin/tree/subversion-1.45

            The first one was successfully built, but then I get tons of test errors while building the second one. Any idea?

            I know I could simply use the build of Kenny (btw, thx! ) but I wanted to try to build it on my own.

            Thanks for any help!

            p.s. Kenny, I had the very same problem mentioned by you here:

            http://jenkins.361315.n4.nabble.com/Compiling-SVNKit-for-subversion-plugin-td4663822.html

            ...and I ended up changing the exec task in the pom file of svnKit from this:

            <exec command="./gradlew clean assemble" />

            ...to this:

            <exec command="./gradlew clean build -x test -x signMaven"/>

            Did you do the same? I then copied the jar file located in the gradle/build folder into my local maven repo, I hope this was not an incredible stupid idea...

            Show
            rocco LuFrija added a comment - - edited Kenny, Bret, Did you do any modification to the pom files of the fork of jenkinsci-svnkit or to the svnPlugin? In fact I'm using the svnkit code patched by Kenny: https://github.com/theotherwhitemeat/svnkit-1 ...and the SVNPlugin 1.45: https://github.com/jenkinsci/subversion-plugin/tree/subversion-1.45 The first one was successfully built, but then I get tons of test errors while building the second one. Any idea? I know I could simply use the build of Kenny (btw, thx! ) but I wanted to try to build it on my own. Thanks for any help! p.s. Kenny, I had the very same problem mentioned by you here: http://jenkins.361315.n4.nabble.com/Compiling-SVNKit-for-subversion-plugin-td4663822.html ...and I ended up changing the exec task in the pom file of svnKit from this: <exec command="./gradlew clean assemble" /> ...to this: <exec command="./gradlew clean build -x test -x signMaven"/> Did you do the same? I then copied the jar file located in the gradle/build folder into my local maven repo, I hope this was not an incredible stupid idea...
            Hide
            batkinson Brent Atkinson added a comment -

            I had similar issues, building on windows.

            Show
            batkinson Brent Atkinson added a comment - I had similar issues, building on windows.
            Hide
            kayers Kenny Ayers added a comment -

            Hey LuFrija,

            If my memory is correct, I had to do the same thing - modify the build command for SVNKit, then copy the resultant jar file into my local maven repo, then build subversion-plugin against that.

            This seems like a hacky workflow, but it's the best I could stitch together given there are no instructions on how to build this properly, and the mailing list didn't respond to my question on how to build the correct way.

            Are you good to go now?

            -Kenny

            Show
            kayers Kenny Ayers added a comment - Hey LuFrija, If my memory is correct, I had to do the same thing - modify the build command for SVNKit, then copy the resultant jar file into my local maven repo, then build subversion-plugin against that. This seems like a hacky workflow, but it's the best I could stitch together given there are no instructions on how to build this properly, and the mailing list didn't respond to my question on how to build the correct way. Are you good to go now? -Kenny
            Hide
            rocco LuFrija added a comment -

            Hi guys,

            Thanks for your fast reply!

            I guess I found the problem...it's all about my company's proxy! In fact having a closer look at the failing tests I could see something like:

            ERROR: Failed to check out https://svn.jenkins-ci.org/trunk/hudson/test-projects/trivial-ant

            I'm now copying with it somehow, in the meantime I'm testing the hpi file uploaded by Kenny.

            Show
            rocco LuFrija added a comment - Hi guys, Thanks for your fast reply! I guess I found the problem...it's all about my company's proxy! In fact having a closer look at the failing tests I could see something like: ERROR: Failed to check out https://svn.jenkins-ci.org/trunk/hudson/test-projects/trivial-ant I'm now copying with it somehow, in the meantime I'm testing the hpi file uploaded by Kenny.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Kenny Ayers
            Path:
            svnkit/src/main/java/org/tmatesoft/svn/core/internal/wc/SVNUpdateEditor15.java
            http://jenkins-ci.org/commit/svnkit/27decb28216ee4fd15b8fcbdb769bf41d81978eb
            Log:
            Removing duplicate myFileFetcher.fetchFile() call. JENKINS-14551

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kenny Ayers Path: svnkit/src/main/java/org/tmatesoft/svn/core/internal/wc/SVNUpdateEditor15.java http://jenkins-ci.org/commit/svnkit/27decb28216ee4fd15b8fcbdb769bf41d81978eb Log: Removing duplicate myFileFetcher.fetchFile() call. JENKINS-14551
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Nicolas De loof
            Path:
            svnkit/src/main/java/org/tmatesoft/svn/core/internal/wc/SVNUpdateEditor15.java
            http://jenkins-ci.org/commit/svnkit/aec5b5033ebcf927978e1764379bcbcb3989b900
            Log:
            Merge pull request #1 from theotherwhitemeat/master

            A fix for JENKINS-14551

            Compare: https://github.com/jenkinsci/svnkit/compare/9bea57e044b4...aec5b5033ebc

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Nicolas De loof Path: svnkit/src/main/java/org/tmatesoft/svn/core/internal/wc/SVNUpdateEditor15.java http://jenkins-ci.org/commit/svnkit/aec5b5033ebcf927978e1764379bcbcb3989b900 Log: Merge pull request #1 from theotherwhitemeat/master A fix for JENKINS-14551 Compare: https://github.com/jenkinsci/svnkit/compare/9bea57e044b4...aec5b5033ebc
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Christoph Kutzinski
            Path:
            pom.xml
            http://jenkins-ci.org/commit/subversion-plugin/3599af503fc0a10bf29e3dfab1619ed53a2e71fc
            Log:
            [FIXED JENKINS-14551] added files merged from a branch results in those files having doubled content

            Compare: https://github.com/jenkinsci/subversion-plugin/compare/5fb0b85fcf2c^...3599af503fc0

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: pom.xml http://jenkins-ci.org/commit/subversion-plugin/3599af503fc0a10bf29e3dfab1619ed53a2e71fc Log: [FIXED JENKINS-14551] added files merged from a branch results in those files having doubled content Compare: https://github.com/jenkinsci/subversion-plugin/compare/5fb0b85fcf2c ^...3599af503fc0

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              cjw296 Chris Withers
              Votes:
              34 Vote for this issue
              Watchers:
              36 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: