-
Bug
-
Resolution: Fixed
-
Critical
-
Windows XP x64, Core i7
The update strategy "Emulate clean checkout by..." does not delete files which are not part of the repository although the log reports that a cleanup is performed.
- is duplicated by
-
-
- Resolved
-
- is related to
-
-
- Closed
-
[JENKINS-13844] "Emulate clean checkout by..." does not work with 1.7 subversion repositories
To clarify this: The error happens when I chose "1.7" as format for the working copy, the server runs subversion "1.6".
Denis Eperonnier
added a comment - Same for me using SVN server 1.5, Jenkins 1.472 + SVN 1.42 plugin, SVN WC 1.7
Issue occurs on a WinXP 32bit slave
Denis Eperonnier
added a comment - Same for me using SVN server 1.5, Jenkins 1.472 + SVN 1.42 plugin, SVN WC 1.7
Issue occurs on a WinXP 32bit slave 
Martin Adam
added a comment - same here, environmen same as in comment above me, only I have jenkins 1.475 and it's a master building node
Martin Adam
added a comment - same here, environmen same as in comment above me, only I have jenkins 1.475 and it's a master building node 
Chris Lee
added a comment - This is caused by changes in SVN 1.7 relating to ignored files - they are no longer reported by default in 'svn status' (the SVNKit API used by Jenkins mirrors this behaviour).
In the workspace for a Jenkins project (after the job has run), no changes are shown:
[root@devemr01 UTIL_TRUNK]# svn status
However, the below command shows files/directories that are marked as ignored:
[root@devemr01 UTIL_TRUNK]# svn status --no-ignore I .gradle I build
...and creating a new file that isn't ignored shows up in 'svn status':
[root@devemr01 UTIL_TRUNK]# touch foo [root@devemr01 UTIL_TRUNK]# svn status ? foo
UpdateWithCleanUpdater.java needs to explicitly request that SVNKit include 'ignored' files such that they can be deleted.
Chris Lee
added a comment - This is caused by changes in SVN 1.7 relating to ignored files - they are no longer reported by default in 'svn status' (the SVNKit API used by Jenkins mirrors this behaviour).
In the workspace for a Jenkins project (after the job has run), no changes are shown:
[root@devemr01 UTIL_TRUNK]# svn status
However, the below command shows files/directories that are marked as ignored:
[root@devemr01 UTIL_TRUNK]# svn status --no-ignore
I .gradle
I build
...and creating a new file that isn't ignored shows up in 'svn status':
[root@devemr01 UTIL_TRUNK]# touch foo
[root@devemr01 UTIL_TRUNK]# svn status
? foo
UpdateWithCleanUpdater.java needs to explicitly request that SVNKit include 'ignored' files such that they can be deleted. Chris Lee
added a comment - Code from UpdateWithCleanUpdater.java:
clientManager.getStatusClient().doStatus(local, null, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler() { public void handleStatus(SVNStatus status) throws SVNException { SVNStatusType s = status.getContentsStatus(); if (s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED) { listener.getLogger().println("Deleting "+status.getFile()); try { File f = status.getFile(); if (f.isDirectory()) hudson.Util.deleteRecursive(f); else f.delete(); } catch (IOException e) { throw new SVNException(SVNErrorMessage.create(SVNErrorCode.UNKNOWN, e)); } } } }, null);
...the fifth parameter is 'includeIgnored' which is strangely set to false (and always has been), although the subsequent code expects ignored elements. Guessing that the inclusion of 'ignored' files was a bug with previous SVN (and SVNKit) versions that Jenkins inadvertently relied on.
SVNKit API: http://svnkit.com/javadoc/org/tmatesoft/svn/core/wc/SVNStatusClient.html#doStatus(java.io.File, org.tmatesoft.svn.core.wc.SVNRevision, org.tmatesoft.svn.core.SVNDepth, boolean, boolean, boolean, boolean, org.tmatesoft.svn.core.wc.ISVNStatusHandler, java.util.Collection)
Chris Lee
added a comment -
Code from UpdateWithCleanUpdater.java:
clientManager.getStatusClient().doStatus(local, null , SVNDepth.INFINITY, false , false , true , false , new ISVNStatusHandler() {
public void handleStatus(SVNStatus status) throws SVNException {
SVNStatusType s = status.getContentsStatus();
if (s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED) {
listener.getLogger().println( "Deleting " +status.getFile());
try {
File f = status.getFile();
if (f.isDirectory())
hudson.Util.deleteRecursive(f);
else
f.delete();
} catch (IOException e) {
throw new SVNException(SVNErrorMessage.create(SVNErrorCode.UNKNOWN, e));
}
}
}
}, null );
...the fifth parameter is 'includeIgnored' which is strangely set to false (and always has been), although the subsequent code expects ignored elements. Guessing that the inclusion of 'ignored' files was a bug with previous SVN (and SVNKit) versions that Jenkins inadvertently relied on.
SVNKit API: http://svnkit.com/javadoc/org/tmatesoft/svn/core/wc/SVNStatusClient.html#doStatus(java.io.File , org.tmatesoft.svn.core.wc.SVNRevision, org.tmatesoft.svn.core.SVNDepth, boolean, boolean, boolean, boolean, org.tmatesoft.svn.core.wc.ISVNStatusHandler, java.util.Collection)
Martin Adam
added a comment - Does this explain not deleting unversioned files as well? Because you're mentioning only ignored files...
Martin Adam
added a comment - Does this explain not deleting unversioned files as well? Because you're mentioning only ignored files... Chris Lee
added a comment - No. There is more to this issue than just the ignored files.
Here's a test case from a simple Jenkins / SVN (Jenkins 1.483, SVN 1.7):
After wiping out the workspace and running an initial build:
[root@devemr01 UTIL_TRUNK]# ll total 24 drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build -rw-rw-r-- 1 jenkins jenkins 376 Sep 30 00:06 build.gradle drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 lib drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 releng -rw-rw-r-- 1 jenkins jenkins 37 Sep 30 00:06 settings.gradle drwxrwxr-x 4 jenkins jenkins 4096 Sep 30 00:06 src
...adding an unversioned file:
[root@devemr01 UTIL_TRUNK]# touch foo [root@devemr01 UTIL_TRUNK]# svn status ? foo
Running the job (note that the unversioned file wasn't detected or removed):
00:07:59 Cleaning up /home/jenkins/workspace/UTIL_TRUNK/.
00:07:59 Updating svn://svn.ma.net/util/trunk to revision '2012-09-30T00:07:58.927 -0700'
00:07:59 At revision 38394
...and the unversioned file, and the ignored dir 'build', are still present:
[root@devemr01 UTIL_TRUNK]# ll total 24 drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build -rw-rw-r-- 1 jenkins jenkins 376 Sep 30 00:06 build.gradle -rw-r--r-- 1 root root 0 Sep 30 00:07 foo drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 lib drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 releng -rw-rw-r-- 1 jenkins jenkins 37 Sep 30 00:06 settings.gradle drwxrwxr-x 4 jenkins jenkins 4096 Sep 30 00:06 src [root@devemr01 UTIL_TRUNK]#
It appears that the SVNKit 'status' API call is working differently (and possibly incorrectly) for SVN 1.7 repositories.
Chris Lee
added a comment - No. There is more to this issue than just the ignored files.
Here's a test case from a simple Jenkins / SVN (Jenkins 1.483, SVN 1.7):
After wiping out the workspace and running an initial build:
[root@devemr01 UTIL_TRUNK]# ll
total 24
drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build
-rw-rw-r-- 1 jenkins jenkins 376 Sep 30 00:06 build.gradle
drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 lib
drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 releng
-rw-rw-r-- 1 jenkins jenkins 37 Sep 30 00:06 settings.gradle
drwxrwxr-x 4 jenkins jenkins 4096 Sep 30 00:06 src
...adding an unversioned file:
[root@devemr01 UTIL_TRUNK]# touch foo
[root@devemr01 UTIL_TRUNK]# svn status
? foo
Running the job (note that the unversioned file wasn't detected or removed):
00:07:59 Cleaning up /home/jenkins/workspace/UTIL_TRUNK/.
00:07:59 Updating svn: //svn.ma.net/util/trunk to revision '2012-09-30T00:07:58.927 -0700'
00:07:59 At revision 38394
...and the unversioned file, and the ignored dir 'build', are still present:
[root@devemr01 UTIL_TRUNK]# ll
total 24
drwxrwxr-x 10 jenkins jenkins 4096 Sep 30 00:06 build
-rw-rw-r-- 1 jenkins jenkins 376 Sep 30 00:06 build.gradle
-rw-r--r-- 1 root root 0 Sep 30 00:07 foo
drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 lib
drwxrwxr-x 2 jenkins jenkins 4096 Sep 30 00:06 releng
-rw-rw-r-- 1 jenkins jenkins 37 Sep 30 00:06 settings.gradle
drwxrwxr-x 4 jenkins jenkins 4096 Sep 30 00:06 src
[root@devemr01 UTIL_TRUNK]#
It appears that the SVNKit 'status' API call is working differently (and possibly incorrectly) for SVN 1.7 repositories. Chris Lee
added a comment - Found the issue; the status objects returned by SVNKit have different values when used against a SVN 1.7 working copy.
Using the original snippet of code, status objects are returned for the correct files, but their status is STATUS_NONE, and hence the subsequent logic for removing files of specific statuses is not triggered:
/home/jenkins/workspace/UTIL_TRUNK/.gradle none /home/jenkins/workspace/UTIL_TRUNK/foo none /home/jenkins/workspace/UTIL_TRUNK/build none
Modifying the code to combine the node & contents status yields the correct results:
/home/jenkins/workspace/UTIL_TRUNK/.gradle ignored Deleting /home/jenkins/workspace/UTIL_TRUNK/.gradle /home/jenkins/workspace/UTIL_TRUNK/foo unversioned Deleting /home/jenkins/workspace/UTIL_TRUNK/foo /home/jenkins/workspace/UTIL_TRUNK/build ignored
The relevant change is:
// for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.)
SVNStatusType s = status.getCombinedNodeAndContentsStatus();
//SVNStatusType s = status.getContentsStatus();
The complete test harness is:
import java.io.File; import java.util.*; import org.tmatesoft.svn.core.SVNDepth; import org.tmatesoft.svn.core.SVNException; import org.tmatesoft.svn.core.wc.ISVNStatusHandler; import org.tmatesoft.svn.core.wc.SVNClientManager; import org.tmatesoft.svn.core.wc.SVNStatus; import org.tmatesoft.svn.core.wc.SVNStatusType; import org.tmatesoft.svn.core.wc.*; import org.tmatesoft.svn.core.internal.wc.*; public class Test { public static void main( String[] args ) throws SVNException { SVNClientManager clientManager = SVNClientManager.newInstance(); Collection<String> changeLists = Collections.emptyList(); //clientManager.getStatusClient().doStatus( new File( args[ 0 ] ).getAbsoluteFile(), SVNRevision.WORKING, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler() clientManager.getStatusClient().doStatus( new File( args[ 0 ] ), null, SVNDepth.INFINITY, false, false, true, false, new ISVNStatusHandler() { @Override public void handleStatus( SVNStatus status ) throws SVNException { // for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.) SVNStatusType s = status.getCombinedNodeAndContentsStatus(); //SVNStatusType s = status.getContentsStatus(); System.out.printf( "%-60s %s\n", status.getFile(), s ); if( s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED ) { System.out.println( "Deleting " + status.getFile() ); } } }, null ); } }
Chris Lee
added a comment -
Found the issue; the status objects returned by SVNKit have different values when used against a SVN 1.7 working copy.
Using the original snippet of code, status objects are returned for the correct files, but their status is STATUS_NONE, and hence the subsequent logic for removing files of specific statuses is not triggered:
/home/jenkins/workspace/UTIL_TRUNK/.gradle none
/home/jenkins/workspace/UTIL_TRUNK/foo none
/home/jenkins/workspace/UTIL_TRUNK/build none
Modifying the code to combine the node & contents status yields the correct results:
/home/jenkins/workspace/UTIL_TRUNK/.gradle ignored
Deleting /home/jenkins/workspace/UTIL_TRUNK/.gradle
/home/jenkins/workspace/UTIL_TRUNK/foo unversioned
Deleting /home/jenkins/workspace/UTIL_TRUNK/foo
/home/jenkins/workspace/UTIL_TRUNK/build ignored
The relevant change is:
// for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.)
SVNStatusType s = status.getCombinedNodeAndContentsStatus();
//SVNStatusType s = status.getContentsStatus();
The complete test harness is:
import java.io.File;
import java.util.*;
import org.tmatesoft.svn.core.SVNDepth;
import org.tmatesoft.svn.core.SVNException;
import org.tmatesoft.svn.core.wc.ISVNStatusHandler;
import org.tmatesoft.svn.core.wc.SVNClientManager;
import org.tmatesoft.svn.core.wc.SVNStatus;
import org.tmatesoft.svn.core.wc.SVNStatusType;
import org.tmatesoft.svn.core.wc.*;
import org.tmatesoft.svn.core.internal.wc.*;
public class Test
{
public static void main( String [] args ) throws SVNException
{
SVNClientManager clientManager = SVNClientManager.newInstance();
Collection< String > changeLists = Collections.emptyList();
//clientManager.getStatusClient().doStatus( new File( args[ 0 ] ).getAbsoluteFile(), SVNRevision.WORKING, SVNDepth.INFINITY, false , false , true , false , new ISVNStatusHandler()
clientManager.getStatusClient().doStatus( new File( args[ 0 ] ), null , SVNDepth.INFINITY, false , false , true , false , new ISVNStatusHandler()
{
@Override
public void handleStatus( SVNStatus status ) throws SVNException
{
// for SVN 1.7 working copies, status.getContentsStatus() is STATUS_NONE; need to use the combined status to get the correct status (UNVERSIONED, IGNORED, etc.)
SVNStatusType s = status.getCombinedNodeAndContentsStatus();
//SVNStatusType s = status.getContentsStatus();
System .out.printf( "%-60s %s\n" , status.getFile(), s );
if ( s == SVNStatusType.STATUS_UNVERSIONED || s == SVNStatusType.STATUS_IGNORED || s == SVNStatusType.STATUS_MODIFIED )
{
System .out.println( "Deleting " + status.getFile() );
}
}
}, null );
}
}
Code changed in jenkins
User: Nicolas De loof
Path:
src/main/java/hudson/scm/subversion/UpdateWithCleanUpdater.java
http://jenkins-ci.org/commit/subversion-plugin/82377581f16cccaed8c4a90169d7d4c11d3f9845
Log:
Merge pull request #22 from forderud/master
JENKINS-13844: "Emulate clean checkout by..." does not work with 1.7 subversion repositories
Compare: https://github.com/jenkinsci/subversion-plugin/compare/3f18badadf37...82377581f16c
Fredrik Orderud
added a comment - Unable to reproduce problem anymore after commit of pull request #22. Therefore resolving.
Fredrik Orderud
added a comment - Unable to reproduce problem anymore after commit of pull request #22. Therefore resolving. 
Linards L
added a comment - Still happens on 1.492 with svn 1.43.
System:
java.vm.version 22.1-b02
line.separator
mail.smtp.sendpartial true
mail.smtps.sendpartial true
os.arch amd64
os.name Windows Server 2008 R2 (Datacenter)
Linards L
added a comment - Still happens on 1.492 with svn 1.43.
System:
java.vm.version 22.1-b02
line.separator
mail.smtp.sendpartial true
mail.smtps.sendpartial true
os.arch amd64
os.name Windows Server 2008 R2 (Datacenter) The fix has now been released with Subversion plugin 1.44.
Linards L: Could you please try to upgrade to SVN plugin 1.44, and try to reproduce afterwards?
Fredrik Orderud
added a comment - - edited The fix has now been released with Subversion plugin 1.44.
Linards L: Could you please try to upgrade to SVN plugin 1.44, and try to reproduce afterwards? 
Linards L
added a comment - Ok. Please update fixVersion field Fredrik Orderud
added a comment - Agree, but for it appears like the version field(s) aren't properly configured. I receive an "1.44 is not a valid version" error message when attempting to update the field.
Fredrik Orderud
added a comment - Agree, but for it appears like the version field(s) aren't properly configured. I receive an "1.44 is not a valid version" error message when attempting to update the field. 
Subversion Plugin DOES NOT support 1.7.