Details
-
Bug
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Won't Fix
-
Jenkins ver. 1.460; Subversion Plugin ver. 1.40, 1.39; Windows Server 2003 R2 x64 SP2; Subversion server ver. 1.6.16.
Description
Updating Subversion Plugin to 1.40 caused multiple bugs. Rolling back to 1.39 resulted to complete inability to perform a checkout. CI server is totally unusable now.
Issue #1
After updating to Subversion Plugin to 1.40 (from 1.39) this exception started occurring sometimes (various salves, various jobs):
Checking out a fresh workspace because there's no workspace at C:_JenkinsCI\workspace\XXX
Cleaning local Directory .
Checking out http://oursvnserver/svn/svnLatest/.../XXX
A ...
ERROR: Failed to check out http://oursvnserver/svn/svnLatest/.../XXX
org.tmatesoft.svn.core.SVNException: svn: E175002: REPORT /svn/svnLatest/!svn/vcc/default failed
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:304)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:289)
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:277)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:696)
at org.tmatesoft.svn.core.internal.io.dav.DAVConnection.doReport(DAVConnection.java:328)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1289)
at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:837)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:216)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:100)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.checkout(SvnNgAbstractUpdate.java:756)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:14)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:9)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1221)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:292)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:781)
at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:85)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:144)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:152)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:121)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:144)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:789)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:770)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:753)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2154)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:287)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: svn: E175002: REPORT /svn/svnLatest/!svn/vcc/default failed
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:208)
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:154)
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:97)
... 35 more
FATAL: null
java.lang.NullPointerException
Issue #2
Emulate clean checkout by first deleting unversioned/ignored files, then 'svn update' no longer deletes unversioned files in 1.40.
Issue #3
Because of #2 and #3 I've decided to roll back to 1.39. So, I've wiped out all workspaces on all nodes and... 1.39 is no longer able to checkout due to:
Checking out a fresh workspace because there's no workspace at C:_JenkinsCI\workspace\update_Tools_on_all_nodes
Cleaning local Directory .
Checking out http://oursvnserver/svn/svnLatest/Tools
ERROR: Failed to check out http://oursvnserver/svn/svnLatest/Tools
org.tmatesoft.svn.core.SVNException: svn: 'C:_JenkinsCI\workspace\update_Tools_on_all_nodes' is not a working copy
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
at org.tmatesoft.svn.core.internal.wc.admin.SVNAdminAreaFactory.open(SVNAdminAreaFactory.java:197)
at org.tmatesoft.svn.core.internal.wc.admin.SVNWCAccess.doOpen(SVNWCAccess.java:379)
at org.tmatesoft.svn.core.internal.wc.admin.SVNWCAccess.open(SVNWCAccess.java:283)
at org.tmatesoft.svn.core.internal.wc.admin.SVNWCAccess.open(SVNWCAccess.java:276)
at org.tmatesoft.svn.core.internal.wc.admin.SVNWCAccess.openAnchor(SVNWCAccess.java:171)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.update(SVNUpdateClient.java:512)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:942)
at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:84)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:136)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:144)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:121)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:136)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:788)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:769)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:753)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2154)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:287)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.tmatesoft.svn.core.SVNErrorMessage: svn: 'C:_JenkinsCI\workspace\update_Tools_on_all_nodes' is not a working copy
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:163)
at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:118)
at org.tmatesoft.svn.core.internal.wc.admin.SVNAdminAreaFactory.open(SVNAdminAreaFactory.java:192)
... 24 more
FATAL: null
java.lang.NullPointerException
at java.util.ArrayList.addAll(Unknown Source)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:743)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:685)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1218)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:581)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:470)
at hudson.model.Run.run(Run.java:1421)
at hudson.matrix.MatrixBuild.run(MatrixBuild.java:248)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:238)
at hudson.model.OneOffExecutor.run(OneOffExecutor.java:66)
Needless to say that 'C:_JenkinsCI\workspace\update_Tools_on_all_nodes' didn't even exist before checkout.
This happens on both slave and master.
Attachments
Issue Links
- is duplicated by
-
-
- Resolved
-
- is related to
-
-
- Resolved
-
-
-
- Resolved
-
Activity
I have not had time to look into this more, but my colleague found an excellent solution. He installed both the monitoring and the groovy plugins, and found that there was now an option to "Execute System Groovy script" available for the build step. The script he ran is as follows:
import net.bull.javamelody.*;
before = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
System.gc();
after = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
println I18N.getFormattedString("ramasse_miette_execute", Math.round((before - after) / 1024));
He set this to run every 10 minutes, so that each garbage collection was small, and we are confident that this will eliminate the problem. Here's hoping that this will help others with similar problems!
pancake, Did you solve your problem by upgrading to newest versions?
This ticket mentions a very old version of Subversion Plugin. There is no feedback for a long time.
So either the Garbage Collector that you run is not ideal for your type of load or you have enough memory, but just barely enough and because of this the GC needs to do lots of work to make some memory available.
In the first case a look at some garbage collection tuning will help, in the second a bit more memory might do the trick. Looking at the Java VM with jvisualvm can give you an idea how memory of the application looks like and which solution might give better results.
Maybe you can provide some facts, e.g. actual memory settings and what the memory graphs look like.