[JENKINS-15098] svnexternals causing issues with concurrent builds - Jenkins Jira

XML

Word

Printable

Details

Type: Bug
Status: Resolved (View Workflow)
Priority: Major
Resolution: Fixed
Component/s: core, subversion-plugin
Labels:
None

Similar Issues:

Show

Description

We're getting these quite often with concurrently running jobs that pull out of SVN:

java.io.IOException: Unable to delete /var/lib/jenkins/jobs/[jobname]/svnexternals.txt
at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:112)
at hudson.XmlFile.write(XmlFile.java:174)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:708)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1195)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:576)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:465)
at hudson.model.Run.run(Run.java:1409)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:238)

Attachments

Activity

Ascending order - Click to sort in descending order

Margaret Lewicka created issue - 2012-09-10 15:35

kutzi added a comment - 2012-11-24 10:52

Is this on Windows?

kutzi added a comment - 2012-11-24 10:52 Is this on Windows?

kutzi added a comment - 2013-02-24 16:17

No feedback from reporter, so closing as Incomplete

kutzi added a comment - 2013-02-24 16:17 No feedback from reporter, so closing as Incomplete

kutzi made changes - 2013-02-24 16:17

Field	Original Value	New Value
Resolution		Incomplete [ 4 ]
Status	Open [ 1 ]	Resolved [ 5 ]

Sai K added a comment - 2013-03-12 23:39

I am also seeing the same issue with concurrent builds of the same job on the same node.
I am using Linux.

Sai K added a comment - 2013-03-12 23:39 I am also seeing the same issue with concurrent builds of the same job on the same node. I am using Linux.

Sai K made changes - 2013-03-12 23:39

Resolution	Incomplete [ 4 ]
Status	Resolved [ 5 ]	Reopened [ 4 ]

Ilya Lebedev added a comment - 2013-03-21 13:58

I see the same issue using Linux. With 3 slaves running the same job simultenously.

Ilya Lebedev added a comment - 2013-03-21 13:58 I see the same issue using Linux. With 3 slaves running the same job simultenously.

Aidan Lawn added a comment - 2013-04-30 17:31 - edited

I see same issue with concurrent jobs.
Jenkins version: Jenkins ver. 1.491
Jenkins Subversion plugin version: 1.45 (was also occurring on 1.43 - I updated to latest but didn't help)

As a workaround I am going to have to make a parent job that does the checkout first, then the concurrent jobs can run. Not ideal and may not be suitable for others situations.

Note: We only see this issue under load, when I guess slower running checkouts makes the chance of collision higher.

Aidan Lawn added a comment - 2013-04-30 17:31 - edited I see same issue with concurrent jobs. Jenkins version: Jenkins ver. 1.491 Jenkins Subversion plugin version: 1.45 (was also occurring on 1.43 - I updated to latest but didn't help) As a workaround I am going to have to make a parent job that does the checkout first, then the concurrent jobs can run. Not ideal and may not be suitable for others situations. Note: We only see this issue under load, when I guess slower running checkouts makes the chance of collision higher.

Wael Darwich added a comment - 2013-05-22 12:34

I am getting the same using Jenkins ver. 1.512 Linux master and Windows slaves

Wael Darwich added a comment - 2013-05-22 12:34 I am getting the same using Jenkins ver. 1.512 Linux master and Windows slaves

Wael Darwich added a comment - 2013-06-20 16:51

I found a solution that worked for me for this bug, in job configuration > Advanced Project Options > Retry Count > SCM checkout retry count for a value greater than zero, I tried 3 and it is working fine, I hope this helps others

Wael Darwich added a comment - 2013-06-20 16:51 I found a solution that worked for me for this bug, in job configuration > Advanced Project Options > Retry Count > SCM checkout retry count for a value greater than zero, I tried 3 and it is working fine, I hope this helps others

Frederik Fromm added a comment - 2013-06-24 11:09

I have the same problem with jenkins LTS 1.509.1. on linux. The path "/var/lib/jenkins/jobs" shows that the svnexternals.txt is located on the master even if the builds are only executed on slaves. That means, every running build of this job uses the same svnexternals.txt at the same time. IMHO each build should have it's own svnexternals.txt or is there a reason, why there is only one for the whole thing?

Frederik Fromm added a comment - 2013-06-24 11:09 I have the same problem with jenkins LTS 1.509.1. on linux. The path "/var/lib/jenkins/jobs" shows that the svnexternals.txt is located on the master even if the builds are only executed on slaves. That means, every running build of this job uses the same svnexternals.txt at the same time. IMHO each build should have it's own svnexternals.txt or is there a reason, why there is only one for the whole thing?

Oleg Nenashev added a comment - 2013-11-05 08:47

The issue can be reproduced on the following environment:

subversion-1.50
jenkins 1.509.4 (master runs on RHEL6.4)
Windows slaves (seems it does not matter)

Even if the workaround exists, the bug is quite painful.

Oleg Nenashev added a comment - 2013-11-05 08:47 The issue can be reproduced on the following environment: subversion-1.50 jenkins 1.509.4 (master runs on RHEL6.4) Windows slaves (seems it does not matter) Even if the workaround exists, the bug is quite painful.

Oleg Nenashev made changes - 2013-11-05 08:48

Assignee

Kohsuke Kawaguchi [ kohsuke ]

kutzi [ kutzi ]

Oleg Nenashev made changes - 2013-11-06 10:15

Assignee

kutzi [ kutzi ]

Oleg Nenashev [ oleg_nenashev ]

Oleg Nenashev added a comment - 2013-11-06 10:37

PR: https://github.com/jenkinsci/subversion-plugin/pull/55

Oleg Nenashev added a comment - 2013-11-06 10:37 PR: https://github.com/jenkinsci/subversion-plugin/pull/55

Oleg Nenashev added a comment - 2014-02-20 21:12

The PR is still waiting for Kutzi's review...

Oleg Nenashev added a comment - 2014-02-20 21:12 The PR is still waiting for Kutzi's review...

Oleg Nenashev made changes - 2014-02-20 21:12

Assignee

Oleg Nenashev [ oleg_nenashev ]

kutzi [ kutzi ]

kutzi made changes - 2014-03-02 09:21

Due Date

2014-03-09

kutzi made changes - 2014-03-28 13:02

Due Date

2014-03-09

SCM/JIRA link daemon added a comment - 2014-05-01 12:50

Code changed in jenkins
User: Oleg Nenashev
Path:
src/main/java/hudson/scm/SubversionSCM.java
src/main/java/hudson/scm/subversion/ExternalsFileManager.java
http://jenkins-ci.org/commit/subversion-plugin/dc5e7a6c58c9d9d05cb89f1a56fa1f7eaf99a569
Log:
[FIXED JENKINS-15098] - Added synchronization of all reads/modifications of svnexternals.txt file.

Resolves https://issues.jenkins-ci.org/browse/JENKINS-15098

Signed-off-by: Oleg Nenashev <nenashev@synopsys.com>

SCM/JIRA link daemon added a comment - 2014-05-01 12:50 Code changed in jenkins User: Oleg Nenashev Path: src/main/java/hudson/scm/SubversionSCM.java src/main/java/hudson/scm/subversion/ExternalsFileManager.java http://jenkins-ci.org/commit/subversion-plugin/dc5e7a6c58c9d9d05cb89f1a56fa1f7eaf99a569 Log: [FIXED JENKINS-15098] - Added synchronization of all reads/modifications of svnexternals.txt file. Resolves https://issues.jenkins-ci.org/browse/JENKINS-15098 Signed-off-by: Oleg Nenashev <nenashev@synopsys.com>

SCM/JIRA link daemon made changes - 2014-05-01 12:50

Resolution		Fixed [ 1 ]
Status	Reopened [ 4 ]	Resolved [ 5 ]

SCM/JIRA link daemon added a comment - 2014-05-01 12:50

Code changed in jenkins
User: Oleg Nenashev
Path:
src/main/java/hudson/scm/SubversionSCM.java
src/main/java/hudson/scm/SvnExternalsFileManager.java
src/main/java/hudson/scm/subversion/ExternalsFileManager.java
http://jenkins-ci.org/commit/subversion-plugin/9e45adf03dde5e24ec19f80f044c266656136875
Log:
~~JENKINS-15098~~ - Refactoring
Moved base class to another package, modified visibility and javadoc.

Related issues: https://issues.jenkins-ci.org/browse/JENKINS-15098

Signed-off-by: Oleg Nenashev <nenashev@synopsys.com>

SCM/JIRA link daemon added a comment - 2014-05-01 12:50 Code changed in jenkins User: Oleg Nenashev Path: src/main/java/hudson/scm/SubversionSCM.java src/main/java/hudson/scm/SvnExternalsFileManager.java src/main/java/hudson/scm/subversion/ExternalsFileManager.java http://jenkins-ci.org/commit/subversion-plugin/9e45adf03dde5e24ec19f80f044c266656136875 Log: JENKINS-15098 - Refactoring Moved base class to another package, modified visibility and javadoc. Related issues: https://issues.jenkins-ci.org/browse/JENKINS-15098 Signed-off-by: Oleg Nenashev <nenashev@synopsys.com>

SCM/JIRA link daemon added a comment - 2014-05-01 12:50

Code changed in jenkins
User: Christoph Kutzinski
Path:
src/main/java/hudson/scm/SubversionSCM.java
src/main/java/hudson/scm/SvnExternalsFileManager.java
http://jenkins-ci.org/commit/subversion-plugin/23168bd2710b672ac81b3eb397402b8fc809d301
Log:
Merge pull request #65 from synopsys-arc-oss/externalsfile-conflicts

[FIXED JENKINS-15098] - Access conflicts to svnexternals.txt

Compare: https://github.com/jenkinsci/subversion-plugin/compare/cd7b24e71725...23168bd2710b

SCM/JIRA link daemon added a comment - 2014-05-01 12:50 Code changed in jenkins User: Christoph Kutzinski Path: src/main/java/hudson/scm/SubversionSCM.java src/main/java/hudson/scm/SvnExternalsFileManager.java http://jenkins-ci.org/commit/subversion-plugin/23168bd2710b672ac81b3eb397402b8fc809d301 Log: Merge pull request #65 from synopsys-arc-oss/externalsfile-conflicts [FIXED JENKINS-15098] - Access conflicts to svnexternals.txt Compare: https://github.com/jenkinsci/subversion-plugin/compare/cd7b24e71725...23168bd2710b

Jenkins IRC Bot made changes - 2014-10-14 19:20

Component/s		core [ 15593 ]
Component/s	concurrent-build [ 15628 ]

R. Tyler Croy made changes - 2016-07-25 23:55

Workflow

JNJira [ 145818 ]

JNJira + In-Review [ 191641 ]

People

Assignee:: kutzi

Reporter:: Margaret Lewicka

Votes:: 7 Vote for this issue

Watchers:: 12 Start watching this issue

Dates

Created:: 2012-09-10 15:35

Updated:: 2014-10-14 19:20

Resolved:: 2014-05-01 12:50