Loading...

XML

Word

Printable

Type: Bug
Resolution: Won't Fix
Priority: Major
Component/s: git-plugin
Labels:
None

Similar Issues:

Show

In our team we are facing the problem of unwanted repetitive builds of the same commit hash.

I looked into the sources of git-plugin, switched on verbose logging with "-Dhudson.plugins.git.GitSCM.verbose=true" and found that storing BuildData map of already built branches in build.xml does not work well if the job is configured to run concurrently ("Execute concurrent builds if necessary"). As discussed in https://github.com/jenkinsci/git-plugin/pull/163, saving the map in the job root could solve the problem. However, currently running builds (one or even more) of the job must also be taken into account to prevent single commit to be built several times in a chain.

Jenkins and plugins versions: 1.532.1, git-plugin 2.0.1, git-client-plugin 1.6.1

Detailed job setup:

maven project - full compilation with all tests
job is started via push hook from git (notifyCommit?...&branch=...)
Branch specifier matches only master, feature and bugfix branches, but ignores personal and other branches.
Branch specifier=":origin/(master|bugfix/S14|feature/S14).*"
It is in regex format because we use name convention for feature/bugfix branches in the form feature/sprint/jiraIssueAndOptionalText, e.g. feature/S14.02/JIRA-123
as there are more developers working independently of each other, the job should allow to start concurrent builds as soon as possible after the developer pushes a new branch/commit
configured Post-build action "Set build description" to set "${GIT_BRANCH}, ${GIT_COMMIT}" for both success and failed builds

I tried to prepare a small repository to demonstrate the problem - without success. It seems the problem is related to the size of the repository - when cloning takes longer time.

We suggest that calls of determineRevisionToBuild and buildData.saveBuild (at least the scheduled commit to build) in hudson.plugins.git.GitSCM#checkout should be as close as possible, before the branch checkout starts (git.checkoutBranch).

Here it is what happens for a copy of real-size repository (workspace about 1.5GB) with just 4 branches, in a clean new Jenkins job (NotifyCommitTest3), just single push "git/notifyCommit?url=...&branch=master":

Build History:

Success > Console Output  #6 	Jan 21, 2014 11:31:17 PM	 13 KB
	origin/feature/S14.02/stash-jenkins-test, 549bbb10e97a90b1f26db59529ba861797771ccc
Success > Console Output  #5 	Jan 21, 2014 11:31:12 PM	 12 KB
	origin/master, 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c
Success > Console Output  #4 	Jan 21, 2014 11:31:07 PM	 12 KB
	origin/master, 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c
Success > Console Output  #3 	Jan 21, 2014 11:31:03 PM	 11 KB
	origin/bugfix/S14.02/stash-jenkins-test, 6fa2f3f32e94997c84c9a49b36c6346565ce8851
Success > Console Output  #2 	Jan 21, 2014 11:30:09 PM	 11 KB
	origin/bugfix/S14.02/stash-jenkins-test, 6fa2f3f32e94997c84c9a49b36c6346565ce8851
Success > Console Output  #1 	Jan 21, 2014 11:28:49 PM	 11 KB
	origin/bugfix/S14.02/stash-jenkins-test, 6fa2f3f32e94997c84c9a49b36c6346565ce8851

Build #1:

Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ch06bl09.mgmt.lmc.cz in workspace /srv/jenkins/workspace/NotifyCommitTest3
Using strategy: Default
Cloning the remote Git repository
Cloning repository ssh://git@stash.int.lmc.cz:7999/rec/g2.git
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
getCandidateRevisions(false,null,,,null,remoteUrls=[ssh://git@stash.int.lmc.cz:7999/rec/g2.git],buildsByBranchName={},lastBuild=null]) considering branches to build
Seen branch in repository origin/bugfix/S14.02/stash-jenkins-test
Seen branch in repository origin/feature/S14.02/stash-jenkins-test
Seen branch in repository origin/master
Seen branch in repository origin/sprint/g2.2
Seen 4 remote branches
Starting with all the branches: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision a8b7d3421db68823b071ca74f269210718929406 (origin/sprint/g2.2)]
Ignoring Branch origin/sprint/g2.2(AnyObjectId[a8b7d3421db68823b071ca74f269210718929406]) because it doesn't match branch specifier
Ignoring Revision a8b7d3421db68823b071ca74f269210718929406 () because we don't care about any of the branches that point to it
After branch filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)]
After non-tip filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)]
Removing what's already been built: {}
After filtering out what's already been built: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)]
Multiple candidate revisions
Scheduling another build to catch up with NotifyCommitTest3
Checking out Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)
First time build. Skipping changelog.
Description set: origin/bugfix/S14.02/stash-jenkins-test, 6fa2f3f32e94997c84c9a49b36c6346565ce8851
Finished: SUCCESS

Build #2 - ignores commit built in #1 and does the same job:

Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ch06bl09.mgmt.lmc.cz in workspace /srv/jenkins/workspace/NotifyCommitTest3@2
Using strategy: Default
Cloning the remote Git repository
Cloning repository ssh://git@stash.int.lmc.cz:7999/rec/g2.git
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
getCandidateRevisions(false,null,,,null,remoteUrls=[ssh://git@stash.int.lmc.cz:7999/rec/g2.git],buildsByBranchName={},lastBuild=null]) considering branches to build
Seen branch in repository origin/bugfix/S14.02/stash-jenkins-test
Seen branch in repository origin/feature/S14.02/stash-jenkins-test
Seen branch in repository origin/master
Seen branch in repository origin/sprint/g2.2
Seen 4 remote branches
Starting with all the branches: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision a8b7d3421db68823b071ca74f269210718929406 (origin/sprint/g2.2)]
Ignoring Branch origin/sprint/g2.2(AnyObjectId[a8b7d3421db68823b071ca74f269210718929406]) because it doesn't match branch specifier
Ignoring Revision a8b7d3421db68823b071ca74f269210718929406 () because we don't care about any of the branches that point to it
After branch filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
After non-tip filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Removing what's already been built: {}
After filtering out what's already been built: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Multiple candidate revisions
Scheduling another build to catch up with NotifyCommitTest3
Checking out Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)
Description set: origin/bugfix/S14.02/stash-jenkins-test, 6fa2f3f32e94997c84c9a49b36c6346565ce8851
Finished: SUCCESS

Build #3:

Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ch06bl09.mgmt.lmc.cz in workspace /srv/jenkins/workspace/NotifyCommitTest3
Using strategy: Default
Fetching changes from the remote Git repository
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
getCandidateRevisions(false,null,,,null,remoteUrls=[ssh://git@stash.int.lmc.cz:7999/rec/g2.git],buildsByBranchName={},lastBuild=null]) considering branches to build
Seen branch in repository origin/bugfix/S14.02/stash-jenkins-test
Seen branch in repository origin/feature/S14.02/stash-jenkins-test
Seen branch in repository origin/master
Seen branch in repository origin/sprint/g2.2
Seen 4 remote branches
Starting with all the branches: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision a8b7d3421db68823b071ca74f269210718929406 (origin/sprint/g2.2)]
Ignoring Branch origin/sprint/g2.2(AnyObjectId[a8b7d3421db68823b071ca74f269210718929406]) because it doesn't match branch specifier
Ignoring Revision a8b7d3421db68823b071ca74f269210718929406 () because we don't care about any of the branches that point to it
After branch filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
After non-tip filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Removing what's already been built: {}
After filtering out what's already been built: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Multiple candidate revisions
Scheduling another build to catch up with NotifyCommitTest3
Checking out Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)
First time build. Skipping changelog.
Description set: origin/bugfix/S14.02/stash-jenkins-test, 6fa2f3f32e94997c84c9a49b36c6346565ce8851
Finished: SUCCESS

Build #4 - skips the already-built-commit for the first time:

Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ch06bl09.mgmt.lmc.cz in workspace /srv/jenkins/workspace/NotifyCommitTest3
Using strategy: Default
Last Built Revision: Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)
Fetching changes from the remote Git repository
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
getCandidateRevisions(false,null,,,null,remoteUrls=[ssh://git@stash.int.lmc.cz:7999/rec/g2.git],buildsByBranchName={origin/bugfix/S14.02/stash-jenkins-test=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)},lastBuild=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)]) considering branches to build
Seen branch in repository origin/bugfix/S14.02/stash-jenkins-test
Seen branch in repository origin/feature/S14.02/stash-jenkins-test
Seen branch in repository origin/master
Seen branch in repository origin/sprint/g2.2
Seen 4 remote branches
Starting with all the branches: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision a8b7d3421db68823b071ca74f269210718929406 (origin/sprint/g2.2)]
Ignoring Branch origin/sprint/g2.2(AnyObjectId[a8b7d3421db68823b071ca74f269210718929406]) because it doesn't match branch specifier
Ignoring Revision a8b7d3421db68823b071ca74f269210718929406 () because we don't care about any of the branches that point to it
After branch filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
After non-tip filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Removing what's already been built: {origin/bugfix/S14.02/stash-jenkins-test=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)}
After filtering out what's already been built: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Multiple candidate revisions
Scheduling another build to catch up with NotifyCommitTest3
Checking out Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master)
First time build. Skipping changelog.
Description set: origin/master, 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c
Finished: SUCCESS

Build #5:

Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ch06bl09.mgmt.lmc.cz in workspace /srv/jenkins/workspace/NotifyCommitTest3@2
Using strategy: Default
Last Built Revision: Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)
Fetching changes from the remote Git repository
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
getCandidateRevisions(false,null,,,null,remoteUrls=[ssh://git@stash.int.lmc.cz:7999/rec/g2.git],buildsByBranchName={origin/bugfix/S14.02/stash-jenkins-test=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)},lastBuild=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)]) considering branches to build
Seen branch in repository origin/bugfix/S14.02/stash-jenkins-test
Seen branch in repository origin/feature/S14.02/stash-jenkins-test
Seen branch in repository origin/master
Seen branch in repository origin/sprint/g2.2
Seen 4 remote branches
Starting with all the branches: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision a8b7d3421db68823b071ca74f269210718929406 (origin/sprint/g2.2)]
Ignoring Branch origin/sprint/g2.2(AnyObjectId[a8b7d3421db68823b071ca74f269210718929406]) because it doesn't match branch specifier
Ignoring Revision a8b7d3421db68823b071ca74f269210718929406 () because we don't care about any of the branches that point to it
After branch filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
After non-tip filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Removing what's already been built: {origin/bugfix/S14.02/stash-jenkins-test=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)}
After filtering out what's already been built: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Multiple candidate revisions
Scheduling another build to catch up with NotifyCommitTest3
Checking out Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master)
Description set: origin/master, 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c
Finished: SUCCESS

Build #6:

Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ch06bl09.mgmt.lmc.cz in workspace /srv/jenkins/workspace/NotifyCommitTest3
Using strategy: Default
Last Built Revision: Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master)
Fetching changes from the remote Git repository
Fetching upstream changes from ssh://git@stash.int.lmc.cz:7999/rec/g2.git
getCandidateRevisions(false,null,,,null,remoteUrls=[ssh://git@stash.int.lmc.cz:7999/rec/g2.git],buildsByBranchName={origin/master=Build #5 of Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), origin/bugfix/S14.02/stash-jenkins-test=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)},lastBuild=Build #5 of Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master)]) considering branches to build
Seen branch in repository origin/bugfix/S14.02/stash-jenkins-test
Seen branch in repository origin/feature/S14.02/stash-jenkins-test
Seen branch in repository origin/master
Seen branch in repository origin/sprint/g2.2
Seen 4 remote branches
Starting with all the branches: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test), Revision a8b7d3421db68823b071ca74f269210718929406 (origin/sprint/g2.2)]
Ignoring Branch origin/sprint/g2.2(AnyObjectId[a8b7d3421db68823b071ca74f269210718929406]) because it doesn't match branch specifier
Ignoring Revision a8b7d3421db68823b071ca74f269210718929406 () because we don't care about any of the branches that point to it
After branch filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)]
After non-tip filtering: [Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test), Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)]
Removing what's already been built: {origin/master=Build #5 of Revision 7c98dcefea27ba90f5dc64710fdcd9f128f5ab5c (origin/master), origin/bugfix/S14.02/stash-jenkins-test=Build #3 of Revision 6fa2f3f32e94997c84c9a49b36c6346565ce8851 (origin/bugfix/S14.02/stash-jenkins-test)}
After filtering out what's already been built: [Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)]
Checking out Revision 549bbb10e97a90b1f26db59529ba861797771ccc (origin/feature/S14.02/stash-jenkins-test)
First time build. Skipping changelog.
Description set: origin/feature/S14.02/stash-jenkins-test, 549bbb10e97a90b1f26db59529ba861797771ccc
Finished: SUCCESS

Assignee:: Mark Waite

Reporter:: Vojta Minarik

Votes:: 8 Vote for this issue

Watchers:: 22 Start watching this issue

Created:: 2014-01-22 12:45

Updated:: 2019-10-23 01:28

Resolved:: 2016-03-10 12:56

Details

Description

Attachments

Activity

People

Dates