Confirmed. On my system. What additional information is required to investigate this?

Are you using the JGit executable (as described in JENKINS-20286) or are you using the command line executable? The executable is selected from a drop down in the plugin configuration

I've not been able to find a repeatable case to duplicate the problem when using the Git command line executable, but I see it consistently when using the JGit executable.

Can you provide a repeatable set of steps which illustrate the problem (and preferably use the Git command line executable)? JENKINS-20286 provides the steps I used to show the problem with the JGit implementation.

1. Command line executable. Git version 1.8.4.4

Other observation: We use the multiple SCMs plugin. Which is known to be problematic. In this case the issue is not isolated to jobs that use that plugin. All jobs are affected.

I've adding the DisableRemotePoll option to all jobs. This resolved the issue for me. The script I used is:

import jenkins.model.*;
import hudson.model.*;
import hudson.tasks.*;
import hudson.plugins.git.*;
import hudson.plugins.git.extensions.impl. DisableRemotePoll;

def updateScm(project, scm) {
    println("updating " + scm.toString() + " for project " + project);
    remote_poll_opt = new DisableRemotePoll();
    scm.extensions.add(remote_poll_opt);
    project.save();
}

for(project in Hudson.instance.items) {
  scm = project.scm;
  if (scm instanceof org.jenkinsci.plugins.multiplescms.MultiSCM) {
    for(actual_scm in scm.getConfiguredSCMs())
      updateScm(project, (GitSCM)actual_scm);
  } else if (scm instanceof hudson.plugins.git.GitSCM) {
      updateScm(project, (GitSCM)scm);    
  } else {
    println("unknown scm type " + scm.toString());
  }
}

Remote polling seems to be broken in Git Plugin 2.0. I have worked around it in my environment by using "Force polling within workspace", which is fine as long as I do not also enable "Delete workspace when build is done". Ideally I would want to delete the workspace after a successful build but I can live with this until remote polling works again.

The workaround of using "Force polling within workspace", a DisableRemotePoll extension instance, does not work for Build Flow projects. Those no longer initialize a workspace. I believe that is an issue with the build flow plugin.

coreyoconnor when you say "confirmed" do you mean that you used command line or that you used jgit?

As indicated in comment #3: command line executable version 1.8.4.4

While writing tests for the git-client-plugin, I found a bug in the command line git implementation used for remote polling. The bug is fixed in git-client-plugin 1.6.2 which was just released.

It is worth trying git-client-plugin 1.6.2 to see if it resolves the problem. Refer to https://github.com/jenkinsci/git-client-plugin/commit/e5e75d5a100905c0d5b332bbde3642e96ffe10a9 for more details on the remote polling bug.

Thanks Mark, I just tried the new git-client plugin and it seems to fix the issue for me.

Tried the new git-client-plugin (1.6.2) and am still seeing this issue.

Zachariah, can you include the output of one of your jobs that detected changes when there were none?

Even better might be to include the output from a build and from the next build which follows it so that they can be compared. If the change in git-client-plugin 1.6.2 fixed the problem for David Merrill and did not fix it for you, then I suspect the problem you're seeing is slightly different than the problem seen by David.

same issue here.
git-client-plugin 1.6.2
build-flow-plugin 0.10

/job/builds.core.master.debug/53/pollingLog/

Polling Log

View as plain text
This page captures the polling log that triggered this build.

Started on Feb 6, 2014 5:14:08 PM
Using strategy: Default
[poll] Last Built Revision: Revision c217dff9f833fa41043a7fe7028c5d600d1bcbfc (origin/master)
using GIT_SSH to set credentials jenkins git access
Done. Took 0.28 sec
Changes found

/job/builds.core.master.debug/52/pollingLog/

Polling Log

View as plain text
This page captures the polling log that triggered this build.

Started on Feb 6, 2014 5:13:08 PM
Using strategy: Default
[poll] Last Built Revision: Revision c217dff9f833fa41043a7fe7028c5d600d1bcbfc (origin/master)
using GIT_SSH to set credentials jenkins git access
Done. Took 0.29 sec
Changes found

when using the workaround "force polling within workspace" this works okay though.
where does it store the SHA1 that have been built when that option is not enabled (ls-remote?) ?

Same issue here using Multiple SCM plugin.
Git 2.0.2.
Git Client 1.6.3.
Multiple SCM 0.3

Started on Feb 25, 2014 4:52:31 PM
Polling SCM changes on master
Using strategy: Default
[poll] Last Built Revision: Revision 18e02f062d1dbe081bc9c0b8bac3f5e7b8e85af6 (origin/master)
using GIT_SSH to set credentials
Using strategy: Default
[poll] Last Built Revision: Revision 18e02f062d1dbe081bc9c0b8bac3f5e7b8e85af6 (origin/master)
using GIT_SSH to set credentials
Done. Took 0.89 sec
Changes found

However the actual revisions are:
Revision: 18e02f062d1dbe081bc9c0b8bac3f5e7b8e85af6
Revision: 8f3e5979c556c117905f46dc1811964d70a21998

It looks like somehow the revision of one repo is used for both repos and maybe that kicks off the build.

Update from my post:
I went back to the job directory and deleted everything but the job configuration. Everything is working fine now. It may be that my original configuration only had one repo and when I added the second repo and issue arose.

Hi,

I identified the reason (at least in our scenario).
We are currently using git-client 1.6.1. But I checked the sources, the bug is also contained in the latest HEAD (20d98ce042e4d9d5ea1f3247567072d8dd0e2e35).

We have a repo with several branches containing namespaces (e.g. "feature1/master").
In the job config as branch specifier we use "remotes/origin/feature1/master"

Having a look at the code you will recognize the following:
hudson.plugins.git.GitSCM.compareRemoteRevisionWithImpl(AbstractProject<?, ?>, Launcher, FilePath, TaskListener, SCMRevisionState)
calls
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getHeadRev(String url, String branch)

In our case branch is "remotes/origin/feature1/master"
branch is split at '/' and only the last segment is used as branch -> "master" in our case (instead of "feature1/master" which would be correct)!

Since we have multiple "feature<x>/master" branches the called command returns multiple revisions and branches in random order.
$ git ls-remote -h ssh://git:29418/xy/main.git master
eb625526cc489b9466a709501550d5adab08e884 refs/heads/featureX/master
97dfc02c8a98745999d7fd1648b9532ac98be225 refs/heads/feature1/master
0e186342e89971ecd46ab58ffb999136326dabce refs/heads/featureY/master

CliGitAPIImpl takes the first entry which is not necessarily the correct one.
Since the hash does not match the hash of the last build the build is triggered again and again!

Kind regards,

Alexander Link
SAP AG, Germany

UPDATE: Fix proposed: https://github.com/jenkinsci/git-client-plugin/pull/120

    public ObjectId getHeadRev(String url, String branch) throws GitException, InterruptedException {
        String[] branchExploded = branch.split("/");
        branch = branchExploded[branchExploded.length-1];
        ArgumentListBuilder args = new ArgumentListBuilder("ls-remote");
        args.add("-h");

        StandardCredentials cred = credentials.get(url);
        if (cred == null) cred = defaultCredentials;

        args.add(url);
        args.add(branch);
        String result = launchCommandWithCredentials(args, null, cred, url);
        return result.length()>=40 ? ObjectId.fromString(result.substring(0, 40)) : null;
    }

in there this part:

String[] branchExploded = branch.split("/");
branch = branchExploded[branchExploded.length-1];

should perhaps be:

branch = "refs/heads/" + branch;

Unfortunately, if that change is made, it breaks multiple unit tests in the git-client-plugin.

Code changed in jenkins
User: Mark Waite
Path:
src/test/java/org/jenkinsci/plugins/gitclient/GitAPITestCase.java
http://jenkins-ci.org/commit/git-client-plugin/bacfa33355b61f089ac71462807f70e8b36ee54b
Log:
Additional tests of wildcards in call to getHeadRev()

Exploring change proposed for JENKINS-20767

I proposed a fix for (and also to demonstrate) the namespace issue: https://github.com/jenkinsci/git-client-plugin/pull/120

Regards,
Alex

Code changed in jenkins
User: Alexander Link
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/8c106e41fac4b928b3abaa524abe91d84c54072f
Log:
Fix for getHeadRev bug when using branch namespaces

JENKINS-20767

git-client-plugin 1.6.4 has been released and includes this change. It should be visible on the update center within the next 8 hours.

Refer to https://wiki.jenkins-ci.org/display/JENKINS/Git+Client+Plugin#GitClientPlugin-ChangeLog for the description of the changes in 1.6.4

Hello Mark,

I tried to test the latest git plugin (2.0.4) with the latest git-client plugin (1.6.4) and due to another change in the git plugin the approach with "remotes/origin/..." does not work!

Commit ea5cad7e2759d0c9f5ed723fe590f982b0bb818d "JENKINS-14026 DefaultBuildChooser should handle smartly branches with /" causes the issue.

When I specify "remotes/origin/feature1/master" the DefaultBuildChooser prepends the repo name (to String fqbn) "origin/remotes/origin/feature1/master" and for this String it is tried to get the head revision - which of course cannot work.

Shall I prepare another Pull Request or do you want to have a look into this?
Be aware that the help adjustments I contributed to the git plugin are already merged. The behaviour described there currently does not work due to above issue...

Thanks and regards,
Alex

That's unfortunate. Thanks for checking. I'll need to defer to Nicolas DeLoof on how to resolve that. I'm not sure if he'll want to have me revert your changes, or if we try to press forward and find some way to make the git plugin and the git client plugin agree on their handling of the specification of branches to build.

I'm quite hesitant to break backwards compatibility, because I don't know all the different ways people are dependent on the original behavior.

This small change should fix the issue: https://github.com/jenkinsci/git-plugin/pull/209

I'm not sure if maybe somewhere else "remotes/" is not handled correctly, but tests on our side look good.

Code changed in jenkins
User: Alexander Link
Path:
src/main/java/hudson/plugins/git/util/DefaultBuildChooser.java
http://jenkins-ci.org/commit/git-plugin/ecfa204bf61783a194f90f4648cff8f5a53dafc1
Log:
Fix to allow "remotes/origin/" in branch spec

https://issues.jenkins-ci.org/browse/JENKINS-20767
See comment:
https://issues.jenkins-ci.org/browse/JENKINS-20767?focusedCommentId=196575&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-196575

Hello Mark,

(Pull Request https://github.com/jenkinsci/git-client-plugin/pull/129)

I tested git-client 1.7.0 together with git 2.1.0 (I assume newer versions are also affected, but I'll check).
I obviously missed something during tests. We have the same issue we had with namespace branches again with the "master" branch. The CI job is triggered again and again.
The problem is that "master" still matches for "refs/heads/master", "refs/heads/abc/master", "refs/heads/xyz/master", etc. Since git ls-remote lists the refs alphabetically the first entry might be the correct one by chance - or not.
I extended the test GitAPITestCase.test_getHeadRev_namespaces().

When I tried to fix this issue I recognized that there are several additional problems with the "Branch Specifier" handling!
First of all you have to be aware that it is possible to create weird branch names containing words which should be reserved by Git but are not.
E.g. you can create a branch "refs/heads/master" (being refs/heads/refs/heads/master) or a branch "origin/master" (being refs/heads/origin/master), etc.
A remote name also can contain weird stuff and slashes! E.g. a remote can be called "origin/master"!

Even without this knowledge it was quite hard to definitely "guess" what is meant by a configured "Branch Specifier". Even git itself has problems with ambiguous specs. Try the following in a repo having a remote "origin" and branch "master":
git branch origin/master
git checkout origin/master
-> warning: refname 'origin/master' is ambiguous.

Therefore it is necessary to specify concrete rules which (from my perspective) did not exist yet.
From my perspective several potential variations have to be checked and depending on if it makes sense in the current repo and a rev could be identified the matching result is taken.
In org.jenkinsci.plugins.gitclient.LegacyCompatibleGitAPIImpl.normalizeBranchSpec(String) you can see how the proposed rules look like:

• If a branch spec starts with "refs/" try to get the revision for exactly this String.
• If a branch spec starts with "remote/" check if a remote with this name exists and build String "refs/heads/<branchName" definite for ls-remote
• If both was not the case check again if the branch spec starts with a valid remote (e.g. "origin/master")
• As ultimate fallbacks try "refs/heads/<branchSpec>" to check if a branch named like this exists (remember the weird possible names - we should not spend too much thoughts, just try)
• ... "refs/heads<branchSpec>" if wildcards are used (e.g. "*/master" -> "refs/heads*/master". "refs/heads/*/master" would not match the "master" branch)
• ... and simply "<branchSpec>" as final fallback.

The patterns are tried starting with the upper one. Ther first one which results in a definite result is considered to be the correct one.

Furthermore I now throw an AmbiguousResultException in case the result is ambiguous.
Until now simply the first entry of "git ls-remote" is taken which is way too randomly considering the alphabetical sorting.

I hope you see my points. I spent over a week now to understand and fix those issues
Maybe the tests and changes in the Pull Request show what I mean.

Thanks a lot and kind regards,
Alex

While considering "Branch specifier" handling, please also consider specifiers with refs/tags/** where one want each matched tag to build disregarding if they dereference to the same commit. This used to work, but has now regressed.
See my comments in JENKINS-14917: Build is not triggered for new tag (without new commit)

The git-client-plugin 1.10.0 includes a change from alexanderlink which allows "refs/heads" and "refs/tags" as a prefix to declare a branch spec in "Branches to build" which is unambiguous.

For example, if the branch to build should be the branch "develop/proposal" and the remote is named "origin", then the "Branches to build" value should be "refs/heads/develop/proposal".

Refer to Alexander's detailed documentation for more information and for descriptions of the cases which still have issues.

Believed to be resolved with git-client-plugin 1.10.0 through the changes from alexanderlink

I can't say I understood half of what the doc said, sorry. Would it be possible for you to explain how I need to configure my job? Currently my "Branches to build" says "master" and all works fine. I tried prepending it with "refs/heads/" and the build failed (ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.).

I use Jenkins 1.570, git-client-plugin 1.10.0.

I constructed a test job and set "Branches to build" to refs/remotes/origin/master. I believe that is an unambiguous declaration of the branch I usually would call "origin/master".

I have done the same, but Jenkins did two builds again: one with changes, and the one following that without any changes. I must be missing something.

That probably indicates that there is some other, additional bug (or condition) which is causing two builds. If you e-mail me your config.xml file for the job, or upload it to this bug report, I can compare your job configuration and my job configuration to see if I can identify any obvious reasons why the plugin would see your job as needing to schedule a second build.

The typical reason why a second build is immediately scheduled is that the plugin thinks the "Branches to build" matches two different branches, and both branches have changes. If polling detects multiple branches to build, it will immediately schedule them both.

You could also add the "Force polling using workspace" behaviour to see if that changes the behavior.

I've attached a file.

Git Build Data page shows identical information for both normal and duplicate builds.

I don't see many differences between your config.xml and the version I used to test the behaviour.

Some of the differences I see (which may be worth you investigating):

1. I enabled "prune stale branches" as an additional behaviour (since I don't like have stale branches lurking in my Jenkins workspaces). If you have stale branches in the Jenkins workspace, that might (conceivably) cause a duplicate build
2. I don't have a Pre-SCM build step. I don't know what's inside the script you use for your Pre-SCM build step, but that might cause enough of a repository change to result in an additional build

Bingo. Thanks for your comments, and sorry for wasting your time. My pre-build step script includes a version update if code changes are detected. It then commits and pushes the new version to the main Git repo. Therefore before the first build is finished Jenkins is already notified about a new commit (by itself) which it reacts to by starting a second build. At least that's how I understand it now.

I have set the job to ignore commits by jenkins user, and will see if the Branch Specifier set to "refs/remotes/origin/master" helps the double build issue.