-
Bug
-
Resolution: Not A Defect
-
Major
-
Jenkins 1.594
Plugins:
git 2.3.4 true false
git-client 1.15.0 true false
When setting up a git job to query a branch for new commits, if there are similarly named branches the git-plugin will query the wrong branch.
I have a job setup to query https://github.com/tork-a/euslisp-release.git The branch specifier is release/indigo/euslisp but when I look at the polling log I see the following:
Started on Feb 24, 2015 2:06:00 PM
Using strategy: Default
[poll] Last Built Revision: Revision 445d613f27d3cf28291919f29ccfbe4482082ae5 (refs/remotes/origin/release/indigo/euslisp)
> git --version # timeout=10
> git -c core.askpass=true ls-remote -h https://github.com/tork-a/euslisp-release.git euslisp # timeout=10
[poll] Latest remote head revision is: c80239beb5b1e524ba8e7ee18f6ea8719a994d44
Done. Took 0.8 sec
Changes found
Running the ls-remote command manually resolves as follows:
$ git -c core.askpass=true ls-remote -h https://github.com/tork-a/euslisp-release.git euslisp
c80239beb5b1e524ba8e7ee18f6ea8719a994d44 refs/heads/debian/hydro/euslisp
fb4412b90baaf2e49754dec1771ca96781c20796 refs/heads/debian/hydro/precise/euslisp
4db89fb3b3c8cd89bccbfcf2de4aba2284e73cf8 refs/heads/debian/hydro/quantal/euslisp
65e602d48960d099c12865ac8dd2d674dca7e0c1 refs/heads/debian/hydro/raring/euslisp
1c9b897308856fce5ba543bdb1c4e3d07c7fd50e refs/heads/debian/indigo/euslisp
521400334fc44c9f7399fabb807c0d71c2be59fa refs/heads/debian/indigo/saucy/euslisp
9f3eabefe587fbb075ac59334f60ab3f609c6a70 refs/heads/debian/indigo/trusty/euslisp
c9ef091b81d6e9e5da7583d4fc390a80dc373987 refs/heads/debian/jade/euslisp
506855f661af2dbc6295e1ae2d84aabf5994cedc refs/heads/debian/jade/trusty/euslisp
6b9e56f4bc148ee1e6e44c1bf3d74a98ad2f7bd8 refs/heads/debian/jade/utopic/euslisp
ddfdd6ee79de4dd9b330dd3328fbf1b7e7383b29 refs/heads/debian/jade/vivid/euslisp
467ec8a124a48a0589b7edf5ab4e89a89cb23f00 refs/heads/patches/debian/hydro/euslisp
d6407dba72dbfef3ae60b47b87d9be263622c830 refs/heads/patches/debian/hydro/precise/euslisp
5db895b78ab206a4f9942156e0c091d2053273e2 refs/heads/patches/debian/hydro/quantal/euslisp
3dba15359404e619a81dddae2d8362c51ba1c1fb refs/heads/patches/debian/hydro/raring/euslisp
d125c5d45eb4c792fe436191af420d668ece18ad refs/heads/patches/debian/indigo/euslisp
8d6645b41893ce19b5c0b4ff908a18a291ad60fd refs/heads/patches/debian/indigo/saucy/euslisp
eee1a1abd8208614aa3edf599bb8a66f5253ec07 refs/heads/patches/debian/indigo/trusty/euslisp
087f11bee271f103a3dc8c9894b46d46b8675fe7 refs/heads/patches/debian/jade/euslisp
6c38bf4e061d96a8e3fedc8e7bd58274fa242b19 refs/heads/patches/debian/jade/trusty/euslisp
a0426bffa153ab4256fb5444984a54d3ec514cc3 refs/heads/patches/debian/jade/utopic/euslisp
6c7fd0d28d98d5b38edff1ac3a1772cd65271b34 refs/heads/patches/debian/jade/vivid/euslisp
cb0c906cb76e1fbdceb5b7dae54f19d57b7e38b6 refs/heads/patches/release/hydro/euslisp
406ac292659b5fba42809e626c3776604028162c refs/heads/patches/release/indigo/euslisp
01a58c693cf44398399a746a0458ac997214d1f7 refs/heads/patches/release/jade/euslisp
ccfb5d2358374c1a2130a6dc105492595900d9c6 refs/heads/patches/rpm/hydro/euslisp
e0bd6c3cb9f3893d362145def530426367e2ef32 refs/heads/patches/rpm/hydro/heisenbug/euslisp
fa79128633b1391e20d7dd2898251cb1053f3316 refs/heads/patches/rpm/indigo/21/euslisp
8b3f34e19b1dc2dcaed704dee6e960af7720d8de refs/heads/patches/rpm/indigo/euslisp
ece33cf90a704dc5592332e239fa8c7d36a0c0e5 refs/heads/patches/rpm/indigo/heisenbug/euslisp
8651f63aa1c498b154448658ba4abe8479c1e825 refs/heads/patches/rpm/jade/21/euslisp
6bbb1d7b592f856060feb8facc7a3e2830a88683 refs/heads/patches/rpm/jade/22/euslisp
002f72bb26b9eedae4791a8d5097dab15b35af30 refs/heads/patches/rpm/jade/euslisp
7e1f06488fc30f898cd7710efcbead2528309512 refs/heads/release/hydro/euslisp
445d613f27d3cf28291919f29ccfbe4482082ae5 refs/heads/release/indigo/euslisp
cba7950078c9f5d6f8324ffe03804023d19c253a refs/heads/release/jade/euslisp
50fa3905158dee647468457b7dab0af0817766a1 refs/heads/rpm/hydro/euslisp
90a74237698b8bb62c55178a283e0f6cd3c257cb refs/heads/rpm/hydro/heisenbug/euslisp
25caabd70bcd3c12b8ba7de5913a1ceee754f4e5 refs/heads/rpm/indigo/21/euslisp
ddf2a287abdaa60214a87c24520e126e7a859044 refs/heads/rpm/indigo/euslisp
eb17f4c21c479d7405a195568a77a6ba56e067af refs/heads/rpm/indigo/heisenbug/euslisp
c612745405d1fdd72944853925e75a8fa6c2c5d4 refs/heads/rpm/jade/21/euslisp
d93805c9b08bfd8ce1d1d1a90cac6707a30feef2 refs/heads/rpm/jade/22/euslisp
82d7cb092ed3b1c0a0378cae4a1288d3c3b43059 refs/heads/rpm/jade/euslisp
As you can see it's matching the wrong branch debian/hydro/euslisp not release/indigo/euslisp when checking for the hashsum.
The first thing that I note is that the branch specifier is being stripped of everything with slashes, from release/indigo/euslisp to just euslisp which is causing overmatching.
If I update the command to use the full branch name it still over matches.
quote
$ git -c core.askpass=true ls-remote -h https://github.com/tork-a/euslisp-release.git release/indigo/euslisp
406ac292659b5fba42809e626c3776604028162c refs/heads/patches/release/indigo/euslisp
445d613f27d3cf28291919f29ccfbe4482082ae5 refs/heads/release/indigo/euslisp
quote
This is because ls-remote only does tail matching: https://github.com/git/git/blob/18d0fec24027ac226dc2c4df2b955eef2a16462a/builtin/ls-remote.c#L130
So to accurately distinguish the correct branch another layer must be added on top to verify the full branch name.
If you pass the full refspec as recommended by the help button, refs/remotes/origin/release/indigo/euslisp it still fails due what appears to be stripping all but the last element based on slashes.
Help button snippet: The safest way is to use the refs/heads/<branchName> syntax. This way the expected branch is unambiguous.
If it was passed through completely it would poll correctly:
quote
$ git ls-remote -h https://github.com/tork-a/euslisp-release.git refs/heads/release/indigo/euslisp
445d613f27d3cf28291919f29ccfbe4482082ae5 refs/heads/release/indigo/euslisp
quote
The polling must pass through the full branch specifier to avoid ambiguous branch resolutions.
This is an issue here: https://github.com/ros/rosdistro/issues/7346 where we have several repositories triggering every hour. Until this is resolved we have resolved to remove these repositories from our continuous integration.
Before this can be used successfully this issue must also be resolved: https://issues.jenkins-ci.org/browse/JENKINS-27114 since it will fail to checkout if you use the full refspec.
[JENKINS-27115] GIt polling queries wrong head
Tully Foote
added a comment - Is this the discussion from Alexander Link you are referring to? https://issues.jenkins-ci.org/browse/JENKINS-20767
When searching for that I also found related tickets: https://issues.jenkins-ci.org/browse/JENKINS-25580 and https://issues.jenkins-ci.org/browse/JENKINS-14026
I've been trying many regex invocations. Using the java regex http://www.tutorialspoint.com/java/java_regular_expressions.htm
But it does not appear that the simple regex :branch/name works. The ls-remote gets passed the entire regex including the : and fails to match anything. And then also passes the full string through to the clone which also fails.
I think I've found that I can make it work with the wildcard evaluation using */full/branch/name, based on the first link above.
I completely understand keeping backwards compatibility so as not to break existing users relying on the behavior. However, I would like to suggest adding an option which defaults to off which can disable the legacy trimming behavior to allow users the option to avoid it if they want.
It's also really frustrating that these things are not documented in the help fields. The trimming is not mentioned in the help fields, and recommending the fully qualified paths to "avoid ambiguity" without mentioning that they are actually going to be trimmed and ambiguous is very frustrating.
From finding the other related issues, it seems that I'm not the only one encountering this problem. It would be really great to get an option to disable the trimming. With or without that updating the documentation to mention the trimming and potential work around solutions would be greatly appreciated.
Tully Foote
added a comment - Is this the discussion from Alexander Link you are referring to? https://issues.jenkins-ci.org/browse/JENKINS-20767
When searching for that I also found related tickets: https://issues.jenkins-ci.org/browse/JENKINS-25580 and https://issues.jenkins-ci.org/browse/JENKINS-14026
I've been trying many regex invocations. Using the java regex http://www.tutorialspoint.com/java/java_regular_expressions.htm
But it does not appear that the simple regex : branch/name works. The ls-remote gets passed the entire regex including the : and fails to match anything. And then also passes the full string through to the clone which also fails.
I think I've found that I can make it work with the wildcard evaluation using */full/branch/name, based on the first link above.
I completely understand keeping backwards compatibility so as not to break existing users relying on the behavior. However, I would like to suggest adding an option which defaults to off which can disable the legacy trimming behavior to allow users the option to avoid it if they want.
It's also really frustrating that these things are not documented in the help fields. The trimming is not mentioned in the help fields, and recommending the fully qualified paths to "avoid ambiguity" without mentioning that they are actually going to be trimmed and ambiguous is very frustrating.
From finding the other related issues, it seems that I'm not the only one encountering this problem. It would be really great to get an option to disable the trimming. With or without that updating the documentation to mention the trimming and potential work around solutions would be greatly appreciated. I think you'd need to use a regular expression like
:.*origin/.*er
as a way to match the branch name containing origin followed by a slash followed by any text which ends in "er".
I like your suggestion on the improvement of the help text. Are you willing to submit a pull request containing the help text that you think would best clarify the situation? The current help text was a result of Alexander Link's significant effort to understand the behavior. I'm sure he'd be glad to see further improvements to that text.
Tully Foote
added a comment - I submitted a pull-request here: https://github.com/jenkinsci/git-plugin/pull/305 For help text mentioning that foo/bar => bar
I've found that the trimming does not happen if you use the full refs/heads/ I think my previous tests used refs/head/ by accident.
The PR takes care of the primary issue here.
Tully Foote
added a comment - I submitted a pull-request here: https://github.com/jenkinsci/git-plugin/pull/305 For help text mentioning that foo/bar => bar
I've found that the trimming does not happen if you use the full refs/heads/ I think my previous tests used refs/head/ by accident.
The PR takes care of the primary issue here. Code changed in jenkins
User: Tully Foote
Path:
src/main/resources/hudson/plugins/git/BranchSpec/help-name.html
http://jenkins-ci.org/commit/git-plugin/a9e34a703da8af95c298c100e640e63335d5a868
Log:
Add comment about trimming before slashes
This is a follow up to https://issues.jenkins-ci.org/browse/JENKINS-27115
Help text improvement will be included in 2.3.6 and later.
The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin. Problems detected in the pre-release should be e-mailed to MarkEWaite and ndeloof.
I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases.
You might try using a regular expression to match the exact branch name pattern you're trying to build.
The rather surprising trimming behavior on "origin/master" and "origin/xxx/master" is a legacy behavior that I'm unwilling to change for fear of breaking far too many of the 50,000 installations of the git plugin. You can read more details about it in Alexander Link's analysis of the alternatives.