-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Jenkins 2.13, Git Plugin 2.5.3
I'm seeing an issue when doing a recursive checkout of submodules in our repository where the Git plugin will attempt to check out a non-existent submodule that was recently added in a different branch but has not yet been merged. This is causing all subsequent builds to fail that have not yet had a merge to include this submodule. Here is an excerpt from our build logs:
I've blanked out sensitive path information with ***, just an FYI:
11:19:09 Checking out Revision *** (origin/master)
11:19:09 > git config core.sparsecheckout # timeout=10
11:19:09 > git checkout -f ***
11:19:10 > git rev-list *** # timeout=10
11:19:10 > git remote # timeout=10
11:19:10 > git submodule init # timeout=10
11:19:12 > git submodule sync # timeout=10
11:19:18 > git config --get remote.origin.url # timeout=10
11:19:18 > git config --get-regexp ^submodule # timeout=10
11:19:19 > git config --get submodule.BuildFiles/***.url # timeout=10
11:19:19 > git submodule update --init --recursive BuildFiles/***
11:19:23 > git config --get submodule.***.url # timeout=10
11:19:23 > git submodule update --init --recursive ***
11:19:27 > git config --get submodule.test/***.url # timeout=10
11:19:27 > git submodule update --init --recursive test/***
11:19:29 FATAL: Command "git submodule update --init --recursive test/***" returned status code 1:
11:19:29 stdout:
11:19:29 stderr: error: pathspec 'test/***' did not match any file(s) known to git.
11:19:29
11:19:29 hudson.plugins.git.GitException: Command "git submodule update --init --recursive test/**" returned status code 1:
11:19:29 stdout:
11:19:29 stderr: error: pathspec 'test/***' did not match any file(s) known to git.
I've traced the issue to the command "git config --get-regexp ^submodule". This is picking up the offending submodule and including it in the recursive update, despite it not being attached to the commit in question. This seems to be a problem with the Git program itself but when I do a recursive update from the command line, I don't see this issue. It seems to be attached to the process through which the Jenkins Git plugin identifies the submodules present. Would it be possible to add a check so the update does not include submodules that don't exist in the commit being targeted? I assume this will be resolved once the submodule gets merged into master but it's very inconvenient for the time being as all our follow-up builds are now failing.
I'm not sure if this is relevant but the current branch setup has it pointed to a detached head on a specific commit. This could be impacting how the submodules are being picked up and returned.
Thanks!
- is related to
-
-
- Closed
-
-
-
- Closed
-
[JENKINS-37419] Git plugin checking out non-existent submodule from different branch
I have to say that this breaking change has rather shaken my faith in the Jenkins QA process.
Up until now I've been very very impressed at how many Jenkins updates I've been able to install over years without anything breaking in any job.
This change however has broken our existing jobs, and has left us with a major headache on a Jenkins server that's been working very well previously.
samdeane it is unfortunate that you (and other heavy users of git submodules) didn't participate in the year-long beta of the submodule changes included in git plugin 3.0. They were announced on the Jenkins user mailing list. They were delivered as a series of 4+ beta releases, trying to assure that the 80 000+ installations of the git plugin would not be harmed by the changes.
Roll back to git plugin 2.x to see if that will resolve the problem, or you can compile the proposed fix in PR225 and see if it resolves your issue. I'm not willing to deliver that proposed fix until I've written more submodule automated tests. There are currently very few submodule automated tests in the git plugin and unfortunately, very few people contribute automated tests to the plugin. My submodule automated tests are being developed on a reference branch and on a prove it fails as expected branch.
> Sam Deane it is unfortunate that you (and other heavy users of git submodules) didn't participate in the year-long beta of the submodule changes included in git plugin 3.0. They were announced on the Jenkins user mailing list. They were delivered as a series of 4+ beta releases, trying to assure that the 80 000+ installations of the git plugin would not be harmed by the changes.
I hear what you're saying, and I understand where you're coming from. However, I also work on software that is used by a lot of people. One thing we've learnt, through sheer empirical evidence, is that betas do not find all the problems in our software. In fact they sometimes seem to find barely any problems. Worse than that, we've learnt that it is vitally important to explain to people that they should not use betas for real work. To do anything other than that is to invite a customer support nightmare. Jenkins is used for real work, for us. We're a small team. I do not have the time or the inclination to install a beta just to see if it works - and then to go through the pain of rolling it back if it doesn't. I do not want to install custom patches and then have to think about managing that when other updates become available. I do not subscribe to the Jenkins mailing list, and would not have time to read the posts if I did. I have enough trouble keeping up with the myriad other things that I'm supposed to be doing. Perhaps my expectations of Jenkins are too high. I get that it's free, and I'm very grateful (although I'd happily pay for it). I don't know who you are or how you come to be working on it - but again, I'm grateful to you and all who do. All I am saying is my particular truth: I've found Jenkins to be very very stable and solid until encountering this problem. Right now, I don't find it to be so, and I'm surprised at the way it's happened. YMMV.
samdeane it sounds like the best answer for you is to rollback to the previous version of the git plugin until the fix is ready. Then you can decide if you want to risk upgrading to the fixed version or stay with the previous version of the git plugin.
Will do. Am I likely to encounter any other problems by rolling back to an older version? Are there any dependencies between the git plugin and things like the GitHub one? That has been my concern about rolling back just one plugin. If that concern is misplaced, then great 
It depends on the plugins you use. Most plugins intentionally depend on older versions of their dependencies so that they don't require user upgrades to a dependent plugin.
The git plugin 3.0 release was also the release which forces users to at least Jenkins 1.625. Most plugins are not dependent on git plugin 3.0 as far as I know. The Jenkins wiki shows the dependencies for the most release release of each plugin, if that helps.
Code changed in jenkins
User: Ethan Trewhitt
Path:
src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java
http://jenkins-ci.org/commit/git-client-plugin/28db63d60dca2c20c714036c26b9208bd2a6bbbb
Log:
Reference the .gitmodules file to get a list of current submodules, rather than the .git/config file, which may have old submodules from other branches. JENKINS-37419
Hi
We are also facing same issue
stderr: error: pathspec 'ProjectA/testrepo' did not match any file(s) known to git.
Please can some one help here how to fix this.