-
Improvement
-
Resolution: Fixed
-
Minor
-
cgit >= 1.8.4
-
Powered by SuggestiMate
Recent versions of cgit support shallow clone support in submodule. Like this
git submodule update --init --depth 1
But it doesn't support this new feature in current version of git-client-plugin.
[JENKINS-21248] Add shallow update init for submodule
Tried this feature in beta git-4.0.0-beta9. But the plugin is refusing the allow the option:
> git submodule init # timeout=10 [WARNING] Git client older than 1.8.4 doesn't support shallow submodule updates. This flag is ignored.
However my Git version should support this, see output of sh 'git --version' in my pipeline script:
+ git --version git version 2.11.0
Bug in the version detection?
There could be a bug in the version detection logic, but you should probably first check that the version of git being used by your job is the same as the version reported in the pipeline script sh step.
I've not seen any other bug reports of broken version detection. There are specific automated tests in the plugin to confirm that git version numbers in various formats are detected correctly. Those tests don't use 2.11.0 specifically, but they use 2.10.0 and other interesting values near it.
If there is a bug in the detection logic, I'll need to update my acceptance test to exercise that bug.
There seem to be some interesting cases in that acceptance test when running on command line git version 2.11.0, but the cases that I'm seeing are not related to version checking logic. In my case, the git submodule update fails with a warning that there are no common commits. The same message does not appear on at least some other command line git versions. I'll need to extend the acceptance test to execute with each of the versions of command line git in my test environment. That will then tell me if there are problems related to command line git version. Note that the issue I'm investigating shows that in my environment I'm able to perform the expected command line git operations. Thus, at least in my environment, the version check is working, since my investigation could not have started without passing that version check.
I've further investigated the JENKINS-21248 acceptance test failures in my environment.
The initial submodule checkout succeeds on the following versions of command line git:
- 2.21
- 2.20
- 2.17
- 2.16
The initial submodule checkout fails on the following versions of command line git:
- 2.11
- 2.10
- 2.7
- 2.1
The stack trace in the checkout failure is:
hudson.plugins.git.GitException: Command "git submodule update --reference /var/lib/git/mwaite/bugs/jenkins-bugs.git --depth=1 modules/JENKINS-46504.url" returned status code 1: stdout: stderr: Cloning into 'modules/JENKINS-46504.url'... warning: no common commits fatal: reference is not a tree: 0736ba35a0d8c05236e3b71584bc4e149aa5f10a Unable to checkout '0736ba35a0d8c05236e3b71584bc4e149aa5f10a' in submodule path 'modules/JENKINS-46504.url' at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2298) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1910) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:81) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$7.lambda$execute$0(CliGitAPIImpl.java:1334) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at java.base/java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:184) at org.jenkinsci.plugins.gitclient.cgit.GitCommandsExecutor.submitRemainingCommand(GitCommandsExecutor.java:75) at org.jenkinsci.plugins.gitclient.cgit.GitCommandsExecutor.invokeAll(GitCommandsExecutor.java:64) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to debian8-mwaite at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:957) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at jdk.internal.reflect.GeneratedMethodAccessor323.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy119.execute(Unknown Source) at hudson.plugins.git.extensions.impl.SubmoduleOption.onCheckoutCompleted(SubmoduleOption.java:148) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1231) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused: hudson.plugins.git.GitException at org.jenkinsci.plugins.gitclient.cgit.GitCommandsExecutor.checkResult(GitCommandsExecutor.java:87) at org.jenkinsci.plugins.gitclient.cgit.GitCommandsExecutor.invokeAll(GitCommandsExecutor.java:68) at org.jenkinsci.plugins.gitclient.cgit.GitCommandsExecutor.invokeAll(GitCommandsExecutor.java:47) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$7.execute(CliGitAPIImpl.java:1337) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused: java.io.IOException: Could not perform submodule update at hudson.plugins.git.extensions.impl.SubmoduleOption.onCheckoutCompleted(SubmoduleOption.java:153) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1231) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)
Prior Failures Everywhere
Failures were happening in that acceptance test prior to my creating a branch on the submodule repository which pointed to the SHA-1 referenced by the parent repository. Once that branch was created, then the newer command line git versions would consistently pass. That earlier failure message was:
hudson.plugins.git.GitException: Command "git submodule update --reference /var/lib/git/mwaite/bugs/jenkins-bugs.git --depth=1 modules/JENKINS-46504.url" returned status code 1: stdout: stderr: Cloning into '/mnt/xvdba/home/jagent/mark-pc2.markwaite.net-agent/workspace/ine-no-localbranch_JENKINS-21248/modules/JENKINS-46504.url'... warning: no common commits error: Server does not allow request for unadvertised object 0736ba35a0d8c05236e3b71584bc4e149aa5f10a Fetched in submodule path 'modules/JENKINS-46504.url', but it did not contain 0736ba35a0d8c05236e3b71584bc4e149aa5f10a. Direct fetching of that commit failed. at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2298) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1910) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:81) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$7.lambda$execute$0(CliGitAPIImpl.java:1334) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253) at java.base/java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:184) at org.jenkinsci.plugins.gitclient.cgit.GitCommandsExecutor.submitRemainingCommand(GitCommandsExecutor.java:75) at org.jenkinsci.plugins.gitclient.cgit.GitCommandsExecutor.invokeAll(GitCommandsExecutor.java:64)
Thanks for the details markewaite. I'm happy to help with testing this useful feature.
How can I check what version of Git is used by the checkout() command? I assume it would be the same I get when I run git --version just above it. Is there a different Git binary used? Is the checkout actually happening on the Master and the sh() command on the agent? Is JGit involved? I'm confused here.
As per your acceptance test, does this point at an issue with the version parsing or not? I've checked the code around where the warning Git client older than 1.8.4 doesn't support shallow submodule updates. This flag is ignored. is raised, the shallow clone arguments are never actually added to the Git command because of the failed version check, so I'm not getting errors from Git on this.
My acceptance test is strong evidence that there is not an issue with version parsing .
Check the jenkins system configuration of the git tool and the agent configuration of the git tool
It is available in git plugiin 4.0.0-beta12 and git client plugin 3.0.0 beta12 now. Please test it in your use case. I intend to release git plugin 4.0.0 and git client plugin 3.0.0 in about 1 week and would sincerely appreciate additional beta testers confirming that the new plugin features are working as expected.
Release Notes
Download
Released with git client plugin 3.0.0 and git plugin 4.0.0 on Nov 2, 2019.
I've added an acceptance test to my regression test suite that runs inside the docker-lfs configuration. It confirms shallow clone support for submodules works as expected.