Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
Jenkins ver. 2.19.3
Git Plugin 3.0.2 to 3.2.0
Description
In a pipeline job from a git repository where the "Branch Specifier" is given as a parameter, I started getting the following failure after updating the git plugin.
hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/$\{GITREF}:refs/remotes/origin/$\{GITREF} --prune" returned status code 128:
stdout:
stderr: fatal: Couldn't find remote ref refs/heads/$\{GITREF}
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1799)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1525)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:65)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:316)
at jenkins.plugins.git.GitSCMFileSystem$BuilderImpl.build(GitSCMFileSystem.java:304)
at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:196)
at jenkins.scm.api.SCMFileSystem.of(SCMFileSystem.java:172)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:99)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:59)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:232)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Finished: FAILURE
I have tested all released version of the git plugin from 3.0.1 to 3.2.0 and the failure seems to have appeared in 3.0.2.
I don't know if it is relevant but with the Git plugin version 3.0.1, I see this at the beginning of the log:
Lightweight checkout support not available, falling back to full checkout.
Attachments
Activity
Sam Duke
added a comment - Ah apologies, I must have misread your activity feed.
We are looking into the reference repository option - is there any documentation on how this checkout method works? (does it use git alternates, for example?)
We are already doing a depth=1 clone of a specific ref (a SHA passed from gerrit) so I don't see how we can narrow it any more. Even with that ur checkout times of the pipeline groovy script is 10 minutes currently (thanks to all the other checking out it needs to do and all our actual building happens on other machines so it really should just be a few seconds to grab the groovy file and execute it.
It seems the Lightweight checkout is seriously stunted in its usability because of the limitation of not being able to expand variables. In fact in anything but the simplest setup ("please build this fixed branch / tag") I can't see how you would use it.
Is there any process to escalate this ticket?
Sam Duke
added a comment - Ah apologies, I must have misread your activity feed.
We are looking into the reference repository option - is there any documentation on how this checkout method works? (does it use git alternates, for example?)
We are already doing a depth=1 clone of a specific ref (a SHA passed from gerrit) so I don't see how we can narrow it any more. Even with that ur checkout times of the pipeline groovy script is 10 minutes currently (thanks to all the other checking out it needs to do and all our actual building happens on other machines so it really should just be a few seconds to grab the groovy file and execute it.
It seems the Lightweight checkout is seriously stunted in its usability because of the limitation of not being able to expand variables. In fact in anything but the simplest setup ("please build this fixed branch / tag") I can't see how you would use it.
Is there any process to escalate this ticket? Sure, you could escalate through one of the organizations that provides commercial support like CloudBees. If you're a CloudBees customer, you can submit a support ticket.
You could escalate it by implementing the desired capability including automated tests and submitting a pull request.
You might also be able to get dramatically faster checkout by choosing a different git provider. If your source code is hosted on GitHub or Bitbucket, you can use the matching branch source provider plugin and it will use a REST API call to read the contents of the Jenkinsfile. I believe a similar technique is also used for Gitea.
The prioritized changes that I've seen as most effective with a large repository with many branches that are mostly independent are:
- Use a reference repository to reduce network data transfer in git fetch
- Narrow the refspec when cloning the repository to only pull changes from the exact branch being built (reduce data transfer in git fetch)
- Shallow clone (depth=1) to only pull most recent changes (reduce data transfer in git fetch)
Arun Subbaramu
added a comment - Is the fix for this issue planned anytime soon or if the issue is not fixed how do we fix the the issue by applying this patch
[^JENKINS-14276.patch]
Any pointers greatly appreciated.
Arun Subbaramu
added a comment - Is the fix for this issue planned anytime soon or if the issue is not fixed how do we fix the the issue by applying this patch
[^JENKINS-14276.patch]
Any pointers greatly appreciated. No fix is planned for this issue by me. I don't recognize your reference to JENKINS-14276.patch. Instructions for building the Git plugin are included in its CONTRIBUTING file. Instructions for building the git client plugin are included in its CONTRIBUTING file.
After looking at the history of JENKINS-14276 and its related bugs, I doubt any patches from that bug or its related bugs will help with this issue. This issue is requesting parameter expansion for pipeline jobs which use lightweight checkout to obtain the Jenkinsfile. I doubt those changes from 2015 will help with lightweight checkout in pipelines.
samskiter I do work on this project during my nights and weekends.
I manage the list of issues I'm actively investigating by unassigning myself from an issue when I'm not actively investigating the issue. I'm not actively investigating this issue.
You may be able to improve checkout performance using a reference repository or by narrowing the refspec of the repository clone or by using a shallow clone. Refer to "Git in the Large" for a presentation that I created to improve git performance with large git repositories in Jenkins.