Hi All,

I am out of sudden facing issue with NEW jobs on existing jenkins server where the job is failing on master node itself while trying to fetch from a relatively big repo. This is a single branch pipeline job.

I am out of ideas after searching on internet, Jira project here and also posted question on stackoverflow after researching the posts. None of them seems to be pointing to the exact issue I am facing. So please accept my sincere apologies for logging this issue here.

Here is some additional info on the jobs itself:

 
1. This is only affecting new jobs using this repo which is a big repo. New job using any other repo under the same GIT org are working fine.

2. Existing jobs that have run in past are still ABLE to fetch and run but I believe on subsequent builds the job only downloads the delta that has changed.

I have tried increasing clone and checkout timeouts but it seems job is not evening trying that long. It barely tries for 1-2mins and immediately bails. I have added logs below, hoping I can at least get some pointers where the issue could be.

I tried additional checkout behavior to the job:

With these I was able to get the job to work but not reliably on other jobs using the same repo. But once the job works one time, then it is working consistently. So it's definitely something to do with first time until it succeeds.

Thanks for your help in advance.

Jenkin version: Jenkins 2.276

git version 2.18.4

Regards,

Vikas

{{ > git fetch --no-tags --progress --depth=1 – https://git.mycompany.com/MYOrg/Myrepo.git +refs/heads/:refs/remotes/origin/ # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://git.mycompany.com/MYOrg/Myrepo.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:998)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1239)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1299)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:125)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:155)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:69)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:309)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --progress --depth=1 – https://git.mycompany.com/MYOrg/Myrepo.git +refs/heads/:refs/remotes/origin/" returned status code 128:
stdout:
stderr: remote: Enumerating objects: 100285, done.
remote: Counting objects: 0% (1/100285)
remote: Counting objects: 1% (1003/100285)
.....................

Receiving objects: 100% (100285/100285), 414.58 MiB | 18.64 MiB/s
Receiving objects: 100% (100285/100285), 418.73 MiB | 17.64 MiB/s, done.
fatal: unable to open .git/objects/pack/tmp_pack_uMRmJb: Operation not permitted
fatal: index-pack failed

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2450)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2051)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:573)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:996)
... 8 more
ERROR: Error fetching remote repo 'origin'
ERROR: Maximum checkout retry attempts reached, aborting
Finished: FAILURE}}