[JENKINS-46921] Git plugin with LFS enabled keeps saying: Bad credentials

Type: Bug
Resolution: Fixed
Priority: Blocker
Component/s: git-plugin
Labels:
None
Environment:
Jenkins ver. 2.78

Similar Issues:
Powered by SuggestiMate

Show

I wanted to add Git LFS support, so I and enabled it as instructed in https://issues.jenkins-ci.org/browse/JENKINS-43158

Screenshot: https://dl.dropboxusercontent.com/s/j1ozouic5p5gyvq/2017-09-15%20at%2018.09.png

But now, all my builds of this project report: `batch response: Bad credentials`.

I set `GIT_TRACE=1` to get more output:

Started by user Kevin van Zonneveld
Building remotely on ci-buildslave-crm-trustymonolith (i-XXXXXXX) (ci-buildslave-crm-trustymonolith) in workspace /srv/content/current
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/transloadit/content # timeout=10
Fetching upstream changes from https://github.com/transloadit/content
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 
 > git fetch --tags --progress https://github.com/transloadit/content +refs/heads/*:refs/remotes/origin/* +refs/pull/*:refs/remotes/origin/pr/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision bd2771432d29f1658df45e477fc3036268eaf7bf (origin/master)
Commit message: "Build plz"
Enabling Git LFS pull
 > git config core.sparsecheckout # timeout=10
 > git checkout -f bd2771432d29f1658df45e477fc3036268eaf7bf
 > git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials 
 > git lfs pull origin
FATAL: Could not checkout bd2771432d29f1658df45e477fc3036268eaf7bf
hudson.plugins.git.GitException: Command "git lfs pull origin" returned status code 2:
stdout: 
Git LFS: (0 of 561 files) 0 B / 1.01 GB                                        
Git LFS: (0 of 561 files) 0 B / 1.01 GB                                        

stderr: trace: exec: 'git-lfs' 'pull' 'origin'
trace: run_command: 'git-lfs' 'pull' 'origin'
trace git-lfs: run_command: 'git' version
trace git-lfs: run_command: 'git' config -l
trace git-lfs: run_command: 'git' -c filter.lfs.smudge=cat -c filter.lfs.clean=cat -c filter.lfs.process= -c filter.lfs.required=false rev-parse HEAD --symbolic-full-name HEAD
trace git-lfs: tq: running as batched queue, batch size of 100
trace git-lfs: run_command: git cat-file --batch
trace git-lfs: fetch assets/demos/inputs/AWS_Cloud_Best_Practices.pdf [463d2d905edb0ca815fb2abddf4c99600407bb0b3832ec0dd584be4d18be53d9]

[.... MANY MORE ASSETS LIKE THESE ...]

trace git-lfs: tq: sending batch of size 100
trace git-lfs: fetch assets/demos/outputs/document-processing/convert-all-pages-of-a-document-into-images/page_0003.png.json [d74009ae6bf47830c36cbd21b1c7ecb04f043f021886dbaf363045fd52c357cf]

[.... MANY MORE ASSETS LIKE THESE ...]

trace git-lfs: api: batch 100 files
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Username for "https://github.com/transloadit/content"
trace git-lfs: fetch assets/demos/outputs/document-processing/convert-all-pages-of-a-document-into-images/page_0007.png.json [df4e18aa5d5e3c24d6ece545648907bb9971ea2016ff804099dc65c93e84eca5]

[.... MANY MORE ASSETS LIKE THESE ...]

trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Password for "https://transloadit-cibot@github.com/transloadit/content"
trace git-lfs: fetch assets/demos/outputs/file-compressing/create-one-zip-file-containing-all-uploaded-files/lolcat.zip.json [4bd5d6d15754b4395180549456329daac9080d357530f2d490c8fa9fd9edfe32]

[.... MANY MORE ASSETS LIKE THESE ...]

trace git-lfs: Filled credentials for https://github.com/transloadit/content
trace git-lfs: fetch assets/demos/outputs/file-exporting/export-uploaded-files-to-microsoft-azure/_assembly-success.json [331642c4cc576767c70944e46e21fdba408d72f4038b39405d41664e604c4f36]
trace git-lfs: HTTP: POST https://github.com/transloadit/content.git/info/lfs/objects/batch
trace git-lfs: fetch assets/demos/outputs/file-exporting/export-uploaded-files-to-microsoft-azure/dennis-snowflake.jpg.json [8ca27e73887e8cfc5d429701f5dfaa4bf49fd106a9921dee3fe4683ab12595a3]

[.... MANY MORE ASSETS LIKE THESE ...]

trace git-lfs: HTTP: 403
trace git-lfs: HTTP: {"message":"Bad credentials","documentation_url":"https://github.com/contact"}
trace git-lfs: api error: Bad credentials
trace git-lfs: tq: sending batch of size 100
trace git-lfs: api: batch 100 files
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Username for "https://github.com/transloadit/content"
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Password for "https://transloadit-cibot@github.com/transloadit/content"
trace git-lfs: Filled credentials for https://github.com/transloadit/content
trace git-lfs: HTTP: POST https://github.com/transloadit/content.git/info/lfs/objects/batch
trace git-lfs: HTTP: 403
trace git-lfs: HTTP: {"message":"Bad credentials","documentation_url":"https://github.com/contact"}
trace git-lfs: api error: Bad credentials
trace git-lfs: tq: sending batch of size 100
trace git-lfs: api: batch 100 files
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Username for "https://github.com/transloadit/content"
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Password for "https://transloadit-cibot@github.com/transloadit/content"
trace git-lfs: Filled credentials for https://github.com/transloadit/content
trace git-lfs: HTTP: POST https://github.com/transloadit/content.git/info/lfs/objects/batch
trace git-lfs: HTTP: 403
trace git-lfs: HTTP: {"message":"Bad credentials","documentation_url":"https://github.com/contact"}
trace git-lfs: api error: Bad credentials
trace git-lfs: tq: sending batch of size 100
trace git-lfs: api: batch 100 files
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Username for "https://github.com/transloadit/content"
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Password for "https://transloadit-cibot@github.com/transloadit/content"
trace git-lfs: Filled credentials for https://github.com/transloadit/content
trace git-lfs: HTTP: POST https://github.com/transloadit/content.git/info/lfs/objects/batch
trace git-lfs: HTTP: 403
trace git-lfs: HTTP: {"message":"Bad credentials","documentation_url":"https://github.com/contact"}
trace git-lfs: api error: Bad credentials
trace git-lfs: tq: sending batch of size 100
trace git-lfs: api: batch 100 files
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Username for "https://github.com/transloadit/content"
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Password for "https://transloadit-cibot@github.com/transloadit/content"
trace git-lfs: Filled credentials for https://github.com/transloadit/content
trace git-lfs: HTTP: POST https://github.com/transloadit/content.git/info/lfs/objects/batch
trace git-lfs: HTTP: 403
trace git-lfs: HTTP: {"message":"Bad credentials","documentation_url":"https://github.com/contact"}
trace git-lfs: api error: Bad credentials
trace git-lfs: tq: sending batch of size 61
trace git-lfs: api: batch 61 files
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Username for "https://github.com/transloadit/content"
trace git-lfs: creds: filling with GIT_ASKPASS: /srv/content/current@tmp/pass8561464566430907902.sh Password for "https://transloadit-cibot@github.com/transloadit/content"
trace git-lfs: Filled credentials for https://github.com/transloadit/content
trace git-lfs: HTTP: POST https://github.com/transloadit/content.git/info/lfs/objects/batch
trace git-lfs: HTTP: 403
trace git-lfs: HTTP: {"message":"Bad credentials","documentation_url":"https://github.com/contact"}
trace git-lfs: api error: Bad credentials
batch response: Bad credentials
batch response: Bad credentials
batch response: Bad credentials
batch response: Bad credentials
batch response: Bad credentials
batch response: Bad credentials
error: failed to fetch some objects from 'https://github.com/transloadit/content.git/info/lfs'

	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1924)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1643)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2225)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
	at hudson.remoting.UserRequest.perform(UserRequest.java:205)
	at hudson.remoting.UserRequest.perform(UserRequest.java:52)
	at hudson.remoting.Request$2.run(Request.java:356)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: hudson.plugins.git.GitException: Could not checkout bd2771432d29f1658df45e477fc3036268eaf7bf
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2237)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
	at hudson.remoting.UserRequest.perform(UserRequest.java:205)
	at hudson.remoting.UserRequest.perform(UserRequest.java:52)
	at hudson.remoting.Request$2.run(Request.java:356)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	at ......remote call to ci-buildslave-crm-trustymonolith (i-ef9a22de)(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1647)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:308)
	at hudson.remoting.Channel.call(Channel.java:896)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
	at com.sun.proxy.$Proxy78.execute(Unknown Source)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167)
	at hudson.scm.SCM.checkout(SCM.java:495)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1212)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:566)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:491)
	at hudson.model.Run.execute(Run.java:1724)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:419)
Finished: FAILURE

(added those `[.... MANY MORE ASSETS LIKE THESE ...]` to cut down on the length of the paste a bit)

When I disable Git-LFS, the checkout and my builds work as expected again.

Is this a misconfiguration on my part? Seems to me LFS should just inherit the working credentials?

PS Using the Attachment feature to include the screenshot inline caused a `JIRA could not attach the file as there was a missing token. Please try attaching the file again.` perhaps there is an issue with JIRA? Just a friendly tip.

is duplicated by

JENKINS-47018 Git LFS pull uses invalid credentials for Bitbucket branch source

Closed

Brian Davidson added a comment - 2017-09-18 19:31

I am seeing the same thing on jenkins slaves which have git lfs version 2.3.0

It works with 2.2.1

Brian Davidson added a comment - 2017-09-18 19:31 I am seeing the same thing on jenkins slaves which have git lfs version 2.3.0 It works with 2.2.1

Steve Baxter added a comment - 2017-09-19 12:08 - edited

I'm seeing exactly the same problem with git-lfs 2.3.0. It goes away if I downgrade to 2.2.1.

Steve Baxter added a comment - 2017-09-19 12:08 - edited I'm seeing exactly the same problem with git-lfs 2.3.0. It goes away if I downgrade to 2.2.1.

Kevin van Zonneveld added a comment - 2017-09-19 15:00

Did you downgrade git-lfs to 2.2.1 via apt-get or similar? I was under the impression lfs was bundled with the git plugin somehow. But when I tried to downgrade the git plugin, I hosed my entire Jenkins installation. Wouldn't boot saying it could not find the github authorization strategy class. Had to remove all plugins and reinstall via cli, then do the whole oauth/clientid dance again :o

Kevin van Zonneveld added a comment - 2017-09-19 15:00 Did you downgrade git-lfs to 2.2.1 via apt-get or similar? I was under the impression lfs was bundled with the git plugin somehow. But when I tried to downgrade the git plugin, I hosed my entire Jenkins installation. Wouldn't boot saying it could not find the github authorization strategy class. Had to remove all plugins and reinstall via cli, then do the whole oauth/clientid dance again :o

Steve Baxter added a comment - 2017-09-19 15:42

On the Mac slave I downloaded the 2.2.1 package and installed manually, on the Win slave I uninstalled 2.3.0 and installed 2.2.1 manually. I'm guessing you are on Linux?

Steve Baxter added a comment - 2017-09-19 15:42 On the Mac slave I downloaded the 2.2.1 package and installed manually, on the Win slave I uninstalled 2.3.0 and installed 2.2.1 manually. I'm guessing you are on Linux?

Kevin van Zonneveld added a comment - 2017-09-20 08:23

Yes on Linux, I'll try downgrading via apt-get, thank you!

I reported this problem upstream here: https://github.com/git-lfs/git-lfs/issues/2601 since it seems a possibility that this problem can be solved there.

Kevin van Zonneveld added a comment - 2017-09-20 08:23 Yes on Linux, I'll try downgrading via apt-get, thank you! I reported this problem upstream here: https://github.com/git-lfs/git-lfs/issues/2601 since it seems a possibility that this problem can be solved there.

Mark Waite added a comment - 2017-09-22 17:21 - edited

kvz LFS is a separately installed component for git. The Jenkins git plugin has been extended to use that separately installed component if requested.

It looks like the git lfs implementers have a pull request to fix the problem in a release after git lfs 2.3.0.

Until a fix is available, the simplest alternative is to downgrade to 2.2.1. Special thanks to bjdavidson, kvz, and stevebaxter for detecting the problem!

Mark Waite added a comment - 2017-09-22 17:21 - edited kvz LFS is a separately installed component for git. The Jenkins git plugin has been extended to use that separately installed component if requested. It looks like the git lfs implementers have a pull request to fix the problem in a release after git lfs 2.3.0. Until a fix is available, the simplest alternative is to downgrade to 2.2.1. Special thanks to bjdavidson , kvz , and stevebaxter for detecting the problem!

Emil Styrke added a comment - 2017-09-25 08:12

I have tested with a git-lfs binary built from the pull request, and it fixes the problem for me.

Emil Styrke added a comment - 2017-09-25 08:12 I have tested with a git-lfs binary built from the pull request, and it fixes the problem for me.

Mark Waite added a comment - 2017-10-13 15:29

Resolved in git lfs 2.3.1 and later. Bug was specific to git-lfs 2.3.0. Special thanks to those who realized that the problem was in git-lfs, not in the plugin!

Mark Waite added a comment - 2017-10-13 15:29 Resolved in git lfs 2.3.1 and later. Bug was specific to git-lfs 2.3.0. Special thanks to those who realized that the problem was in git-lfs, not in the plugin!

Assignee:: Unassigned

Reporter:: Kevin van Zonneveld

Votes:: 3 Vote for this issue

Watchers:: 6 Start watching this issue

Created:: 2017-09-18 08:48

Updated:: 2019-10-23 01:29

Resolved:: 2017-10-13 15:29

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Brian Davidson added a comment - 2017-09-18 19:31

Expand comment: Brian Davidson added a comment - 2017-09-18 19:31

Collapse comment: Steve Baxter added a comment - 2017-09-19 12:08, Edited by Steve Baxter - 2017-09-19 14:38

Expand comment: Steve Baxter added a comment - 2017-09-19 12:08, Edited by Steve Baxter - 2017-09-19 14:38

Collapse comment: Kevin van Zonneveld added a comment - 2017-09-19 15:00

Expand comment: Kevin van Zonneveld added a comment - 2017-09-19 15:00

Collapse comment: Steve Baxter added a comment - 2017-09-19 15:42

Expand comment: Steve Baxter added a comment - 2017-09-19 15:42

Collapse comment: Kevin van Zonneveld added a comment - 2017-09-20 08:23

Expand comment: Kevin van Zonneveld added a comment - 2017-09-20 08:23

Collapse comment: Mark Waite added a comment - 2017-09-22 17:21, Edited by Mark Waite - 2017-09-23 11:56

Expand comment: Mark Waite added a comment - 2017-09-22 17:21, Edited by Mark Waite - 2017-09-23 11:56

Collapse comment: Emil Styrke added a comment - 2017-09-25 08:12

Expand comment: Emil Styrke added a comment - 2017-09-25 08:12

Collapse comment: Mark Waite added a comment - 2017-10-13 15:29

Expand comment: Mark Waite added a comment - 2017-10-13 15:29

People

Dates