Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-74788

git LFS breaks for intial checkout from github

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • git-plugin
    • None
    • Ubuntu 24

      Context: I'm trying to create a new upgraded Jenkins server based on an older server.

      The original Jenkins server has a (scan org repos on github, use webhooks for auto builds) setup.

      I've copied what look like all the relevent secrets over, and it can do Jenkinsfile based builds from the repos fine.... in some repos.

      But for others, it dies due to LFS.
      Console errors shown below.
      TL;DR: its complaining about "bad credentials". But how can it be "bad" for lfs checkout, when it successfully checked out the rest of the repo pre-lfs?

      Cloning the remote Git repository
      Cloning with configured refspecs honoured and without tags
      Cloning repository https://github.com/xxxx.git
       > git init /tmp/workspace/ithub_xxxx_master # timeout=10
      Fetching upstream changes from|https://github.com/xxxx.git
      > git --version # timeout=10
      > git --version # 'git version 2.47.0'
      using GIT_ASKPASS to set credentials xxxxxxxx
      > git fetch --no-tags --force --progress – https://github.com/xxxx.git +refs/heads/master:refs/remotes/origin/master # timeout=10
      Avoid second fetch
      Checking out Revision 026f701df5d02981ab063fd889e1ef316258d843 (master)
      > git config remote.origin.url https://github.com/xxxx.git # timeout=10
      > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master # timeout=10
      > git config core.sparsecheckout # timeout=10
      > git checkout -f 026f701df5d02981ab063fd889e1ef316258d843 # timeout=10
      ERROR: Checkout failed
      hudson.plugins.git.GitException: Command "git checkout -f 026f701df5d02981ab063fd889e1ef316258d843" returned status code 128:
      stdout: 
      stderr: Downloading xxxx/diag_minimizer/doc/diag_cloud_logger_component.png (54 KB)
      Error downloading object: xxxx/diag_minimizer/doc/diag_cloud_logger_component.png (220c7db): Smudge error: Error downloading xxxx/diag_minimizer/doc/diag_cloud_logger_component.png (220c7db45e0515fea9f948153a35f3bdfa6df3226fa1174d1a1d44a30f473dce): batch response: Bad credentials

          [JENKINS-74788] git LFS breaks for intial checkout from github

          Mark Waite added a comment -

          The git checkout operation is usually a purely local operation, reading from the local file system and writing to the local file system. When git lfs is added, then the git checkout command is no longer a purely local operation. When git lfs is enabled, if a large file is needed by a checkout operation and is not already stored locally, then git lfs must request the large file from the remote server. If the remote server requires credentials, then that request to the remote server must provide those credentials.

          When the Jenkins job definition declares that it is enabling git LFS, the git plugin provides credentials to the git checkout command. Are you sure that your Jenkins job on the new server declares that it needs git LFS?

          Mark Waite added a comment - The git checkout operation is usually a purely local operation, reading from the local file system and writing to the local file system. When git lfs is added, then the git checkout command is no longer a purely local operation. When git lfs is enabled, if a large file is needed by a checkout operation and is not already stored locally, then git lfs must request the large file from the remote server. If the remote server requires credentials, then that request to the remote server must provide those credentials. When the Jenkins job definition declares that it is enabling git LFS, the git plugin provides credentials to the git checkout command. Are you sure that your Jenkins job on the new server declares that it needs git LFS?

          P added a comment -

          I'm confused as to the context of the question.
          Maybe I didnt provided enough context for this bug report?

          Oh I see: I confused myself jumping back and forward between the "working" old server, which shws some git lfs calls immediately after this point, and the new not-working one.. which hasnt gotten to the ''git lfs pull" stage yet.

          (but either way, its actually the github plugin that calls git lfs, not our Jenkinsfile)

          I guess this is more of a "sparse checkout isnt working properly" type question, not a git LFS question after all, maybe.

          Maybe I also thought git lfs, because the errors above include "smudge errror", and I thought that was to do with git lfs?

          =============

          Bottom line is: the branch-specific checkout for the autobuild is failing, as seen above, before it even gets to my specific Jenkinsfile calls.
          I cant figure out any reason why it is failing, since the initial init works.

          P added a comment - I'm confused as to the context of the question. Maybe I didnt provided enough context for this bug report? Oh I see: I confused myself jumping back and forward between the "working" old server, which shws some git lfs calls immediately after this point, and the new not-working one.. which hasnt gotten to the ''git lfs pull" stage yet. (but either way, its actually the github plugin that calls git lfs, not our Jenkinsfile) I guess this is more of a "sparse checkout isnt working properly" type question, not a git LFS question after all, maybe. Maybe I also thought git lfs, because the errors above include "smudge errror", and I thought that was to do with git lfs? ============= Bottom line is: the branch-specific checkout for the autobuild is failing, as seen above, before it even gets to my specific Jenkinsfile calls. I cant figure out any reason why it is failing, since the initial init works.

          Mark Waite added a comment -

          Bottom line is: the branch-specific checkout for the autobuild is failing, as seen above, before it even gets to my specific Jenkinsfile calls. I cant figure out any reason why it is failing, since the initial init works.

          My theory is that the working system has added the "Git LFS" additional behavior in the job configuration as part of its checkout while the failing system has not added the "Git LFS" additional behavior.

          Mark Waite added a comment - Bottom line is: the branch-specific checkout for the autobuild is failing, as seen above, before it even gets to my specific Jenkinsfile calls. I cant figure out any reason why it is failing, since the initial init works. My theory is that the working system has added the "Git LFS" additional behavior in the job configuration as part of its checkout while the failing system has not added the "Git LFS" additional behavior.

          P added a comment -

          That was it! Thank you so much!

          The configuration interface for this kind of stuff is.. a little wierd :-/

          I didnt notice that the original, working one, had a git "behaviour" configured, of

          "Git LFS pull after checkout"

          So  I went to the new one, dug into the Org-folder Configuration, found the Behaviors section, and added that one.
          Now the previously blocked build, proceeds through

          P added a comment - That was it! Thank you so much! The configuration interface for this kind of stuff is.. a little wierd :-/ I didnt notice that the original, working one, had a git "behaviour" configured, of " Git LFS pull after checkout" So  I went to the new one, dug into the Org-folder Configuration, found the Behaviors section, and added that one. Now the previously blocked build, proceeds through

            Unassigned Unassigned
            pbrownrobo P
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: