Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45228

Git merge requires authentication in LFS merges, plugin does not authenticate the git merge command

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • None
    • Jenkins version: 2.46.3
      Git plugin version: 3.3.0
    • git client plugin 3.1.0

      In case of changes in LFS data git merge requires authentication, but authentication data is not provided.

      Pipeline step:

      def mr_checkout_with_merge(url, account) {
        checkout changelog: true, poll: true, scm: [
          $class: 'GitSCM',
          branches: [[name: '${gitlabSourceBranch}']],
          doGenerateSubmoduleConfigurations: false,
          extensions: [
            [$class: 'RelativeTargetDirectory', relativeTargetDir: 'src'],
            [$class: 'UserIdentity', email: 'jenkins', name: 'jenkins'],
            [$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true, recursiveSubmodules: true, reference: '', trackingSubmodules: false],
            [$class: 'PreBuildMerge', options: [fastForwardMode: 'NO_FF', mergeRemote: 'origin', mergeStrategy: 'default', mergeTarget: '${gitlabTargetBranch}']],
            [$class: 'CloneOption', depth: 0, noTags: false, reference: '', shallow: false, timeout: 60],
            [$class: 'GitLFSPull']
          ],
          submoduleCfg: [],
          userRemoteConfigs: [[credentialsId: account, url: url]]
        ]
      }

       Log output:

      ERROR: Branch not suitable for integration as it does not merge cleanly: Command "git merge --no-ff 6af4f83cc9f2fc96d68df589e9822acf2fbf7fee" returned status code 128:
       stdout: 
       stderr: Downloading data/test_data.tar.gz (106.24 MB)
       Error downloading object: data/test_data.tar.gz (1b3d845): Smudge error: Error downloading data/test_data.tar.gz (1b3d845ad0f78327a16db63e03a7bc60dd2f7134bbaeeb117c3602d037893eef): batch request: Permission denied (publickey).: exit status 255
      Errors logged to /home/jenkins/workspace/Precommits/Ubuntu16/src/.git/lfs/objects/logs/20170626T055343.268201821.log
       Use `git lfs logs last` to view the log.
       error: external filter git-lfs smudge – %f failed 2
       error: external filter git-lfs smudge – %f failed
       fatal: data/test_data.tar.gz: smudge filter lfs failed

          [JENKINS-45228] Git merge requires authentication in LFS merges, plugin does not authenticate the git merge command

          Andrey Babushkin added a comment - - edited

          I have the same situation viviendelmon has.

          whitty it seems that the only workaround is to add direct 'sh' steps with direct 'git' invocation as asmorkalov suggested:

          sshagent(credentials: [account]) {
             sh "git clone ${repo_url} ."
             sh "git lfs install"
             sh "git checkout ${source_branch}"
             sh "git checkout ${target_branch}"
             sh "git merge --no-ff ${source_branch}"
             sh "git submodule update --init --recursive"
             sh "git lfs pull"
          }
          
          

          Andrey Babushkin added a comment - - edited I have the same situation viviendelmon has. whitty it seems that the only workaround is to add direct 'sh' steps with direct 'git' invocation as asmorkalov  suggested: sshagent(credentials: [account]) { sh "git clone ${repo_url} ." sh "git lfs install" sh "git checkout ${source_branch}" sh "git checkout ${target_branch}" sh "git merge --no-ff ${source_branch}" sh "git submodule update --init --recursive" sh "git lfs pull" }

          Greg Whiteley added a comment -

          Thanks oxygenxo - for me that can't work because the problem is on the "master" node as it tries to retrieve the Jenkinsfile - I'm using the Multi-branch Pipeline configuration, which (for a pull request) does:

          1. (On master) retrieve Jenkinsfile checkout repo (or update)
            1. checkout PR branch
            2. Merge to target branch
            3. Read Jenkinsfile for pipeline
          2. Execute Jenkinsfile
            1. Allocate a node node()
            2. Run scm functions checkout(scm)
              1. checkout repo (or update)
              2. checkout PR branch
              3. merge to target branch
            3. .. rest of Jenkinsfile ...

          Step 1.2 fails (before the Jenkinsfile is available). Changing the Jenkinsfile would allow me to override step 2.2 with no problems, but 1.2 is executed without user/jenkinsfile intervention - it comes directly from the Multi-branch pipeline configuration and is executed on the master (not on a node) without reading the Jenkinsfile (it's a chicken and egg issue)

          Greg Whiteley added a comment - Thanks oxygenxo - for me that can't work because the problem is on the "master" node as it tries to retrieve the Jenkinsfile - I'm using the Multi-branch Pipeline configuration, which (for a pull request) does: (On master) retrieve Jenkinsfile checkout repo (or update) checkout PR branch Merge to target branch Read Jenkinsfile for pipeline Execute Jenkinsfile Allocate a node node() Run scm functions checkout(scm) checkout repo (or update) checkout PR branch merge to target branch .. rest of Jenkinsfile ... Step 1.2 fails (before the Jenkinsfile is available). Changing the Jenkinsfile would allow me to override step 2.2 with no problems, but 1.2 is executed without user/jenkinsfile intervention - it comes directly from the Multi-branch pipeline configuration and is executed on the master (not on a node ) without reading the Jenkinsfile (it's a chicken and egg issue)

          Geoffroy Jabouley added a comment - - edited

          Hello

          also facing this issue in some cases where files where modified/added into source branch of PR.

          Any idea when/if LFS credentials support will be added to the merge portion of the git plugin? Maybe at least providing some implementation guidance for a future contribution

          BR

           

          Edit: https://github.com/jenkinsci/git-client-plugin/pull/433

          Geoffroy Jabouley added a comment - - edited Hello also facing this issue in some cases where files where modified/added into source branch of PR. Any idea when/if LFS credentials support will be added to the merge portion of the git plugin? Maybe at least providing some implementation guidance for a future contribution BR   Edit:  https://github.com/jenkinsci/git-client-plugin/pull/433

          Jonas Hanschke added a comment - - edited

          A way to work around this problem is to rebase the PR's source branch onto the target branch. The problem occurs only when there are new LFS files in the target branch, which will not be the case anymore after a rebase (they will already be present in the source branch, then).

          Still, this is a very annoying bug, which blocks our workflow (that usually doesn't involve rebasing). Any heads-up on when a fix can be expected? Thanks!

          Jonas Hanschke added a comment - - edited A way to work around this problem is to rebase the PR's source branch onto the target branch. The problem occurs only when there are new LFS files in the target branch, which will not be the case anymore after a rebase (they will already be present in the source branch, then). Still, this is a very annoying bug, which blocks our workflow (that usually doesn't involve rebasing). Any heads-up on when a fix can be expected? Thanks!

          jhanschke, you can check my ongoing PR (https://github.com/jenkinsci/git-client-plugin/pull/433, branch is based on v2.8.1), and recompile/install the plugin while change is being reviewed and integrated by markewaite

          Geoffroy Jabouley added a comment - jhanschke , you can check my ongoing PR ( https://github.com/jenkinsci/git-client-plugin/pull/433,  branch is based on v2.8.1), and recompile/install the plugin while change is being reviewed and integrated by markewaite

          René Scheibe added a comment -

          I would like to ask everybody interested to test the beta version markewaite just released - see https://github.com/jenkinsci/git-client-plugin/releases/tag/git-client-3.1.0-beta

          René Scheibe added a comment - I would like to ask everybody interested to test the beta version markewaite just released - see https://github.com/jenkinsci/git-client-plugin/releases/tag/git-client-3.1.0-beta

          Kalle Niemitalo added a comment - - edited

          Thanks, I see the git-client 3.1.0-beta binary is available from the experimental update center.

          It seems to be working OK so far, when a multibranch pipeline with Bitbucket Branch Source merges a pull request to a branch in which LFS files have been modified. I also verified that attempting the same merges with git-client 3.0.0 causes the build to hang because of repetitive authentication errors. (To verify this, I had to delete the workspace first, as Git LFS would otherwise have reused the LFS files that it cached when git-client 3.1.0-beta provided the credential.)

          I have not tested running checkout as a pipeline step from within Jenkinsfile, with either version of git-client.

          I should perhaps file a separate issue about the GIT_ASKPASS=echo environment variable.

          Kalle Niemitalo added a comment - - edited Thanks, I see the git-client 3.1.0-beta binary is available from the experimental update center . It seems to be working OK so far, when a multibranch pipeline with Bitbucket Branch Source merges a pull request to a branch in which LFS files have been modified. I also verified that attempting the same merges with git-client 3.0.0 causes the build to hang because of repetitive authentication errors. (To verify this, I had to delete the workspace first, as Git LFS would otherwise have reused the LFS files that it cached when git-client 3.1.0-beta provided the credential.) I have not tested running checkout as a pipeline step from within Jenkinsfile, with either version of git-client. I should perhaps file a separate issue about the GIT_ASKPASS=echo environment variable.

          Mark Waite added a comment -

          Fixed in git client plugin 3.1.0 release Jan 18, 2020.

          Mark Waite added a comment - Fixed in git client plugin 3.1.0 release Jan 18, 2020.

          Bharath R added a comment - - edited

          jhanschke geoffroyjabouley markewaite asmorkalov is this really fixed? I still face the problem.
          The only workaround is to rebase the PR every time, which is annoying. The problem also occurs on master node as kon & whitty point out, in pre-pipeline stage in a Bitbucket Multi branch pipeline.

          The problem started after we enabled LFS.
          I use GIT LFS Pull after checkout as an additional step in Jenkins configuration. Have tried 'Clean before/after checkout' and 'Wipeout repository and force clone' options too. None helps.
          So i see this option doing a spare checkout as below, though spare checkout is disabled in the job. Looks like Git LFS pull is doing a spare checkout for some reason.
          Enabling Git LFS pull
          > git config core.sparsecheckout # timeout=10
          And looks like as per the discussion https://issues.jenkins.io/browse/JENKINS-44598 , with spare checkout, merge won't work because complete history isn't avaialble?

          And I get the following error:
          stderr: error: Your local changes to the following files would be overwritten by merge: LFSfile.zip

          I have git client plugin version 3.6.0 installed, Git plugin 4.5.2.

          Is there something, I could be doing wrong? Any help would be appreciated.

          Bharath R added a comment - - edited jhanschke geoffroyjabouley markewaite asmorkalov is this really fixed? I still face the problem. The only workaround is to rebase the PR every time, which is annoying. The problem also occurs on master node as kon & whitty point out, in pre-pipeline stage in a Bitbucket Multi branch pipeline. The problem started after we enabled LFS. I use GIT LFS Pull after checkout as an additional step in Jenkins configuration. Have tried 'Clean before/after checkout' and 'Wipeout repository and force clone' options too. None helps. So i see this option doing a spare checkout as below, though spare checkout is disabled in the job. Looks like Git LFS pull is doing a spare checkout for some reason. Enabling Git LFS pull > git config core.sparsecheckout # timeout=10 And looks like as per the discussion https://issues.jenkins.io/browse/JENKINS-44598 , with spare checkout, merge won't work because complete history isn't avaialble? And I get the following error: stderr: error: Your local changes to the following files would be overwritten by merge: LFSfile.zip I have git client plugin version 3.6.0 installed, Git plugin 4.5.2. Is there something, I could be doing wrong? Any help would be appreciated.

          stderr: error: Your local changes to the following files would be overwritten by merge: LFSfile.zip

          That does not look like an authentication error. I suggest filing a separate issue.

          Kalle Niemitalo added a comment - stderr: error: Your local changes to the following files would be overwritten by merge: LFSfile.zip That does not look like an authentication error. I suggest filing a separate issue.

            Unassigned Unassigned
            asmorkalov Alexander Smorkalov
            Votes:
            15 Vote for this issue
            Watchers:
            25 Start watching this issue

              Created:
              Updated:
              Resolved: