Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27082

Sparse checkouts and submodules: Error on checkout, submodule missing

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • git-plugin
    • None
    • jenkins 1.580.3, git-plugin-2.3.5

      When using a repository with (relative) submodules and sparse checkouts, the GIT plugin fails on the initial (and subsequent) checkouts and the resulting working copy does not include the files of the submodule:

      Started by user Ronny Schütz
      Building on master in workspace /var/lib/jenkins/jobs/testrs_git/workspace
      Using strategy: Default
      Last Built Revision: Revision dfe102e4a93d71f242bd23d67363c9398f01ea54 (refs/remotes/origin/master)
      Cloning the remote Git repository
      Using no checkout clone with sparse checkout.
      Cloning repository ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git
       > /usr/bin/git init /var/lib/jenkins/jobs/testrs_git/workspace # timeout=360
      Fetching upstream changes from ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git
       > /usr/bin/git --version # timeout=360
      using GIT_SSH to set credentials ttbsbldmac101.ttg.global
       > /usr/bin/git -c core.askpass=true fetch --tags --progress ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git +refs/heads/*:refs/remotes/origin/*
      
       > /usr/bin/git config remote.origin.url ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git # timeout=360
       > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=360
       > /usr/bin/git config remote.origin.url ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git # timeout=360
      Cleaning workspace
       > /usr/bin/git rev-parse --verify HEAD # timeout=360
      No valid HEAD. Skipping the resetting
       > /usr/bin/git clean -fdx # timeout=360
      Fetching upstream changes from ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git
      using GIT_SSH to set credentials ttbsbldmac101.ttg.global
       > /usr/bin/git -c core.askpass=true fetch --tags --progress ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git +refs/heads/*:refs/remotes/origin/*
      getCandidateRevisions(false,origin/master,,,,remoteUrls=[ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git],buildsByBranchName={refs/remotes/origin/master=Build #15 of Revision dfe102e4a93d71f242bd23d67363c9398f01ea54 (refs/remotes/origin/master)},lastBuild=Build #15 of Revision dfe102e4a93d71f242bd23d67363c9398f01ea54 (refs/remotes/origin/master)]) considering branches to build
      Qualifying origin/master as a branch in repository origin -> refs/remotes/origin/master
      Qualifying origin/master as a branch in repository origin -> refs/remotes/origin/origin/master
       > /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=360
      rev-parse refs/remotes/origin/master -> AnyObjectId[dfe102e4a93d71f242bd23d67363c9398f01ea54]
      Found a new commit AnyObjectId[dfe102e4a93d71f242bd23d67363c9398f01ea54] to be built on refs/remotes/origin/master
       > /usr/bin/git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=360
      Failed to rev-parse: refs/remotes/origin/origin/master
      Checking out Revision dfe102e4a93d71f242bd23d67363c9398f01ea54 (refs/remotes/origin/master)
       > /usr/bin/git config core.sparsecheckout # timeout=360
       > /usr/bin/git config core.sparsecheckout true # timeout=360
       > /usr/bin/git read-tree -mu HEAD # timeout=360
      Command "/usr/bin/git read-tree -mu HEAD" returned status code 128:
      stdout: 
      stderr: fatal: Not a valid object name HEAD
      
       > /usr/bin/git checkout -f dfe102e4a93d71f242bd23d67363c9398f01ea54
       > /usr/bin/git rev-list dfe102e4a93d71f242bd23d67363c9398f01ea54 # timeout=360
      [workspace] $ /bin/sh -xe /var/lib/jenkins/tmp/hudson5727280207521334905.sh
      + find . '!' -path '*/.git/*'
      + sort
      .
      ./dir1
      ./dir1/test1
      ./dir2
      ./dir2/test2
      ./.git
      Started calculate disk usage of build
      Finished Calculation of disk usage of build in 0 seconds
      Started calculate disk usage of workspace
      Finished Calculation of disk usage of workspace in 0 seconds
      Finished: SUCCESS
      

      Repositories (attached):

      • testcommon
        • commonfile1
        • commonfile2
      • testmain
        • dir1/test1
        • dir2/test2
        • dir3/test3
        • testcommon (submodule; referencing testcommon via ../testcommon.git)

      Jenkins job settings:

      • Source code management: GIT
        • Repository: ssh://git@localhost:19999/mnt/fatdisk/git/testmain.git
        • Branches: */master
        • Clean after checkout
        • Advanced submodules behavior
          • Recursively update submodules: checked
        • Sparse checkout paths
          • dir1
          • dir2
      • Execute shell (to log working copy)
        • find . ! -path "*/.git/*" | sort

      If I remove the sparse checkout option, the submodule is checked out correctly.

          [JENKINS-27082] Sparse checkouts and submodules: Error on checkout, submodule missing

          ASHOK MOHANTY added a comment - - edited

          I am getting the same error - when trying to use 'sparsecheckout' to checkout only a directory not full git-project.

           > git config core.sparsecheckout # timeout=10
           > git config core.sparsecheckout true # timeout=10
           > git read-tree -mu HEAD # timeout=10
          Command "git read-tree -mu HEAD" returned status code 128:
          stdout: 
          stderr: fatal: Not a valid object name HEAD
          

          git plugin 4.0.0 , git-client  3.0.0 , gitlab-plugin 1.4.3  .

           

          OR do we have any alternate solution to checkout directory (not complete project) in Jenkins job.

          ASHOK MOHANTY added a comment - - edited I am getting the same error - when trying to use 'sparsecheckout' to checkout only a directory not full git-project. > git config core.sparsecheckout # timeout=10 > git config core.sparsecheckout true # timeout=10 > git read-tree -mu HEAD # timeout=10 Command "git read-tree -mu HEAD" returned status code 128: stdout: stderr: fatal: Not a valid object name HEAD git plugin 4.0.0 , git-client  3.0.0 , gitlab-plugin 1.4.3  .   OR do we have any alternate solution to checkout directory (not complete project) in Jenkins job.

          Mark Waite added a comment -

          akmjenkins I can't tell from your message if you're using a submodule with the sparse checkout or not.

          If you're not using a submodule with a sparse checkout, then this issue does not apply to you. You may want to check the version of command line git on the system that has the issue. If you're using Red Hat Linux 6, CentOS 6, Oracle Linux 6, Scientific Linux 6, Red Hat Linux 7, CentOS 7, Oracle Linux 7, or Scientific Linux 7, you may have a version of command line git that is too old to support sparse checkout. I believe reliable sparse checkout support requires at least command line git 1.9.

          If your command line git version is not too old and you are not using a submodule, then you should open a new issue. This issue is specifically related to sparse checkout interaction with submodules.

          Mark Waite added a comment - akmjenkins I can't tell from your message if you're using a submodule with the sparse checkout or not. If you're not using a submodule with a sparse checkout, then this issue does not apply to you. You may want to check the version of command line git on the system that has the issue. If you're using Red Hat Linux 6, CentOS 6, Oracle Linux 6, Scientific Linux 6, Red Hat Linux 7, CentOS 7, Oracle Linux 7, or Scientific Linux 7, you may have a version of command line git that is too old to support sparse checkout. I believe reliable sparse checkout support requires at least command line git 1.9. If your command line git version is not too old and you are not using a submodule, then you should open a new issue. This issue is specifically related to sparse checkout interaction with submodules.

          ASHOK MOHANTY added a comment -

          Thanks Mark. Yes I am trying to use submodule (sparse checkout). But ya found the git version is 1.8.x, will upgrade to 1.9.x and re-try.

          FYI - OS is "Oracle Linux Server 7.7".

          ASHOK MOHANTY added a comment - Thanks Mark. Yes I am trying to use submodule (sparse checkout). But ya found the git version is 1.8.x, will upgrade to 1.9.x and re-try. FYI - OS is "Oracle Linux Server 7.7".

          Mark Waite added a comment -

          akmjenkins if you are upgrading to a newer version of git, please consider upgrading to a much more recent version than 1.9.x. Git 2.26.0 recently released. Git 2.26.0 and the preceding 25+ releases are significant improvements over command line git 1.9.

          Mark Waite added a comment - akmjenkins if you are upgrading to a newer version of git, please consider upgrading to a much more recent version than 1.9.x. Git 2.26.0 recently released. Git 2.26.0 and the preceding 25+ releases are significant improvements over command line git 1.9.

          Is there any solution/workaround to this, because without this feature (checkout with sparse and submodules) one needs to implement the checkout in the `jnlp` container or any manually with shell commands and also setting up credentials into the credential manager `cache` etc which is totally cumbersome. Any help appreciated.

          Gabriel Nützi added a comment - Is there any solution/workaround to this, because without this feature (checkout with sparse and submodules) one needs to implement the checkout in the `jnlp` container or any manually with shell commands and also setting up credentials into the credential manager `cache` etc which is totally cumbersome. Any help appreciated.

          Mark Waite added a comment -

          gabyx there has been no work in the Jenkins git plugin or git client plugin to change this behavior or add additional support for sparse checkout with submodules. There is no automated test in either the git client plugin or the git plugin that attempts to perform a sparse checkout with submodules. The withCredentials Pipeline wrapper makes it easier to perform your own git commands in sh, bat, and powershell steps if you need something more sophisticated than the plugin provides.

          See the documentation at https://plugins.jenkins.io/git/#plugin-content-git-credentials-binding

          Mark Waite added a comment - gabyx there has been no work in the Jenkins git plugin or git client plugin to change this behavior or add additional support for sparse checkout with submodules. There is no automated test in either the git client plugin or the git plugin that attempts to perform a sparse checkout with submodules. The withCredentials Pipeline wrapper makes it easier to perform your own git commands in sh , bat , and powershell steps if you need something more sophisticated than the plugin provides. See the documentation at https://plugins.jenkins.io/git/#plugin-content-git-credentials-binding

          Ah tnhanks a lot, that reference is actually very helpful!

          Gabriel Nützi added a comment - Ah tnhanks a lot, that reference is actually very helpful!

          Gabriel Nützi added a comment - - edited

          markewaite : Somehow this features crashes when I use

           container('jnlp') {
              withCredentials([gitUsernamePassword(credentialsId: 'my-creds',
                                                   gitToolName: 'git-tool')]) {
                sh 'git submodule update --init --recursive'
              }
            }
          

          with

          [2022-08-11T12:56:38.341Z] + git submodule update --init --recursive
          [2022-08-11T12:56:38.341Z] Submodule '**********' (*************.git) registered for path 'bla bla bla'
          [2022-08-11T12:56:38.341Z] Submodule '**********' (*************..git) registered for path 'bla bla bla 2'
          [2022-08-11T12:56:38.341Z] Cloning into '*************....
          [2022-08-11T12:56:38.341Z] ****: line 3: syntax error near unexpected token `('
          [2022-08-11T12:56:38.341Z] ****: line 3: `        Password*) echo XXXXXXXXXXXXXXXXXXXX                ;;'
          [2022-08-11T12:56:38.341Z] error: unable to read askpass response from ****
          

           

          Where XXXXXXXXX contains the password!!! Is that another bug of the gitUsernamePassword feature??

          we have 2.37 Git version inside the jnlp container which is the jenkins agent.

          Gabriel Nützi added a comment - - edited markewaite : Somehow this features crashes when I use container( 'jnlp' ) { withCredentials([gitUsernamePassword(credentialsId: 'my-creds' , gitToolName: 'git-tool' )]) { sh 'git submodule update --init --recursive' } } with [2022-08-11T12:56:38.341Z] + git submodule update --init --recursive [2022-08-11T12:56:38.341Z] Submodule '**********' (*************.git) registered for path 'bla bla bla' [2022-08-11T12:56:38.341Z] Submodule '**********' (*************..git) registered for path 'bla bla bla 2' [2022-08-11T12:56:38.341Z] Cloning into '*************.... [2022-08-11T12:56:38.341Z] ****: line 3: syntax error near unexpected token `(' [2022-08-11T12:56:38.341Z] ****: line 3: ` Password*) echo XXXXXXXXXXXXXXXXXXXX ;;' [2022-08-11T12:56:38.341Z] error: unable to read askpass response from ****   Where XXXXXXXXX contains the password!!! Is that another bug of the gitUsernamePassword feature?? we have 2.37 Git version inside the jnlp container which is the jenkins agent.

          Mark Waite added a comment -

          Create a personal access token as the credential so that it does not contain special characters that may confuse the shell. Command line git reads the password from a program. That program is implemented in the git plugin as a shell script. If the password you're using contains characters that are special to the shell, they may not be correctly handled in that shell script.

          Mark Waite added a comment - Create a personal access token as the credential so that it does not contain special characters that may confuse the shell. Command line git reads the password from a program. That program is implemented in the git plugin as a shell script. If the password you're using contains characters that are special to the shell, they may not be correctly handled in that shell script.

          Gabriel Nützi added a comment - - edited

          The shell script if it is a core.askpass is clearly wrong. It should be something to:

          #!/bin/bash
          if [[ "$1" =~ Password ]]; then
              echo "$GIT_PASSWORD"
          else
              echo "$GIT_USERNAME"
          fi
          

          Or how does it do it?

          basically does it wrap the git executable to be `git -c core.askpass=script.sh ...` ???

          I tried somehting as

          sh 'git config --global --add "credential.helper" ""'
                writeFile(file: 'askpass.sh', text: """#!/bin/bash
                  if [[ "\$1" =~ Password ]]; then
                      echo "\$GIT_PASSWORD"
                  else
                      echo "\$GIT_USERNAME"
                  fi
                """)
                sh 'chmod +x askpass.sh'
                sh 'git -c core.askpass="askpass.sh" submodule update --init --recursive'
          

          which did also not work, also it should...

          Completely wrong here:
          https://github.com/jenkinsci/git-plugin/blob/6525c2714322563fb3ae339e6b1d43500311f31c/src/main/java/jenkins/plugins/git/GitUsernamePasswordBinding.java#L145
          I make a PR.

          Gabriel Nützi added a comment - - edited The shell script if it is a core.askpass is clearly wrong. It should be something to: #!/bin/bash if [[ "$1" =~ Password ]]; then echo "$GIT_PASSWORD" else echo "$GIT_USERNAME" fi Or how does it do it? basically does it wrap the git executable to be `git -c core.askpass=script.sh ...` ??? I tried somehting as sh 'git config --global --add "credential.helper" ""' writeFile(file: 'askpass.sh' , text: """#!/bin/bash if [[ "\$1" =~ Password ]]; then echo "\$GIT_PASSWORD" else echo "\$GIT_USERNAME" fi """) sh 'chmod +x askpass.sh' sh 'git -c core.askpass= "askpass.sh" submodule update --init --recursive' which did also not work, also it should... Completely wrong here: https://github.com/jenkinsci/git-plugin/blob/6525c2714322563fb3ae339e6b1d43500311f31c/src/main/java/jenkins/plugins/git/GitUsernamePasswordBinding.java#L145 I make a PR.

            Unassigned Unassigned
            rschuetz Ronny Schuetz
            Votes:
            6 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: