• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • git-plugin
    • None
    • git plugin 4.4.0

      When performing a git clone operation,

      git-client and git-plugin both triggers a git fetch.

       

      Below are the two lines of code running in the same git clone command both doing fetches.

       

      The second link is part of the GITSCM step, it triggers the git-client-plugin's CloneCommand (first link) to perform the clone operation. And afterwards, the GITSCM step performs a fetch once the CloneCommand Completes.

      However, within the CloneCommand (first link) itself, a fetch command was already done.

       

      https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609

       

      https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153

       

      I am aware there are needs to support for git clone to fetch all refs first -JENKINS-31393-. The suggestion provided in that ticket was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

       

      Thanks,

      Sam

          [JENKINS-49757] Git plugin calls fetch twice per checkout

          sam leung created issue -
          sam leung made changes -
          Description Original: When performing a git clone operation,

          git-client and git-plugin both triggers a git fetch.

           

          Below are the two lines of code running in the same git clone command both doing fetches.

          [https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609]

           

          [https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153]

           

          I am aware there are needs to support for git clone to fetch all refs first [JENKINS-31393|https://issues.jenkins-ci.org/browse/JENKINS-31393]. The suggestion provided was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

           

          Thanks,

          Sam
          New: When performing a git clone operation,

          git-client and git-plugin both triggers a git fetch.

           

          Below are the two lines of code running in the same git clone command both doing fetches.

          [https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609]

           

          [https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153]

           

          I am aware there are needs to support for git clone to fetch all refs first JENKINS-31393. The suggestion provided in that ticket was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

           

          Thanks,

          Sam
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]

          Mark Waite added a comment - - edited

          I appreciate the analysis very much.

          Unfortunately, the analysis doesn't seem (to me) like evidence of a bug. The git plugin (your second link) uses the git client plugin (your first link) to perform the fetch. I don't see from your description that two fetch operations are being performed for each repository being cloned.

          Please provide log output which shows two fetches being performed.

          Since you've provided those call sites, I assume you could also include stack traces to show that a clone is being performed twice.

          Mark Waite added a comment - - edited I appreciate the analysis very much. Unfortunately, the analysis doesn't seem (to me) like evidence of a bug. The git plugin (your second link) uses the git client plugin (your first link) to perform the fetch. I don't see from your description that two fetch operations are being performed for each repository being cloned. Please provide log output which shows two fetches being performed. Since you've provided those call sites, I assume you could also include stack traces to show that a clone is being performed twice.

          sam leung added a comment -

          Hi,

           

          The clone itself isn't being performed twice, however it's triggering fetch twice. 

          Both fetches are being trigger by the same Clone operation. Once by the git-plugin after doing a clone, and once more by the git-client-plugin after calling the git-plugin to do the clone.

           

          Below is the log output

           

          00:06:32 Cloning repository ssh://[git project]
          00:06:32  > /usr/bin/git init /var/lib/jenkins/workspace/workspace # timeout=10
          00:06:32 Fetching upstream changes from ssh://[git project]
          00:06:32  > /usr/bin/git --version # timeout=10
          00:06:32 using GIT_SSH to set credentials Gerrit SSH Key
          00:06:32  > /usr/bin/git fetch --tags --progress ssh://[git project] +refs/heads/*:refs/remotes/origin/* # timeout=20
          00:07:49  > /usr/bin/git config remote.origin.url ssh://[git project] # timeout=10
          00:07:49  > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
          00:07:49  > /usr/bin/git config remote.origin.url ssh://[git project] # timeout=10
          00:07:49 Fetching upstream changes from ssh://[git project]
          00:07:49 using GIT_SSH to set credentials fpajenkins Gerrit SSH Key
          00:07:49  > /usr/bin/git fetch --tags --progress ssh://[git project] refs/changes/14/3174614/2 # timeout=20
          00:08:18  > /usr/bin/git rev-parse cc8266a2cecb79d475d5c3b1dbaa5b2ea3171400^{commit} # timeout=10
          00:08:18 Checking out Revision cc8266a2cecb79d475d5c3b1dbaa5b2ea3171400 (detached)
          00:08:18  > /usr/bin/git config core.sparsecheckout # timeout=10
          00:08:18  > /usr/bin/git config core.sparsecheckout true # timeout=10
          00:08:18  > /usr/bin/git read-tree -mu HEAD # timeout=10
          

           

          sam leung added a comment - Hi,   The clone itself isn't being performed twice, however it's triggering fetch twice.  Both fetches are being trigger by the same Clone operation. Once by the git-plugin after doing a clone, and once more by the git-client-plugin after calling the git-plugin to do the clone.   Below is the log output   00:06:32 Cloning repository ssh: //[git project] 00:06:32 > /usr/bin/git init / var /lib/jenkins/workspace/workspace # timeout=10 00:06:32 Fetching upstream changes from ssh: //[git project] 00:06:32 > /usr/bin/git --version # timeout=10 00:06:32 using GIT_SSH to set credentials Gerrit SSH Key 00:06:32 > /usr/bin/git fetch --tags --progress ssh: //[git project] +refs/heads/*:refs/remotes/origin/* # timeout=20 00:07:49 > /usr/bin/git config remote.origin.url ssh: //[git project] # timeout=10 00:07:49 > /usr/bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 00:07:49 > /usr/bin/git config remote.origin.url ssh: //[git project] # timeout=10 00:07:49 Fetching upstream changes from ssh: //[git project] 00:07:49 using GIT_SSH to set credentials fpajenkins Gerrit SSH Key 00:07:49 > /usr/bin/git fetch --tags --progress ssh: //[git project] refs/changes/14/3174614/2 # timeout=20 00:08:18 > /usr/bin/git rev-parse cc8266a2cecb79d475d5c3b1dbaa5b2ea3171400^{commit} # timeout=10 00:08:18 Checking out Revision cc8266a2cecb79d475d5c3b1dbaa5b2ea3171400 (detached) 00:08:18 > /usr/bin/git config core.sparsecheckout # timeout=10 00:08:18 > /usr/bin/git config core.sparsecheckout true # timeout=10 00:08:18 > /usr/bin/git read-tree -mu HEAD # timeout=10  

          sam leung added a comment - - edited

          If you look at the first link. It's calling the fetchcommand.execute directly and skipping all the fetchCommand decorators.

          sam leung added a comment - - edited If you look at the first link. It's calling the fetchcommand.execute directly and skipping all the fetchCommand decorators.
          sam leung made changes -
          Description Original: When performing a git clone operation,

          git-client and git-plugin both triggers a git fetch.

           

          Below are the two lines of code running in the same git clone command both doing fetches.

          [https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609]

           

          [https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153]

           

          I am aware there are needs to support for git clone to fetch all refs first JENKINS-31393. The suggestion provided in that ticket was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

           

          Thanks,

          Sam
          New: When performing a git clone operation,

          git-client and git-plugin both triggers a git fetch.

           

          Below are the two lines of code running in the same git clone command both doing fetches.

           

          The first link is part of the Git-client clone step, it triggers the git-plugin CloneCommand (second link) to perform the clone operations. And afterwards, the Git-client plugin performs a fetch once cloneCommand Completes.

          However, within the cloneCommand (second link) itself, a fetch command was already done.

           

          [https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609]

           

          [https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153]

           

          I am aware there are needs to support for git clone to fetch all refs first -JENKINS-31393-. The suggestion provided in that ticket was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

           

          Thanks,

          Sam

          sam leung added a comment - - edited

          markewaite

          The second link is part of the GITSCM step, it triggers the git-client-plugin's CloneCommand (first link) to perform the clone operation. And afterwards, the GITSCM step performs a fetch once the CloneCommand Completes.

          However, within the CloneCommand (first link) itself, a fetch command was already done.

           

          see GitSCM#1140 on the call from GITSCM into CloneCommand,

          see CliGitAPIImpl#609 inside CloneCommand for the first fetch.

          see GitSCM#1153 for the second fetch

           

          Thanks a lot for looking into this

          sam leung added a comment - - edited markewaite The second link is part of the GITSCM step, it triggers the git-client-plugin's CloneCommand (first link) to perform the clone operation. And afterwards, the GITSCM step performs a fetch once the CloneCommand Completes. However, within the CloneCommand (first link) itself, a fetch command was already done.   see GitSCM#1140 on the call from GITSCM into CloneCommand, see CliGitAPIImpl#609 inside CloneCommand for the first fetch. see GitSCM#1153 for the second fetch   Thanks a lot for looking into this
          sam leung made changes -
          Description Original: When performing a git clone operation,

          git-client and git-plugin both triggers a git fetch.

           

          Below are the two lines of code running in the same git clone command both doing fetches.

           

          The first link is part of the Git-client clone step, it triggers the git-plugin CloneCommand (second link) to perform the clone operations. And afterwards, the Git-client plugin performs a fetch once cloneCommand Completes.

          However, within the cloneCommand (second link) itself, a fetch command was already done.

           

          [https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609]

           

          [https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153]

           

          I am aware there are needs to support for git clone to fetch all refs first -JENKINS-31393-. The suggestion provided in that ticket was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

           

          Thanks,

          Sam
          New: When performing a git clone operation,

          git-client and git-plugin both triggers a git fetch.

           

          Below are the two lines of code running in the same git clone command both doing fetches.

           

          The second link is part of the GITSCM step, it triggers the git-client-plugin's CloneCommand (first link) to perform the clone operation. And afterwards, the GITSCM step performs a fetch once the CloneCommand Completes.

          However, within the CloneCommand (first link) itself, a fetch command was already done.

           

          [https://github.com/jenkinsci/git-client-plugin/blob/4241e818dbe27fb78093094f29ebcd2203166610/src/main/java/org/jenkinsci/plugins/gitclient/CliGitAPIImpl.java#L609]

           

          [https://github.com/jenkinsci/git-plugin/blob/cb58c74efafafd08bc03bba2787997c6a0272df0/src/main/java/hudson/plugins/git/GitSCM.java#L1153]

           

          I am aware there are needs to support for git clone to fetch all refs first --JENKINS-31393--. The suggestion provided in that ticket was to use "honor git refspec", so shouldn't "honor git refspec" skip the first fetch here?

           

          Thanks,

          Sam

          Mark Waite added a comment - - edited

          I think I see the same behavior in the log file of my jobs. I have a Freestyle job configured which generates the following log output:

          > git init /home/beemarkwaite/mark-pc2.markwaite.net-agent/workspace/Bugs-Individual/JENKINS-48064-Fisheye-url-validator-ignored # timeout=10
          Fetching upstream changes from https://github.com/aeshell/aesh.git
           > git --version # timeout=10
           > git fetch --tags --progress https://github.com/aeshell/aesh.git +refs/heads/master:refs/remotes/origin/master +refs/heads/0.56.1:refs/remotes/origin/0.56.1
           > git config remote.origin.url https://github.com/aeshell/aesh.git # timeout=10
           > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master # timeout=10
           > git config --add remote.origin.fetch +refs/heads/0.56.1:refs/remotes/origin/0.56.1 # timeout=10
           > git config remote.origin.url https://github.com/aeshell/aesh.git # timeout=10
          Fetching upstream changes from https://github.com/aeshell/aesh.git
           > git fetch --tags --progress https://github.com/aeshell/aesh.git +refs/heads/master:refs/remotes/origin/master +refs/heads/0.56.1:refs/remotes/origin/0.56.1
          

          That shows two calls are made to git fetch. Since git fetch is an incremental operation, I'm not planning to make any change in this area of the code. The second fetch operation is not free, but it is usually very lightweight, since it will only request changes made since the preceding fetch.

          For the sake of precision, your earlier comments are not describing the implementation of the code. You said:

          The first link is part of the Git-client clone step, it triggers the git-plugin CloneCommand (second link) to perform the clone operations. And afterwards, the Git-client plugin performs a fetch.

          As far as I can tell, that's not the way the code works. The git client plugin does not call the git plugin. It does not have any dependency on the git plugin and does not know about the git plugin implementation. The git client plugin provides an API that is called by the git plugin.

          The stack trace you provided shows two calls to git fetch, but the two calls are using different credentials. I'm assuming that the difference in credentials is because you edited the log output to remove identifying information.

          Mark Waite added a comment - - edited I think I see the same behavior in the log file of my jobs. I have a Freestyle job configured which generates the following log output: > git init /home/beemarkwaite/mark-pc2.markwaite.net-agent/workspace/Bugs-Individual/JENKINS-48064-Fisheye-url-validator-ignored # timeout=10 Fetching upstream changes from https://github.com/aeshell/aesh.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/aeshell/aesh.git +refs/heads/master:refs/remotes/origin/master +refs/heads/0.56.1:refs/remotes/origin/0.56.1 > git config remote.origin.url https://github.com/aeshell/aesh.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master # timeout=10 > git config --add remote.origin.fetch +refs/heads/0.56.1:refs/remotes/origin/0.56.1 # timeout=10 > git config remote.origin.url https://github.com/aeshell/aesh.git # timeout=10 Fetching upstream changes from https://github.com/aeshell/aesh.git > git fetch --tags --progress https://github.com/aeshell/aesh.git +refs/heads/master:refs/remotes/origin/master +refs/heads/0.56.1:refs/remotes/origin/0.56.1 That shows two calls are made to git fetch . Since git fetch is an incremental operation, I'm not planning to make any change in this area of the code. The second fetch operation is not free, but it is usually very lightweight, since it will only request changes made since the preceding fetch. For the sake of precision, your earlier comments are not describing the implementation of the code. You said: The first link is part of the Git-client clone step, it triggers the git-plugin CloneCommand (second link) to perform the clone operations. And afterwards, the Git-client plugin performs a fetch. As far as I can tell, that's not the way the code works. The git client plugin does not call the git plugin. It does not have any dependency on the git plugin and does not know about the git plugin implementation. The git client plugin provides an API that is called by the git plugin. The stack trace you provided shows two calls to git fetch , but the two calls are using different credentials. I'm assuming that the difference in credentials is because you edited the log output to remove identifying information.

            Unassigned Unassigned
            slcy sam leung
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: