Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45447

Git checkout with many tags is much slower with 3.3.0 release if pre-build merge is used

      We have 62 thousands of tags in our git repo.
      After upgrade of git plugin from 3.1.0 to 3.3.0 checkout of the repo with PreBuildMerge option takes about an hour.

      14:54:44  > git rev-parse HEAD^{commit} # timeout=10
      14:54:45 Seen branch in repository origin/10098_update_header_link_and_my_account_menu
      ...
      ...
      14:54:45 Seen 1,592 remote branches
      14:54:45  > git tag -l # timeout=10
      14:54:46  > git rev-parse refs/tags/test01wxunit/20160617_094816^{commit} # timeout=10
      ...
      ...
      15:49:57  > git rev-parse refs/tags/b2bAATpipeline/tag/2015-03-06_12-08-07^{commit} # timeout=10
      15:49:57 Checking out Revision ab591971099f42107cbd7915120ee1381a206047 (origin/hybris551aws)
      

      It looks like the root cause of the issue is commit 31fedce9c41c9006c886835e03c9fe825d49aba6.

          [JENKINS-45447] Git checkout with many tags is much slower with 3.3.0 release if pre-build merge is used

          Mykhaylo Yehorov created issue -
          Mark Waite made changes -
          Summary Original: Git checkout lasts too much time New: Git checkout with many tags is much slower with 3.3.0 release
          Mark Waite made changes -
          Summary Original: Git checkout with many tags is much slower with 3.3.0 release New: Git checkout with many tags is much slower with 3.3.0 release if pre-build merge is used
          Mykhaylo Yehorov made changes -
          Description Original: We have 62 thousands of tags in out git repo.
          After upgrade of git plugin from 3.1.0 to 3.3.0 checkout of the repo with PreBuildMerge option takes about an hour.

          {noformat}
          14:54:44 > git rev-parse HEAD^{commit} # timeout=10
          14:54:45 Seen branch in repository origin/10098_update_header_link_and_my_account_menu
          ...
          ...
          14:54:45 Seen 1,592 remote branches
          14:54:45 > git tag -l # timeout=10
          14:54:46 > git rev-parse refs/tags/test01wxunit/20160617_094816^{commit} # timeout=10
          ...
          ...
          15:49:57 > git rev-parse refs/tags/b2bAATpipeline/tag/2015-03-06_12-08-07^{commit} # timeout=10
          15:49:57 Checking out Revision ab591971099f42107cbd7915120ee1381a206047 (origin/hybris551aws)
          {noformat}


          It looks like the root cause of the issue is commit 31fedce9c41c9006c886835e03c9fe825d49aba6.
          New: We have 62 thousands of tags in our git repo.
          After upgrade of git plugin from 3.1.0 to 3.3.0 checkout of the repo with PreBuildMerge option takes about an hour.

          {noformat}
          14:54:44 > git rev-parse HEAD^{commit} # timeout=10
          14:54:45 Seen branch in repository origin/10098_update_header_link_and_my_account_menu
          ...
          ...
          14:54:45 Seen 1,592 remote branches
          14:54:45 > git tag -l # timeout=10
          14:54:46 > git rev-parse refs/tags/test01wxunit/20160617_094816^{commit} # timeout=10
          ...
          ...
          15:49:57 > git rev-parse refs/tags/b2bAATpipeline/tag/2015-03-06_12-08-07^{commit} # timeout=10
          15:49:57 Checking out Revision ab591971099f42107cbd7915120ee1381a206047 (origin/hybris551aws)
          {noformat}


          It looks like the root cause of the issue is commit 31fedce9c41c9006c886835e03c9fe825d49aba6.
          Mark Waite made changes -
          Link New: This issue is duplicated by JENKINS-47169 [ JENKINS-47169 ]

          Jakob Unterwurzacher added a comment - - edited

          The rev-parse thing also breaks building the linux kernel. The reason is interesting: the v2.6.11  tag ( https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tag/?h=v2.6.11 ) does not have a backing commit:

           > git rev-parse refs/tags/v2.6.11^{commit} # timeout=10
          FATAL: Command "git rev-parse refs/tags/v2.6.11^{commit}" returned status code 128:
          stdout: refs/tags/v2.6.11^{commit}
          
          stderr: error: refs/tags/v2.6.11^{commit}: expected commit type, but the object dereferences to tree type
          error: refs/tags/v2.6.11^{commit}: expected commit type, but the object dereferences to tree type
          fatal: ambiguous argument 'refs/tags/v2.6.11^{commit}': unknown revision or path not in the working tree.
          Use '--' to separate paths from revisions, like this:
          'git <command> [<revision>...] -- [<file>...]'
          

          As for the workaround: Downgrading the git plugin fixes the issue. The newest version that I found was working is 3.1.0, which can be downloaded from http://updates.jenkins-ci.org/download/plugins/git/ .

          Jakob Unterwurzacher added a comment - - edited The rev-parse thing also breaks building the linux kernel. The reason is interesting: the v2.6.11  tag ( https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tag/?h=v2.6.11  ) does not have a backing commit: > git rev-parse refs/tags/v2.6.11^{commit} # timeout=10 FATAL: Command "git rev-parse refs/tags/v2.6.11^{commit}" returned status code 128: stdout: refs/tags/v2.6.11^{commit} stderr: error: refs/tags/v2.6.11^{commit}: expected commit type, but the object dereferences to tree type error: refs/tags/v2.6.11^{commit}: expected commit type, but the object dereferences to tree type fatal: ambiguous argument 'refs/tags/v2.6.11^{commit}' : unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this : 'git <command> [<revision>...] -- [<file>...]' As for the workaround: Downgrading the git plugin fixes the issue. The newest version that I found was working is 3.1.0, which can be downloaded from http://updates.jenkins-ci.org/download/plugins/git/  .

          Mark Waite added a comment -

          jakob_tsd please file a separate bug report for that issue. It has a very different cause than this bug and is an interesting case that I believe is also noted in the JGit tests. A tag that points to a tree rather than a commit is an interesting tagging case.

          Mark Waite added a comment - jakob_tsd please file a separate bug report for that issue. It has a very different cause than this bug and is an interesting case that I believe is also noted in the JGit tests. A tag that points to a tree rather than a commit is an interesting tagging case.

          Jakob Unterwurzacher added a comment - Yes, that's true. Done:  https://issues.jenkins-ci.org/browse/JENKINS-47350

          Mark Waite added a comment -

          I have a duplication case for the problem in the JENKINS-32577-regex-branch-to-build-does-not-ignore-user-exclusion job in my regression test kit. That doesn't help anyone else work on it (private repo), but it shows that I can duplicate it.

          If others want to develop a fix, I can provide the job definition.

          Mark Waite added a comment - I have a duplication case for the problem in the JENKINS-32577-regex-branch-to-build-does-not-ignore-user-exclusion job in my regression test kit. That doesn't help anyone else work on it (private repo), but it shows that I can duplicate it. If others want to develop a fix, I can provide the job definition.
          Mark Waite made changes -
          Comment [ I have a duplication case for the problem in the [JENKINS-32577-regex-branch-to-build-does-not-ignore-user-exclusion|https://github.com/MarkEWaite/docker-private-lfs/blob/lts-with-plugins-add-credentials/ref/jobs/Bugs-Individual/jobs/JENKINS-32577-regex-branch-to-build-does-not-ignore-user-exclusion/config.xml] job in my regression test kit. That doesn't help anyone else work on it (private repo), but it shows that I can duplicate it.

          If others want to develop a fix, I can provide the job definition.
          ]

            markewaite Mark Waite
            yehorov Mykhaylo Yehorov
            Votes:
            6 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: