Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56063

git-plugin fails to expand variables in refspec on initial clone

    • Git plugin 4.5.1 released 26 Dec 2020

      Git plugin fails to expand variables in the refspec setting. A sure way to trigger the bug is to enable both "Honor refspec on initial clone" and "Wipe out repository & force clone". The config is attached.

      Here's an excerpt from the log to the top-level job:

       

       > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
       > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
       > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
       > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
      Cleaning workspace
       > git rev-parse --verify HEAD # timeout=10
      No valid HEAD. Skipping the resetting
       > git clean -fdx # timeout=10
      Pruning obsolete local branches
      Fetching upstream changes from http://stash/scm/rob/testrepository.git
       > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master6 --prune # timeout=10
      
      

      Note that ${BUILD_ID} is not expanded in the first commit, but is expanded at the end where branch specifier is being used.

       

      Remove "Wipe out repository & force clone", and the variable is being expanded now.

       

       > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master9 --prune # timeout=10
       > git rev-parse refs/remotes/origin/master9^{commit} # timeout=10
       > git rev-parse refs/remotes/origin/refs/heads/master9^{commit} # timeout=10
      

       

      It may seem a trivial issue with ${BUILD_ID}, but it's actually a big problem when I try pulling ${sourceCommitHash} with stash-pull-request-builder. That's why I consider it a major bug. It prevents me from using the correct commit received from Stash REST API and makes me use a suboptimal solution that involves a race condition.

      I believe git-plugin should always expand variables in the refspec setting on its own, without relying on the shell.

       

          [JENKINS-56063] git-plugin fails to expand variables in refspec on initial clone

          Mark Waite added a comment -

          If you'll use Pipeline jobs rather than Freestyle jobs, you can allow Groovy to expand the refspec for you and have all the power of the Jenkins Declarative Pipeline and (if needed) scripted Pipeline. I realize that doesn't resolve your issue, but it gives you a path that may allow you to satisfy your use case without waiting for a change in the git plugin.

          Mark Waite added a comment - If you'll use Pipeline jobs rather than Freestyle jobs, you can allow Groovy to expand the refspec for you and have all the power of the Jenkins Declarative Pipeline and (if needed) scripted Pipeline. I realize that doesn't resolve your issue, but it gives you a path that may allow you to satisfy your use case without waiting for a change in the git plugin.

          Pavel Roskin added a comment -

          Thank you. I was able to set up my Stash pull request builders in a way that they don't trigger this issue. But I lost a lot of time before I came to a working setup. I wanted to use strict settings during testing to avoid interference with the previously run tests. It was very disorienting when that strict setup triggered this issue with variable expansion.

          It's good that git commands are logged when the installed git binary is used. With jgit, nothing is logged, making it unclear what is expanded and what is not.

          Pavel Roskin added a comment - Thank you. I was able to set up my Stash pull request builders in a way that they don't trigger this issue. But I lost a lot of time before I came to a working setup. I wanted to use strict settings during testing to avoid interference with the previously run tests. It was very disorienting when that strict setup triggered this issue with variable expansion. It's good that git commands are logged when the installed git binary is used. With jgit, nothing is logged, making it unclear what is expanded and what is not.

          Rishabh Budhouliya added a comment - PR for the fix: https://github.com/jenkinsci/git-plugin/pull/830

          Stephane Odul added a comment -

          Thanks a lot for fixing this issue, we had resorted to write a git wrapper to avoid cloning the hundreds of branches we have in our repo, which was causing every single stage to download 5x more data than we needed from git (we run our workload on ephemeral containers).

          Stephane Odul added a comment - Thanks a lot for fixing this issue, we had resorted to write a git wrapper to avoid cloning the hundreds of branches we have in our repo, which was causing every single stage to download 5x more data than we needed from git (we run our workload on ephemeral containers).

            rishabhbudhouliya Rishabh Budhouliya
            proski Pavel Roskin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: