Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56063

git-plugin fails to expand variables in refspec on initial clone

    • Git plugin 4.5.1 released 26 Dec 2020

      Git plugin fails to expand variables in the refspec setting. A sure way to trigger the bug is to enable both "Honor refspec on initial clone" and "Wipe out repository & force clone". The config is attached.

      Here's an excerpt from the log to the top-level job:

       

       > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
       > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
       > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
       > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
      Cleaning workspace
       > git rev-parse --verify HEAD # timeout=10
      No valid HEAD. Skipping the resetting
       > git clean -fdx # timeout=10
      Pruning obsolete local branches
      Fetching upstream changes from http://stash/scm/rob/testrepository.git
       > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master6 --prune # timeout=10
      
      

      Note that ${BUILD_ID} is not expanded in the first commit, but is expanded at the end where branch specifier is being used.

       

      Remove "Wipe out repository & force clone", and the variable is being expanded now.

       

       > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master9 --prune # timeout=10
       > git rev-parse refs/remotes/origin/master9^{commit} # timeout=10
       > git rev-parse refs/remotes/origin/refs/heads/master9^{commit} # timeout=10
      

       

      It may seem a trivial issue with ${BUILD_ID}, but it's actually a big problem when I try pulling ${sourceCommitHash} with stash-pull-request-builder. That's why I consider it a major bug. It prevents me from using the correct commit received from Stash REST API and makes me use a suboptimal solution that involves a race condition.

      I believe git-plugin should always expand variables in the refspec setting on its own, without relying on the shell.

       

          [JENKINS-56063] git-plugin fails to expand variables in refspec on initial clone

          Pavel Roskin created issue -
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]

          Mark Waite added a comment -

          If you'll use Pipeline jobs rather than Freestyle jobs, you can allow Groovy to expand the refspec for you and have all the power of the Jenkins Declarative Pipeline and (if needed) scripted Pipeline. I realize that doesn't resolve your issue, but it gives you a path that may allow you to satisfy your use case without waiting for a change in the git plugin.

          Mark Waite added a comment - If you'll use Pipeline jobs rather than Freestyle jobs, you can allow Groovy to expand the refspec for you and have all the power of the Jenkins Declarative Pipeline and (if needed) scripted Pipeline. I realize that doesn't resolve your issue, but it gives you a path that may allow you to satisfy your use case without waiting for a change in the git plugin.

          Pavel Roskin added a comment -

          Thank you. I was able to set up my Stash pull request builders in a way that they don't trigger this issue. But I lost a lot of time before I came to a working setup. I wanted to use strict settings during testing to avoid interference with the previously run tests. It was very disorienting when that strict setup triggered this issue with variable expansion.

          It's good that git commands are logged when the installed git binary is used. With jgit, nothing is logged, making it unclear what is expanded and what is not.

          Pavel Roskin added a comment - Thank you. I was able to set up my Stash pull request builders in a way that they don't trigger this issue. But I lost a lot of time before I came to a working setup. I wanted to use strict settings during testing to avoid interference with the previously run tests. It was very disorienting when that strict setup triggered this issue with variable expansion. It's good that git commands are logged when the installed git binary is used. With jgit, nothing is logged, making it unclear what is expanded and what is not.
          Mark Waite made changes -
          Labels Original: expansion git-plugin variables New: expansion git-plugin newbie-friendly variables
          Rishabh Budhouliya made changes -
          Assignee New: Rishabh Budhouliya [ rishabhbudhouliya ]
          Rishabh Budhouliya made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

          Rishabh Budhouliya added a comment - PR for the fix: https://github.com/jenkinsci/git-plugin/pull/830
          Rishabh Budhouliya made changes -
          Status Original: In Progress [ 3 ] New: In Review [ 10005 ]
          Deepansh Nagaria made changes -
          Description Original: Git plugin fails to expand variables in the refspec setting. A sure way to trigger the bug is to enable both "Honor refspec on initial clone" and "Wipe out repository & force clone". The config is attached.

          Here's an excerpt from the log to the top-level job:

           
          {code:java}
           > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
           > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
           > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
           > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
          Cleaning workspace
           > git rev-parse --verify HEAD # timeout=10
          No valid HEAD. Skipping the resetting
           > git clean -fdx # timeout=10
          Pruning obsolete local branches
          Fetching upstream changes from http://stash/scm/rob/testrepository.git
           > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master6 --prune # timeout=10

          {code}
          Note that ${BUILD_ID} is not expanded in the first commit, but is expanded at the end where branch specifier is being used.

           

          Remove "Wipe out repository & force clone", and the variable is being expanded now.

           
          {code:java}
           > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master9 --prune # timeout=10
           > git rev-parse refs/remotes/origin/master9^{commit} # timeout=10
           > git rev-parse refs/remotes/origin/refs/heads/master9^{commit} # timeout=10
          {code}
           

          It may seem a trivial issue with ${BUILD_ID}, but it's actually a big problem when I try pulling ${sourceCommitHash} with stash-pull-request-builder. That's why I consider it a major bug. It prevents me from using the correct commit received from Stash REST API and makes me use a suboptimal solution that involves a race condition.

          I believe git-plugin should always expand variables in the refspec setting on its own, without relying on the shell.

           
          New:
          0...........................................................................0Git plugin fails to expand variables in the refspec setting. A sure way to trigger the bug is to enable both "Honor refspec on initial clone" and "Wipe out repository & force clone". The config is attached.

          Here's an excerpt from the log to the top-level job:

           
          {code:java}
           > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
           > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
           > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master${BUILD_ID} # timeout=10
           > git config remote.origin.url http://stash/scm/rob/testrepository.git # timeout=10
          Cleaning workspace
           > git rev-parse --verify HEAD # timeout=10
          No valid HEAD. Skipping the resetting
           > git clean -fdx # timeout=10
          Pruning obsolete local branches
          Fetching upstream changes from http://stash/scm/rob/testrepository.git
           > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master6 --prune # timeout=10

          {code}
          Note that ${BUILD_ID} is not expanded in the first commit, but is expanded at the end where branch specifier is being used.

           

          Remove "Wipe out repository & force clone", and the variable is being expanded now.

           
          {code:java}
           > git fetch --no-tags --progress http://stash/scm/rob/testrepository.git +refs/heads/master:refs/remotes/origin/master9 --prune # timeout=10
           > git rev-parse refs/remotes/origin/master9^{commit} # timeout=10
           > git rev-parse refs/remotes/origin/refs/heads/master9^{commit} # timeout=10
          {code}
           

          It may seem a trivial issue with ${BUILD_ID}, but it's actually a big problem when I try pulling ${sourceCommitHash} with stash-pull-request-builder. That's why I consider it a major bug. It prevents me from using the correct commit received from Stash REST API and makes me use a suboptimal solution that involves a race condition.

          I believe git-plugin should always expand variables in the refspec setting on its own, without relying on the shell.

           

            rishabhbudhouliya Rishabh Budhouliya
            proski Pavel Roskin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: