Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70155

ReadTrusted fails when using Pipeline from SCM with a specific commit / Lightweight checkout fails when specifying single commit

      We have a pipeline job that specifies a commit SHA as branch spec.

      As described at plugin/job-dsl/api-viewer/index.html#path/pipelineJob-definition-cpsScm-scm-gitSCM-branches-branchSpec-name

      • <commitId>Checks out the specified commit.E.g. 5062ac843f2b947733e6a3b105977056821bd3525062ac84, ...

       

      This works as long as lightweight checkout is disabled, which means SCMFileSystem is not used. 

      See here: https://github.com/jenkinsci/workflow-cps-plugin/blob/ef35de2396200112c03fd56359f7691a6de23dc3/plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java#L117 

      When lightweight checkout is enabled the error described below happens. 

      hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune – origin +refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f:refs/remotes/origin/15b834bad5c89087e90b2dcee263ece1af328a9f" returned status code 128: stdout: stderr: fatal: couldn't find remote ref refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f

      The same thing happens with readTrusted, which is used by the Kubernetes plugin to read yaml files and which actually sent me on this hunt.

      Our pipeline job was always started correctly, the pipeline file was pulled from the right commit and then it failed to pull the yaml file with the error message shown above.

      Fixing readTrusted can be done by specifying 

      Dorg.jenkinsci.plugins.workflow.multibranch.SCMBinder.USE_HEAVYWEIGHT_CHECKOUT=true

      To force it to not use SCMFileSystem.

      This could be fixed more cleanly by having GitSCMFileSystem work with commits, but I am not sure how.

      Another way would be to somehow change the supports() method to return false in case a commit is given.

      https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L257

      But I don't see a way to differentiate between a commit SHA and a branch name. 

      The actual wrong behavior in all of this is happening here:
      https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L404

      instead of a proper ref the commit SHA will be used here.

       

      The following two links describe the same symptoms and will probably be the same issue.

      https://issues.jenkins.io/browse/JENKINS-60697

      https://stackoverflow.com/questions/69116849/gerrit-refspec-not-recognized-during-scm-git-checkout-jenkinsfile-declarative 

          [JENKINS-70155] ReadTrusted fails when using Pipeline from SCM with a specific commit / Lightweight checkout fails when specifying single commit

          Mark Waite added a comment -

          Recent changes in git plugin 4.14.0 may help when combined with an unreleased change in the workflow cps plugin and with changes that are available in the most recent release of the scm api plugin.  The unreleased workflow cps plugin change is available from https://ci.jenkins.io/job/Plugins/job/workflow-cps-plugin/job/PR-633/lastSuccessfulBuild/

          Mark Waite added a comment - Recent changes in git plugin 4.14.0 may help when combined with an unreleased change in the workflow cps plugin and with changes that are available in the most recent release of the scm api plugin.  The unreleased workflow cps plugin change is available from https://ci.jenkins.io/job/Plugins/job/workflow-cps-plugin/job/PR-633/lastSuccessfulBuild/

          Neri Marschik added a comment - - edited

          Thanks for the quick reply!

          What you mean is the following?

          Add a parameter 'BRANCH' and set branch spec to '${BRANCH}'. The build is passed down (above mentioned PR) to SCMFileSystem -> GitSCMFileSystem, which uses the env out of build to create HeadNameResult which is used in the fetch.

          I might misunderstand so this is just a guess, but I think this will also result in the following string 

          +refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f:refs/remotes/origin/15b834bad5c89087e90b2dcee263ece1af328a9f

          Which is the failing one described above.

          In case the branch spec is a commit SHA, we probably cannot just fetch it (depends on the Git version I think). 

          Neri Marschik added a comment - - edited Thanks for the quick reply! What you mean is the following? Add a parameter 'BRANCH' and set branch spec to '${BRANCH}'. The build  is passed down (above mentioned PR) to SCMFileSystem -> GitSCMFileSystem, which uses the env  out of  build to create  HeadNameResult which is used in the fetch. I might misunderstand so this is just a guess, but I think this will also result in the following string  +refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f:refs/remotes/origin/15b834bad5c89087e90b2dcee263ece1af328a9f Which is the failing one described above. In case the branch spec is a commit SHA, we probably cannot just fetch it (depends on the Git version I think). 

          Jesse Glick added a comment -

          Not sure offhand (really need a JenkinsRule test case) but just passing the new Run parameter in a couple of places might fix issues like this.

          Jesse Glick added a comment - Not sure offhand (really need a JenkinsRule test case) but just passing the new Run parameter in a couple of places might fix issues like this.

          The same thing happens if you use `FETCH_HEAD` in the branch spec and have only one ref in refspec.

          Victor Balakine added a comment - The same thing happens if you use `FETCH_HEAD` in the branch spec and have only one ref in refspec.

          The fix that the author of JENKINS-71506 proposed is a step in the right direction, but I don't see it being merged any time soon in its current form https://github.com/jenkinsci/git-plugin/pull/1468

          Victor Balakine added a comment - The fix that the author of JENKINS-71506 proposed is a step in the right direction, but I don't see it being merged any time soon in its current form https://github.com/jenkinsci/git-plugin/pull/1468

            Unassigned Unassigned
            codesuki Neri Marschik
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: