Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70155

ReadTrusted fails when using Pipeline from SCM with a specific commit / Lightweight checkout fails when specifying single commit

      We have a pipeline job that specifies a commit SHA as branch spec.

      As described at plugin/job-dsl/api-viewer/index.html#path/pipelineJob-definition-cpsScm-scm-gitSCM-branches-branchSpec-name

      • <commitId>Checks out the specified commit.E.g. 5062ac843f2b947733e6a3b105977056821bd3525062ac84, ...

       

      This works as long as lightweight checkout is disabled, which means SCMFileSystem is not used. 

      See here: https://github.com/jenkinsci/workflow-cps-plugin/blob/ef35de2396200112c03fd56359f7691a6de23dc3/plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java#L117 

      When lightweight checkout is enabled the error described below happens. 

      hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune – origin +refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f:refs/remotes/origin/15b834bad5c89087e90b2dcee263ece1af328a9f" returned status code 128: stdout: stderr: fatal: couldn't find remote ref refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f

      The same thing happens with readTrusted, which is used by the Kubernetes plugin to read yaml files and which actually sent me on this hunt.

      Our pipeline job was always started correctly, the pipeline file was pulled from the right commit and then it failed to pull the yaml file with the error message shown above.

      Fixing readTrusted can be done by specifying 

      Dorg.jenkinsci.plugins.workflow.multibranch.SCMBinder.USE_HEAVYWEIGHT_CHECKOUT=true

      To force it to not use SCMFileSystem.

      This could be fixed more cleanly by having GitSCMFileSystem work with commits, but I am not sure how.

      Another way would be to somehow change the supports() method to return false in case a commit is given.

      https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L257

      But I don't see a way to differentiate between a commit SHA and a branch name. 

      The actual wrong behavior in all of this is happening here:
      https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L404

      instead of a proper ref the commit SHA will be used here.

       

      The following two links describe the same symptoms and will probably be the same issue.

      https://issues.jenkins.io/browse/JENKINS-60697

      https://stackoverflow.com/questions/69116849/gerrit-refspec-not-recognized-during-scm-git-checkout-jenkinsfile-declarative 

          [JENKINS-70155] ReadTrusted fails when using Pipeline from SCM with a specific commit / Lightweight checkout fails when specifying single commit

          Neri Marschik created issue -
          Neri Marschik made changes -
          Description Original: We have a pipeline job that specifies a commit SHA as branch spec.

          As described at _plugin/job-dsl/api-viewer/index.html#path/pipelineJob-definition-cpsScm-scm-gitSCM-branches-branchSpec-name_
           *
          {quote}{*}{{<commitId>}}{*}Checks out the specified commit.E.g. {{{}5062ac843f2b947733e6a3b105977056821bd352{}}}, {{{}5062ac84{}}}, ...{quote}

           

          This works as long as lightweight checkout is disabled, which means _SCMFileSystem_ is not used. 

          See here: [https://github.com/jenkinsci/workflow-cps-plugin/blob/ef35de2396200112c03fd56359f7691a6de23dc3/plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java#L117

          When lightweight checkout is enabled the error described below happens. 
          {quote}hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune -- origin +refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f:refs/remotes/origin/15b834bad5c89087e90b2dcee263ece1af328a9f" returned status code 128: stdout: stderr: fatal: couldn't find remote ref refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f
          {quote}
          The same thing happens with readTrusted, which is used by the Kubernetes plugin to read yaml files and which actually sent me on this hunt.

          Our pipeline job was always started correctly, the pipeline file was pulled from the right commit and then it failed to pull the yaml file with the message:

          Fixing readTrusted can be done by specifying 
          {quote}{{_Dorg.jenkinsci.plugins.workflow.multibranch.SCMBinder.USE_HEAVYWEIGHT_CHECKOUT=true_}}{quote}
          To force it to not use SCMFileSystem.

          This could be fixed more cleanly by having GitSCMFileSystem work with commits, but I am not sure how.

          Another way would be to somehow change the _supports()_ method to return false in case a commit is given.

          [https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L257]

          But I don't see a way to differentiate between a commit SHA and a branch name. 

          The actual wrong behavior in all of this is happening here:
          [https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L404|https://github.com/jenkinsci/git-plugin/blob/c1aa477b8a715a7235a8c63e3b0b9266a5db1c42/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L404]

          instead of a proper ref the commit SHA will be used here.

           

          The following two links describe the same symptoms and will probably be the same issue.

          https://issues.jenkins.io/browse/JENKINS-60697

          [https://stackoverflow.com/questions/69116849/gerrit-refspec-not-recognized-during-scm-git-checkout-jenkinsfile-declarative
          New: We have a pipeline job that specifies a commit SHA as branch spec.

          As described at _plugin/job-dsl/api-viewer/index.html#path/pipelineJob-definition-cpsScm-scm-gitSCM-branches-branchSpec-name_
           *
          {quote}{*}{{<commitId>}}{*}Checks out the specified commit.E.g. {{{}5062ac843f2b947733e6a3b105977056821bd352{}}}, {{{}5062ac84{}}}, ...
          {quote}

           

          This works as long as lightweight checkout is disabled, which means _SCMFileSystem_ is not used. 

          See here: [https://github.com/jenkinsci/workflow-cps-plugin/blob/ef35de2396200112c03fd56359f7691a6de23dc3/plugin/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinition.java#L117

          When lightweight checkout is enabled the error described below happens. 
          {quote}hudson.plugins.git.GitException: Command "git fetch --tags --force --progress --prune – origin +refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f:refs/remotes/origin/15b834bad5c89087e90b2dcee263ece1af328a9f" returned status code 128: stdout: stderr: fatal: couldn't find remote ref refs/heads/15b834bad5c89087e90b2dcee263ece1af328a9f
          {quote}
          The same thing happens with readTrusted, which is used by the Kubernetes plugin to read yaml files and which actually sent me on this hunt.

          Our pipeline job was always started correctly, the pipeline file was pulled from the right commit and then it failed to pull the yaml file with the error message shown above.

          Fixing readTrusted can be done by specifying 
          {quote}{{_Dorg.jenkinsci.plugins.workflow.multibranch.SCMBinder.USE_HEAVYWEIGHT_CHECKOUT=true_}}
          {quote}
          To force it to not use SCMFileSystem.

          This could be fixed more cleanly by having GitSCMFileSystem work with commits, but I am not sure how.

          Another way would be to somehow change the _supports()_ method to return false in case a commit is given.

          [https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L257]

          But I don't see a way to differentiate between a commit SHA and a branch name. 

          The actual wrong behavior in all of this is happening here:
          [https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L404|https://github.com/jenkinsci/git-plugin/blob/c1aa477b8a715a7235a8c63e3b0b9266a5db1c42/src/main/java/jenkins/plugins/git/GitSCMFileSystem.java#L404]

          instead of a proper ref the commit SHA will be used here.

           

          The following two links describe the same symptoms and will probably be the same issue.

          https://issues.jenkins.io/browse/JENKINS-60697

          [https://stackoverflow.com/questions/69116849/gerrit-refspec-not-recognized-during-scm-git-checkout-jenkinsfile-declarative
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]
          Jesse Glick made changes -
          Component/s Original: git-plugin [ 15543 ]
          Component/s Original: workflow-cps-plugin [ 21713 ]
          Jesse Glick made changes -
          Link New: This issue relates to JENKINS-42971 [ JENKINS-42971 ]
          Jesse Glick made changes -
          Link New: This issue relates to JENKINS-60697 [ JENKINS-60697 ]
          Victor Balakine made changes -
          Link New: This issue is caused by JENKINS-71506 [ JENKINS-71506 ]

            Unassigned Unassigned
            codesuki Neri Marschik
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: