Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22537

Seemingly random failures of GitHub plugin PR builder

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • ghprb-plugin
    • None
    • Linux

      Sometimes when trying to build PR after GitHub hook has triggered, job fails while trying to checkout non-existent commit:

      Cloning the remote Git repository
      Cloning repository <private repo>
      Fetching upstream changes from <private repo>
      using GIT_SSH to set credentials GitHub jenkins-admin
      Fetching upstream changes from <private repo>
      using GIT_SSH to set credentials GitHub jenkins-admin
      Checking out Revision 58ef9bff662b45d099f0b7faba822661cbc144b6 (detached)
      FATAL: Could not checkout null with start point 58ef9bff662b45d099f0b7faba822661cbc144b6
      hudson.plugins.git.GitException: Could not checkout null with start point 58ef9bff662b45d099f0b7faba822661cbc144b6
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$8.execute(CliGitAPIImpl.java:1448)
      at hudson.plugins.git.GitSCM.checkout(GitSCM.java:896)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:1411)
      at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
      at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
      at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561)
      at hudson.model.Run.execute(Run.java:1665)
      at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:246)
      Caused by: hudson.plugins.git.GitException: Command "git checkout -f 58ef9bff662b45d099f0b7faba822661cbc144b6" returned status code 128:
      stdout:
      stderr: fatal: reference is not a tree: 58ef9bff662b45d099f0b7faba822661cbc144b6

      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1276)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1253)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1249)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1065)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1075)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$8.execute(CliGitAPIImpl.java:1443)
      ... 9 more

      Launching parametrised build with manually defined ${sha1} always runs fine for same PR's. Exact trigger criteria is unclear but it seems to be related to force pushed to PR branch.

          [JENKINS-22537] Seemingly random failures of GitHub plugin PR builder

          Mihails Strasuns created issue -

          BTW, the checkout fails because for some reason is trying to checkout an old for that pull request (that was force-pushed). Sometimes it works, sometimes it doesn't.

          Leandro Lucarella added a comment - BTW, the checkout fails because for some reason is trying to checkout an old for that pull request (that was force-pushed). Sometimes it works, sometimes it doesn't.

          Mark Waite added a comment - - edited

          I wonder if that is related to JENKINS-21980 ? That bug reports that polling can fail when a prior detached head commit no longer exists in the repository.

          Mark Waite added a comment - - edited I wonder if that is related to JENKINS-21980 ? That bug reports that polling can fail when a prior detached head commit no longer exists in the repository.

          Hm, exactly the case quoted in the issue description does not have any detached commits, it is a normal pull request branch. With this specific branch I can reliably trigger the issue by just doing `git commit --amend; git push -f origin branch-name` with no actual changes. Every new job triggered in such way fails. But I can't reproduce it on any artificially crafted branch.

          Mihails Strasuns added a comment - Hm, exactly the case quoted in the issue description does not have any detached commits, it is a normal pull request branch. With this specific branch I can reliably trigger the issue by just doing `git commit --amend; git push -f origin branch-name` with no actual changes. Every new job triggered in such way fails. But I can't reproduce it on any artificially crafted branch.

          Mark Waite added a comment -

          I thought that if the commit which was created prior to "git commit --amend" were seen by the Jenkins job, then after the "git commit --amend" that prior commit would now be "detached" because there are no longer any references to the prior commit.

          I'm not a gerrit user, so I may misunderstand the use cases, but I'm accustomed to "git commit --amend" leaving an "orphan" commit in the git reflog.

          Mark Waite added a comment - I thought that if the commit which was created prior to "git commit --amend" were seen by the Jenkins job, then after the "git commit --amend" that prior commit would now be "detached" because there are no longer any references to the prior commit. I'm not a gerrit user, so I may misunderstand the use cases, but I'm accustomed to "git commit --amend" leaving an "orphan" commit in the git reflog.

          Yeah, there is one. But I am not passing it to job manually, where it could possibly take the hash from? It is not referenced from any of branch commits anymore.

          Mihails Strasuns added a comment - Yeah, there is one. But I am not passing it to job manually, where it could possibly take the hash from? It is not referenced from any of branch commits anymore.

          Mark Waite added a comment -

          I thought that the git plugin remembered the SHA1 of the preceding commit it processed so that it could use that earlier SHA1 to generate the list of changes. If the SHA1 of the preceding commit was in the Jenkins job, then was somehow removed, it seems like that might cause the failure you're describing.

          Is there any chance the job moved from one node to another between the passing and the failing run?

          Was the workspace wiped between the passing and the failing case? If it was wiped, then the plugin might remember the SHA1 which no longer exists in the newly cloned repository.

          Mark Waite added a comment - I thought that the git plugin remembered the SHA1 of the preceding commit it processed so that it could use that earlier SHA1 to generate the list of changes. If the SHA1 of the preceding commit was in the Jenkins job, then was somehow removed, it seems like that might cause the failure you're describing. Is there any chance the job moved from one node to another between the passing and the failing run? Was the workspace wiped between the passing and the failing case? If it was wiped, then the plugin might remember the SHA1 which no longer exists in the newly cloned repository.

          Yeah, that's what JENKINS-21980 seems to be suggesting, that Jenkins remembers the hash of the last build (now detached) to check if there are changes or not.

          Leandro Lucarella added a comment - Yeah, that's what JENKINS-21980 seems to be suggesting, that Jenkins remembers the hash of the last build (now detached) to check if there are changes or not.

          Shouldn't it be present in reflog of local copy then? git reflog | grep 58ef has shown me nothing.
          Or it is last successful build?

          Mihails Strasuns added a comment - Shouldn't it be present in reflog of local copy then? git reflog | grep 58ef has shown me nothing. Or it is last successful build?

          Mark Waite added a comment -

          I think it should have been present in the repository reflog, unless the repository was wiped from one job to the next. At a minimum, that SHA1 should be somewhere in the Jenkins jobs directory, since the value needs to be remembered on disc somewhere.

          Mark Waite added a comment - I think it should have been present in the repository reflog, unless the repository was wiped from one job to the next. At a minimum, that SHA1 should be somewhere in the Jenkins jobs directory, since the value needs to be remembered on disc somewhere.

            Unassigned Unassigned
            dicebot Mihails Strasuns
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: