Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-73680

Failures to fetch Pull request refs from a mirror

      A Jenkins job with the Discover Pull Request feature enabled can fail to retrieve PR refs when it is configured to clone data from a Mirror.

      According to Troubleshooting Smart Mirroring, under Pull request refs are not synchronized section, those refs are not automatically synced to the mirrors, causing the workflow to often fail.

      This is a suggestion to handle this situation:

      1. If the job is set to clone from the mirror, maybe the event needs to be sent from the mirror instead of the primary server.
      2. If the event is sent from the primary server, the Jenkins plugin should have some kind of wait state until the mirror is ready.

      Thanks!

          [JENKINS-73680] Failures to fetch Pull request refs from a mirror

          Youcef added a comment - - edited

          Here is what I think happens, Bitbucket stores commits relevant to PRs in refs/pull-request implementation. This structure is not updated in real-time to the mirrors, or with delay, when Bitbucket notifies Jenkins that there is an update to a PR, Jenkins tries to fetch that information from the mirror which is not synced yet from the primary.

           

          Here is the error I see:

           Cloning repository ssh://git@<bitbucket.something>:7999/proj/repo1.git
            > git.exe init D:\Jenkins\workspace\PR-74 # timeout=10
           Fetching upstream changes from ssh://git@<bitbucket.something>:7999/proj/repo1.git   > git.exe --version # timeout=10
            > git --version # 'git version 2.45.2.windows.1'
           using GIT_SSH to set credentials ssh-svc_bbAdmin
           Verifying host key using known hosts file, will automatically accept unseen keys
            > git.exe fetch --no-tags --force --progress -- ssh://git@<bitbucket.something>:7999/proj/repo1.git+refs/heads/*:refs/remotes/origin/* +refs/pull-requests/74/from:refs/remotes/origin/PR-74 # timeout=10
           ERROR: Error cloning remote repo 'origin'
           hudson.plugins.git.GitException: Command "git.exe fetch --no-tags --force --progress -- ssh://git@<bitbucket.something>:7999/proj/repo1.git +refs/heads/*:refs/remotes/origin/* +refs/pull-requests/74/from:refs/remotes/origin/PR-74" returned status code 128:
           stdout: 
           stderr: fatal: couldn't find remote ref refs/pull-requests/74/from
           
           	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2846)
           	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2185)
           	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:635)
           	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:871)
           	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:170)
           	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
           	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
           	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
           	at hudson.remoting.Request$2.run(Request.java:377)
           	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
           	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
           	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
           	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
           	at java.base/java.lang.Thread.run(Thread.java:1583)
           	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenk1
           		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1826)
           		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
           		at hudson.remoting.Channel.call(Channel.java:1042)
           		at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:153)
           		at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
           		at java.base/java.lang.reflect.Method.invoke(Method.java:580)
           		at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:138)
           		at PluginClassLoader for git-client/jdk.proxy137/jdk.proxy137.$Proxy250.execute(Unknown Source)
           		at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1220)
           		at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1303)
           		at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
           		at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:101)
           		at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:88)
           		at PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
           		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
           		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
           		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
           		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
           		at java.base/java.lang.Thread.run(Thread.java:1583)
           ERROR: Error cloning remote repo 'origin'
           ERROR: Maximum checkout retry attempts reached, aborting 

          Youcef added a comment - - edited Here is what I think happens, Bitbucket stores commits relevant to PRs in refs/pull-request implementation. This structure is not updated in real-time to the mirrors, or with delay, when Bitbucket notifies Jenkins that there is an update to a PR, Jenkins tries to fetch that information from the mirror which is not synced yet from the primary.   Here is the error I see: Cloning repository ssh: //git@<bitbucket.something>:7999/proj/repo1.git > git.exe init D:\Jenkins\workspace\PR-74 # timeout=10 Fetching upstream changes from ssh: //git@<bitbucket.something>:7999/proj/repo1.git   > git.exe --version # timeout=10 > git --version # 'git version 2.45.2.windows.1' using GIT_SSH to set credentials ssh-svc_bbAdmin Verifying host key using known hosts file, will automatically accept unseen keys > git.exe fetch --no-tags --force --progress -- ssh: //git@<bitbucket.something>:7999/proj/repo1.git+refs/heads/*:refs/remotes/origin/* +refs/pull-requests/74/from:refs/remotes/origin/PR-74 # timeout=10 ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Command "git.exe fetch --no-tags --force --progress -- ssh: //git@<bitbucket.something>:7999/proj/repo1.git +refs/heads/*:refs/remotes/origin/* +refs/pull-requests/74/from:refs/remotes/origin/PR-74" returned status code 128: stdout: stderr: fatal: couldn't find remote ref refs/pull-requests/74/from at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2846) at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2185) at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:635) at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:871) at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:170) at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161) at hudson.remoting.UserRequest.perform(UserRequest.java:211) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:377) at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang. Thread .run( Thread .java:1583) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenk1 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1826) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) at hudson.remoting.Channel.call(Channel.java:1042) at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:153) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at PluginClassLoader for git-client //org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:138) at PluginClassLoader for git-client/jdk.proxy137/jdk.proxy137.$Proxy250.execute(Unknown Source) at PluginClassLoader for git //hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1220) at PluginClassLoader for git //hudson.plugins.git.GitSCM.checkout(GitSCM.java:1303) at PluginClassLoader for workflow-scm-step //org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136) at PluginClassLoader for workflow-scm-step //org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:101) at PluginClassLoader for workflow-scm-step //org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:88) at PluginClassLoader for workflow-step-api //org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang. Thread .run( Thread .java:1583) ERROR: Error cloning remote repo 'origin' ERROR: Maximum checkout retry attempts reached, aborting

            Unassigned Unassigned
            ulissesn Ulisses
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: