Minor We have configured a multi phase job where any changes in Git SCM will trigger two jobs. Among the two jobs any one of the job getting failed randomly while fetching the git changes. And, during the next build it's getting passed. Here is my error log.
11:41:52 Started by upstream project "1_CPU_Xtreme_Feedback" build number 3092 11:41:52 originally caused by: 11:41:52 Started by an SCM change 11:41:52 [EnvInject] - Loading node environment variables. 11:41:52 Building remotely on kcebuilder2 in workspace c:\jenkins_keivmcsi\workspace\CPU_XF_ARM9 11:41:55 Fetching changes from the remote Git repository 11:41:55 Fetching upstream changes from git:lift 11:42:14 FATAL: Failed to fetch from git:lift 11:42:14 hudson.plugins.git.GitException: Failed to fetch from git:lift 11:42:14 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:612) 11:42:14 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:836) 11:42:14 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:861) 11:42:14 at hudson.model.AbstractProject.checkout(AbstractProject.java:1411) 11:42:14 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652) 11:42:14 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) 11:42:14 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:561) 11:42:14 at hudson.model.Run.execute(Run.java:1665) 11:42:14 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) 11:42:14 at hudson.model.ResourceController.execute(ResourceController.java:88) 11:42:14 at hudson.model.Executor.run(Executor.java:246) 11:42:14 Caused by: hudson.plugins.git.GitException: Command "fetch -t git:lift +refs/heads/*:refs/remotes/origin/*" returned status code 128: 11:42:14 stdout: 11:42:14 stderr: ssh: Could not resolve hostname git: no address associated with name 11:42:14 fatal: Could not read from remote repository. 11:42:14 11:42:14 Please make sure you have the correct access rights 11:42:14 and the repository exists. 11:42:14 11:42:14 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:928) 11:42:14 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:902) 11:42:14 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.fetch(CliGitAPIImpl.java:163) 11:42:14 at hudson.plugins.git.GitAPI.fetch(GitAPI.java:229) 11:42:14 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 11:42:14 at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) 11:42:14 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 11:42:14 at java.lang.reflect.Method.invoke(Unknown Source) 11:42:14 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:299) 11:42:14 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:280) 11:42:14 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:239) 11:42:14 at hudson.remoting.UserRequest.perform(UserRequest.java:118) 11:42:14 at hudson.remoting.UserRequest.perform(UserRequest.java:48) 11:42:14 at hudson.remoting.Request$2.run(Request.java:328) 11:42:14 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) 11:42:14 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) 11:42:14 at java.util.concurrent.FutureTask.run(Unknown Source) 11:42:14 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 11:42:14 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 11:42:14 at java.lang.Thread.run(Unknown Source)
[JENKINS-28551] Git fetch fails randomly with status code 128
The message "could not resolve hostname" seems to hint that your DNS server is sometimes not responding to the request to convert a hostname to an IP address. It seems unlikely that the git plugin or the git client plugin can help with unreliable DNS service.
If you need to confirm the DNS service reliability, you could write a batch command (Windows) or shell script (Linux) to check that your DNS server consistently and reliably returns addresses throughout the day. If your DNS server is outside your control, you could consider installing a caching DNS server on each slave so that they will remember the successfully resolved names and return them from the cache, rather than asking your central DNS server.
But, out of two jobs which starts simultaneously only one job is getting this error and the other job is getting executed successfully. Both are started in the same node. Why did both the jobs didn't get the same message "could not resolve hostname"? And, If I start the upstream job also in the same node I am not getting this error.
If the DNS server is not responding reliably, then difference could happen at any time, either within the same job or between different jobs when the DNS server is asked to convert a hostname to an IP address.
Have you attempted to change the git source code URL from a hostname based URL to an IP address based URL in order to confirm that the DNS server is the problem?
Since the message that is being reported is coming from inside the command line git program, there is not much the git plugin can do for you except report the message.
You could also try JGit by adding it as a git implementation in the system configuration, then switch to use that implementation in the job definition. If the problem is intermittent misbehavior of the DNS server, then both git and JGit should be affected, since they both require hostname to IP address conversion.
Now I configured all the jobs to be run on a single "Node", I am still getting these errors. Now three out of one jobs fails with this error.
Actually this is our git config file, we are using IP address only
{{Host git
HostName 10.8.152.132
User git
identityFile ~/.ssh/my_git_key}}
I will try your suggestions and let you know. Thanks.
If the upstream job is started in "node 1" and it triggers two jobs, if both the job starts running in "node 2", at least one job fails with this message. When triggered manually both the jobs are running successfully next time.