-
Bug
-
Resolution: Unresolved
-
Major
-
None
While fetching branch indexing in a multibrach pipeline job, it getting stuck at scanning branch for days in GitlabEE Server Error.(As shown in the attached image)
Problmatic thread got stuck in SocketInputStream.socketread0 method
Thread Stack Trace: Executor #-1 for Built-In Node : executing BranchIndexing[Faceplay/UnityClient] java.base@11.0.13/java.net.SocketInputStream.socketRead0(Native Method) java.base@11.0.13/java.net.SocketInputStream.socketRead(SocketInputStream.java:115) java.base@11.0.13/java.net.SocketInputStream.read(SocketInputStream.java:168) java.base@11.0.13/java.net.SocketInputStream.read(SocketInputStream.java:140) java.base@11.0.13/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478) java.base@11.0.13/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472) java.base@11.0.13/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70) java.base@11.0.13/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1374) java.base@11.0.13/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:985) java.base@11.0.13/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.13/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.13/java.io.BufferedInputStream.read(BufferedInputStream.java:351) java.base@11.0.13/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:754) java.base@11.0.13/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:689) java.base@11.0.13/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1615) java.base@11.0.13/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) java.base@11.0.13/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) java.base@11.0.13/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:334) org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:378) org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:267) org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:297) org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:662) org.glassfish.jersey.client.JerseyInvocation$$Lambda$1124/0x0000000800ef4840.call(Unknown Source) org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:697) org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:691) org.glassfish.jersey.client.JerseyInvocation$$Lambda$1127/0x0000000800ef5440.call(Unknown Source) org.glassfish.jersey.internal.Errors.process(Errors.java:292) org.glassfish.jersey.internal.Errors.process(Errors.java:274) org.glassfish.jersey.internal.Errors.process(Errors.java:205) org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390) org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:691) org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:661) org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:413) org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:313) org.gitlab4j.api.GitLabApiClient.get(GitLabApiClient.java:416) org.gitlab4j.api.GitLabApiClient.get(GitLabApiClient.java:404) org.gitlab4j.api.AbstractApi.get(AbstractApi.java:214) org.gitlab4j.api.RepositoryFileApi.getFile(RepositoryFileApi.java:169) org.gitlab4j.api.RepositoryFileApi.getFile(RepositoryFileApi.java:122) io.jenkins.plugins.gitlabbranchsource.GitLabSCMFile.type(GitLabSCMFile.java:93) jenkins.scm.api.SCMFile.getType(SCMFile.java:265) io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource$4.stat(GitLabSCMSource.java:686) org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory$1.isHead(WorkflowBranchProjectFactory.java:75) jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342) jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:286) io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.retrieve(GitLabSCMSource.java:357) jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373) jenkins.scm.api.SCMSource.fetch(SCMSource.java:283) jenkins.branch.MultiBranchProject.computeChildren(MultiBranchProject.java:641) com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:278) com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:166) jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:1032) hudson.model.ResourceController.execute(ResourceController.java:101) hudson.model.Executor.run(Executor.java:442) )
Timeouts setting(org.gitlab4j.api.GitLabApiClien setRequestTimeout API call in gitlabbranchsource plugin) are needed to solve the above problem.
The same issue can be identified in Bitbuckt/Github source plugin(https://issues.jenkins.io/browse/JENKINS-36724, https://github.com/jenkinsci/bitbucket-branch-source-plugin/pull/135))
Additonally, it is necessary to provide a jvm system property to set the gitlab timeout for timeout value.(example, java -Dio.jenkins.plugins.gitlabbranchsource.config.readTimeout -Dio.jenkins.plugins.gitlabbranchsource.config.connectTimeout)
If you can, we propose a common SCMSourceRequest interface. This issue occurs in many jenkins sCM source plugin such as gitblab/bitbucket/github, so we need a common interface at Jenkins level that sets the request timeout based on system properties
