[JENKINS-36029] multibranch-job deleted when bitbucket communication fails

Type: Bug
Resolution: Fixed
Priority: Critical
Component/s: bitbucket-branch-source-plugin
Labels:
None
Environment:
Jenkins 2.8

Similar Issues:
Powered by SuggestiMate

Show

I'm trying out the "Bitbucket team/project" feature: I have an issue where some of my jobs keeps being deleted and recreated randomly, and it seems to be related to situations where communicating with bitbucket fails. I would not expect Jenkins to delete my multibranch-job when e.g. Bitbucket or some network link is down.

Relevant log:

Proposing ansible-docker
Connecting to https://bitbucket.org using hidden-org-name/****** (hidden-org-name bitbucket credentials)
Looking up hidden-org-name/ansible-docker for branches
Checking branch jenkins-test from hidden-org-name/ansible-docker
Met criteria
ERROR: Failed to create or update a subproject ansible-docker
com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: Communication error: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
	at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.getRequest(BitbucketCloudApiClient.java:421)
	at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.getRepository(BitbucketCloudApiClient.java:173)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.getRepositoryType(BitbucketSCMSource.java:254)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.observe(BitbucketSCMSource.java:372)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieveBranches(BitbucketSCMSource.java:326)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:279)
	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:146)
	at jenkins.scm.api.SCMSource.retrieve(SCMSource.java:230)
	at jenkins.scm.api.SCMSource.fetch(SCMSource.java:175)
	at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria$1.call(MultiBranchProjectFactory.java:157)
	at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria$1.call(MultiBranchProjectFactory.java:154)
	at jenkins.branch.OrganizationFolder.withSCMSourceCriteria(OrganizationFolder.java:255)
	at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria.recognizes(MultiBranchProjectFactory.java:154)
	at jenkins.branch.OrganizationFolder$1$1.complete(OrganizationFolder.java:165)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator.add(BitbucketSCMNavigator.java:198)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator.visitSources(BitbucketSCMNavigator.java:175)
	at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:125)
	at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:157)
	at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:122)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:747)
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
	at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(HttpConnection.java:828)
	at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2116)
	at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
	at com.cloudbees.jenkins.plugins.bitbucket.client.BitbucketCloudApiClient.getRequest(BitbucketCloudApiClient.java:412)
	... 20 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
	at sun.security.ssl.InputRecord.read(InputRecord.java:505)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
	... 33 more

  Evaluating orphaned items in hidden-org-name on Bitbucket
  Will remove ansible-docker as it is #1 in the list
  Finished: SUCCESS

is duplicated by

JENKINS-41863 Folder computation / branch indexing can unexpectedly delete things when bitbucket goes down

Resolved

relates to

JENKINS-40767 Failures in scanning / indexing should prevent the orphaned item strategy from running

Closed

JENKINS-42000 GitHub organization job losing build history

Closed

links to

CloudBees Internal OSS-1424

Asgeir Frimannsson created issue - 2016-06-17 03:23

Antonio Muñiz added a comment - 2016-06-20 07:51

Counterpart of ~~JENKINS-34776~~

Antonio Muñiz added a comment - 2016-06-20 07:51 Counterpart of JENKINS-34776

Vladimir Rozhkov added a comment - 2016-07-11 08:52

I have a lot of repositories in our team account (kinda 30) and I'm very annoyed that I have to re-run folder computation 10 times to get all my repos configured.
Will some retry logic work here?

Vladimir Rozhkov added a comment - 2016-07-11 08:52 I have a lot of repositories in our team account (kinda 30) and I'm very annoyed that I have to re-run folder computation 10 times to get all my repos configured. Will some retry logic work here?

Michael Fowler made changes - 2016-07-11 19:50

Priority

Original: Major [ 3 ]

New: Critical [ 2 ]

Javier Martín added a comment - 2016-07-13 06:36 - edited

We are experiencing this issue as well. As we delegate our User Directory to JIRA, when this instance is not available an Unauthorized Error 401 appears rising the following error:

com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: 401: No Autorizado.
{"errors":[

{"context":null,"message":"Authentication failed. Please check your credentials and try again.","exceptionName":"com.atlassian.bitbucket.auth.AuthenticationSystemException"}

]}
at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:382)
at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getBranches(BitbucketServerAPIClient.java:258)
at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:384)
at jenkins.scm.api.SCMSource.fetch(SCMSource.java:245)
at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:75)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:206)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Finished: FAILURE

Then we need to run the folder computation again and all build numbers are restarted. We use this parameter as a complement to version number so there is a way to retain build numbers when this happens? This is not a case when a Jenkinsfile is not present in a branch (I assume when this happens the multibranch job is deleted), this is a temporary issue with the scan credentials and I think it wouln't be that way.

Javier Martín added a comment - 2016-07-13 06:36 - edited We are experiencing this issue as well. As we delegate our User Directory to JIRA, when this instance is not available an Unauthorized Error 401 appears rising the following error: com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: 401: No Autorizado. {"errors":[ {"context":null,"message":"Authentication failed. Please check your credentials and try again.","exceptionName":"com.atlassian.bitbucket.auth.AuthenticationSystemException"} ]} at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:382) at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getBranches(BitbucketServerAPIClient.java:258) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.retrieve(BitbucketSCMSource.java:384) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:245) at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:75) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:206) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:410) Finished: FAILURE Then we need to run the folder computation again and all build numbers are restarted. We use this parameter as a complement to version number so there is a way to retain build numbers when this happens? This is not a case when a Jenkinsfile is not present in a branch (I assume when this happens the multibranch job is deleted), this is a temporary issue with the scan credentials and I think it wouln't be that way.

Vladimir Rozhkov added a comment - 2016-07-13 08:34

jaomp this is not the same issue. The issue is about ssl handshake failure and yours is about auth failure. Make sure you've setup credentials correctly.

Meanwhile, I'm working on PR to fix the SSL handshake issue.

Vladimir Rozhkov added a comment - 2016-07-13 08:34 jaomp this is not the same issue. The issue is about ssl handshake failure and yours is about auth failure. Make sure you've setup credentials correctly. Meanwhile, I'm working on PR to fix the SSL handshake issue.

Javier Martín added a comment - 2016-07-13 09:41

The title of the issue is "multibranch-job deleted when bitbucket communication fails" and I think this is the same issue. The log says unauthorized request, but this is due to a communication problem between bitbucket and jira and therefore bitbucket-branch-source-plugin behaves in the same way as the open issue. The credentials are placed correctly and works fine almost always.

Maybe I neet to create another issue because this is only because SSL handshake, but seems pretty close to me, the problem is that when bitbucket communication fails jenkins deletes all multibranch-jobs and restarts build numbers when the computation folder runs again.

Javier Martín added a comment - 2016-07-13 09:41 The title of the issue is "multibranch-job deleted when bitbucket communication fails" and I think this is the same issue. The log says unauthorized request, but this is due to a communication problem between bitbucket and jira and therefore bitbucket-branch-source-plugin behaves in the same way as the open issue. The credentials are placed correctly and works fine almost always. Maybe I neet to create another issue because this is only because SSL handshake, but seems pretty close to me, the problem is that when bitbucket communication fails jenkins deletes all multibranch-jobs and restarts build numbers when the computation folder runs again.

Antonio Muñiz added a comment - 2016-07-13 10:36

Right, the fix is the same for any "connection issue".

Antonio Muñiz added a comment - 2016-07-13 10:36 Right, the fix is the same for any "connection issue".

Vladimir Rozhkov added a comment - 2016-07-13 11:04

javi_m: sorry then for misunderstanding.
Does simple retry fix the issue, how do you think?

Vladimir Rozhkov added a comment - 2016-07-13 11:04 javi_m : sorry then for misunderstanding. Does simple retry fix the issue, how do you think?

Antonio Muñiz added a comment - 2016-07-13 11:21 - edited

Already fixed in github-branch-source https://github.com/jenkinsci/github-branch-source-plugin/pull/57

The fix needed here is similar.

Antonio Muñiz added a comment - 2016-07-13 11:21 - edited Already fixed in github-branch-source https://github.com/jenkinsci/github-branch-source-plugin/pull/57 The fix needed here is similar.

Assignee:: Stephen Connolly

Reporter:: Asgeir Frimannsson

Votes:: 19 Vote for this issue

Watchers:: 24 Start watching this issue

Created:: 2016-06-17 03:23

Updated:: 2017-12-05 05:45

Resolved:: 2017-02-15 19:26

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Antonio Muñiz added a comment - 2016-06-20 07:51

Expand comment: Antonio Muñiz added a comment - 2016-06-20 07:51

Collapse comment: Vladimir Rozhkov added a comment - 2016-07-11 08:52

Expand comment: Vladimir Rozhkov added a comment - 2016-07-11 08:52

Collapse comment: Javier Martín added a comment - 2016-07-13 06:36, Edited by Javier Martín - 2016-07-13 06:37

Expand comment: Javier Martín added a comment - 2016-07-13 06:36, Edited by Javier Martín - 2016-07-13 06:37

Collapse comment: Vladimir Rozhkov added a comment - 2016-07-13 08:34

Expand comment: Vladimir Rozhkov added a comment - 2016-07-13 08:34

Collapse comment: Javier Martín added a comment - 2016-07-13 09:41

Expand comment: Javier Martín added a comment - 2016-07-13 09:41

Collapse comment: Antonio Muñiz added a comment - 2016-07-13 10:36

Expand comment: Antonio Muñiz added a comment - 2016-07-13 10:36

Collapse comment: Vladimir Rozhkov added a comment - 2016-07-13 11:04

Expand comment: Vladimir Rozhkov added a comment - 2016-07-13 11:04

Collapse comment: Antonio Muñiz added a comment - 2016-07-13 11:21, Edited by Antonio Muñiz - 2016-07-13 11:21

Expand comment: Antonio Muñiz added a comment - 2016-07-13 11:21, Edited by Antonio Muñiz - 2016-07-13 11:21

People

Dates