Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64711

Build status notification fails. But works without the Bitbucket server consumer configured?

      Recently our Bitbucket integration stopped working

      From Jenkins job we see this (some parts redacted):

      Posting build status of INPROGRESS to ****** for commit id [********] and ref 'refs/heads/production'
      Failed to post build status, additional information: The server failed to service request

      On Bitbucket side we see this (some parts redacted):

      2021-01-22 09:14:53,050 WARN [http-nio-7990-exec-10] ******* @P7WCJOx554x294432x0 1pokq4f 172.23.194.42,172.23.6.75,0:0:0:0:0:0:0:1 "POST /rest/api/1.0/projects/TST/repos/test/commits/3d3aaae526b9ff09f8478d6de13772a1e74624fa/builds HTTP/1.1" c.a.b.i.b.j.JenkinsBuildServerProvider$JenkinsPublicKeyCacheLoader Unable to load Jenkins public key for applicationId: ************
      java.io.IOException: No header found for http://*************

      2021-01-22 09:14:53,052 ERROR [http-nio-7990-exec-10] ******* @P7WCJOx554x294432x0 1pokq4f 172.23.194.42,172.23.6.75,0:0:0:0:0:0:0:1 "POST /rest/api/1.0/projects/TST/repos/test/commits/3d3aaae526b9ff09f8478d6de13772a1e74624fa/builds HTTP/1.1" c.a.s.i.r.e.DefaultUnhandledExceptionMapperHelper Unhandled exception while processing REST request: "POST /rest/api/1.0/projects/TST/repos/test/commits/3d3aaae526b9ff09f8478d6de13772a1e74624fa/builds HTTP/1.1"
      com.atlassian.cache.CacheException: Failed to fetch public key from Jenkins please ensure your application link is configured correctly

       Caused by: java.io.IOException: No header found for http://***********

      We were experimenting with setting everything up again from scratch. And when we did a test WITHOUT the Bitbucket Server consumer configured it does work ??

      We investigated the communication with wireshark, and managed to trace the cause back to some part of the http headers.

      Doing a manual call of the status rest post, WITHOUT the BBS-Signature and BBS-Signature-Algorithm does work. But the moment these 2 headers are added the error occurs.

       

          [JENKINS-64711] Build status notification fails. But works without the Bitbucket server consumer configured?

          Martin Henschke added a comment - - edited

          Hi Koen Dierckx, thanks for the report.

          From the symptoms you described, this actually appears to be a bug in Bitbucket Server rather than the plugin specifically- for that reason I've raised a tracking ticket in our public bug tracker here: https://jira.atlassian.com/browse/BSERV-12886

          To properly triage and diagnose this issue, can you please provide more detailed replication details in the tracking ticket? Specifically the process you went through setting up and app-linking Bitbucket and Jenkins. If after the bug is investigated it's found to be on the Bitbucket side, we'll close this ticket.

          Thank you!

          Martin Henschke added a comment - - edited Hi Koen Dierckx, thanks for the report. From the symptoms you described, this actually appears to be a bug in Bitbucket Server rather than the plugin specifically- for that reason I've raised a tracking ticket in our public bug tracker here: https://jira.atlassian.com/browse/BSERV-12886 To properly triage and diagnose this issue, can you please provide more detailed replication details in the tracking ticket? Specifically the process you went through setting up and app-linking Bitbucket and Jenkins. If after the bug is investigated it's found to be on the Bitbucket side, we'll close this ticket. Thank you!

          Koen Dierckx added a comment -

          We've done some further testing, and managed to find a workaround. Disable Kerberos SSO

           

          The issues seem to be caused by / related to this plugin: https://plugins.jenkins.io/kerberos-sso/

          I think it messes with the consumer setup and communication.

          If we enable the Kerberos SSO plugin, everything works initially, but then starts failing after a while. Making it hard to debug.

           

          From the plugin page:

          The authentication can be bypassed for specific request by setting a Bypass-Kerberos header in the request. It doesn't matter which value it has, the user will be authenticated as anonymous.

           

          Would it be possible to configure the Bitbucket Server requests,  so we could add this header there ?

           

          Koen Dierckx added a comment - We've done some further testing, and managed to find a workaround. Disable Kerberos SSO   The issues seem to be caused by / related to this plugin: https://plugins.jenkins.io/kerberos-sso/ I think it messes with the consumer setup and communication. If we enable the Kerberos SSO plugin, everything works initially, but then starts failing after a while. Making it hard to debug.   From the plugin page: The authentication can be bypassed for specific request by setting a Bypass-Kerberos header in the request. It doesn't matter which value it has, the user will be authenticated as anonymous.   Would it be possible to configure the Bitbucket Server requests,  so we could add this header there ?  

          Spanaird added a comment -

          kndx how did you come to a conclusion that this problem was caused by Kerberos SSO? We also have a similar problem, but couldn't get our head around on what's causing it. We don't have Kerberos SSO plugin installed, but use Azure AD plugin, and frankly I am not sure if Azure AD plugin is adding header BBS-Signature and BBS-Signature-Algorithm to the call.

          Spanaird added a comment - kndx how did you come to a conclusion that this problem was caused by Kerberos SSO? We also have a similar problem, but couldn't get our head around on what's causing it. We don't have Kerberos SSO plugin installed, but use Azure AD plugin, and frankly I am not sure if Azure AD plugin is adding header BBS-Signature and BBS-Signature-Algorithm to the call.

          For posterity's sake, I just wanted to briefly mention here that I went into some more detail on why this issue can happen, and some suggestions on how to potentially fix this issue over at the BSERV ticket linked earlier by mhenschke_atlassian.

          As mentioned there, there's some Bitbucket tweaks that we're making to help avoid this issue happening in the first place. But for those hitting this problem, I recommend running through some of the potential causes I've listed on that BSERV ticket to see if those will help.

          Evan Slaughter added a comment - For posterity's sake, I just wanted to briefly mention here that I went into some more detail on why this issue can happen, and some suggestions on how to potentially fix this issue over at the BSERV ticket linked earlier by mhenschke_atlassian . As mentioned there, there's some Bitbucket tweaks that we're making to help avoid this issue happening in the first place. But for those hitting this problem, I recommend running through some of the potential causes I've listed on that BSERV ticket to see if those will help.

            mhenschke_atlassian Martin Henschke
            kndx Koen Dierckx
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: