It seems this JENKINS-47891 is about Jenkins receiving notifications from Bitbucket Server built-in webhooks, and JENKINS-46946 is about Jenkins configuring Bitbucket Server to send those notifications.

Is there any workaround?

paderka Yes, disable the built-in webhooks and use the add-in bitbucket-webhooks. Its configuration page in the repo settings is called "Post webhooks".

estyrke Thank you for your reply, but what is the right URL for that webhook if

<JENKINS_URL>/bitbucket-scmsource-hook/notify

doesn't work?

That is what I use, it should be correct. Possibly, you have a broken version of the Bitbucket branch source plugin in Jenkins. I seem to recall that the webhook triggering was broken in one of the recent versions. I'm running 2.2.5 here.

I'm using Bitbucket branch source plugin version 2.2.7 and Jenkins 2.92

I'm also using '<JENKINS_URL>/bitbucket-scmsource-hook/notify/' successfully with:
Jenkins 2.85 &  cloudbees-bitbucket-branch-source 2.2.7

Even with 2.2.5 version I got exception below.

java.lang.Exception: X-Event-Key HTTP header not found
	at org.kohsuke.stapler.HttpResponses.error(HttpResponses.java:83)
	at com.cloudbees.jenkins.plugins.bitbucket.hooks.BitbucketSCMSourcePushHookReceiver.doNotify(BitbucketSCMSourcePushHookReceiver.java:94)

Are you using this Bitbucket add-on - https://marketplace.atlassian.com/plugins/nl.topicus.bitbucket.bitbucket-webhooks/server/overview  ?

Ah, I've used https://marketplace.atlassian.com/plugins/com.atlassian.stash.plugin.stash-web-post-receive-hooks-plugin/server/overview Thanks for your advice. I'm going to try it!

paderka I'm having the exact same issue and I exclusively use the plugin namraclik mentioned.  A bug must have been introduced.  I've had this working for the better part of 8 months and it stopped with some of the latest Jenkins/Bitbucket plugin updates.

this really is becoming a nightmare. I'm trying to get this to work with jenkins 2.107.1 and Bitbucket 5.8.0 which has webhooks built-in. And now I should install another webhook plugin to get it to work in combination with jenkins and the latest Bitbucket branch source plugin ...

But it seems to work with Eernies plugin. Hopefully it stays free.

Still no feedback on https://github.com/jenkinsci/bitbucket-branch-source-plugin/compare/master...kwin:bbserver-use-native-webhooks ?

We unfortunately do not have the liberty to install third-party plugins on Bitbucket Server, so it would be really great to use (and manage) the native webhooks.

Any update here?
I get the error, even using the plugin: https://marketplace.atlassian.com/plugins/nl.topicus.bitbucket.bitbucket-webhooks/server/overview
I use bitbucket 5.9.1 and jenkins 2.107.2

java.lang.Exception: X-Event-Key HTTP header not found
	at org.kohsuke.stapler.HttpResponses.error(HttpResponses.java:83)
	at com.cloudbees.jenkins.plugins.bitbucket.hooks.BitbucketSCMSourcePushHookReceiver.doNotify(BitbucketSCMSourcePushHookReceiver.java:94)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:184)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:117)
	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:129)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
	at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:225)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:105)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:564)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
	at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)
	at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199)
	at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Hey, unfortunately, I'm not a Jenkins plugin developer nor a Java developer, so I really don't have the necessary tools at hand. But I'll try to contribute to resolving this issue anyways. 

Currently, I believe supported hook types are handled here: https://github.com/jenkinsci/bitbucket-branch-source-plugin/blob/master/src/main/java/com/cloudbees/jenkins/plugins/bitbucket/hooks/HookEventType.java

Event types seem unchanged and thus correct for Bitbucket Cloud (see https://confluence.atlassian.com/bitbucket/event-payloads-740262817.html). However, they seem to have changed for Bitbucket Server.

From https://confluence.atlassian.com/bitbucketserver/event-payload-938025882.html I conclude the following event type names (for Bitbucket Server):

• ``repo:push`` becomes ``repo:refs_changed``
• ``pullrequest:created`` becomes ``pr:opened``
• ``pullrequest:updated`` becomes ``pr:opened`` (assuming it corresponds to Atlassian's understanding of 'Modifying' a PR)
• ``pullrequest:fulfilled`` becomes ``pr:merged``
• ``pullrequest:rejected`` becomes ``pr:declined``

If I interpret the code correctly, one can now change behaviour in https://github.com/jenkinsci/bitbucket-branch-source-plugin/blob/master/src/main/java/com/cloudbees/jenkins/plugins/bitbucket/hooks/PullRequestHookProcessor.java in order to handle Bitbucket Cloud and Bitbuket Server events differently. 

Unfortunately, it looks like the POST request body has changed considerably as well, so I guess existing Interfaces (mainly the two ending on *Event.java in https://github.com/jenkinsci/bitbucket-branch-source-plugin/tree/master/src/main/java/com/cloudbees/jenkins/plugins/bitbucket/api) need to be adapted. 

Anyone knows, whether Atlassian is transiting towards the format they're using for Bitbucket Server? It looks cleaner to me. 

Hope this helps in some way.

PS: I have that same issue and I believe that it would make things simpler and cleaner if this plugin would take over changes that were applied to Bitbucket Server. 

Ok, let me try to put a bit of context here:

Why are we using a "custom" BB Server plugin for webhooks?

Because when bb-branch-source development started there were no built-in webhooks support in BB Server. However BB Cloud already had it. So the most sensible solution by that time was to use a plugin in BB Server to somehow "mimic" webhook messages from BB Cloud. That way the same code could handle webhooks from both BB Server and BB Cloud.

The "official" plugin, as many of you already said, is: https://marketplace.atlassian.com/plugins/nl.topicus.bitbucket.bitbucket-webhooks/server/overview - which is OSS and the code is on GitHub: https://github.com/Eernie/bitbucket-webhooks-plugin

BB Server comes with built-in webhooks now

Yes. And that's causing confusion, sorry for that. I'll try to write fresh and clear docs on this in the short term.
In the long term we have to decide what to do about this. Options are:

1. Support both webhook types (plugin generated and built-in)
2. Support built-in webhooks only
3. Support plugin webhook only

Each options has pros and cons:

1. Supporting both has significant maintenance overhead
2. Supporting built-in only will probably break most of existent setups, and we would lose any control over the evolution of the webhooks source (the current plugin is OSS, actively maintained and accepting contributions).
3. Supporting plugin webhooks only, it seems to be producing confusion and installing a plugin is not always well-seen (specially on rigid BB Server installations) when there is a built-in alternative.

IMO, until BB Server webhooks are "compatible" with BB Cloud webhooks payload we should stick to the "custom" plugin approach.

IMO, until BB Server webhooks are "compatible" with BB Cloud webhooks payload we should stick to the "custom" plugin approach.

This seems a very naive approach. Who says Atlassian will make them "compatible"? Or can you point us to some information that they will do the above at some point?
BB Server webhooks is newer code than the BB cloud webhooks, so they could've just sticked to the more used one.
If anything, BB cloud would move to the same code as BB server uses - but that would still be an assumption/wild guess.

IMHO it's better to support the official webhooks provided by Atlassian.
This will make the most sense to all end-users and foster greater adoption of Jenkins and the bitbucket-branch-source plugin in general.

In this way you keep support the "custom plugin" method too, as you say that uses the same payload as BB cloud.
As you said here:

So the most sensible solution by that time was to use a plugin in BB Server to somehow "mimic" webhook messages from BB Cloud

twz123 would actually be great if you would open a PR cause I definitely have some input  the changes look good! Few minor tweaks I would do.

For now please stop misusing jelly enums. You can do this a lot better with a field and descriptor with doFill.

See here for a good example use: 

https://github.com/jenkinsci/violation-comments-to-stash-plugin/blob/06424100d8fe25c1dd15c1ff7476ac2d735694f1/src/main/java/org/jenkinsci/plugins/jvctb/config/ViolationConfig.java#L120-L127

https://github.com/jenkinsci/violation-comments-to-stash-plugin/blob/06424100d8fe25c1dd15c1ff7476ac2d735694f1/src/main/resources/org/jenkinsci/plugins/jvctb/config/ViolationConfig/config.jelly#L5-L7

Pull request #138 opened. Looking forward to get your feedback casz

FYI, my company uses Bitbucket Server. With several teams asking about webhook support for Jenkins and speculation running wild, our Engineering Tools manager sought an official answer from Atlassian. Here is his report:

I've been in contact with Atlassian Premier Support and the lead developers of Bitbucket, who reviewed JENKINS-47891 at my request. They confirm that there are differences in the web hook behaviors between Server and Cloud. This is because although the two products share a name (Bitbucket) and general purpose (remote Git repository management), they serve different customer needs and their code bases are distinct. They said that making Bitbucket Server and Bitbucket Cloud to be exactly the same is not currently planned. From their perspective, Bitbucket Server web hooks are working as designed and customizing the functionality to meet the specific needs of individual third party applications is not on the roadmap.

The PR has now been completed, merged and made its way into the new release of the plugin.

This worked nicely for our Multi-Branch Pipelines generated via Bitbucket team folder "jobs" that are the one item we configure "manually" to get our repos and branches discovered to spawn the pipeline jobs. A rescan of the organization "job" in Jenkins (after setting the global config to use the "Native" webhook implementation in the connector for our BB server) made configurations in all repos that had a Jenkinsfile in at least one branch, but no configurations in repos that had no such branches (corollary: to initially introduce pipeline support in codebase of a component, one has to rescan that organization or MBP job).

One caveat I hit in our local setup (constrained by a web of corporate IT policies) was that our lab's Jenkins server uses a self-signed certificate which the centrally managed corporate Bitbucket Server instance reasonably does not trust, so it can not HTTP POST to. Our other consumers (developers' browsers and internal build farm) have no issues with trusting this certificate.

I had to change the (apache httpd) reverse proxy handling to let it through as plaintext:

RedirectMatch 302 ^/?((?!neo.*$|bitbucket-scmsource-hook$|bitbucket-scmsource-hook/.*$).*)$ https://jenkins.domain.com/$1
ProxyPass         /bitbucket-scmsource-hook  http://jenkins.domain.com:8080/bitbucket-scmsource-hook/ nocanon connectiontimeout=3600 timeout=3600 retry=0

...and had to change the Jenkins global config to announce the server URL as the "http" (non-"https") version. UPDATE: Added RFE JENKINS-55423 for a better solution to this sort of problem.

Thanks to everybody involved in making this feature!

I can't get native hooks to be auto registered, is there something special needed compared to the plugin?

Plugin has always worked fine but does not (yet) support Bitbucket Server 6.0.0

After adding a logger in Jenkins for com.cloudbees.jenkins.plugins.bitbucket I figured out that the configured user did not have Admin access for those repositories/projects in Bitbucket. This was not required with the plugin and is not easy to figure out. It would be nice to have some warning message in the scan log for this case. 

I'm on bitbucket-branch-source 2.4.1 and Bitbucket Server 5.16. Native Webhooks does not work for Pull requests here:

tkleiber wrote:

Native Webhooks does not work for Pull requests here:

In your screenshot, the webhook URI ends with "/bitbucket-hook/". That path seems to be handled by Bitbucket plugin and Bitbucket Push and Pull Request plugin, which do not support Bitbucket Server native webhooks. Bitbucket Branch Source plugin instead uses a URI like "https://jenkins.example/bitbucket-scmsource-hook/notify?server_url=http://bitbucket.example:7990", where the server_url parameter must match what you have in "Bitbucket Endpoints" in the Jenkins configuration.

Thank you, that was the missing piece, not it works!