Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65624

Webhook failures after upgrading jetty to 9.4.39.v20210325 in 2.277.3

    XMLWordPrintable

Details

    • 2.290, 2.289.2

    Description

      After upgrading Jenkins from 2.249.2 to 2.277.3, webhooks being fired from github enterprise by way of the ghrpb started randomly failing with errors indicating that the payload was missing from the request. Messages were similar to below:

      SEVERE: Request doesn't contain payload. You are sending url encoded request, so you should pass github payload through 'payload' request parameter

      We saw similarly worded errors coming from the Github plugin as well.

      I believe I've traced the failure to this issue in Jetty which looks like it's been corrected with this PR

      The underlying error was exposed after enabling Jenkins logging on jetty:

       

      May 12, 2021 11:34:33 AM FINE org.eclipse.jetty.server.HttpConnection fillRequestBuffer
      javax.net.ssl.SSLHandshakeException: Encrypted buffer max length exceeded

      Which further down in the stack indicated it was bubbling up from ghprb:

      org.jenkinsci.plugins.ghprb.GhprbRootAction.extractRequestBody(GhprbRootAction.java:286)

       

      Attachments

        Issue Links

          Activity

            oleg_nenashev Oleg Nenashev added a comment -

            CC olamy

             

            oleg_nenashev Oleg Nenashev added a comment - CC olamy  
            olamy Olivier Lamy added a comment -

            _wayne thanks for the report. this should be fixed with Jetty 9.4.40 which is coming with winstone 5.17 from jenkins-2.282 

            olamy Olivier Lamy added a comment - _wayne  thanks for the report. this should be fixed with Jetty 9.4.40 which is coming with winstone 5.17 from  jenkins-2.282 
            olamy Olivier Lamy added a comment -

            let us know if upgrading fix your issue

            olamy Olivier Lamy added a comment - let us know if upgrading fix your issue
            _wayne wayne goyer added a comment -

            Appreciate the info olamy !

            From release list it looks like 9.4.40 is available starting in Jenkins release 2.290. I'll test it in that release when I get a chance and add the results here. If this isn't what you had in mind lemme know.

            _wayne wayne goyer added a comment - Appreciate the info  olamy  ! From release list it looks like 9.4.40 is available starting in Jenkins release 2.290. I'll test it in that release when I get a chance and add the results here. If this isn't what you had in mind lemme know.
            evanbluhm Evan Bluhm added a comment -

             We've been running into the same bug in 2.277.3 (as have some other users in https://issues.jenkins.io/browse/JENKINS-65581). I can confirm that downgrading to 2.277.2 or upgrading to 2.290 both successfully resolve the problem. This does mean we're currently blocked from receiving any LTS security updates until the Winstone updates are backported or included in a new LTS version (it looks like the next RC is 2.289, so we might be waiting a long time).

            evanbluhm Evan Bluhm added a comment -  We've been running into the same bug in 2.277.3 (as have some other users in https://issues.jenkins.io/browse/JENKINS-65581 ). I can confirm that downgrading to 2.277.2 or upgrading to 2.290 both successfully resolve the problem. This does mean we're currently blocked from receiving any LTS security updates until the Winstone updates are backported or included in a new LTS version (it looks like the next RC is 2.289, so we might be waiting a long time).

            timja how do you feel about getting this one backported to 2.289.2?

            batmat Baptiste Mathus added a comment - timja  how do you feel about getting this one backported to 2.289.2?

            I realize the related PR was not linked here, it's https://github.com/jenkinsci/jenkins/pull/5437

            batmat Baptiste Mathus added a comment - I realize the related PR was not linked here, it's  https://github.com/jenkinsci/jenkins/pull/5437

            People

              olamy Olivier Lamy
              _wayne wayne goyer
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: