Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65624

Webhook failures after upgrading jetty to 9.4.39.v20210325 in 2.277.3

    • 2.290, 2.289.2

      After upgrading Jenkins from 2.249.2 to 2.277.3, webhooks being fired from github enterprise by way of the ghrpb started randomly failing with errors indicating that the payload was missing from the request. Messages were similar to below:

      SEVERE: Request doesn't contain payload. You are sending url encoded request, so you should pass github payload through 'payload' request parameter

      We saw similarly worded errors coming from the Github plugin as well.

      I believe I've traced the failure to this issue in Jetty which looks like it's been corrected with this PR

      The underlying error was exposed after enabling Jenkins logging on jetty:

       

      May 12, 2021 11:34:33 AM FINE org.eclipse.jetty.server.HttpConnection fillRequestBuffer
      javax.net.ssl.SSLHandshakeException: Encrypted buffer max length exceeded

      Which further down in the stack indicated it was bubbling up from ghprb:

      org.jenkinsci.plugins.ghprb.GhprbRootAction.extractRequestBody(GhprbRootAction.java:286)

       

          [JENKINS-65624] Webhook failures after upgrading jetty to 9.4.39.v20210325 in 2.277.3

          wayne goyer created issue -
          wayne goyer made changes -
          Description Original: After upgrading Jenkins from 2.249.2 to 2.277.3, webhooks being fired from github enterprise by way of the ghrpb started randomly failing with errors indicating that the payload was missing from the request. Messages were similar to below:

           
          {code:java}
          SEVERE: Request doesn't contain payload. You are sending url encoded request, so you should pass github payload through 'payload' request parameter{code}
           

          We saw similarly worded errors coming from the Github plugin as well.

          I believe I've traced the failure to [this issue|https://github.com/eclipse/jetty.project/issues/6121] in Jetty which looks like it's been corrected with [this PR|https://github.com/eclipse/jetty.project/pull/6142]

          The underlying error was exposed after enabling Jenkins logging on jetty:

           
          {noformat}
          May 12, 2021 11:34:33 AM FINE org.eclipse.jetty.server.HttpConnection fillRequestBuffer
          javax.net.ssl.SSLHandshakeException: Encrypted buffer max length exceeded{noformat}
          Which further down in the stack indicated it was bubbling up from ghprb:
          {noformat}
          org.jenkinsci.plugins.ghprb.GhprbRootAction.extractRequestBody(GhprbRootAction.java:286){noformat}
           
          New: After upgrading Jenkins from 2.249.2 to 2.277.3, webhooks being fired from github enterprise by way of the ghrpb started randomly failing with errors indicating that the payload was missing from the request. Messages were similar to below:
          {noformat}
          SEVERE: Request doesn't contain payload. You are sending url encoded request, so you should pass github payload through 'payload' request parameter{noformat}
          We saw similarly worded errors coming from the Github plugin as well.

          I believe I've traced the failure to [this issue|https://github.com/eclipse/jetty.project/issues/6121] in Jetty which looks like it's been corrected with [this PR|https://github.com/eclipse/jetty.project/pull/6142]

          The underlying error was exposed after enabling Jenkins logging on jetty:

           
          {noformat}
          May 12, 2021 11:34:33 AM FINE org.eclipse.jetty.server.HttpConnection fillRequestBuffer
          javax.net.ssl.SSLHandshakeException: Encrypted buffer max length exceeded{noformat}
          Which further down in the stack indicated it was bubbling up from ghprb:
          {noformat}
          org.jenkinsci.plugins.ghprb.GhprbRootAction.extractRequestBody(GhprbRootAction.java:286){noformat}
           
          Oleg Nenashev made changes -
          Component/s New: winstone-jetty [ 20645 ]
          Oleg Nenashev made changes -
          Labels Original: webhook winstone New: regression webhook winstone

          Oleg Nenashev added a comment -

          CC olamy

           

          Oleg Nenashev added a comment - CC olamy  
          Oleg Nenashev made changes -
          Assignee New: Olivier Lamy [ olamy ]

          Olivier Lamy added a comment -

          _wayne thanks for the report. this should be fixed with Jetty 9.4.40 which is coming with winstone 5.17 from jenkins-2.282 

          Olivier Lamy added a comment - _wayne  thanks for the report. this should be fixed with Jetty 9.4.40 which is coming with winstone 5.17 from  jenkins-2.282 

          Olivier Lamy added a comment -

          let us know if upgrading fix your issue

          Olivier Lamy added a comment - let us know if upgrading fix your issue

          wayne goyer added a comment -

          Appreciate the info olamy !

          From release list it looks like 9.4.40 is available starting in Jenkins release 2.290. I'll test it in that release when I get a chance and add the results here. If this isn't what you had in mind lemme know.

          wayne goyer added a comment - Appreciate the info  olamy  ! From release list it looks like 9.4.40 is available starting in Jenkins release 2.290. I'll test it in that release when I get a chance and add the results here. If this isn't what you had in mind lemme know.
          Tim Jacomb made changes -
          Labels Original: regression webhook winstone New: lts-candidate regression webhook winstone

            olamy Olivier Lamy
            _wayne wayne goyer
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: