Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60409

Form Submission Length Cap

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      Jenkins 2.204.5, Jenkins 2.224, Winstone 5.4.3, Winstone 5.9

      Description

      *Jenkins LTS Notice*: Jenkins LTS 2.204.3 and 2.204.4 are also affected due to the Winstone upgrade which was introduced as a part of the JENKINS-57888 fix backporting. Please see https://groups.google.com/forum/#!topic/jenkinsci-dev/M_RtDuDXtbU for the discussion and retrospective

      In Jenkins Version 2.205, PR #4339 moved the cloud configuration from Configure System into is own configuration form on the Manage Nodes page. There is a cap to the length of this form (200000) and prevents me from adding additional docker clouds into the settings.

      java.lang.IllegalStateException: Form is larger than max length 200000
      	at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:562)
      	at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:519)
      	at org.eclipse.jetty.server.Request.getParameters(Request.java:430)
      Caused: org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
      	at org.eclipse.jetty.server.Request.getParameters(Request.java:434)
      	at org.eclipse.jetty.server.Request.getParameterNames(Request.java:1077)
      	at hudson.security.csrf.CrumbFilter.extractCrumbFromRequest(CrumbFilter.java:112)
      	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:81)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
      	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
      	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
      	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
      	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1592)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
      	at org.eclipse.jetty.server.Server.handle(Server.java:500)
      	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
      	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:562)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
      	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
      	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
      	at java.lang.Thread.run(Thread.java:748)
      

        Attachments

          Issue Links

            Activity

            mastershihochief Joshua Hunter created issue -
            mahmoud_ksemtini Mahmoud Ksemtini made changes -
            Field Original Value New Value
            Attachment Screenshot 2019-12-10 at 15.27.37.png [ 49732 ]
            Hide
            mahmoud_ksemtini Mahmoud Ksemtini added a comment - - edited

            I get the same error when trying to replay a job from the UI.  (Jenkins Version 2.205)

            Show
            mahmoud_ksemtini Mahmoud Ksemtini added a comment - - edited I get the same error when trying to replay a job from the UI.  (Jenkins Version 2.205)
            Hide
            dean Dean Smith added a comment -

            We have also been experiencing the same error when replaying builds since 2.205.

            It seems to only be for jobs with a large Jenkinsfile (84,100 characters/ ~1800 lines) but not jobs with smaller Jenkinsfiles, although even the large ones are below 200,000 characters.

            Show
            dean Dean Smith added a comment - We have also been experiencing the same error when replaying builds since 2.205. It seems to only be for jobs with a large Jenkinsfile (84,100 characters/ ~1800 lines) but not jobs with smaller Jenkinsfiles, although even the large ones are below 200,000 characters.
            Hide
            pedkoz Pedro Cuello Orozco added a comment -

            Hi guys,

            I am facing the same issue when I try to give access to a new user into my permission matrix Role. I guess it happens in cloud version, but I am facing it in the not cloud version 2.207. I am forced to get back to my previous version 2.196 

             

            Regards,

            Show
            pedkoz Pedro Cuello Orozco added a comment - Hi guys, I am facing the same issue when I try to give access to a new user into my permission matrix Role. I guess it happens in cloud version, but I am facing it in the not cloud version 2.207. I am forced to get back to my previous version 2.196    Regards,
            Hide
            jwnmulder jwnmulder added a comment -

            Running jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=1000000" did resolve the issue for us

            Show
            jwnmulder jwnmulder added a comment - Running jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=1000000" did resolve the issue for us
            Hide
            oleg_nenashev Oleg Nenashev added a comment -
            Show
            oleg_nenashev Oleg Nenashev added a comment - Daniel Beck
            Hide
            danielbeck Daniel Beck added a comment -

            Oleg Nenashev

            A comment from a week ago stated:

            We have also been experiencing the same error when replaying builds since 2.205.

            Clearly this isn't my PR that only moved some form fields in the global configs around.


            I suggest you look at the Winstone/Jetty update. This looks interesting:

             + 3856 Different behaviour with maxFormContentSize=0 if Content-Length header is present/missing

            Show
            danielbeck Daniel Beck added a comment - Oleg Nenashev A comment from a week ago stated: We have also been experiencing the same error when replaying builds since 2.205. Clearly this isn't my PR that only moved some form fields in the global configs around. I suggest you look at the Winstone/Jetty update. This looks interesting:  + 3856 Different behaviour with maxFormContentSize=0 if Content-Length header is present/missing
            Hide
            danielbeck Daniel Beck added a comment - - edited

            I just backported 4339 onto 2.204 and the form works like a charm with the same content that makes 2.205 fail. Whatever is going on here has absolutely nothing to do with that PR.

            Show
            danielbeck Daniel Beck added a comment - - edited I just backported 4339 onto 2.204 and the form works like a charm with the same content that makes 2.205 fail. Whatever is going on here has absolutely nothing to do with that PR.
            Hide
            dean Dean Smith added a comment -

            Running jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=1000000" did resolve the issue for us

            This resolved the issue for us as well. 

            I agree that it looks to be nothing to do with any core Jenkins changes but instead Winstone/Jetty library changes that curtailed this limit.

            Show
            dean Dean Smith added a comment - Running jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=1000000" did resolve the issue for us This resolved the issue for us as well.  I agree that it looks to be nothing to do with any core Jenkins changes but instead Winstone/Jetty library changes that curtailed this limit.
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-61226 [ JENKINS-61226 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-60984 [ JENKINS-60984 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-60968 [ JENKINS-60968 ]
            oleg_nenashev Oleg Nenashev made changes -
            Component/s winstone-jetty [ 20645 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            We just received a bunch of new reports, linked them to this ticket.

            Olivier Lamy any hints on the Jetty side? "+ 3856 Different behaviour with maxFormContentSize=0 if Content-Length header is present/missing" mentioned by Daniel Beck makes me think we need to alter the defaults in Winstone to restore the original behavior

            Show
            oleg_nenashev Oleg Nenashev added a comment - We just received a bunch of new reports, linked them to this ticket. Olivier Lamy any hints on the Jetty side? "+ 3856 Different behaviour with maxFormContentSize=0 if Content-Length header is present/missing" mentioned by Daniel Beck makes me think we need to alter the defaults in Winstone to restore the original behavior
            Hide
            olamy Olivier Lamy added a comment -

            I guess we need to update  the default value. maybe something around 500000? I reckon it's a bit sensible to change that. but a default with no size limit is weird.....

            Show
            olamy Olivier Lamy added a comment - I guess we need to update  the default value. maybe something around 500000? I reckon it's a bit sensible to change that. but a default with no size limit is weird.....
            Hide
            danielbeck Daniel Beck added a comment -

            Would go an order of magnitude up or so to give us breathing room. Given the UI of Jenkins, long form submissions are expected (and 4339 actually helps with that instead of causing the problem…).

            I wonder whether there's a good location to document the system property and the need to set it in instances with very complex configurations.

            Show
            danielbeck Daniel Beck added a comment - Would go an order of magnitude up or so to give us breathing room. Given the UI of Jenkins, long form submissions are expected (and 4339 actually helps with that instead of causing the problem…). I wonder whether there's a good location to document the system property and the need to set it in instances with very complex configurations.
            oleg_nenashev Oleg Nenashev made changes -
            Labels lts-candidate
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Marked as an lts-candidate. Taking the number of reports, we would be interested to backport a fix to the next LTS .1 if we have one

            Show
            oleg_nenashev Oleg Nenashev added a comment - Marked as an lts-candidate. Taking the number of reports, we would be interested to backport a fix to the next LTS .1 if we have one
            Hide
            brianodriscoll brian odriscoll added a comment -

            Hi Oleg,

            So we used the workaround advised in JENKINS-60409.
            First our senior picked it up to 400k but it didn't fix the issue.- (still too small for the POST size we are doing), then picked it up to 1M and it began to work properly.
            Kind Regards
            Brian

            Show
            brianodriscoll brian odriscoll added a comment - Hi Oleg, So we used the workaround advised in JENKINS-60409 . First our senior picked it up to 400k but it didn't fix the issue.- (still too small for the POST size we are doing), then picked it up to 1M and it began to work properly. Kind Regards Brian
            Hide
            raihaan Raihaan Shouhell added a comment -

            https://github.com/eclipse/jetty.project/blob/65a22e5e809ecbd20b3475cd8644fdf7aa5e8490/jetty-server/src/main/java/org/eclipse/jetty/server/handler/ContextHandler.java#L148

             

            This looks like where the default size is set. Even if we do bump it, we would require that jetty releases and we then update winstone.

            Show
            raihaan Raihaan Shouhell added a comment - https://github.com/eclipse/jetty.project/blob/65a22e5e809ecbd20b3475cd8644fdf7aa5e8490/jetty-server/src/main/java/org/eclipse/jetty/server/handler/ContextHandler.java#L148   This looks like where the default size is set. Even if we do bump it, we would require that jetty releases and we then update winstone.
            Hide
            olamy Olivier Lamy added a comment -

            Raihaan Shouhell why? we do not need a Jetty release. Jetty has his own default (a bit low to prevent DOS attack and this probably will not be changed in Jetty code).

            But we can our own higher default here in Winstone.

            Show
            olamy Olivier Lamy added a comment - Raihaan Shouhell why? we do not need a Jetty release. Jetty has his own default (a bit low to prevent DOS attack and this probably will not be changed in Jetty code). But we can our own higher default here in Winstone.
            Hide
            raihaan Raihaan Shouhell added a comment -

            Olivier Lamy you're right. I did not know we could override that bit. Definitely should be in winstone

            Show
            raihaan Raihaan Shouhell added a comment - Olivier Lamy you're right. I did not know we could override that bit. Definitely should be in winstone
            danielbeck Daniel Beck made changes -
            Summary Cloud Configuration Length Cap Form Submission Length Cap
            danielbeck Daniel Beck made changes -
            Labels lts-candidate lts-candidate regression
            slide_o_mix Alex Earl made changes -
            Link This issue is blocked by JENKINS-60984 [ JENKINS-60984 ]
            slide_o_mix Alex Earl made changes -
            Link This issue is duplicated by JENKINS-61307 [ JENKINS-61307 ]
            Hide
            jglick Jesse Glick added a comment -

            So Winstone does attempt to set this parameter to be unlimited; see JENKINS-20327 and winstone #20. Perhaps something about jetty.project #3899 broke this hack, though it is not obvious what: the same key is being used, and in the same way from what I can tell.

            (Note that maven-hpi-plugin, jenkins-test-harness, and jenkinsfile-runner all attempt to set this too but with apparently obsolete property names. The comment in RunMojo is obsolete as well: we no longer accept client-side update center JSON.)

            Show
            jglick Jesse Glick added a comment - So Winstone does attempt to set this parameter to be unlimited; see JENKINS-20327 and winstone #20 . Perhaps something about jetty.project #3899 broke this hack, though it is not obvious what: the same key is being used, and in the same way from what I can tell. (Note that maven-hpi-plugin , jenkins-test-harness , and jenkinsfile-runner all attempt to set this too but with apparently obsolete property names. The comment in RunMojo is obsolete as well: we no longer accept client-side update center JSON.)
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-20327 [ JENKINS-20327 ]
            jglick Jesse Glick made changes -
            Assignee Jesse Glick [ jglick ]
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            jglick Jesse Glick added a comment -

            Working on a fix in winstone #95.

            Show
            jglick Jesse Glick added a comment - Working on a fix in winstone #95.
            jglick Jesse Glick made changes -
            Remote Link This issue links to "winstone #95 (Web Link)" [ 24617 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            olamy Olivier Lamy made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            Hide
            alexhraber Alex Raber added a comment - - edited

            Jesse Glick fyi this is happening in 2.204.4 LTS as well, can we get a patch bump to 2.204.5 with your winstone fix please.

            Show
            alexhraber Alex Raber added a comment - - edited Jesse Glick fyi this is happening in 2.204.4 LTS as well, can we get a patch bump to 2.204.5 with your winstone fix please.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Winstone 5.9 with fixes was released: https://github.com/jenkinsci/winstone/releases/tag/winstone-5.9

            Jenkins Core update pull request: https://github.com/jenkinsci/jenkins/pull/4542

            Show
            oleg_nenashev Oleg Nenashev added a comment - Winstone 5.9 with fixes was released:  https://github.com/jenkinsci/winstone/releases/tag/winstone-5.9 Jenkins Core update pull request:  https://github.com/jenkinsci/jenkins/pull/4542
            oleg_nenashev Oleg Nenashev made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            oleg_nenashev Oleg Nenashev made changes -
            Environment Jenkins 2.205+, Jenkins LTS 2.204.3 and 2.204.4
            Winstone 5.5 to 5.8
            oleg_nenashev Oleg Nenashev made changes -
            Description In Jenkins Version 2.205, PR #4339 moved the cloud configuration from Configure System into is own configuration form on the Manage Nodes page. There is a cap to the length of this form (200000) and prevents me from adding additional docker clouds into the settings.
            {code:java}
            java.lang.IllegalStateException: Form is larger than max length 200000
            at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:562)
            at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:519)
            at org.eclipse.jetty.server.Request.getParameters(Request.java:430)
            Caused: org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
            at org.eclipse.jetty.server.Request.getParameters(Request.java:434)
            at org.eclipse.jetty.server.Request.getParameterNames(Request.java:1077)
            at hudson.security.csrf.CrumbFilter.extractCrumbFromRequest(CrumbFilter.java:112)
            at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:81)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
            at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
            at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1592)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
            at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
            at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
            at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.Server.handle(Server.java:500)
            at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
            at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:562)
            at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
            at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
            at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
            at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
            at java.lang.Thread.run(Thread.java:748)
            {code}
            **Jenkins LTS Notice**: Jenkins LTS 2.204.3 and 2.204.4 are also affected due to the Winstone upgrade which was backported as a part of the JENKINS-57888 fix backporting. Please see [https://groups.google.com/forum/#!topic/jenkinsci-dev/M_RtDuDXtbU] for the discussion and retrospective

            In Jenkins Version 2.205, PR #4339 moved the cloud configuration from Configure System into is own configuration form on the Manage Nodes page. There is a cap to the length of this form (200000) and prevents me from adding additional docker clouds into the settings.
            {code:java}
            java.lang.IllegalStateException: Form is larger than max length 200000
            at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:562)
            at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:519)
            at org.eclipse.jetty.server.Request.getParameters(Request.java:430)
            Caused: org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
            at org.eclipse.jetty.server.Request.getParameters(Request.java:434)
            at org.eclipse.jetty.server.Request.getParameterNames(Request.java:1077)
            at hudson.security.csrf.CrumbFilter.extractCrumbFromRequest(CrumbFilter.java:112)
            at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:81)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
            at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
            at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1592)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
            at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
            at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
            at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.Server.handle(Server.java:500)
            at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
            at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:562)
            at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
            at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
            at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
            at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
            at java.lang.Thread.run(Thread.java:748)
            {code}
            oleg_nenashev Oleg Nenashev made changes -
            Description **Jenkins LTS Notice**: Jenkins LTS 2.204.3 and 2.204.4 are also affected due to the Winstone upgrade which was backported as a part of the JENKINS-57888 fix backporting. Please see [https://groups.google.com/forum/#!topic/jenkinsci-dev/M_RtDuDXtbU] for the discussion and retrospective

            In Jenkins Version 2.205, PR #4339 moved the cloud configuration from Configure System into is own configuration form on the Manage Nodes page. There is a cap to the length of this form (200000) and prevents me from adding additional docker clouds into the settings.
            {code:java}
            java.lang.IllegalStateException: Form is larger than max length 200000
            at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:562)
            at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:519)
            at org.eclipse.jetty.server.Request.getParameters(Request.java:430)
            Caused: org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
            at org.eclipse.jetty.server.Request.getParameters(Request.java:434)
            at org.eclipse.jetty.server.Request.getParameterNames(Request.java:1077)
            at hudson.security.csrf.CrumbFilter.extractCrumbFromRequest(CrumbFilter.java:112)
            at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:81)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
            at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
            at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1592)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
            at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
            at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
            at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.Server.handle(Server.java:500)
            at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
            at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:562)
            at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
            at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
            at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
            at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
            at java.lang.Thread.run(Thread.java:748)
            {code}
            **Jenkins LTS Notice**: Jenkins LTS 2.204.3 and 2.204.4 are also affected due to the Winstone upgrade which was introduced as a part of the JENKINS-57888 fix backporting. Please see [https://groups.google.com/forum/#!topic/jenkinsci-dev/M_RtDuDXtbU] for the discussion and retrospective

            In Jenkins Version 2.205, PR #4339 moved the cloud configuration from Configure System into is own configuration form on the Manage Nodes page. There is a cap to the length of this form (200000) and prevents me from adding additional docker clouds into the settings.
            {code:java}
            java.lang.IllegalStateException: Form is larger than max length 200000
            at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:562)
            at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:519)
            at org.eclipse.jetty.server.Request.getParameters(Request.java:430)
            Caused: org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
            at org.eclipse.jetty.server.Request.getParameters(Request.java:434)
            at org.eclipse.jetty.server.Request.getParameterNames(Request.java:1077)
            at hudson.security.csrf.CrumbFilter.extractCrumbFromRequest(CrumbFilter.java:112)
            at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:81)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
            at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
            at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
            at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
            at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
            at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
            at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1592)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
            at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
            at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
            at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.Server.handle(Server.java:500)
            at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
            at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:562)
            at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
            at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
            at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
            at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
            at java.lang.Thread.run(Thread.java:748)
            {code}
            jglick Jesse Glick made changes -
            Status Reopened [ 4 ] Open [ 1 ]
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            I have just released an alternate Winstone 5.4.1 release for 2.204.x LTS. This patch reverts Jetty to older versions https://github.com/jenkinsci/winstone/releases/tag/winstone-5.4.1 but keeps other regression fixes. It should be more stable than upgrade to Winstone 5.9 with just another Jetty upgrade and a risk of new regressions. 

            Pull request with the 2.204.x baseline update: https://github.com/jenkinsci/jenkins/pull/4545

            Show
            oleg_nenashev Oleg Nenashev added a comment - I have just released an alternate Winstone 5.4.1 release for 2.204.x LTS. This patch reverts Jetty to older versions  https://github.com/jenkinsci/winstone/releases/tag/winstone-5.4.1  but keeps other regression fixes. It should be more stable than upgrade to Winstone 5.9 with just another Jetty upgrade and a risk of new regressions.  Pull request with the 2.204.x baseline update:  https://github.com/jenkinsci/jenkins/pull/4545
            oleg_nenashev Oleg Nenashev made changes -
            Released As Winstone 5.4.1, Winstone 5.9
            oleg_nenashev Oleg Nenashev made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            danielbeck Daniel Beck made changes -
            Labels lts-candidate regression lts-candidate lts-candidate-for-2.204-and-2.222 regression
            Hide
            always_fail Always Fail added a comment -

            Will this fix be included in 2.224?

            Show
            always_fail Always Fail added a comment - Will this fix be included in 2.224?
            Hide
            oleg_nenashev Oleg Nenashev added a comment - - edited

            I believe so. https://github.com/jenkinsci/jenkins/pull/4542 is waiting for the 24hrs merge timeout which ends in 1 hour or so. Taking the approvals, I am pretty confident that the next weekly release will include the fix. LTS is a separate story, I am waiting for responses from Oliver Gondža and Kohsuke Kawaguchi about out-of-order 2.204.5 LTS 

            Show
            oleg_nenashev Oleg Nenashev added a comment - - edited I believe so.  https://github.com/jenkinsci/jenkins/pull/4542  is waiting for the 24hrs merge timeout which ends in 1 hour or so. Taking the approvals, I am pretty confident that the next weekly release will include the fix. LTS is a separate story, I am waiting for responses from Oliver Gondža and Kohsuke Kawaguchi about out-of-order 2.204.5 LTS 
            Hide
            batmat Baptiste Mathus added a comment - - edited

            Curious, did someone ever test

            -Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1

            to remove any limit instead of just bumping it higher?

             

            UPDATE: just tested it. This works. I think this should be the recommendation instead of any high number. FWIW, this is what Jenkins normally does internally.

            Show
            batmat Baptiste Mathus added a comment - - edited Curious, did someone ever test -Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1 to remove any limit instead of just bumping it higher?   UPDATE: just tested it. This works. I think this should be the recommendation instead of any high number. FWIW, this is what Jenkins normally does internally.
            Hide
            jglick Jesse Glick added a comment -

            Minimal test in context to reproduce: run Jenkins with Winstone 5.8 on a fresh user dir. Go through setup wizard, installing no plugins. Create an API token for admin. Then run

            x=1; while :; do echo trying $x; (echo description=; seq -s. $x | tr -d '[:digit:]') > /tmp/$x-dots.txt; curl -f -u admin:YOURTOKEN -d @/tmp/$x-dots.txt http://localhost:8080/submitDescription || break; x=$((x * 3)); done
            

            You should see it fail after 200000:

            trying 1
            trying 3
            trying 9
            trying 27
            trying 81
            trying 243
            trying 729
            trying 2187
            trying 6561
            trying 19683
            trying 59049
            trying 177147
            trying 531441
            curl: (22) The requested URL returned error: 500 Server Error
            

            If you now start Jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1, or with Winstone 5.9, it keeps on going.

            Show
            jglick Jesse Glick added a comment - Minimal test in context to reproduce: run Jenkins with Winstone 5.8 on a fresh user dir. Go through setup wizard, installing no plugins. Create an API token for admin . Then run x=1; while :; do echo trying $x ; (echo description=; seq -s. $x | tr -d '[:digit:]' ) > /tmp/ $x -dots.txt; curl -f -u admin:YOURTOKEN -d @/tmp/ $x -dots.txt http://localhost:8080/submitDescription || break; x=$((x * 3)); done You should see it fail after 200000: trying 1 trying 3 trying 9 trying 27 trying 81 trying 243 trying 729 trying 2187 trying 6561 trying 19683 trying 59049 trying 177147 trying 531441 curl: (22) The requested URL returned error: 500 Server Error If you now start Jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1 , or with Winstone 5.9, it keeps on going.
            Hide
            byteenable Byte Enable added a comment -

            I am experiencing this issue as well.  Source file is 200K in size.  I whittled it down to 177K and still experience the error.  Is there a work-around?

            Show
            byteenable Byte Enable added a comment - I am experiencing this issue as well.  Source file is 200K in size.  I whittled it down to 177K and still experience the error.  Is there a work-around?
            jglick Jesse Glick made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Fixed but Unreleased [ 10203 ]
            danielbeck Daniel Beck made changes -
            Labels lts-candidate lts-candidate-for-2.204-and-2.222 regression 2.204.5-fixed lts-candidate lts-candidate-for-2.204-and-2.222 regression
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

             

            Show
            oleg_nenashev Oleg Nenashev added a comment - Jenkins LTS 2.204.5 with fixes is out:  https://github.com/jenkinsci/jenkins/releases/tag/jenkins-2.204.5  . Official Changelogs are coming soon. ETA for Jenkins weekly is today  
            oleg_nenashev Oleg Nenashev made changes -
            Released As Winstone 5.4.1, Winstone 5.9 Jenkins 2.204.5, Jenkins 2.224, Winstone 5.4.3, Winstone 5.9
            oleg_nenashev Oleg Nenashev made changes -
            Status Fixed but Unreleased [ 10203 ] Resolved [ 5 ]
            olivergondza Oliver Gondža made changes -
            Labels 2.204.5-fixed lts-candidate lts-candidate-for-2.204-and-2.222 regression 2.204.5-fixed 2.222.1-fixed regression
            Hide
            jonesbusy Valentin Delaye added a comment -

            Hi,

            I have the same issue on 2.222 (non-LTS) with websocket connection payload. I know the feature is still in beta, but I guess the 2.221. should fix the websocket layer as well.

            2020-03-24 09:47:21.616+0000 [id=219730] INFO j.s.DefaultJnlpSlaveReceiver#channelClosed: Jetty (winstone)-219730 for ************* terminated: java.nio.channels.ClosedChannelException
            2020-03-24 09:47:27.142+0000 [id=219727] WARNING j.agents.WebSocketAgents$Session#error
            org.eclipse.jetty.websocket.api.MessageTooLargeException: Binary message size [69632] exceeds maximum size [65536]
            at org.eclipse.jetty.websocket.api.WebSocketPolicy.assertValidBinaryMessageSize(WebSocketPolicy.java:128)
            at org.eclipse.jetty.websocket.common.message.SimpleBinaryMessage.appendFrame(SimpleBinaryMessage.java:57)
            at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:61)
            at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.onContinuationFrame(AbstractEventDriver.java:183)
            at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onContinuationFrame(JettyListenerEventDriver.java:255)
            at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:155)
            at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:322)
            at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202)
            at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225)
            at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259)
            at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:460)
            at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:441)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
            at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
            at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
            at java.lang.Thread.run(Thread.java:748)
            
            

            Will try again when the 222.1 is released

            Thanks

            Show
            jonesbusy Valentin Delaye added a comment - Hi, I have the same issue on 2.222 (non-LTS) with websocket connection payload. I know the feature is still in beta, but I guess the 2.221. should fix the websocket layer as well. 2020-03-24 09:47:21.616+0000 [id=219730] INFO j.s.DefaultJnlpSlaveReceiver#channelClosed: Jetty (winstone)-219730 for ************* terminated: java.nio.channels.ClosedChannelException 2020-03-24 09:47:27.142+0000 [id=219727] WARNING j.agents.WebSocketAgents$Session#error org.eclipse.jetty.websocket.api.MessageTooLargeException: Binary message size [69632] exceeds maximum size [65536] at org.eclipse.jetty.websocket.api.WebSocketPolicy.assertValidBinaryMessageSize(WebSocketPolicy.java:128) at org.eclipse.jetty.websocket.common.message.SimpleBinaryMessage.appendFrame(SimpleBinaryMessage.java:57) at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:61) at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.onContinuationFrame(AbstractEventDriver.java:183) at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onContinuationFrame(JettyListenerEventDriver.java:255) at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:155) at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:322) at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202) at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225) at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259) at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:460) at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:441) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.lang. Thread .run( Thread .java:748) Will try again when the 222.1 is released Thanks
            Hide
            danielbeck Daniel Beck added a comment -

            Valentin Delaye

            same issue

            It's not. Note how the error message is completely different. You're looking for JENKINS-61409.

            Show
            danielbeck Daniel Beck added a comment - Valentin Delaye same issue It's not. Note how the error message is completely different. You're looking for JENKINS-61409 .
            Hide
            jonesbusy Valentin Delaye added a comment -

            Jesse Glick Ok thanks

            Show
            jonesbusy Valentin Delaye added a comment - Jesse Glick Ok thanks

              People

              Assignee:
              jglick Jesse Glick
              Reporter:
              mastershihochief Joshua Hunter
              Votes:
              12 Vote for this issue
              Watchers:
              22 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: