Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-20473

line-ending mismatch between the Jenkins HTTP server implementation and the HTTP specification

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • window senvironment, windows 2008 server

      When I submit using json the server responds with error 400, the same exact submittal with jenkins version 1.5.30 and below gives me a success. We looked at this some more and found this out .

      The root of this problem is a line-ending mismatch between the Jenkins HTTP server implementation and the HTTP specification. The HTTP RFC specifies \r\n (CRLF, (Windows line endings)) to be the line endings for HTTP. However, almost all servers also support \n (UNIX line endings). Invoking the Jenkins' HTTP API on a linux machine directly via the 'curl' command line tool is happy with the \n line-endings (which curl defaults to on linux). However, forcing curl to use \r\n as its line endings causes the transaction to fail using the same error code the obs.py fails with.

      Now, for the fun part (why we can't work around this easily) - We checked the underlying Python library httplib we use has \r\n hardcoded as that's what the HTTP specification calls for. There is no option to switch it to \n.

          [JENKINS-20473] line-ending mismatch between the Jenkins HTTP server implementation and the HTTP specification

          Simple test case on linux:

          $ curl http://105.128.26.118:8080/job/TOP_TEST/build/token=BA -F json='{"parameter": {"badparam":"blah"}}' --user xxx:xxx -v --crlf
          Returns HTTP 400 expects form submission

          $ curl http://105.128.26.118:8080/job/TOP_TEST/build/token=BA -F json='{"parameter": {"badparam":"blah"}}' --user xxx:xxx -v
          Returns a Jenkins JSON parsing error as it should.

          Aaron Fitzgerald added a comment - Simple test case on linux: $ curl http://105.128.26.118:8080/job/TOP_TEST/build/token=BA -F json='{"parameter": {"badparam":"blah"}}' --user xxx:xxx -v --crlf Returns HTTP 400 expects form submission $ curl http://105.128.26.118:8080/job/TOP_TEST/build/token=BA -F json='{"parameter": {"badparam":"blah"}}' --user xxx:xxx -v Returns a Jenkins JSON parsing error as it should.

          Note that this issue was noticed because we invoke builds with a Python script which worked on builds 1.5.30 and below.

          Aaron Fitzgerald added a comment - Note that this issue was noticed because we invoke builds with a Python script which worked on builds 1.5.30 and below.

          Dmitry Danilson added a comment - - edited

          Parameters

          In order to make sure for me (and prove for you and others) that I am getting everything right, let me describe my explanation of the issue you have.
          So, we have 3 completely independent parameters that influence the issue:

          • OS (either Windows or Linux);
          • Jenkins version (only 2 variants that we have to take into account: either less or equal 1.530 or bigger than that);
          • Line endings of request client (again only 2 variants: either CRLF or LF).

          Prediction

          So here is the table of results you are getting together with my prediction based on my understanding of the issue:

          OS/Version(crlf/non-crlf) <=1.530 >1.530
          Linux Sucess/Success Error 400/JSON parsing error
          Windows Success/Success Error 400/JSON parsing error

          Please make sure that everything is as described in the table.

          Reasons

          Why it fails for Version>1.530 for both OSes

          According to https://wiki.jenkins-ci.org/display/JENKINS/Quick+and+Simple+Security , you should either grant additional permission for anonymous users or use a suggested plugin:

          Anonymous read access required
          The preceding instructions require that anonymous users have read access to Jenkins. If you disable overall read access for anonymous users, the Build Token Root Plugin allows use of tokens.

          Why it works for Version<=1.530 for both OSes

          Among the changes done after 1.530 release (see http://jenkins-ci.org/changelog for reference) there was one called "Use fine-grained permissions for node manipulation via REST API & CLI (issue 18485)". It changed the way how permission check works - so you need additional effort.

          Why for Version>1.530 for both OSes it fails with 400 Error and not 403 or 404, which looks more appropriate in case if it is permission issue?

          I guess it is due to multipart submission. I believe that in case if you use regular call it will return either 403 or 404 Error.

          Hope it helps.
          Feel free to post additional info in case if suggested recipe doesn't solve your issue.
          Until that, I am going to consider it as solved.

          Dmitry Danilson added a comment - - edited Parameters In order to make sure for me (and prove for you and others) that I am getting everything right, let me describe my explanation of the issue you have. So, we have 3 completely independent parameters that influence the issue: OS (either Windows or Linux); Jenkins version (only 2 variants that we have to take into account: either less or equal 1.530 or bigger than that); Line endings of request client (again only 2 variants: either CRLF or LF). Prediction So here is the table of results you are getting together with my prediction based on my understanding of the issue: OS/Version(crlf/non-crlf) <=1.530 >1.530 Linux Sucess/Success Error 400/JSON parsing error Windows Success/Success Error 400/JSON parsing error Please make sure that everything is as described in the table. Reasons Why it fails for Version>1.530 for both OSes According to https://wiki.jenkins-ci.org/display/JENKINS/Quick+and+Simple+Security , you should either grant additional permission for anonymous users or use a suggested plugin : Anonymous read access required The preceding instructions require that anonymous users have read access to Jenkins. If you disable overall read access for anonymous users, the Build Token Root Plugin allows use of tokens. Why it works for Version<=1.530 for both OSes Among the changes done after 1.530 release (see http://jenkins-ci.org/changelog for reference) there was one called "Use fine-grained permissions for node manipulation via REST API & CLI (issue 18485)". It changed the way how permission check works - so you need additional effort. Why for Version>1.530 for both OSes it fails with 400 Error and not 403 or 404, which looks more appropriate in case if it is permission issue? I guess it is due to multipart submission. I believe that in case if you use regular call it will return either 403 or 404 Error. Hope it helps. Feel free to post additional info in case if suggested recipe doesn't solve your issue. Until that, I am going to consider it as solved.

            Unassigned Unassigned
            narayankamath Narayan Kamath
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: