Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62200

MF Application Automation Tools plugin: violation of RFC7230

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      The plugin is unable to authenticate inside ALM during "Execute tests using ALM Lab Management" step if ALM server is behind haproxy v2.0 and above.

      The root cause is that the plugin expects Case-Sensitive http headers and by doing that violates RFC7230:
      https://tools.ietf.org/html/rfc7230#section-3.2

      Each header field consists of a case-insensitive field name followed
      by a colon (":"), optional leading whitespace, the field value, and
      optional trailing whitespace.

      And as we see in code, there are many places that violate this:

      For Set-Cookie header:

      For WWW-Authenticate header:

      Maybe there are other places and headers as well.

      We've faced the issue because newer versions of haproxy (2.0+) now use the new http processing mechanism internally (h2) by default. And because of this, all the http headers are now lowercased by default. So haproxy outputs "www-authenticate" instead of the original "WWW-Authenticate" which is perfectly compliant with RFC. But not with the plugin.

      Workaround

      There is a haproxy config option to override this behavior for some headers: https://cbonte.github.io/haproxy-dconv/2.0/configuration.html#3.1-h1-case-adjust. You should use 2 haproxy configuration options in conjunction: "h1-case-adjust" and "option h1-case-adjust-bogus-client".

      We used it and I can prove that this is a valid workaround.

        Attachments

          Activity

          Hide
          roy_lu Roy Lu added a comment - - edited

          Hi Fedor Radzievskiy,

          I resolved the illegal issue. Replacing with 3rd client lib is proceeding.

          An event log query api still got problem such as:

          http://host:port/qcbin/rest/domains/DEFAULT/projects/p1/event-log-reads?query=

          {context['*Timeslot: 1010*']}

          &fields=id

          Show
          roy_lu Roy Lu added a comment - - edited Hi Fedor Radzievskiy , I resolved the illegal issue. Replacing with 3rd client lib is proceeding. An event log query api still got problem such as: http://host:port/qcbin/rest/domains/DEFAULT/projects/p1/event-log-reads?query= {context['*Timeslot: 1010*']} &fields=id
          Hide
          fff Fedor Radzievskiy added a comment -

          Hi Roy Lu,
          Unfortunately I don't see anything on email. Maybe it's a Jira issue...
          Anyway I'll ask our Support to reach you to send it via FTP.

          some URLs would be considered illegal by it

          Can you please specify these URLs? We use our own java wrappers for ALM REST internally and I haven't encountered such issues so far. But our implementation does not cover all the endpoints.

          Show
          fff Fedor Radzievskiy added a comment - Hi Roy Lu , Unfortunately I don't see anything on email. Maybe it's a Jira issue... Anyway I'll ask our Support to reach you to send it via FTP. some URLs would be considered illegal by it Can you please specify these URLs? We use our own java wrappers for ALM REST internally and I haven't encountered such issues so far. But our implementation does not cover all the endpoints.
          Hide
          roy_lu Roy Lu added a comment -

          Hi Fedor Radzievskiy

          You're right. I found camel case is a behavior of ALM server. And for getting headers it always needs ignore case.

          // Get header in Apache http client
          public Header[] getHeaders(String name) {
              List<Header> headersFound = null;
          
              for(int i = 0; i < this.headers.size(); ++i) {
                  Header header = (Header)this.headers.get(i);
                  if (header.getName().equalsIgnoreCase(name)) {
                      if (headersFound == null) {
                          headersFound = new ArrayList();
                      }
          
                      headersFound.add(header);
                  }
              }
          
              return headersFound != null ? (Header[])headersFound.toArray(new Header[headersFound.size()]) : this.EMPTY;
          }
          

          Actually I wanted to replace our client with 3rd http client long ago. But because some URLs would be considered illegal by it. So I haven't apply it yet.

          I sent you the pb by email. Please check.

          Thanks.

           

           

          Show
          roy_lu Roy Lu added a comment - Hi Fedor Radzievskiy You're right. I found camel case is a behavior of ALM server. And for getting headers it always needs ignore case. // Get header in Apache http client public Header[] getHeaders( String name) { List<Header> headersFound = null ; for ( int i = 0; i < this .headers.size(); ++i) { Header header = (Header) this .headers.get(i); if (header.getName().equalsIgnoreCase(name)) { if (headersFound == null ) { headersFound = new ArrayList(); } headersFound.add(header); } } return headersFound != null ? (Header[])headersFound.toArray( new Header[headersFound.size()]) : this .EMPTY; } Actually I wanted to replace our client with 3rd http client long ago. But because some URLs would be considered illegal by it. So I haven't apply it yet. I sent you the pb by email. Please check. Thanks.    
          Hide
          fff Fedor Radzievskiy added a comment - - edited

          Hi Roy Lu,

          No matter what header I set in ALM's response, client will always get 'WWW-Authenticate'.

          The main question is: How did you manage to change it on ALM's side? I thought it is hardcoded there...

          I think this is the meaning of RFC-7230

          Well I treat it like this: "any client should be capable of parsing headers regardless of the case of their keys". From what you described, it seems that headers are still parsed case-sensitively, but for some reason their case is always the same (i.e. exactly WWW-Authenticate) and I don't understand how did you manage to achieve that.

          I can't test this at my environment. Could you help to test my build at your side?

          Sure. And I can also make a simple haproxy config for you so that you are able to test it on your environment because this should be way faster. The haproxy installation and configuration is pretty straightforward.

          Show
          fff Fedor Radzievskiy added a comment - - edited Hi Roy Lu , No matter what header I set in ALM's response, client will always get 'WWW-Authenticate'. The main question is: How did you manage to change it on ALM's side? I thought it is hardcoded there... I think this is the meaning of RFC-7230 Well I treat it like this: "any client should be capable of parsing headers regardless of the case of their keys". From what you described, it seems that headers are still parsed case-sensitively, but for some reason their case is always the same (i.e. exactly WWW-Authenticate) and I don't understand how did you manage to achieve that. I can't test this at my environment. Could you help to test my build at your side? Sure. And I can also make a simple haproxy config for you so that you are able to test it on your environment because this should be way faster. The haproxy installation and configuration is pretty straightforward.
          Hide
          roy_lu Roy Lu added a comment - - edited

          Hi Fedor Radzievskiy,

          After my investigation I have some points:

          1. No matter what header I set in ALM's response, client will always get 'WWW-Authenticate'. I tested 'www-authenticate', 'WWW-AUTHENTICATE', 'wWw-auTHenticaTE'. For client, no matter client app or browser, the header would always be 'WWW-Authenticate'. I think this is the meaning of RFC-7230.
          2. Same behavior for all other headers. You can check that.
          3. Even I tried to use Apache http client, nothing different. Using http client can not solve this issue. Besides, we have some special URL format that would be consider illegal by these client api. And in Jenkins plugin, these clients can not use Jenkins Proxy configuration.
          4. I believe this is a special behavior of haproxy. Because it seems that other apps follow camel case rule. No matter how you set the name of header, it would always be in camel case at client side.
          5. I will make a patch for only your environment on condition that it would not produce regression on other environments.
          6. I can't test this at my environment. Could you help to test my build at your side?
          Show
          roy_lu Roy Lu added a comment - - edited Hi Fedor Radzievskiy , After my investigation I have some points: No matter what header I set in ALM's response, client will always get 'WWW-Authenticate'. I tested 'www-authenticate', 'WWW-AUTHENTICATE', 'wWw-auTHenticaTE'. For client, no matter client app or browser, the header would always be 'WWW-Authenticate'. I think this is the meaning of RFC-7230. Same behavior for all other headers. You can check that. Even I tried to use Apache http client, nothing different. Using http client can not solve this issue. Besides, we have some special URL format that would be consider illegal by these client api. And in Jenkins plugin, these clients can not use Jenkins Proxy configuration. I believe this is a special behavior of haproxy. Because it seems that other apps follow camel case rule. No matter how you set the name of header, it would always be in camel case at client side. I will make a patch for only your environment on condition that it would not produce regression on other environments. I can't test this at my environment. Could you help to test my build at your side?

            People

            Assignee:
            roy_lu Roy Lu
            Reporter:
            fff Fedor Radzievskiy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: