Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-11543

UTF-8 encoding issue of build parameters as soon as including File parameter

      String parameter UTF-8 encoding crash issue when include File paramemter

      for example,

      setting up File parameter and String parameter. > String parameter is "가나다라마바" (in korean) > build

      parameter result is "가나다라" (encoding crashed)

      please check it
      thanks

          [JENKINS-11543] UTF-8 encoding issue of build parameters as soon as including File parameter

          Han SeungHoon created issue -

          Ryo Amano added a comment -

          I encountered similar problem.
          My case was 'Select parameter' with 'File parameter' instead of 'String parameter'.

          It seems to be crashed when input value includes non-ascii string if job has 'File parameter'.

          Ryo Amano added a comment - I encountered similar problem. My case was 'Select parameter' with 'File parameter' instead of 'String parameter'. It seems to be crashed when input value includes non-ascii string if job has 'File parameter'.

          krizleebear added a comment -

          The issue still exists with the newest version (1.577) today. This makes File parameters practically unusable for internationalized use cases!

          I've attached a minimal repro case config.xml. When you build it with default parameters (String parameter is defaulted to "äöü"), then this build will fail. Removing the File parameter restores the build.

          Please fix this issue. I was pretty surprised when I found this issue being known for 3 years.

          krizleebear added a comment - The issue still exists with the newest version (1.577) today. This makes File parameters practically unusable for internationalized use cases! I've attached a minimal repro case config.xml. When you build it with default parameters (String parameter is defaulted to "äöü"), then this build will fail. Removing the File parameter restores the build. Please fix this issue. I was pretty surprised when I found this issue being known for 3 years.
          krizleebear made changes -
          Attachment New: config.xml [ 26725 ]
          krizleebear made changes -
          Description Original: String parameter UTF-8 encoding crash issue when include File paramemter

          for example,

          setting up File paramemter and String parameter. > String parameter is "가나다라마바" (in korean) > build

          parameter result is "가나다라" (encoding crashed)

          please check it
          thanks
          New: String parameter UTF-8 encoding crash issue when include File paramemter

          for example,

          setting up File parameter and String parameter. > String parameter is "가나다라마바" (in korean) > build

          parameter result is "가나다라" (encoding crashed)

          please check it
          thanks
          Environment Original: none New: Linux
          Summary Original: UTF-8 encoding crash issue when include File paramemter New: UTF-8 encoding issue of build parameters as soon as including File parameter
          Daniel Beck made changes -
          Component/s New: core [ 15593 ]
          Affects Version/s Original: current [ 10162 ]
          Due Date Original: 2011-11-11

          Daniel Beck added a comment -

          When a file upload form field is included, it changes the request content type from application/x-www-form-urlencoded to multipart/form-data.

          Daniel Beck added a comment - When a file upload form field is included, it changes the request content type from application/x-www-form-urlencoded to multipart/form-data.

          Daniel Beck added a comment -

          hudson-behavior.js seems to build a UTF-8 encoded JSON string, while the browser sets no charset, making it to default to Latin 1 when parsed on the server.

          Not sure who's to blame here (although I'd tentatively guess YUI/behavior should make sure the browser sets a charset for all the multipart text parts), but the following patch in Stapler works for me by simply parsing the JSON string in multipart form data as UTF-8 if no other charset is set:

          $ git diff
          diff --git a/core/src/main/java/org/kohsuke/stapler/RequestImpl.java b/core/src/main/java/org/kohsuke/stapler/RequestImpl.java
          index b8fb81c..230cef2 100644
          --- a/core/src/main/java/org/kohsuke/stapler/RequestImpl.java
          +++ b/core/src/main/java/org/kohsuke/stapler/RequestImpl.java
          @@ -852,8 +852,19 @@ public class RequestImpl extends HttpServletRequestWrapper implements StaplerReq
                           isSubmission=true;
                           parseMultipartFormData();
                           FileItem item = parsedFormData.get("json");
          -                if(item!=null)
          -                    p = item.getString();
          +                if(item!=null) {
          +                    if (item.getContentType() == null) {
          +                        // JENKINS-11543: Client doesn't set charset per part, so
          +                        // default to UTF-8 if no other charset is specified.
          +                        try {
          +                            p = item.getString("UTF-8");
          +                        } catch (java.io.UnsupportedEncodingException uee) {
          +                            throw new AssertionError("UTF-8 unsupported")
          +                        }
          +                    } else {
          +                        p = item.getString();
          +                    }
          +                }
                       } else {
                           p = getParameter("json");
                           isSubmission = !getParameterMap().isEmpty();

          Of course anything not relying on structured form submission will continue to be broken.

          Asked KK on IRC where to file issues with Stapler, as the GitHub issue tracker seems abandoned.

          Daniel Beck added a comment - hudson-behavior.js seems to build a UTF-8 encoded JSON string, while the browser sets no charset, making it to default to Latin 1 when parsed on the server. Not sure who's to blame here (although I'd tentatively guess YUI/behavior should make sure the browser sets a charset for all the multipart text parts), but the following patch in Stapler works for me by simply parsing the JSON string in multipart form data as UTF-8 if no other charset is set: $ git diff diff --git a/core/src/main/java/org/kohsuke/stapler/RequestImpl.java b/core/src/main/java/org/kohsuke/stapler/RequestImpl.java index b8fb81c..230cef2 100644 --- a/core/src/main/java/org/kohsuke/stapler/RequestImpl.java +++ b/core/src/main/java/org/kohsuke/stapler/RequestImpl.java @@ -852,8 +852,19 @@ public class RequestImpl extends HttpServletRequestWrapper implements StaplerReq isSubmission= true ; parseMultipartFormData(); FileItem item = parsedFormData.get( "json" ); - if (item!= null ) - p = item.getString(); + if (item!= null ) { + if (item.getContentType() == null ) { + // JENKINS-11543: Client doesn't set charset per part, so + // default to UTF-8 if no other charset is specified. + try { + p = item.getString( "UTF-8" ); + } catch (java.io.UnsupportedEncodingException uee) { + throw new AssertionError( "UTF-8 unsupported" ) + } + } else { + p = item.getString(); + } + } } else { p = getParameter( "json" ); isSubmission = !getParameterMap().isEmpty(); Of course anything not relying on structured form submission will continue to be broken. Asked KK on IRC where to file issues with Stapler, as the GitHub issue tracker seems abandoned.

          krizleebear added a comment -

          Thanks for the quick reply!
          To be honest I didn't expect to get an update on a topic this old.
          I'm looking forward to see further updates to this issue and of course a fix.

          krizleebear added a comment - Thanks for the quick reply! To be honest I didn't expect to get an update on a topic this old. I'm looking forward to see further updates to this issue and of course a fix.

          Daniel Beck added a comment -

          krizleebear: It was easy to reproduce and someone confirmed it's still a problem. What's not to love?


          Possibly related library issue: https://issues.apache.org/jira/browse/FILEUPLOAD-206

          FileUpload fails to handle requests with a form enctype of multipart/form-data;charset=UTF-8, so I doubt this can be resolved client-side (happy to be proven wrong!), so the Stapler fix above seems to be the correct solution for now.

          Daniel Beck added a comment - krizleebear : It was easy to reproduce and someone confirmed it's still a problem. What's not to love? Possibly related library issue: https://issues.apache.org/jira/browse/FILEUPLOAD-206 FileUpload fails to handle requests with a form enctype of multipart/form-data;charset=UTF-8 , so I doubt this can be resolved client-side (happy to be proven wrong!), so the Stapler fix above seems to be the correct solution for now.

            danielbeck Daniel Beck
            senicy Han SeungHoon
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: