Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24132

MSBuild Output encoding vs Jenkins' encoding vs Browser Encoding

    • Icon: Improvement Improvement
    • Resolution: Done
    • Icon: Trivial Trivial
    • None
    • Win8 in French, Jenkins ver. 1.574, Chrome Version 37.0.2062.58 beta-m

      When using Jenkins with MSBuild/MSTests, the output console and the plain text have encoding issues.

      My OS is in French, so we use accents, which are not well displayed in Jenkins' MSBuild/MSTest output (see last line in the enclosed screenshot ; the word "Génération" became "G,n,ration")

      The MS CMD output is said to be encoded in CP 850.

      My Jenkins/SystemInfo, regarding encoding says :
      file.encoding Cp1252
      file.encoding.pkg sun.io
      sun.io.unicode.encoding UnicodeLittle
      sun.java.command C:\Program Files (x86)\Jenkins\jenkins.war --httpPort=8082 --ajp13Port=-1 -Dfile.encoding=UTF-8
      sun.jnu.encoding Cp1252

      I tried to use the "-Dfile.encoding=" argument with "UTF8","UTF-8", "Cp850" "ISO8859_1" but without any luck.

      I guess as long as there will be differences between MSDOS/MSBuild's encoding and Jenkins' encoding, I won't be able to have proper outputs.

          [JENKINS-24132] MSBuild Output encoding vs Jenkins' encoding vs Browser Encoding

          Please check you encoding on Jenkins build node.

          Gregory Boissinot added a comment - Please check you encoding on Jenkins build node.

          Screenshot from a CMD line in Windows, outside of Jenkins: French accents are correct. Screenshot from Jenkins with a simple Windows Batch Step: accents are not correctly displayed.

          Romuald Ginhoux added a comment - Screenshot from a CMD line in Windows, outside of Jenkins: French accents are correct. Screenshot from Jenkins with a simple Windows Batch Step: accents are not correctly displayed.

          Hi. I am facing with the same problem and I don't understand how you resolved it. Jenkins 1.583 on Windows Server 2008 R2.
          When I run msbuild from a cmd line in Windows outside of jenkins, French accents are correctly displayed. When I run the same command from a Windows Batch Step in Jenkins, I have the same misformatting as Hubert described.

          Romuald Ginhoux added a comment - Hi. I am facing with the same problem and I don't understand how you resolved it. Jenkins 1.583 on Windows Server 2008 R2. When I run msbuild from a cmd line in Windows outside of jenkins, French accents are correctly displayed. When I run the same command from a Windows Batch Step in Jenkins, I have the same misformatting as Hubert described.

          Duarte Carreira added a comment - - edited

          Just adding my solution in case anyone is stuck with this.

          I had the same issue running psql queries with some characters.

          What ended working was setting in my "Windows batch command" jobs the code page to the same code as Jenkins' default 1252:

          {{...
          set PGCLIENTENCODING="WIN1252"
          chcp 1252
          ...}}

          The relevant part is the chcp 1252 command. This corresponds to ANSI Latin 1; Western European (Windows), according to this: https://msdn.microsoft.com/en-us/library/windows/desktop/dd317756(v=vs.85).aspx.

          The PGCLIENTENCODING is specific to psql so applies only to my specific case.

          Hope this helps.

          Duarte Carreira added a comment - - edited Just adding my solution in case anyone is stuck with this. I had the same issue running psql queries with some characters. What ended working was setting in my "Windows batch command" jobs the code page to the same code as Jenkins' default 1252: {{... set PGCLIENTENCODING="WIN1252" chcp 1252 ...}} The relevant part is the chcp 1252 command. This corresponds to ANSI Latin 1; Western European (Windows), according to this: https://msdn.microsoft.com/en-us/library/windows/desktop/dd317756(v=vs.85).aspx . The PGCLIENTENCODING is specific to psql so applies only to my specific case. Hope this helps.

          Robert Lieske added a comment - - edited

          I had the same problem with umlauts not displaying correctly when doing a MSBuild from Jenkins. If someone comes across the same problem, I present a working solution here:
          I found out, that the root cause of the problem was the cmd.exe called not having the correct codepage. Even though the characters appeared to be correct when running from console, they were not (you can check by redirecting output to file).
          I added an additional Windows Batch Command Build Step, where I called MSBuild.exe after changing the codepage:

          chcp 1252
          cmd.exe /C " C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe HelloWorld/HelloWorld.sln " && exit %%ERRORLEVEL%%

          The umlauts here were displayed correctly (while with the MSBuild Build step they are not).
          Since I prefered to use the MSBuild Build step over pure Shell Command, I had to change the codepage of cmd. Jenkins does not allow to supply addional parameters to be passed to cmd.exe, so I had to look for other ways. There are many offered (some of them can cause Windows failing to boot, and some didn't work with Jenkins - like using Autorun key). The one that has the least impact on the system configuration and I use therefore is to add the following registry key as Jenkins user (as it is working only for "current user"):

          [HKEY_CURRENT_USER\Console]
          "CodePage"=dword:4e4

          Please note, that you should also use a font, that can display umlauts properly (like Lucida Console).

          Robert Lieske added a comment - - edited I had the same problem with umlauts not displaying correctly when doing a MSBuild from Jenkins. If someone comes across the same problem, I present a working solution here: I found out, that the root cause of the problem was the cmd.exe called not having the correct codepage. Even though the characters appeared to be correct when running from console, they were not (you can check by redirecting output to file). I added an additional Windows Batch Command Build Step, where I called MSBuild.exe after changing the codepage: chcp 1252 cmd.exe /C " C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe HelloWorld/HelloWorld.sln " && exit %%ERRORLEVEL%% The umlauts here were displayed correctly (while with the MSBuild Build step they are not). Since I prefered to use the MSBuild Build step over pure Shell Command, I had to change the codepage of cmd. Jenkins does not allow to supply addional parameters to be passed to cmd.exe, so I had to look for other ways. There are many offered (some of them can cause Windows failing to boot, and some didn't work with Jenkins - like using Autorun key). The one that has the least impact on the system configuration and I use therefore is to add the following registry key as Jenkins user (as it is working only for "current user"): [HKEY_CURRENT_USER\Console] "CodePage"=dword:4e4 Please note, that you should also use a font, that can display umlauts properly (like Lucida Console).

          Provided by chichimotsu in PR #29 and included in release 1.27

          Lionel Cabasson added a comment - Provided by chichimotsu in PR #29 and included in release 1.27

          rake kniven added a comment -

          Hello,

          does the new chcp call has any impact on binary build output? Will my exe differ after calling chcp?
          I have a Delphi build slave and it omplains about not finding chcp command. I know how to fix this and thought about any side effects.

          Cheers

          rake kniven added a comment - Hello, does the new chcp call has any impact on binary build output? Will my exe differ after calling chcp? I have a Delphi build slave and it omplains about not finding chcp command. I know how to fix this and thought about any side effects. Cheers

          Hello,

          This change won't impact the binary built by msbuild.
          However, it changes the encoding of the log generated by msbuild. If anything in your build process parses this output, it will be impacted and might behave differently.

          I hope this answers your question.

          Regards

          Lionel Cabasson added a comment - Hello, This change won't impact the binary built by msbuild. However, it changes the encoding of the log generated by msbuild. If anything in your build process parses this output, it will be impacted and might behave differently. I hope this answers your question. Regards

          rake kniven added a comment -

          Hello Lionel,

          thank you for your response!
          Will give it a try.

          Cheers.

          rake kniven added a comment - Hello Lionel, thank you for your response! Will give it a try. Cheers.

          Tecno Speed added a comment - - edited

          Hello.

          I was tried to use this solution but my problem keeping existing.

          I use Windows server 2012 and jenkins.

          My Windows has page code 860, i tried to charge to chcp 1252, to same from jenkins, not success...
          I only need to output my logs with especial characteres(In portuguese PT-BR) to jenkins console.

          I also tried to change jenkins xml, including this argument: "-Dfile.encoding=UTF8", like is instructed in this link: https://medium.com/pacroy/how-to-fix-jenkins-console-log-encoding-issue-on-windows-a1f4b26e0db4

          No success also.

          Tecno Speed added a comment - - edited Hello. I was tried to use this solution but my problem keeping existing. I use Windows server 2012 and jenkins. My Windows has page code 860, i tried to charge to chcp 1252, to same from jenkins, not success... I only need to output my logs with especial characteres(In portuguese PT-BR) to jenkins console. I also tried to change jenkins xml, including this argument: "-Dfile.encoding=UTF8", like is instructed in this link: https://medium.com/pacroy/how-to-fix-jenkins-console-log-encoding-issue-on-windows-a1f4b26e0db4 No success also.

            marshall777 Lionel Cabasson
            hub Hubert Chauvat
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: