Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-66731

Pipeline Nodes and Processes: PowerShell Core Script UTF-8 encoding doesn't seem to work

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • durable-task-plugin
    • None
    • Jenkins 2.303.1, Pipeline Nodes and Processes 2.40 Linux hosted Jenkins, Windows 10 Node.

      Unicode doesn't render properly.  Come back as  � character.  When running a python script that queries the environment, the encoding comes back as encoding='cp1252'.

      I think I'm explicitly saying to use UTF-8  in the pipeline also I think PowerShell Core is using UTF-8 by default. 

      pwsh(encoding: 'UTF-8',
      label: 'runscript',
      script: '''''')

      master and nodes have are using the -Dfile.encoding=UTF-8 JAVA Tool Options.

          [JENKINS-66731] Pipeline Nodes and Processes: PowerShell Core Script UTF-8 encoding doesn't seem to work

          Mark Waite added a comment -

          Be sure that the locale of the Jenkins process supports UTF-8. We have a known issue in the 2.303.1 Docker image that the locale is set to POSIX instead of C.UTF-8. The workaround is to assure that the locale is set (environment variables LC_ALL, LANG, and LANGUAGE) to C.UTF-8 explicitly in the image you build that is derived from the jenkins/jenkins:2.303.1 images.

          The "Manage Jenkins" -> "System Information" page displays the locale information of the controller. The "System Information" page for each agent displays the locale information for that agent

          Mark Waite added a comment - Be sure that the locale of the Jenkins process supports UTF-8. We have a known issue in the 2.303.1 Docker image that the locale is set to POSIX instead of C.UTF-8. The workaround is to assure that the locale is set (environment variables LC_ALL, LANG, and LANGUAGE) to C.UTF-8 explicitly in the image you build that is derived from the jenkins/jenkins:2.303.1 images. The "Manage Jenkins" -> "System Information" page displays the locale information of the controller. The "System Information" page for each agent displays the locale information for that agent

          Francis Kwok added a comment -

          For the controller Properties:

          file.encoding UTF-8
          sun.jnu.encoding UTF-8
          user.language en

          Controller Environmental Vars:

          LANG C.UTF-8

           

          Node Properties:

          file.encoding UTF-8
          sun.jnu.encoding utf-8

           

          user.language en

           

          Node Environmental Vars:

          JAVA_TOOL_OPTIONS -Dfile.encoding=UTF-8
          LANG C.UTF-8
          LANGUAGE C.UTF-8
          LC_ALL C.UTF-8

           

          The console still has  charters and the encoding still returns cp1252

           


          Installing pyqt5 (5.15.4)

          Installing pywin32 (301)

          Installing rich (10.10.0)

          Installing scons (4.2.0)[Pipeline] }[Pipeline] // stage[Pipeline] stage[Pipeline] { (Build Software)[Pipeline] echobuilding...[Pipeline] pwsh (runscript)---------------------- <class 'rich.console.Console'> -----------------------

          A high level console interface.
           
          -------------------------------------------------------------------------
            <console width=79 None>  
          -------------------------------------------------------------------------
           
          color_system = None
          encoding = 'cp1252'
          file = <_io.TextIOWrapper name='<stdout>' mode='w'
          encoding='cp1252'>

          Francis Kwok added a comment - For the controller Properties: file.encoding UTF-8 sun.jnu.encoding UTF-8 user.language en Controller Environmental Vars: LANG C.UTF-8   Node Properties: file.encoding UTF-8 sun.jnu.encoding utf-8   user.language en   Node Environmental Vars: JAVA_TOOL_OPTIONS -Dfile.encoding=UTF-8 LANG C.UTF-8 LANGUAGE C.UTF-8 LC_ALL C.UTF-8   The console still has  �  charters and the encoding still returns cp1252   � Installing pyqt5 (5.15.4) � Installing pywin32 (301) � Installing rich (10.10.0) � Installing scons (4.2.0) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Build Software) [Pipeline] echobuilding... [Pipeline] pwsh (runscript) ---------------------- <class 'rich.console.Console'> ----------------------- A high level console interface.   -------------------------------------------------------------------------   <console width=79 None>   -------------------------------------------------------------------------   color_system = None encoding = 'cp1252' file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp1252'>

          Alex Earl added a comment -

          If you are using Python 3.7 or above, you could use Python UTF-8 mode: https://peps.python.org/pep-0540/

          Alex Earl added a comment - If you are using Python 3.7 or above, you could use Python UTF-8 mode: https://peps.python.org/pep-0540/

            Unassigned Unassigned
            jumpingmule Francis Kwok
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: