Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-18414

Jenkins Subversion plugin checkouts in something NON-UTF-8

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • subversion-plugin
    • None

      Subversion checkouts of a name with some UTF-8 chars are done incorrectly from the jenkins console ONLY.

      For instance "Descripción.txt" gets downloaded as "Descripci?n.txt"

      If you do them with svn on the command line everything is fine, you get the proper filename.
      If you do them with jsvn (svnkit cli ofrom Ubunut) everything is fine, you get the proper filename.

      Workaround:
      1) You have to write a bash script to be executed just after the subversion checkout BUT before the maven build.

      2) This script must remove all badly checked out files like "Descripci?n.txt"

      3) Finally you do an "svn update" within the checked out project directory, so the files are checked out correctly now "Descripción.txt".

      4) Everything else from here works as expected.

          [JENKINS-18414] Jenkins Subversion plugin checkouts in something NON-UTF-8

          kutzi added a comment -

          Can you provide a minimal subversion repository to reproduce the problem?

          kutzi added a comment - Can you provide a minimal subversion repository to reproduce the problem?

          The problem is that this is configuration issue (yesterday I hadn't time to explain it all)

          I have another Jenkins installation (that I need to switch off) on a Ubuntu Linux 10.04 working against the same repository and it works fine with the same repo.

          The problem is that, for some reason, the Subversion plugin within Jenkins, is working on an incorrect encoding, contrary to the OS environment and Jenkins settings. I have managed to pass proper values ('UTF-8') to "file.encoding" and "sun.jnu.encoding" at Jenkins command line and it now displays accented words properly (like my name "José Luis Vázquez González") BUT the subversion plugin does NOT use this config and checkouts the repository in something non UTF-8 (Latin1 maybe?)

          It is NOT a Subversion plugin version problem either. I tested the same version that was working on the other server and it failed as well.

          I am sure it is a subversion plugin configuration or environment problem, and JUST in that plugin. (Remember, the svn and jsvn command line tools executed as the jenkins user will work just fine, checking out "descripción.txt" as expected)

          How can I FORCE the subversion plugin to use UTF-8 instead of whatever it is using now?

          Jose L Vazquez added a comment - The problem is that this is configuration issue (yesterday I hadn't time to explain it all) I have another Jenkins installation (that I need to switch off) on a Ubuntu Linux 10.04 working against the same repository and it works fine with the same repo. The problem is that, for some reason, the Subversion plugin within Jenkins, is working on an incorrect encoding, contrary to the OS environment and Jenkins settings. I have managed to pass proper values ('UTF-8') to "file.encoding" and "sun.jnu.encoding" at Jenkins command line and it now displays accented words properly (like my name "José Luis Vázquez González") BUT the subversion plugin does NOT use this config and checkouts the repository in something non UTF-8 (Latin1 maybe?) It is NOT a Subversion plugin version problem either. I tested the same version that was working on the other server and it failed as well. I am sure it is a subversion plugin configuration or environment problem, and JUST in that plugin. (Remember, the svn and jsvn command line tools executed as the jenkins user will work just fine, checking out "descripción.txt" as expected) How can I FORCE the subversion plugin to use UTF-8 instead of whatever it is using now?

          kutzi added a comment -

          AFAIK the subversion-plugin should use the system encoding.
          Have you checked that that the node (slave or master) which actually checks out the code has really UTF8 set?

          In any case this sounds more like a configuration problem on your side than a bug in the plugin. So I advise you to ask for help on the users mailing list.

          kutzi added a comment - AFAIK the subversion-plugin should use the system encoding. Have you checked that that the node (slave or master) which actually checks out the code has really UTF8 set? In any case this sounds more like a configuration problem on your side than a bug in the plugin. So I advise you to ask for help on the users mailing list.

          Yes, it is a config problem, but I still don't get it...

          What do you mean "that node (slave or master) which actually checks out the code"?

          1) The code is checked out locally, on the same server Jenkins is installed.

          2) I have logged as "jenkins" into the workspace directory that it is working on. And I know it is correct cause:

          • It tells me the local path ("Building in workspace /var/lib/jenkins/workspace/modcaja-a-test")
          • If I remove the workspace files from command line at that location it disappears from jenkins.
          • I can see the incorrect checkouts from jenkins and...
          • Jenkins can see proper checkouts if I do them on the command line instead.

          3) If I go to the workspace directory as jenkins I should have the same environment as the jenkins webapp, right?
          If there I do locale I get this:

          $ locale
          LANG=es_ES.UTF-8
          LANGUAGE=en_US.UTF-8
          LC_CTYPE="en_US.UTF-8"
          LC_NUMERIC="en_US.UTF-8"
          LC_TIME="en_US.UTF-8"
          LC_COLLATE="en_US.UTF-8"
          LC_MONETARY="en_US.UTF-8"
          LC_MESSAGES="en_US.UTF-8"
          LC_PAPER="en_US.UTF-8"
          LC_NAME="en_US.UTF-8"
          LC_ADDRESS="en_US.UTF-8"
          LC_TELEPHONE="en_US.UTF-8"
          LC_MEASUREMENT="en_US.UTF-8"
          LC_IDENTIFICATION="en_US.UTF-8"
          LC_ALL=en_US.UTF-8

          Everything is UTF-8, yes LANG is different from the rest, but the point is that ALL is UTF-8!
          right?

          4) If I do a svn co ... from there in the command line it is done correctly. So the environment seems ok right?

          5) EVEN if I do jsvn co ... from there it is done correctly. And jsvn is the command line of svnkit, which is what the jenkins plugin is using. right?

          6) SO how come that, with jenkins having file.encoding=UTF-8 and sun.jnu.encoding=UTF-8, the subversion plugin DOES NOT use UTF-8?
          What environment is it seen? (the jenkins linux user sees a different utf-8 one for sure)
          Can I override it?

          Jose L Vazquez added a comment - Yes, it is a config problem, but I still don't get it... What do you mean "that node (slave or master) which actually checks out the code"? 1) The code is checked out locally, on the same server Jenkins is installed. 2) I have logged as "jenkins" into the workspace directory that it is working on. And I know it is correct cause: It tells me the local path ("Building in workspace /var/lib/jenkins/workspace/modcaja-a-test") If I remove the workspace files from command line at that location it disappears from jenkins. I can see the incorrect checkouts from jenkins and... Jenkins can see proper checkouts if I do them on the command line instead. 3) If I go to the workspace directory as jenkins I should have the same environment as the jenkins webapp, right? If there I do locale I get this: $ locale LANG=es_ES.UTF-8 LANGUAGE=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=en_US.UTF-8 Everything is UTF-8, yes LANG is different from the rest, but the point is that ALL is UTF-8! right? 4) If I do a svn co ... from there in the command line it is done correctly. So the environment seems ok right? 5) EVEN if I do jsvn co ... from there it is done correctly. And jsvn is the command line of svnkit, which is what the jenkins plugin is using. right? 6) SO how come that, with jenkins having file.encoding=UTF-8 and sun.jnu.encoding=UTF-8, the subversion plugin DOES NOT use UTF-8? What environment is it seen? (the jenkins linux user sees a different utf-8 one for sure) Can I override it?

          kutzi added a comment -

          Maybe related to http://issues.tmatesoft.com/issue/SVNKIT-301

          Which jsvn version have you used to try it on the commandline?

          kutzi added a comment - Maybe related to http://issues.tmatesoft.com/issue/SVNKIT-301 Which jsvn version have you used to try it on the commandline?

          Thanks,

          It is the one that comes with Linux 12.04:

          $ dpkg -l |grep svnkit
          ii libsvnkit-java 1.3.5+dfsg-1 pure Java Subversion client library
          ii svnkit 1.3.5+dfsg-1 pure Java Subversion client

          1.3.5 according to the command line.

          Jose L Vazquez added a comment - Thanks, It is the one that comes with Linux 12.04: $ dpkg -l |grep svnkit ii libsvnkit-java 1.3.5+dfsg-1 pure Java Subversion client library ii svnkit 1.3.5+dfsg-1 pure Java Subversion client 1.3.5 according to the command line.

          kutzi added a comment -

          Can yout try it with 1.7.5?

          kutzi added a comment - Can yout try it with 1.7.5?

          Yes, I downloaded it with wget like this:

          wget http://www.svnkit.com/org.tmatesoft.svn_1.7.5.standalone.zip
          unzip org.tmatesoft.svn_1.7.5.standalone.zip

          Then I tested it on a workspace tmp directory as the jenkins user in the command line like this:

          /home/admin/svnkit-1.7.5/bin/jsvn co svn+ssh://svn.rflabs.com/data/subversion/modcaja/trunk/modcaja

          And it checked out the file correctly:
          $ ls modcaja/especs/ -l
          total 9968
          ...
          rw-rw-r- 1 jenkins jenkins 2577064 Jun 21 15:34 WG S2S - Descripción Protocolo 3GS.PDF

          As expected.

          But withint Jenkins the plugin is unable!

          Jose L Vazquez added a comment - Yes, I downloaded it with wget like this: wget http://www.svnkit.com/org.tmatesoft.svn_1.7.5.standalone.zip unzip org.tmatesoft.svn_1.7.5.standalone.zip Then I tested it on a workspace tmp directory as the jenkins user in the command line like this: /home/admin/svnkit-1.7.5/bin/jsvn co svn+ssh://svn.rflabs.com/data/subversion/modcaja/trunk/modcaja And it checked out the file correctly: $ ls modcaja/especs/ -l total 9968 ... rw-rw-r - 1 jenkins jenkins 2577064 Jun 21 15:34 WG S2S - Descripción Protocolo 3GS.PDF As expected. But withint Jenkins the plugin is unable!

          Tobias Peters added a comment -

          I had the same problem and the issue was with environment variables
          the slave was running on a linux machine, started via ssh from master but the jvm process on the slave didn't get the environment parameter set up for the slave
          you can check by doing

          sudo cat /proc/[slave_pid]/environ

          so I added 'Prefix Start Slave Command' of

          export LANG=en_US.UTF-8;

          restarted slave, and checked to see if it was set in /proc/?/environ

          after that it did do the svn co correctly

          Tobias Peters added a comment - I had the same problem and the issue was with environment variables the slave was running on a linux machine, started via ssh from master but the jvm process on the slave didn't get the environment parameter set up for the slave you can check by doing sudo cat /proc/ [slave_pid] /environ so I added 'Prefix Start Slave Command' of export LANG=en_US.UTF-8; restarted slave, and checked to see if it was set in /proc/?/environ after that it did do the svn co correctly

          Thanks Tobias,
          That sounds like it would be the solution for me as well, but I have a couple of questions...

          1- How do I know the PID of the slave task that Jenkins is running so that I could check its environment?

          2- Where is the option 'Prefix Start Slave Command'?
          my Jenkins is in Spanish and I am having trouble finding that option. Is that a Jenkins general option or a per-project config attribute?

          I don't seem to find anything about configuring slaves on Jenkins. My config is a standalone one the web and the managed server is the same machine.

          Thanks again,

          Jose L Vazquez added a comment - Thanks Tobias, That sounds like it would be the solution for me as well, but I have a couple of questions... 1- How do I know the PID of the slave task that Jenkins is running so that I could check its environment? 2- Where is the option 'Prefix Start Slave Command'? my Jenkins is in Spanish and I am having trouble finding that option. Is that a Jenkins general option or a per-project config attribute? I don't seem to find anything about configuring slaves on Jenkins. My config is a standalone one the web and the managed server is the same machine. Thanks again,

          Tobias Peters added a comment -

          1)
          the pid should be easy to find, just by checking which java processes run on the node

          ps -aux | grep slave.jar

          should output something including a similar line

          jenkins 16699 0.1 11.9 1709700 245440 ? Sl Jul16 1:02 /usr/lib/jvm/sun-jdk-1.6/bin/java -jar slave.jar

          where as 2nd column (16699) is the pid

          2)
          the config entry is found in the node configuration page

          • launch method: "... via SSH..."
          • Advanced
          • "Prefix Start Slave Command", just beneath the "JVM Options"

          Tobias Peters added a comment - 1) the pid should be easy to find, just by checking which java processes run on the node ps -aux | grep slave.jar should output something including a similar line jenkins 16699 0.1 11.9 1709700 245440 ? Sl Jul16 1:02 /usr/lib/jvm/sun-jdk-1.6/bin/java -jar slave.jar where as 2nd column (16699) is the pid 2) the config entry is found in the node configuration page launch method: "... via SSH..." Advanced "Prefix Start Slave Command", just beneath the "JVM Options"

          Something is wrong here...

          ps aux |grep slave.jar
          never renders anything (and using the -aux instead makes my ubuntu box complain about ps syntax) It seems I don't have or use a slave.jar at all!?!

          Whereas using:

          ps aux |grep jenkins
          instead I get an extra process when a task is running on Jenkins (I guess that is the slave running):

          $ ps aux |grep jenkins

          jenkins 924 0.3 14.9 1332708 617724 ? Ssl Jul09 35:15 /usr/bin/java -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -jar /usr/share/jenkins/jenkins.war --webroot=/var/run/jenkins/war --httpPort=7070 --prefix=/jenkins --ajp13Port=-1 --preferredClassLoader=java.net.URLClassLoader --logfile=/var/log/jenkins/jenkins.log

          jenkins 26158 119 0.9 1194144 40652 ? Sl 13:21 0:03 /usr/lib/jvm/java-6-sun/bin/java -cp /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.3.jar:/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn/boot/plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main /var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn /run/jenkins/war/WEB-INF/lib/remoting-2.24.jar /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.3.jar 49861

          Maybe I have a strange jenkins version? Mine is Jenkins ver. 1.519

          When I do sudo cat /proc/[slave_pid]/environ I get something like:

          TERM=linuxJENKINS_HOME=/var/lib/jenkinsJAVA_HOME=/usr/lib/jvm/java-6-sunMAVEN_HOME=/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvnNODE_LABELS=masterXFILESEARCHPATH=/usr/dt/app-defaults/%L/DtUPSTART_JOB=jenkinsHUDSON_URL=http://rflabs.com/jenkins/JOB_URL=http://rflabs.com/jenkins/job/modcaja/PWD=/BUILD_ID=2013-07-17_13-18-44BUILD_TAG=jenkins-modcaja-274BUILD_URL=http://rflabs.com/jenkins/job/modcaja/274/NODE_NAME=masterNLSPATH=/usr/dt/lib/nls/msg/%L/%N.catBUILD_NUMBER=274LD_LIBRARY_PATH=/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/../lib/i386UPSTART_EVENTS=runlevelEXECUTOR_NUMBER=1JENKINS_SERVER_COOKIE=69dd0d0208e62d55LANGUAGE=en_US.UTF-8BUILD_DISPLAY_NAME=#274WORKSPACE=/var/lib/jenkins/workspace/modcajaM2_HOME=/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvnPATH=/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn/bin:/usr/lib/jvm/java-6-sun/bin:/usr/lib/jvm/java-6-sun/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/binJENKINS_URL=http://rflabs.com/jenkins/LC_ALL=en_US.UTF-8RUNLEVEL=2PREVLEVEL=NHOME=/var/lib/jenkinsJOB_NAME=modcajaHUDSON_COOKIE=76876651-d2a6-4b47-9be6-47aedb62179aHUDSON_SERVER_COOKIE=69dd0d0208e62d55SVN_REVISION_1=1114SVN_REVISION=1114HUDSON_HOME=/var/lib/jenkinsSVN_URL=svn+ssh://svn.rflabs.com/data/subversion/modcaja/trunk/modcajaSVN_URL_1=svn+ssh://svn.rflabs.com/data/subversion/modcaja/trunk/modcajaLANG=en_US.UTF-8

          In which every environment variable about locale seems to be correctly pointing to en_US.UTF-8

          I can't find the "node configuration page" either, I guess I don't know how it was translated to Spanish or maybe I am using a different Jenkins version.

          Can I force my jenkins into English? how?

          Jose L Vazquez added a comment - Something is wrong here... ps aux |grep slave.jar never renders anything (and using the -aux instead makes my ubuntu box complain about ps syntax) It seems I don't have or use a slave.jar at all!?! Whereas using: ps aux |grep jenkins instead I get an extra process when a task is running on Jenkins (I guess that is the slave running): $ ps aux |grep jenkins jenkins 924 0.3 14.9 1332708 617724 ? Ssl Jul09 35:15 /usr/bin/java -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -jar /usr/share/jenkins/jenkins.war --webroot=/var/run/jenkins/war --httpPort=7070 --prefix=/jenkins --ajp13Port=-1 --preferredClassLoader=java.net.URLClassLoader --logfile=/var/log/jenkins/jenkins.log jenkins 26158 119 0.9 1194144 40652 ? Sl 13:21 0:03 /usr/lib/jvm/java-6-sun/bin/java -cp /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.3.jar:/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn/boot/plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main /var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn /run/jenkins/war/WEB-INF/lib/remoting-2.24.jar /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.3.jar 49861 Maybe I have a strange jenkins version? Mine is Jenkins ver. 1.519 When I do sudo cat /proc/ [slave_pid] /environ I get something like: TERM=linuxJENKINS_HOME=/var/lib/jenkinsJAVA_HOME=/usr/lib/jvm/java-6-sunMAVEN_HOME=/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvnNODE_LABELS=masterXFILESEARCHPATH=/usr/dt/app-defaults/%L/DtUPSTART_JOB=jenkinsHUDSON_URL= http://rflabs.com/jenkins/JOB_URL=http://rflabs.com/jenkins/job/modcaja/PWD=/BUILD_ID=2013-07-17_13-18-44BUILD_TAG=jenkins-modcaja-274BUILD_URL=http://rflabs.com/jenkins/job/modcaja/274/NODE_NAME=masterNLSPATH=/usr/dt/lib/nls/msg/%L/%N.catBUILD_NUMBER=274LD_LIBRARY_PATH=/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/../lib/i386UPSTART_EVENTS=runlevelEXECUTOR_NUMBER=1JENKINS_SERVER_COOKIE=69dd0d0208e62d55LANGUAGE=en_US.UTF-8BUILD_DISPLAY_NAME=#274WORKSPACE=/var/lib/jenkins/workspace/modcajaM2_HOME=/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvnPATH=/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/mvn/bin:/usr/lib/jvm/java-6-sun/bin:/usr/lib/jvm/java-6-sun/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/binJENKINS_URL=http://rflabs.com/jenkins/LC_ALL=en_US.UTF-8RUNLEVEL=2PREVLEVEL=NHOME=/var/lib/jenkinsJOB_NAME=modcajaHUDSON_COOKIE=76876651-d2a6-4b47-9be6-47aedb62179aHUDSON_SERVER_COOKIE=69dd0d0208e62d55SVN_REVISION_1=1114SVN_REVISION=1114HUDSON_HOME=/var/lib/jenkinsSVN_URL=svn+ssh://svn.rflabs.com/data/subversion/modcaja/trunk/modcajaSVN_URL_1=svn+ssh://svn.rflabs.com/data/subversion/modcaja/trunk/modcajaLANG=en_US.UTF-8 In which every environment variable about locale seems to be correctly pointing to en_US.UTF-8 I can't find the "node configuration page" either, I guess I don't know how it was translated to Spanish or maybe I am using a different Jenkins version. Can I force my jenkins into English? how?

          Tobias Peters added a comment -

          I'm running ver. 1.520

          jenkins uses the browsers language setting to determin gui language, in Firefox it is under
          Firefox > Preferences > Content > Languages

          the node configuration page is, Jenkins > Manage Jenkins > Manage Nodes > [node_name] > Configuration
          http://[jenkins_host]:7070/jenkins/computer/[node_name]/configure

          I'm not quite sure if I did grasp your server distibution right though.

          My Configuration is 'master' on windows, which is checking out svn just fine, and multiple slaves
          started via ssh from the master on linux (gentoo I think) which did have the utf-8 issues when
          checking out svn repos. the prefix command is only a setting for starting slaves via ssh

          You running multiple 'master' jenkins servers?
          from the looks of it the 2 proccesses you have is

          • (pid: 924) jenkins master server, the one with jenkins.war
          • (pid: 26158) jenkins job process

          and the environment variables you posted seem to be from the job process (pid: 26158), the svn checkout, at least
          for the slaves but I assume this goes for the master as well, are executed in the environment / process
          from the node in your case it should be the pid 924, so you would have to check that process and maybe
          adjust the startupto include language setting

          Tobias Peters added a comment - I'm running ver. 1.520 jenkins uses the browsers language setting to determin gui language, in Firefox it is under Firefox > Preferences > Content > Languages the node configuration page is, Jenkins > Manage Jenkins > Manage Nodes > [node_name] > Configuration http://[jenkins_host]:7070/jenkins/computer/[node_name]/configure I'm not quite sure if I did grasp your server distibution right though. My Configuration is 'master' on windows, which is checking out svn just fine, and multiple slaves started via ssh from the master on linux (gentoo I think) which did have the utf-8 issues when checking out svn repos. the prefix command is only a setting for starting slaves via ssh You running multiple 'master' jenkins servers? from the looks of it the 2 proccesses you have is (pid: 924) jenkins master server, the one with jenkins.war (pid: 26158) jenkins job process and the environment variables you posted seem to be from the job process (pid: 26158), the svn checkout, at least for the slaves but I assume this goes for the master as well, are executed in the environment / process from the node in your case it should be the pid 924, so you would have to check that process and maybe adjust the startupto include language setting

          Thanks, I use Chrome and I changed its language preferences so that now I get Jenkins in English.

          But, as you suspected, we have different configurations. I just have a single standalone jenkins installation, just ONE master, no slaves.

          I only have http://jenkins.rflabs.com/jenkins/computer/(master)/configure which only allows me to set:

          1. of executors Help for feature: # of executors
            Labels Help for feature: Labels
            Usage Help for feature: Usage
            Node Properties
            Environment variables

          I'll try to set "Environment variables" to see if that make any difference, but I don't have 'Prefix Start Slave Command'.

          I'll also try to check the environment for the main master process, as you suggested, but I already set up some environment variables for that.

          Jose L Vazquez added a comment - Thanks, I use Chrome and I changed its language preferences so that now I get Jenkins in English. But, as you suspected, we have different configurations. I just have a single standalone jenkins installation, just ONE master, no slaves. I only have http://jenkins.rflabs.com/jenkins/computer/(master)/configure which only allows me to set: of executors Help for feature: # of executors Labels Help for feature: Labels Usage Help for feature: Usage Node Properties Environment variables I'll try to set "Environment variables" to see if that make any difference, but I don't have 'Prefix Start Slave Command'. I'll also try to check the environment for the main master process, as you suggested, but I already set up some environment variables for that.

          I finally fixed it!,
          thanks to Tobias Peters indications, and the problem was configuration related but just for the Ubuntu Jenkins distribution.

          I checked the main process environment, as Tobias suggested, and it was wrong, something like this:
          JENKINS_HOME=/var/lib/jenkinsUPSTART_INSTANCE=UPSTART_JOB=jenkinsTERM=linuxPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/binPWD=/HOME=/var/lib/jenkinsLD_LIBRARY_PATH=/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/../lib/i386

          No LANG, no LANGUAGE, no LC_ALL configured and probably using the default which is not UTF-8.

          I noticed this ubuntu process is managed by upstart, so I end up finding that the environment for the process can be changed at /etc/init/jenkins.conf

          I added some lines at the script section:
          ...
          script
          [ -r /etc/default/jenkins ] && . /etc/default/jenkins
          export JENKINS_HOME
          + export LANG=en_US.UTF-8
          + export LANGUAGE=en_US.UTF-8
          + export LC_ALL=en_US.UTF-8
          exec start-stop-daemon --start -c $JENKINS_USER --exec $JAVA --name jenkins \
          – $JAVA_ARGS -jar $JENKINS_WAR $JENKINS_ARGS --logfile=$JENKINS_LOG
          end script

          And now it works!

          A better solution yet is just to add those three lines at the end of /etc/default/jenkins like this:

          export LANG=en_US.UTF-8
          export LANGUAGE=en_US.UTF-8
          export LC_ALL=en_US.UTF-8

          I am not sure if I need all of them but with these I get an environment such as:

          JENKINS_HOME=/var/lib/jenkinsUPSTART_INSTANCE=LANGUAGE=en_US.UTF-8UPSTART_JOB=jenkinsTERM=linuxPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/binLANG=en_US.UTF-8PWD=/LC_ALL=en_US.UTF-8HOME=/var/lib/jenkinsLD_LIBRARY_PATH=/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/../lib/i386

          That checks out files correctly and probably avoids other encoding problems in other plugins.

          Thanks for the tips Tobias!

          Jose L Vazquez added a comment - I finally fixed it!, thanks to Tobias Peters indications, and the problem was configuration related but just for the Ubuntu Jenkins distribution. I checked the main process environment, as Tobias suggested, and it was wrong, something like this: JENKINS_HOME=/var/lib/jenkinsUPSTART_INSTANCE=UPSTART_JOB=jenkinsTERM=linuxPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/binPWD=/HOME=/var/lib/jenkinsLD_LIBRARY_PATH=/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/../lib/i386 No LANG, no LANGUAGE, no LC_ALL configured and probably using the default which is not UTF-8. I noticed this ubuntu process is managed by upstart, so I end up finding that the environment for the process can be changed at /etc/init/jenkins.conf I added some lines at the script section: ... script [ -r /etc/default/jenkins ] && . /etc/default/jenkins export JENKINS_HOME + export LANG=en_US.UTF-8 + export LANGUAGE=en_US.UTF-8 + export LC_ALL=en_US.UTF-8 exec start-stop-daemon --start -c $JENKINS_USER --exec $JAVA --name jenkins \ – $JAVA_ARGS -jar $JENKINS_WAR $JENKINS_ARGS --logfile=$JENKINS_LOG end script And now it works! A better solution yet is just to add those three lines at the end of /etc/default/jenkins like this: export LANG=en_US.UTF-8 export LANGUAGE=en_US.UTF-8 export LC_ALL=en_US.UTF-8 I am not sure if I need all of them but with these I get an environment such as: JENKINS_HOME=/var/lib/jenkinsUPSTART_INSTANCE=LANGUAGE=en_US.UTF-8UPSTART_JOB=jenkinsTERM=linuxPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/binLANG=en_US.UTF-8PWD=/LC_ALL=en_US.UTF-8HOME=/var/lib/jenkinsLD_LIBRARY_PATH=/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.45/jre/../lib/i386 That checks out files correctly and probably avoids other encoding problems in other plugins. Thanks for the tips Tobias!

          Tobias Peters added a comment - - edited

          sure glad u could fix it

          arguable though if it is a bug or not, since the svn process runs in the node process with the node's environment and the environment variables you setup for the node are only used inside the job not for the nodes main process

          could someone comment on that?
          environment variables configured for a node aren't set for the main process of the node itself and svn checkout is done in that process, maybe the svn shouldn't run in the node's main process?

          Tobias Peters added a comment - - edited sure glad u could fix it arguable though if it is a bug or not, since the svn process runs in the node process with the node's environment and the environment variables you setup for the node are only used inside the job not for the nodes main process could someone comment on that? environment variables configured for a node aren't set for the main process of the node itself and svn checkout is done in that process, maybe the svn shouldn't run in the node's main process?

          Many years later...

          I just updated to latest Jenkins (1.615) and the Subversion plugin is 2.5, however:

          The checkout process does not properly checkout a file which has a unicode(ish) cahracter in it, specifically: "bigletter_人.png" comes out as "bigletter_?.png". I think this is related to the problem in this issue.
          I'm running on OS X Yosemite, installed through Homebrew (afaik). I think I did setup utf8 in the startup parameters like below, but it does not fix the problem:

          <?xml version="1.0" encoding="UTF-8"?>
          <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
          <plist version="1.0">
          <dict>
          	<key>Label</key>
          	<string>homebrew.mxcl.jenkins</string>
          	<key>ProgramArguments</key>
          	<array>
          		<string>/usr/bin/java</string>
          		<string>-Dmail.smtp.starttls.enable=true</string>
          		<string>-Dfile.encoding=utf-8</string>
          		<string>-Dsun.jnu.encoding=UTF-8</string>
          		<string>-jar</string>
          		<string>/usr/local/opt/jenkins/libexec/jenkins.war</string>
          		<string>--httpListenAddress=127.0.0.1</string>
          		<string>--httpPort=8088</string>
          	</array>
          	<key>RunAtLoad</key>
          	<true/>
          </dict>
          </plist>
          
          

          Jonny Bergstrom added a comment - Many years later... I just updated to latest Jenkins (1.615) and the Subversion plugin is 2.5, however: The checkout process does not properly checkout a file which has a unicode(ish) cahracter in it, specifically: "bigletter_人.png" comes out as "bigletter_?.png". I think this is related to the problem in this issue. I'm running on OS X Yosemite, installed through Homebrew (afaik). I think I did setup utf8 in the startup parameters like below, but it does not fix the problem: <?xml version= "1.0" encoding= "UTF-8" ?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd" > <plist version= "1.0" > <dict> <key> Label </key> <string> homebrew.mxcl.jenkins </string> <key> ProgramArguments </key> <array> <string> /usr/bin/java </string> <string> -Dmail.smtp.starttls.enable=true </string> <string> -Dfile.encoding=utf-8 </string> <string> -Dsun.jnu.encoding=UTF-8 </string> <string> -jar </string> <string> /usr/local/opt/jenkins/libexec/jenkins.war </string> <string> --httpListenAddress=127.0.0.1 </string> <string> --httpPort=8088 </string> </array> <key> RunAtLoad </key> <true/> </dict> </plist>

          Issue still exists with subversion plugin 2.7.2.

          Daniel Fischer added a comment - Issue still exists with subversion plugin 2.7.2.

            Unassigned Unassigned
            josvazg Jose L Vazquez
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: