-
Bug
-
Resolution: Cannot Reproduce
-
Minor
-
Jenkins ver. 2.46.2
git 3.3.0
git-client 2.4.5
git-server 1.7
-
Powered by SuggestiMate
Please help with this case: We clone a project that contains some characters in the filename that are altered, this only happens with the uppercase letters.
The job ends without problems but the workspace is wrong.
If you do a git clone on the server console it will work and download without problems with the correct filenames.
We checked the environment variables to be es_ES.UTF-8, appending information to the environment variables and the result of the workspace
file.encoding = UTF8
sun.jnu.encoding UTF8
sun.management.compiler HotSpot 64-Bit Tiered Compilers
sun.os.patch.level unknown
svnkit.http.methods Digest,Basic,NTLM,Negotiate
svnkit.ssh2.persistent false
user.country ES
user.dir /
user.language es
Env
LANG es_ES.UTF-8
LANGUAGE es_ES:es
LC_ALL es_ES.UTF-8
Workspace ls:
drwxr-xr-x 4 user_jen jen 1024 ago 24 10:55 . drwxr-xr-x 5 user_jen jen 1024 ago 24 10:55 .. -rw-r--r-- 1 user_jen jen 11975 ago 24 10:55 configuración_de_código.docx drwxr-xr-x 8 user_jen jen 1024 ago 24 10:55 .git drwxr-xr-x 2 user_jen jen 1024 ago 24 10:55 histórico -rw-r--r-- 1 user_jen jen 11961 ago 24 10:55 LA_CONFIGURACI?N_DE_C?DIGO.docx
- error_git.jpg
- 29 kB
- error_git2.jpg
- 158 kB
[JENKINS-46422] Git plugin Wrong clone uppercase characters
regards,
In the job I have printed the variables and they are correct, I show the job log and the problem is only in uppercase
Lanzada por el usuario user_jen
[EnvInject] - Loading node environment variables.
[EnvInject] - Preparing an environment for the build.
[EnvInject] - Keeping Jenkins system variables.
[EnvInject] - Keeping Jenkins build variables.
[EnvInject] - Injecting as environment variables the properties content
LC_ALL=es_ES.UTF-8
LANG=es_ES.UTF-8
LANGUAGE=es_ES:es
[EnvInject] - Variables injected successfully.
[EnvInject] - Injecting contributions.
Ejecutando.en el espacio de trabajo /data/jenkins/jobs/test_wiki/workspace
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository git@xxxxxx:root/ChartestMTP.git
> /usr/bin/git init /data/jenkins/jobs/test_wiki/workspace # timeout=10
Fetching upstream changes from git@xxxxxxx:root/ChartestMTP.git
> /usr/bin/git --version # timeout=10
using GIT_ASKPASS to set credentials Usuario administrador GitLab
> /usr/bin/git fetch --tags --progress git@xxxxxxxxxx:root/ChartestMTP.git +refs/heads/:refs/remotes/origin/
> /usr/bin/git config remote.origin.url git@1xxxxxxxx:root/ChartestMTP.git # timeout=10
> /usr/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10
> /usr/bin/git config remote.origin.url git@xxxxxxxx:root/ChartestMTP.git # timeout=10
Cleaning workspace
> /usr/bin/git rev-parse --verify HEAD # timeout=10
No valid HEAD. Skipping the resetting
> /usr/bin/git clean -fdx # timeout=10
Fetching upstream changes from git@xxxxxxxx:root/ChartestMTP.git
using GIT_ASKPASS to set credentials Usuario administrador GitLab
> /usr/bin/git fetch --tags --progress git@xxxxxxxxx:root/ChartestMTP.git +refs/heads/:refs/remotes/origin/
> /usr/bin/git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> /usr/bin/git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision d0dca457b0ade1e8422dd50d63218a9ce8605cde (refs/remotes/origin/master)
> /usr/bin/git config core.sparsecheckout # timeout=10
> /usr/bin/git checkout -f d0dca457b0ade1e8422dd50d63218a9ce8605cde
> /usr/bin/git rev-list d0dca457b0ade1e8422dd50d63218a9ce8605cde # timeout=10
[workspace] $ /bin/sh -xe /tmp/hudson8097104860677771449.sh
+ echo es_ES.UTF-8
es_ES.UTF-8
+ echo es_ES.UTF-8
es_ES.UTF-8
+ env
BUILD_URL=http://xxxxxxxxxx:80/job/test_wiki/40/
XDG_SESSION_ID=c2
HOSTNAME=xxxxxxxxxx
HUDSON_SERVER_COOKIE=26981e101d10e5f6
SHELL=/bin/bash
HISTSIZE=1000
BUILD_TAG=jenkins-test_wiki-40
GIT_PREVIOUS_COMMIT=d0dca457b0ade1e8422dd50d63218a9ce8605cde
ROOT_BUILD_CAUSE=MANUALTRIGGER
WORKSPACE=/data/jenkins/jobs/test_wiki/workspace
LC_ALL=es_ES.UTF-8
JOB_URL=http://xxxxxxxxx:80/job/test_wiki/
RUN_CHANGES_DISPLAY_URL=http://xxxxxxxx:80/job/test_wiki/40/display/redirect?page=changes
USER=user_jen
NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat
GIT_COMMIT=d0dca457b0ade1e8422dd50d63218a9ce8605cde
JENKINS_HOME=/data/jenkins
MAIL=/var/spool/mail/user_jen
PATH=/usr/java/jdk1.8.0_131/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/x02tch01/.local/bin:/home/user_jen/bin
RUN_DISPLAY_URL=http://xxxxxxx:80/job/test_wiki/40/display/redirect
BUILD_CAUSE_MANUALTRIGGER=true
PWD=/data/jenkins/jobs/test_wiki/workspace
JAVA_HOME=/usr/java/jdk1.8.0_131
HUDSON_URL=http://xxxxxxxxx:80/
LANG=es_ES.UTF-8
JOB_NAME=test_wiki
XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt
BUILD_DISPLAY_NAME=#40
BUILD_CAUSE=MANUALTRIGGER
BUILD_ID=40
JENKINS_URL=http://xxxxxxx:80/
JOB_BASE_NAME=test_wiki
GIT_PREVIOUS_SUCCESSFUL_COMMIT=d0dca457b0ade1e8422dd50d63218a9ce8605cde
HISTCONTROL=ignoredups
HOME=/home/user_jen
SHLVL=2
LANGUAGE=es_ES:es
GIT_BRANCH=origin/master
EXECUTOR_NUMBER=6
JENKINS_SERVER_COOKIE=26981e101d10e5f6
GIT_URL=git@xxxxxxxx:root/ChartestMTP.git
NODE_LABELS=master
LOGNAME=user_jen
HUDSON_HOME=/data/jenkins
NODE_NAME=master
LESSOPEN=||/usr/bin/lesspipe.sh %s
BUILD_NUMBER=40
JOB_DISPLAY_URL=http://xxxxxxxx:80/job/test_wiki/display/redirect
JVM_OPTIONS=-Dfile.encoding=UTF8 -Dsun.jnu.encoding=UTF8 -Duser.country=ES -Duser.language=es
XDG_RUNTIME_DIR=/run/user/3744
ROOT_BUILD_CAUSE_MANUALTRIGGER=true
_=/bin/env
+ echo --------
--------
+ java -XshowSettings:properties -version
Property settings:
awt.toolkit = sun.awt.X11.XToolkit
file.encoding = UTF-8
file.encoding.pkg = sun.io
file.separator = /
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
java.awt.printerjob = sun.print.PSPrinterJob
java.class.path = .
java.class.version = 52.0
java.endorsed.dirs = /usr/java/jdk1.8.0_131/jre/lib/endorsed
java.ext.dirs = /usr/java/jdk1.8.0_131/jre/lib/ext
/usr/java/packages/lib/ext
java.home = /usr/java/jdk1.8.0_131/jre
java.io.tmpdir = /tmp
java.library.path = /usr/java/packages/lib/amd64
/usr/lib64
/lib64
/lib
/usr/lib
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.8.0_131-b11
java.specification.name = Java Platform API Specification
java.specification.vendor = Oracle Corporation
java.specification.version = 1.8
java.vendor = Oracle Corporation
java.vendor.url = http://java.oracle.com/
java.vendor.url.bug = http://bugreport.sun.com/bugreport/
java.version = 1.8.0_131
java.vm.info = mixed mode
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.name = Java Virtual Machine Specification
java.vm.specification.vendor = Oracle Corporation
java.vm.specification.version = 1.8
java.vm.vendor = Oracle Corporation
java.vm.version = 25.131-b11
line.separator = \n
os.arch = amd64
os.name = Linux
os.version = 3.10.0-514.el7.x86_64
path.separator = :
sun.arch.data.model = 64
sun.boot.class.path = /usr/java/jdk1.8.0_131/jre/lib/resources.jar
/usr/java/jdk1.8.0_131/jre/lib/rt.jar
/usr/java/jdk1.8.0_131/jre/lib/sunrsasign.jar
/usr/java/jdk1.8.0_131/jre/lib/jsse.jar
/usr/java/jdk1.8.0_131/jre/lib/jce.jar
/usr/java/jdk1.8.0_131/jre/lib/charsets.jar
/usr/java/jdk1.8.0_131/jre/lib/jfr.jar
/usr/java/jdk1.8.0_131/jre/classes
sun.boot.library.path = /usr/java/jdk1.8.0_131/jre/lib/amd64
sun.cpu.endian = little
sun.cpu.isalist =
sun.io.unicode.encoding = UnicodeLittle
sun.java.launcher = SUN_STANDARD
sun.jnu.encoding = UTF-8
sun.management.compiler = HotSpot 64-Bit Tiered Compilers
sun.os.patch.level = unknown
user.country = ES
user.dir = /data/jenkins/jobs/test_wiki/workspace
user.home = /home/user_jen
user.language = es
user.name = user_jen
user.timezone =
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
+ echo --------
--------
+ locale
LANG=es_ES.UTF-8
LC_CTYPE="es_ES.UTF-8"
LC_NUMERIC="es_ES.UTF-8"
LC_TIME="es_ES.UTF-8"
LC_COLLATE="es_ES.UTF-8"
LC_MONETARY="es_ES.UTF-8"
LC_MESSAGES="es_ES.UTF-8"
LC_PAPER="es_ES.UTF-8"
LC_NAME="es_ES.UTF-8"
LC_ADDRESS="es_ES.UTF-8"
LC_TELEPHONE="es_ES.UTF-8"
LC_MEASUREMENT="es_ES.UTF-8"
LC_IDENTIFICATION="es_ES.UTF-8"
LC_ALL=es_ES.UTF-8
Finished: SUCCESS
Are you certain that those characters are stored as the correct UTF-8 encoded characters in the repository? It could be that they were written on a Windows system in the Windows encoding, and thus are encoded in the git object store with a non UTF-8 character that appears correct on a Windows system using a non-UTF-8 character encoding, but not on Linux?
https://stackoverflow.com/questions/22827239/how-to-make-git-properly-display-utf-8-encoded-pathnames-in-the-console-window indicates that there were still character encoding fixes being made in 2015. If your command line git version is older than 2.13.0, you could consider updating to the latest version of command line git.
https://stackoverflow.com/questions/7113849/git-character-encoding-with-spanish-characters may also offer some help.
https://stackoverflow.com/questions/682691/git-dealing-with-mixed-encoding-filenames likewise may help.
If the Linux computer where you're hosting these builds is Red Hat or CentOS or Oracle Linux, then it likely has a very old command line git implementation. Since you indicate that command line git on the machine is well behaved, I assumed that was not the problem, but you may want to confirm on a newer git version, just in case that is relevant.
regards,
Thanks for the help but the problem persists, if I make a clone in the operating system console it works, that's why I think the problem is in the jenkins or the plugin.
Attachment image from console
I've been unable to duplicate your issue in my test case. The test case includes a file in the repository which I confirmed uses UTF-8 encoded characters for the file name "buildo.number", where each lower case vowel is replaced with the accented upper case character.
That test has executed on Ubuntu 16, CentOS 6, and Debian 8 successfully.
I did see an unexpected behavior when performing the checkout of that repository on Windows. Windows git has unicode support, but for reasons which are unclear to me, the checkout on WIndows reports a different file name than the checkout on Linux. I assume that is because my default Linux locale (en_US.UTF-8) supports UTF-8 encoded Spanish characters, while my default Windows locale (??) likely is not a UTF-8 locale.
Since you're reporting the problem on Linux, not Windows, I didn't spend further time investigating the issue on Windows. I suspect that it can be resolved by the correct Windows git settings.
I appreciate that you believe that the problem must be in the plugin, but I don't see an issue in my testing of the plugin. Please provide a repository which shows the problem, or precise steps to duplicate the problem beginning with an empty repository.
As far as I can tell, this is a problem in the environment of the agent where the build is running. It is not something that the git plugin can alter.
Assure that the agent running the build has the correct setting for LANG (or LC_ALL, or ...)