Major I have a Jenkins set up that uses slaves and archives files after a build. What I've found is that if there is a file that uses multi-byte characters in the name (like 日本語ファイル) then the name gets altered when uploaded to the Jenkins master. Using the master node does not have the issue, so I did some research and found that the problem actually lies in the version of Ant that is being used (1.8.3) and how it reads file names and converts the multi-byte characters down to one byte. It looks like the issue was fixed in Apache Ant 1.9.0
- is duplicated by
-
JENKINS-24918 Filenames of archived artifacts become garbled when they contain non-ascii characters
-
- Resolved
-
[JENKINS-24138] Archived file names can change when moved from slave to master
Chris Rees
created issue -
| Component/s | New: core [ 15593 ] | |
| Component/s | Original: ant [ 15488 ] | |
| Affects Version/s | Original: current [ 10162 ] |
Chris Rees
added a comment - I tried setting the file.encoding property and it didn't change anything. The issue looked like it was in TarEntry and TarUtils within Ant when digging down to see why file names were being modified.
Chris Rees
added a comment - I tried setting the file.encoding property and it didn't change anything. The issue looked like it was in TarEntry and TarUtils within Ant when digging down to see why file names were being modified. Chris Rees
added a comment - I should probably add in that the Jenkins server is being run in WebSphere Liberty Profile 8.5.5.2 and Java is an IBM 1.6.0. We tracked the issue down to TarUtils by debugging the running Jenkins server
WLP Info (dev machine):
ps aux | grep wlp
crr 1460 0.0 0.0 4444 780 ? S 10:37 0:00 /bin/sh /opt/ibm/wlp/bin/server run Home
crr 1523 53.7 2.1 4906124 343144 ? Sl 10:37 1:00 /usr/lib/j2sdk1.6-ibm/jre/bin/java -XX:MaxPermSize=256m -Darchaius.deployment.applicationId=pipeline -Darchaius.deployment.environment=dev -Darchaius.dynamicPropertyFactory.registerConfigWithJMX=true -Xdebug -Xrunjdwp:transport=dt_socket,address=localhost:10000,server=y,suspend=n -Dcom.sun.management.jmxremote -Dfile.encoding=UTF-8 -Xquickstart -Dwas4d.error.page=localhost:2425/108267124 -javaagent:/opt/ibm/wlp/bin/tools/ws-javaagent.jar -jar /opt/ibm/wlp/bin/tools/ws-server.jar Home
Java:
java version "1.6.0"
Java(TM) SE Runtime Environment (build pxa6460sr16-20140418_01(SR16))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr16-20140416_196573 (JIT enabled, AOT enabled)
J9VM - 20140416_196573
JIT - r9_20130920_46510ifx5
GC - GA24_Java6_SR16_20140416_1614_B196573)
JCL - 20140406_01
Chris Rees
added a comment - I should probably add in that the Jenkins server is being run in WebSphere Liberty Profile 8.5.5.2 and Java is an IBM 1.6.0. We tracked the issue down to TarUtils by debugging the running Jenkins server
WLP Info (dev machine):
ps aux | grep wlp
crr 1460 0.0 0.0 4444 780 ? S 10:37 0:00 /bin/sh /opt/ibm/wlp/bin/server run Home
crr 1523 53.7 2.1 4906124 343144 ? Sl 10:37 1:00 /usr/lib/j2sdk1.6-ibm/jre/bin/java -XX:MaxPermSize=256m -Darchaius.deployment.applicationId=pipeline -Darchaius.deployment.environment=dev -Darchaius.dynamicPropertyFactory.registerConfigWithJMX=true -Xdebug -Xrunjdwp:transport=dt_socket,address=localhost:10000,server=y,suspend=n -Dcom.sun.management.jmxremote -Dfile.encoding=UTF-8 -Xquickstart -Dwas4d.error.page=localhost:2425/108267124 -javaagent:/opt/ibm/wlp/bin/tools/ws-javaagent.jar -jar /opt/ibm/wlp/bin/tools/ws-server.jar Home
Java:
java version "1.6.0"
Java(TM) SE Runtime Environment (build pxa6460sr16-20140418_01(SR16))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr16-20140416_196573 (JIT enabled, AOT enabled)
J9VM - 20140416_196573
JIT - r9_20130920_46510ifx5
GC - GA24_Java6_SR16_20140416_1614_B196573)
JCL - 20140406_01 Chris Rees
made changes -
| Environment | Original: Tested on Ubuntu and RHEL6. Currently using Jenkins 1.553 but the issue would be present in all current versions | New: Tested on Ubuntu and RHEL6. Currently using Jenkins 1.553 but the issue would be present in all current versions. The server runs in WebSphere Liberty Profile 8.5.5.1 and 8.5.5.2 and Java is an IBM JDK 1.6.0 |
Chris Rees
added a comment - We patched the 1.8.3 ant with the fixes from 1.9.0 and dropped it into the Jenkins war and confirmed that Ant 1.9.0 would solve the issue.
Chris Rees
added a comment - We patched the 1.8.3 ant with the fixes from 1.9.0 and dropped it into the Jenkins war and confirmed that Ant 1.9.0 would solve the issue. Would probably be best if you could post the fix as a pull request on Github.
Chris Rees
added a comment - I'll see what I can do. The fix was to change some of the Ant 1.8.3 behavior to match that of 1.9.0, which means Jenkins would either need to keep a modified copy of the file in source or update dependencies.
Chris Rees
added a comment - I'll see what I can do. The fix was to change some of the Ant 1.8.3 behavior to match that of 1.9.0, which means Jenkins would either need to keep a modified copy of the file in source or update dependencies. Chris Rees
added a comment - Another option would be to add Apache Commons Compress as a dependency and use the Tar classes there instead of the ones in Ant 1.8.3
Chris Rees
added a comment - Another option would be to add Apache Commons Compress as a dependency and use the Tar classes there instead of the ones in Ant 1.8.3
Setting file.encoding system property has no effect?