Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24138

Archived file names can change when moved from slave to master

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • Tested on Ubuntu and RHEL6. Currently using Jenkins 1.553 but the issue would be present in all current versions. The server runs in WebSphere Liberty Profile 8.5.5.1 and 8.5.5.2 and Java is an IBM JDK 1.6.0

      I have a Jenkins set up that uses slaves and archives files after a build. What I've found is that if there is a file that uses multi-byte characters in the name (like 日本語ファイル) then the name gets altered when uploaded to the Jenkins master. Using the master node does not have the issue, so I did some research and found that the problem actually lies in the version of Ant that is being used (1.8.3) and how it reads file names and converts the multi-byte characters down to one byte. It looks like the issue was fixed in Apache Ant 1.9.0

        1. JapaneseName.png
          JapaneseName.png
          5 kB
        2. JapaneseNameBroken.png
          JapaneseNameBroken.png
          12 kB
        3. Output.png
          Output.png
          16 kB

          [JENKINS-24138] Archived file names can change when moved from slave to master

          Chris Rees added a comment -

          I should probably add in that the Jenkins server is being run in WebSphere Liberty Profile 8.5.5.2 and Java is an IBM 1.6.0. We tracked the issue down to TarUtils by debugging the running Jenkins server

          WLP Info (dev machine):
          ps aux | grep wlp
          crr 1460 0.0 0.0 4444 780 ? S 10:37 0:00 /bin/sh /opt/ibm/wlp/bin/server run Home
          crr 1523 53.7 2.1 4906124 343144 ? Sl 10:37 1:00 /usr/lib/j2sdk1.6-ibm/jre/bin/java -XX:MaxPermSize=256m -Darchaius.deployment.applicationId=pipeline -Darchaius.deployment.environment=dev -Darchaius.dynamicPropertyFactory.registerConfigWithJMX=true -Xdebug -Xrunjdwp:transport=dt_socket,address=localhost:10000,server=y,suspend=n -Dcom.sun.management.jmxremote -Dfile.encoding=UTF-8 -Xquickstart -Dwas4d.error.page=localhost:2425/108267124 -javaagent:/opt/ibm/wlp/bin/tools/ws-javaagent.jar -jar /opt/ibm/wlp/bin/tools/ws-server.jar Home

          Java:
          java version "1.6.0"
          Java(TM) SE Runtime Environment (build pxa6460sr16-20140418_01(SR16))
          IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr16-20140416_196573 (JIT enabled, AOT enabled)
          J9VM - 20140416_196573
          JIT - r9_20130920_46510ifx5
          GC - GA24_Java6_SR16_20140416_1614_B196573)
          JCL - 20140406_01

          Chris Rees added a comment - I should probably add in that the Jenkins server is being run in WebSphere Liberty Profile 8.5.5.2 and Java is an IBM 1.6.0. We tracked the issue down to TarUtils by debugging the running Jenkins server WLP Info (dev machine): ps aux | grep wlp crr 1460 0.0 0.0 4444 780 ? S 10:37 0:00 /bin/sh /opt/ibm/wlp/bin/server run Home crr 1523 53.7 2.1 4906124 343144 ? Sl 10:37 1:00 /usr/lib/j2sdk1.6-ibm/jre/bin/java -XX:MaxPermSize=256m -Darchaius.deployment.applicationId=pipeline -Darchaius.deployment.environment=dev -Darchaius.dynamicPropertyFactory.registerConfigWithJMX=true -Xdebug -Xrunjdwp:transport=dt_socket,address=localhost:10000,server=y,suspend=n -Dcom.sun.management.jmxremote -Dfile.encoding=UTF-8 -Xquickstart -Dwas4d.error.page=localhost:2425/108267124 -javaagent:/opt/ibm/wlp/bin/tools/ws-javaagent.jar -jar /opt/ibm/wlp/bin/tools/ws-server.jar Home Java: java version "1.6.0" Java(TM) SE Runtime Environment (build pxa6460sr16-20140418_01(SR16)) IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr16-20140416_196573 (JIT enabled, AOT enabled) J9VM - 20140416_196573 JIT - r9_20130920_46510ifx5 GC - GA24_Java6_SR16_20140416_1614_B196573) JCL - 20140406_01

          Chris Rees added a comment -

          We patched the 1.8.3 ant with the fixes from 1.9.0 and dropped it into the Jenkins war and confirmed that Ant 1.9.0 would solve the issue.

          Chris Rees added a comment - We patched the 1.8.3 ant with the fixes from 1.9.0 and dropped it into the Jenkins war and confirmed that Ant 1.9.0 would solve the issue.

          Daniel Beck added a comment -

          Would probably be best if you could post the fix as a pull request on Github.

          Daniel Beck added a comment - Would probably be best if you could post the fix as a pull request on Github.

          Chris Rees added a comment -

          I'll see what I can do. The fix was to change some of the Ant 1.8.3 behavior to match that of 1.9.0, which means Jenkins would either need to keep a modified copy of the file in source or update dependencies.

          Chris Rees added a comment - I'll see what I can do. The fix was to change some of the Ant 1.8.3 behavior to match that of 1.9.0, which means Jenkins would either need to keep a modified copy of the file in source or update dependencies.

          Chris Rees added a comment -

          Another option would be to add Apache Commons Compress as a dependency and use the Tar classes there instead of the ones in Ant 1.8.3

          Chris Rees added a comment - Another option would be to add Apache Commons Compress as a dependency and use the Tar classes there instead of the ones in Ant 1.8.3

          Yoerg added a comment -

          Four years later, I still have this Issue in 2.121.1 when archiving artifacts from a Linux agent (file.encoding UTF-8, sun.io.unicode.encoding UnicodeLittle, sun.jnu.encoding UTF-8) to a Windows master (file.encoding Cp1252, sun.io.unicode.encoding UnicodeLittle, sun.jnu.encoding Cp1252).

          Yoerg added a comment - Four years later, I still have this Issue in 2.121.1 when archiving artifacts from a Linux agent (file.encoding UTF-8, sun.io.unicode.encoding UnicodeLittle, sun.jnu.encoding UTF-8) to a Windows master (file.encoding Cp1252, sun.io.unicode.encoding UnicodeLittle, sun.jnu.encoding Cp1252).

          Kim Abbott added a comment -

          I too am still having this issue.  Running with:

          Jenkins 2.176.1
          Copy Artifact Plugin 1.42.1
          Master Linux
          Slave Windows 8 Server

          File pulls from repo onto Windows slave with correct filename, but when archived back to master the ™ gets garbled.  So the rest of my deploy steps fail because I now have a file with an invalid filename.  

          Its unclear if any changes from 2014 were ever implemented?  Even version 1.9.0 is pretty dated, so seems like core is due to update and update dependencies?  I don't know of any workaround at this point.

          Kim Abbott added a comment - I too am still having this issue.  Running with: Jenkins 2.176.1 Copy Artifact Plugin  1.42.1 Master Linux Slave Windows 8 Server File pulls from repo onto Windows slave with correct filename, but when archived back to master the ™ gets garbled.  So the rest of my deploy steps fail because I now have a file with an invalid filename.   Its unclear if any changes from 2014 were ever implemented?  Even version 1.9.0 is pretty dated, so seems like core is due to update and update dependencies?  I don't know of any workaround at this point.

          This article presented a good workaround for this issue. Worked for me.

          Tomasz Fijarczyk added a comment - This article presented a good workaround for this issue. Worked for me.

          Kim Abbott added a comment -

          Thank you, looks promising, will take a look.

          Did you add this via global properties or via EnvInject API Plugin?

          Kim Abbott added a comment - Thank you, looks promising, will take a look. Did you add this via global properties or via  EnvInject API Plugin ?

          I just followed article guidelines and used Windows env variables .

          Tomasz Fijarczyk added a comment - I just followed article guidelines and used Windows env variables .

            Unassigned Unassigned
            serneum Chris Rees
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: