Jenkins 2.255

      P4Plugin for Jenkins 1.11.0

       

      When setting utf8-bom in charset, files with unicode filetype (see unicode.png) are written in the workspace without any utf8 BOM (see utf8_bom_missing.png). This is not reproducible with p4v (see utf8_bom.png).

       

      I reproduced this issue with the following Pipeline script:

       

      pipeline {

          agent any

          stages {

              stage('sync'){

                  steps

      { p4sync charset: 'utf8-bom', credential: '192.168.1.7', populate: autoClean(delete: true, modtime: false, parallel: [enable: false,   minbytes: '1024',   minfiles: '1',   threads: '4'], pin: '', quiet: true, replace: true, tidy: false), source: depotSource('//depot/...')     }

              }

          }

      }

        1. unicode.png
          unicode.png
          84 kB
        2. utf8_bom.png
          utf8_bom.png
          88 kB
        3. utf8_missing_bom.png
          utf8_missing_bom.png
          89 kB
        4. utf8.txt
          0.0 kB

          [JENKINS-63619] BOM is missing on unicode file.

          Kevin Park added a comment -

          Kevin Park added a comment - This could be related to https://issues.jenkins-ci.org/browse/JENKINS-49141

          Karl Wirth added a comment -

          Reproduced same  problem. Tried testing with a checkout step also.

          (1) Create a Perforce unicode enabled server (p4d -r . -xi to vonvert a normal server)

          (2) Create a workspace with P4CHARSET=utf8.

          (3) Submit the attached file called utf8.txt. Its contents when check with 'od -a' is:

          0000000   o   ;   ?   a   a   a   a   a   a   a   a  nl
          0000014
          

          (4) Create a build job with the following code:

          pipeline {
              agent any
              stages {
                  stage('sync'){
                      steps
          	{ 
          	checkout perforce(credential: 'MasterUnicode', populate: forceClean(have: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: false), workspace: manualSpec(charset: 'utf8-bom', cleanup: false, name: 'jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}', pinHost: false, spec: clientSpec(allwrite: false, backup: true, changeView: '', clobber: true, compress: false, line: 'LOCAL', locked: false, modtime: false, rmdir: false, serverID: '', streamName: '', type: 'WRITABLE', view: '//depot/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}/...')))
          
          	sh label: 'OD -A OUTPUT:', script: 'od -a utf8.txt'	
          	  }
                  }
              }}
          
          

          (5) Create the pipeline job with a Linux slave (or build on Linux master).

          (6) The 'od -a' will shows the contents of the file. Note it is missing the 3 first characters (UTF8 BOM):

          [Pipeline] sh (OD -A OUTPUT:)
          + od -a utf8.txt
          0000000   a   a   a   a   a   a   a   a  nl
          0000011
          

          (7) Force sync the same file back to the utf8 workspace (outside Jenkins) where P4CHARSET=utf8 and the BOM is there:

          $ od -a utf8.txt 
          0000000   o   ;   ?   a   a   a   a   a   a   a   a  nl
          0000014

          (8) Force sync the same file with P4CHARSET=utf8-bom and the BOM is there:

          $ od -a utf8.txt 
          0000000   o   ;   ?   a   a   a   a   a   a   a   a  nl
          0000014

           

           

          Karl Wirth added a comment - Reproduced same  problem. Tried testing with a checkout step also. (1) Create a Perforce unicode enabled server (p4d -r . -xi to vonvert a normal server) (2) Create a workspace with P4CHARSET=utf8. (3) Submit the attached file called utf8.txt. Its contents when check with 'od -a' is: 0000000 o ; ? a a a a a a a a nl 0000014 (4) Create a build job with the following code: pipeline { agent any stages { stage( 'sync' ){ steps { checkout perforce(credential: 'MasterUnicode' , populate: forceClean(have: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: '', quiet: false ), workspace: manualSpec(charset: ' utf8-bom ', cleanup: false , name: ' jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER} ', pinHost: false , spec: clientSpec(allwrite: false , backup: true , changeView: ' ', clobber: true , compress: false , line: ' LOCAL ', locked: false , modtime: false , rmdir: false , serverID: ' ', streamName: ' ', type: ' WRITABLE ', view: ' //depot/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}/...'))) sh label: 'OD -A OUTPUT:' , script: 'od -a utf8.txt' } } }} (5) Create the pipeline job with a Linux slave (or build on Linux master). (6) The 'od -a' will shows the contents of the file. Note it is missing the 3 first characters (UTF8 BOM): [Pipeline] sh (OD -A OUTPUT:) + od -a utf8.txt 0000000 a a a a a a a a nl 0000011 (7) Force sync the same file back to the utf8 workspace (outside Jenkins) where P4CHARSET=utf8 and the BOM is there: $ od -a utf8.txt 0000000 o ; ? a a a a a a a a nl 0000014 (8) Force sync the same file with P4CHARSET=utf8-bom and the BOM is there: $ od -a utf8.txt 0000000 o ; ? a a a a a a a a nl 0000014    

            Unassigned Unassigned
            s3037560 Kevin Park
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: