Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37584

P4 plugin : P4_CHANGELIST not available in workflow (pipeline)

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • p4-plugin
    • None
    • jenkins 2.7.2 LTS
      p4-plgin 1.3.8
      jdk 1.8 (sun)
      OS : redhat 7.2

      I'm using p4 plugin for several months in freestyle jobs, and I'm moving to 'pipelines' implementation (workflow if you like)

      According to this documentation, there are some known limitations :
      https://github.com/jenkinsci/p4-plugin/blob/master/SETUP.md

      especially :

      No access to Environment ${VAR} variables

      On the other hand, one line below is written :

      Exposed Variables :

      • P4_CHANGELIST - current changelist (valid for this to be blank when there are no changes).

      Within my pipelines, what I see is that 'P4_CHANGELIST' is not available after a call to p4sync.

      Is it expected ?

      If so, how do I retrieve the changelist at which the sync is done ? (in order to propagate it to further steps and stages in the pipeline)

          [JENKINS-37584] P4 plugin : P4_CHANGELIST not available in workflow (pipeline)

          p4paul Would you care to comment on how this was fixed? Do we have to upgrade to 2.60+ for this to work? Do we need other syntax?

          Staffan Forsell added a comment - p4paul Would you care to comment on how this was fixed? Do we have to upgrade to 2.60+ for this to work? Do we need other syntax?

          Paul Allen added a comment -

          Jenkins added a fix in workflow-scm-step 2.6 and requires core 2.60+

          https://wiki.jenkins.io/display/JENKINS/Pipeline+SCM+Step+Plugin

          This is called 'Pipeline: SCM Step' in Installed plugins list.

          Paul Allen added a comment - Jenkins added a fix in workflow-scm-step 2.6 and requires core 2.60+ https://wiki.jenkins.io/display/JENKINS/Pipeline+SCM+Step+Plugin This is called 'Pipeline: SCM Step' in Installed plugins list.

          Staffan Forsell added a comment - - edited

          p4paul Ok, so now I have updated my jenkins to 2.60.3 and p4 plugin 1.7.4.

          I'm still not seeing the environment variables being populated. I thought this was supposed to happen. Otherwise https://github.com/jenkinsci/p4-plugin/blob/master/SETUP.md#exposed-variables should probably be updated.

          I also tested with:

          def p4vars = p4sync charset: 'none', credential: 'cred', populate: forceClean(have: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: 'now', quiet: true), source: depotSource('//depot/stuff/...')
          

          In this case, p4vars.P4_CHANGELIST returns 'now', previously P4_CHANGELIST was expanded to the actual numerical changelist.
          I can't find the proper way to get the actual P4_CHANGELIST that was synced now.

          Staffan Forsell added a comment - - edited p4paul Ok, so now I have updated my jenkins to 2.60.3 and p4 plugin 1.7.4. I'm still not seeing the environment variables being populated. I thought this was supposed to happen. Otherwise https://github.com/jenkinsci/p4-plugin/blob/master/SETUP.md#exposed-variables  should probably be updated. I also tested with: def p4vars = p4sync charset: 'none' , credential: 'cred' , populate: forceClean(have: false , parallel: [enable: false , minbytes: '1024' , minfiles: '1' , threads: '4' ], pin: 'now' , quiet: true ), source: depotSource( ' //depot/stuff/...' ) In this case, p4vars.P4_CHANGELIST returns 'now', previously P4_CHANGELIST was expanded to the actual numerical changelist. I can't find the proper way to get the actual P4_CHANGELIST that was synced now.

          Paul Allen added a comment -

          Please do not use pin: 'now' this will override the change number and break polling.  Just leave pin as an empty string ('') to build the latest; for example:

          def env = p4sync credential:'phooey1666',
             format:"jenkins-${NODE_NAME}-${JOB_NAME}",
             stream:'//streams/st1-main',
             populate:[$class:'AutoCleanImpl', delete:true, have:true, modtime:false, quiet:false, pin:'']
          
          echo env.P4_CHANGELIST
          echo env.P4_CLIENT

          output: 

          [Pipeline] echo
          11593
          [Pipeline] echo
          jenkins-master-dsl1

          Paul Allen added a comment - Please do not use pin: 'now'  this will override the change number and break polling.  Just leave pin as an empty string ( '')  to build the latest; for example: def env = p4sync credential: 'phooey1666' , format: "jenkins-${NODE_NAME}-${JOB_NAME}" , stream: ' //streams/st1-main' , populate:[$class: 'AutoCleanImpl' , delete: true , have: true , modtime: false , quiet: false , pin:''] echo env.P4_CHANGELIST echo env.P4_CLIENT output:  [Pipeline] echo 11593 [Pipeline] echo jenkins-master-dsl1

          Staffan Forsell added a comment - - edited

          p4paul Ok, thanks for the feedback.
          Got the scripts working again after this change.
          Problem with some of the latest changes is that they break compatibility with previous scripts and thus we need to change Jenkinsfile on multiple old branches just to get them to build again. Also updating the steps references (https://jenkins.io/doc/pipeline/steps/p4/), especially p4sync would be appreciated.
          Currently there are a lot of similar options that are not documented properly see e.g. p4sync arguments depotPath vs source. Was 'depotPath' deprecated etc... Also some kind of semantic versioning scheme would be nice as my Jenkinsfile scripts tend to break even on one of these 1.7.0 -> 1.7.4 upgrades.

          Staffan Forsell added a comment - - edited p4paul Ok, thanks for the feedback. Got the scripts working again after this change. Problem with some of the latest changes is that they break compatibility with previous scripts and thus we need to change Jenkinsfile on multiple old branches just to get them to build again. Also updating the steps references ( https://jenkins.io/doc/pipeline/steps/p4/ ), especially p4sync would be appreciated. Currently there are a lot of similar options that are not documented properly see e.g. p4sync arguments depotPath vs source. Was 'depotPath' deprecated etc... Also some kind of semantic versioning scheme would be nice as my Jenkinsfile scripts tend to break even on one of these 1.7.0 -> 1.7.4 upgrades.

          Paul Allen added a comment -

          Thank you for letting me know about the URL (https://jenkins.io/doc/pipeline/steps/p4/).  Jenkins created this based on the code, but I am not sure how they update it?  I agree it is confusing to read, I normally use the 'Pipeline Syntax' snippet generator and help bubbles to workout what options are available.  The older depotPath vs source should be backwards compatible.

          For example 3 ways to define a stream on p4sync:

          stream: '//streams/foo'
          source: [$class: 'StreamSource', stream: '//streams/foo']
          source: streamSource('//streams/foo')

           

          Paul Allen added a comment - Thank you for letting me know about the URL ( https://jenkins.io/doc/pipeline/steps/p4/ ).  Jenkins created this based on the code, but I am not sure how they update it?  I agree it is confusing to read, I normally use the 'Pipeline Syntax' snippet generator and help bubbles to workout what options are available.  The older depotPath vs source should be backwards compatible. For example 3 ways to define a stream on p4sync : stream: ' //streams/foo' source: [$class: 'StreamSource' , stream: ' //streams/foo' ] source: streamSource( ' //streams/foo' )  

          I think the docs at the URL is generated from the same stuff as 'Pipeline syntax'. Basically there some options that are undocumented in 'Pipeline syntax' too. But the jenkins steps references could definitely be made more readable. Couldn't find a jira for that though... might file one tomorrow in jenkins-io.

          Our previous job looked essentially like:

          def p4Pin = 'now'
          p4sync credential: 'cred', depotPath: env.P4_DEPOT, populate: [$class: 'ForceCleanImpl', quiet: true], pin: p4Pin
          

          P4_DEPOT ="//depot/releases/3.4.x" was set in the job as a environment variable.
          The depotPath had no ending '/..' before and after the upgrade this started failing due to p4-plugin generating the incorrect view client view:

          View:
          	//depot/releases/3.4.x //jenkins-somenodename-thejobname/3.4.x
          

          I.e. it started mapping the wanted depot dir to a client sub dir instead of the root of the client sometime between 1.7.0 and 1.7.4.

          Staffan Forsell added a comment - I think the docs at the URL is generated from the same stuff as 'Pipeline syntax'. Basically there some options that are undocumented in 'Pipeline syntax' too. But the jenkins steps references could definitely be made more readable. Couldn't find a jira for that though... might file one tomorrow in jenkins-io. Our previous job looked essentially like: def p4Pin = 'now' p4sync credential: 'cred' , depotPath: env.P4_DEPOT, populate: [$class: 'ForceCleanImpl' , quiet: true ], pin: p4Pin P4_DEPOT ="//depot/releases/3.4.x" was set in the job as a environment variable. The depotPath had no ending '/..' before and after the upgrade this started failing due to p4-plugin generating the incorrect view client view: View: //depot/releases/3.4.x //jenkins-somenodename-thejobname/3.4.x I.e. it started mapping the wanted depot dir to a client sub dir instead of the root of the client sometime between 1.7.0 and 1.7.4.

          Ben Hines added a comment - - edited

          This doesn't work for me at all with the latest p4 plugin. (jenkins version: 2.68 which is higher than the 2.6x mentioned above) All other plugins up to date.  I just get null.

          I'm not using 'populate' like above though, i'm passing a workspace and spec:

           

          def p4env = p4sync poll: true, credential: '',
           workspace: [$class: 'org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl', charset: "", pinHost: false, name: client,
           spec: [$class: 'org.jenkinsci.plugins.p4.workspace.WorkspaceSpec', view: view]
           ]
          echo "Sync p4changelist:"
           echo p4env.P4_CHANGELIST
          

           

          also tried this as above:

          p4env = p4sync poll:true, credential:'e6b4d2aa-df16-4ab5-aa9d-e8a149f1096f',
          format: "<redacted>-${NODE_NAME}-${JOB_NAME}",
          stream:'<mystream>',
          populate:[$class:'AutoCleanImpl', delete:true, have:true, modtime:false, quiet:false, pin:'']
          
          echo "Sync p4changelist:"
          echo p4env.P4_CHANGELIST

           

           

          results for both:
          [Pipeline] echo
          Sync p4changelist:
          [Pipeline] echo
          null

          Ben Hines added a comment - - edited This doesn't work for me at all with the latest p4 plugin. (jenkins version: 2.68 which is higher than the 2.6x mentioned above) All other plugins up to date.  I just get null. I'm not using 'populate' like above though, i'm passing a workspace and spec:   def p4env = p4sync poll: true , credential: '', workspace: [$class: 'org.jenkinsci.plugins.p4.workspace.ManualWorkspaceImpl' , charset: "", pinHost: false , name: client, spec: [$class: 'org.jenkinsci.plugins.p4.workspace.WorkspaceSpec' , view: view] ] echo "Sync p4changelist:" echo p4env.P4_CHANGELIST   also tried this as above: p4env = p4sync poll: true , credential: 'e6b4d2aa-df16-4ab5-aa9d-e8a149f1096f' , format: "<redacted>-${NODE_NAME}-${JOB_NAME}" , stream: '<mystream>' , populate:[$class: 'AutoCleanImpl' , delete: true , have: true , modtime: false , quiet: false , pin:''] echo "Sync p4changelist:" echo p4env.P4_CHANGELIST     results for both: [Pipeline] echo Sync p4changelist: [Pipeline] echo null

          Code changed in jenkins
          User: Paul Allen
          Path:
          src/main/java/org/jenkinsci/plugins/p4/build/P4EnvironmentContributor.java
          http://jenkins-ci.org/commit/p4-plugin/65279c9d1158fa247e406d6146ef8395d39906c6
          Log:
          Copy environment map to EnvVars.

          JENKINS-37584

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Paul Allen Path: src/main/java/org/jenkinsci/plugins/p4/build/P4EnvironmentContributor.java http://jenkins-ci.org/commit/p4-plugin/65279c9d1158fa247e406d6146ef8395d39906c6 Log: Copy environment map to EnvVars. JENKINS-37584

          Paul Allen added a comment -

          You can download the latest HPI build here: https://ci.jenkins.io/job/Plugins/job/p4-plugin/job/master/100/

          Paul Allen added a comment - You can download the latest HPI build here:  https://ci.jenkins.io/job/Plugins/job/p4-plugin/job/master/100/

            p4karl Karl Wirth
            squalou squalou jenkins
            Votes:
            7 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated:
              Resolved: