Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55315

Obtain target changelist from PerforceScm object in pipeline

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin

      We have a multibranch pipeline which after being triggered immediately splits into 6 stages for various linux and windows platforms. The sequence of each of the pipelines goes like this:

      • Obtain branch source (checkout scm)
      • Obtain platform specific thirdparties (p4sync)
      • Obtain common thirdparties (p4sync)
      • Build/Test/Archive/Etc

      The problem I am having is pinning the thirdparty syncs to the same changelist as the branch, because each instance of a sync can set the P4_CHANGELIST environment variable (which seems to be updated globally) to a different value, so unless I perform a single sync ahead of all the others just to get the changelist and store it, I have a race condition.

      What I would like to do is be able to put this at the start of my script, before I enter the pipeline block:

      env.GLOBAL_P4_CHANGELIST = scm.getChangelist()
      

      Which would ensure I have a global reference to the changelist that triggered the build to use for pinning the subsequent thirdparty syncs,  that isn't overridden by any subsequent uses of p4sync.

      I see that PerforceScm has a property P4Ref revision. Can this be exposed in the manner I'd like?

       

      My current workaround consists of either wrapping an initial checkout in a lock step so that I can set the global variable after a first sync has completed (this delays all other parallel stages by the time it takes to sync, which can be up to 10 minutes currently)

      lock("$env.JOB_NAME-GLOBAL_P4_CHANGELIST"){
          if ( !env.GLOBAL_P4_CHANGELIST ){
              checkout scm
              env.GLOBAL_P4_CHANGELIST = "$env.P4_CHANGELIST"
              env.GLOBAL_P4_CHANGELIST_STAGE = "$env.STAGE_NAME"
          }
          echo "GLOBAL_P4_CHANGELIST = $env.GLOBAL_P4_CHANGELIST"
          echo "GLOBAL_P4_CHANGELIST_STAGE = $env.GLOBAL_P4_CHANGELIST_STAGE"
      }
      if ( ! (env.STAGE_NAME == $env.GLOBAL_P4_CHANGELIST_STAGE) ){
          checkout scm
      }
      

      or by trying to use a lock on updating the variable immediately after a checkout scm, which still has a race condition.

          [JENKINS-55315] Obtain target changelist from PerforceScm object in pipeline

          Alisdair Robertson created issue -
          Alisdair Robertson made changes -
          Description Original: We have a multibranch pipeline which after being triggered immediately splits into 6 stages for various linux and windows platforms. The sequence of each of the pipelines goes like this:
           - Obtain branch source ({{checkout scm}})
           - Obtain platform specific thirdparties ({{p4sync}})
           - Obtain common thirdparties ({{p4sync}})
           - Build/Test/Archive/Etc

          The problem I am having is pinning the thirdparty syncs to the same changelist as the branch, because each instance of a sync can set the {{P4_CHANGELIST}} environment variable (which seems to be updated globally) to a different value, so unless I perform a single sync ahead of all the others just to get the changelist and store it, I have a race condition.

          What I would like to do is be able to put this at the start of my script, before I enter the pipeline block:
          {code}
          env.GLOBAL_P4_CHANGELIST = scm.getChangelist()
          {code}
          Which would ensure I have a global reference to the changelist that triggered the build to use for pinning the subsequent thirdparty syncs,  that isn't overridden by any subsequent uses of {{p4sync}}.

          I see that {{PerforceScm}} has a property {{P4Ref revision}}. Can this be exposed in the manner I'd like?

           

          My current workaround consists of either wrapping an initial checkout in a {{lock}} step so that I can set the global variable after a first sync has completed (this delays all other parallel stages by the time it takes to sync, which can be up to 10 minutes currently)
          {code:groovy}
          lock("$env.JOB_NAME-GLOBAL_P4_CHANGELIST"){
              if ( !env.GLOBAL_P4_CHANGELIST ){
                  checkout source_control_management
                  env.GLOBAL_P4_CHANGELIST = "$env.P4_CHANGELIST"
                  env.GLOBAL_P4_CHANGELIST_STAGE = "$env.STAGE_NAME"
              }
              echo "GLOBAL_P4_CHANGELIST = $env.GLOBAL_P4_CHANGELIST"
              echo "GLOBAL_P4_CHANGELIST_STAGE = $env.GLOBAL_P4_CHANGELIST_STAGE"
          }
          if ( ! (env.STAGE_NAME == $env.GLOBAL_P4_CHANGELIST_STAGE) ){
              checkout source_control_management
          }
          {code}
          or by trying to use a lock on updating the variable immediately after a {{checkout scm}}, which still has a race condition.
          New: We have a multibranch pipeline which after being triggered immediately splits into 6 stages for various linux and windows platforms. The sequence of each of the pipelines goes like this:
           - Obtain branch source ({{checkout scm}})
           - Obtain platform specific thirdparties ({{p4sync}})
           - Obtain common thirdparties ({{p4sync}})
           - Build/Test/Archive/Etc

          The problem I am having is pinning the thirdparty syncs to the same changelist as the branch, because each instance of a sync can set the {{P4_CHANGELIST}} environment variable (which seems to be updated globally) to a different value, so unless I perform a single sync ahead of all the others just to get the changelist and store it, I have a race condition.

          What I would like to do is be able to put this at the start of my script, before I enter the pipeline block:
          {code:java}
          env.GLOBAL_P4_CHANGELIST = scm.getChangelist()
          {code}
          Which would ensure I have a global reference to the changelist that triggered the build to use for pinning the subsequent thirdparty syncs,  that isn't overridden by any subsequent uses of {{p4sync}}.

          I see that {{PerforceScm}} has a property {{P4Ref revision}}. Can this be exposed in the manner I'd like?

           

          My current workaround consists of either wrapping an initial checkout in a {{lock}} step so that I can set the global variable after a first sync has completed (this delays all other parallel stages by the time it takes to sync, which can be up to 10 minutes currently)
          {code}
          lock("$env.JOB_NAME-GLOBAL_P4_CHANGELIST"){
              if ( !env.GLOBAL_P4_CHANGELIST ){
                  checkout scm
                  env.GLOBAL_P4_CHANGELIST = "$env.P4_CHANGELIST"
                  env.GLOBAL_P4_CHANGELIST_STAGE = "$env.STAGE_NAME"
              }
              echo "GLOBAL_P4_CHANGELIST = $env.GLOBAL_P4_CHANGELIST"
              echo "GLOBAL_P4_CHANGELIST_STAGE = $env.GLOBAL_P4_CHANGELIST_STAGE"
          }
          if ( ! (env.STAGE_NAME == $env.GLOBAL_P4_CHANGELIST_STAGE) ){
              checkout scm
          }
          {code}
          or by trying to use a lock on updating the variable immediately after a {{checkout scm}}, which still has a race condition.
          Karl Wirth made changes -
          Assignee New: Karl Wirth [ p4karl ]
          Karl Wirth made changes -
          Labels New: P4_SUPPORT
          Karl Wirth made changes -
          Assignee Original: Karl Wirth [ p4karl ]
          Karl Wirth made changes -
          Labels Original: P4_SUPPORT New: P4_VERIFY
          Paul Allen made changes -
          Labels Original: P4_VERIFY New: P4_C
          Paul Allen made changes -
          Labels Original: P4_C New: P4_DOC
          Paul Allen made changes -
          Assignee New: Kevin Williamson [ p4kevin ]
          Kevin Williamson made changes -
          Assignee Original: Kevin Williamson [ p4kevin ]

            Unassigned Unassigned
            alisdair_robertson Alisdair Robertson
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: