Our pipeline resembles this pseudo seen below. The code is over-simplified and will likely not run as-is but it does give the essential of what is happening.

Basically here we need to ensure that all stages in the parallel to build at the same CL. By default each will sync to head at the time they start, since each gets assigned to a different agent it's quite possible one or more is put in the queue to wait for available resources. During that time it's very likely that new commits occur thus having them all sync to head creates a "staggered" build across multiple changes which is undesirable.

We used to do it in the way you suggests where one is not pinned, we struggled a bit to get the information across other stage but eventually made it work. however getting that changelist ID to propagate to other steps prooved to be challenging and it became much simpler to just fetch the change eagerly and pass it along the entire pipeline.

Now we have averything working just dandy and all other steps that need access to the changelist can do so naturally... except polling no longer work because of this side effect. And since there are no obvious alternative ways to sync through the plugin that would not generate this side effect we are basically stuck. Temporary solution was to change the polling with a cron and force trigger the job each 30 mins or so.

We experimented with perforce triggers and ran in another issue. Since the changelist is not (least wasnot at the time we tried) attached to the job being triggered we ran in situations where each commit woudl trigger a job but build system restrictions caused long queues. To make things worst when a job started it would pick head CL thus many jobs ended running at the same CL because of this which was very wasteful use of resources.

Last thing we had not tried yet was to simply do without the populate and run the p4 sync directly with a p4.run.

def pinnedchangelist = p4.run("changes", "-s", "submitted", "-m1", "${stream_location}/...")[0]['change']
pipeline{
  options{
    skipDefaultCheckout()
  }
  parallel{
    stages{
      stage('someplatform'){
	    agent('able-tagged'){}
        step('sync'){
          //see below for actual code
        }
		step('build'){
		  ///
		}
      }
      stage('someotherplatform'){
	    agent('able-tagged'){}
        step('sync'){
          //see below for actual code
        }
		step('build'){
		  ///
		}
      }
      stage('yetanother'){
	    agent('able-tagged'){}
        step('sync'){
          //see below for actual code
        }
		step('build'){
		  ///
		}
      }
    }
  }
}

the steps sync contains this code which does the actual sync

    def popImpl = [$class: "SyncOnlyImpl",
            force: needs_force_sync(config),
            have: true,
            modtime: true,
            parallel: null,
            pin: pinnedChangelist,
            quiet: false,
            revert: false]

    echo "==  Workspace preparation complete, performing a sync ${popImpl}"
    checkout([
            $class: 'PerforceScm',
            credential: "${config.p4_credential}".toString(),
            populate: popImpl,
            workspace: ws
            ])

Edit:
added skipDefaultCheckout() in option block at start of pipeline
reformat pipeline code

Hi newtopian - A quick aplogy. I owe you a response on this one but I'm still working on it. Will get back to you when I have something.

Hi p4karl & newtopian

Karl, for general context on how we are currently doing parallel builds & enforcing all parallel machines are synced to the same CL for a given build see this blog post from Riot Games, under the "Multiple Syncs in A Pipeline" section of the article:

https://technology.riotgames.com/news/using-perforce-complex-jenkins-pipeline

That's the core idea of what we're currently doing succesfully (with some tradeoffs and limitations that Eric is working on tackling with what he describes above).  Run a job with multiple seperate syncs occuring on parallel job parts & ensure they all sync to the same CL regardless of whether new CLs come into Perforce before some of the parallel parts begin (ex: waiting on an available agent).

Apologies for the tangent but it gives an idea of what we're aiming for in a broader sense.

Cheers.

I resolved the same problem by creating a separate polling pipeline, which just does the polling, and triggers the main pipeline with:

build(job: 'main_pipeline_job', wait: false)

Any news on this p4karl ?

I'm am updating this to highlight to the devs. Still a major cause of frustration.

I'd like to refine / clarify the use-case at the top of the issue description to match the way this issue is affecting myself & newtopian

Parallel Pipeline Where All Parallel Job Parts Sync Same Perforce Path

Polled builds (pipeline) should sync to "@now" when polling occurred not "@now" when sync occurs.

Usage case - On a busy system a job with long-running parallel parts each syncing Perforce Path1,  will sync Path1 at different ages on the different parallel parts of the Jennkins job if new CLs are submitted while some parallel parts are still waiting in queue for available agents (or have begun but have not yet reached their sync step)

Example - Each triggered build job has parallel parts Part1 and Part2

• Poll finds latest change 123 on Path1
• Build triggered by polling & added to Jenkins queue to wait for available agents.  Perforce @now for Path1 is equal to CL 123
• Change 124 is submitted to Path1
• Agent1 becomes available. Build job Part1 starts on Agent1. Sync to @now occurs on Agent1 when @now of Path1 is equal to CL 124
• Job Part2 is still waiting in the Jenkins queue for an available agent.
• Change 125 and 126 submitted to Path1
• Agent2 becomes available. Build Job Part2 stars on Agent2. Sync to @now occurs on Agent2 when @now of Path1 is equal to CL 126
• For a single build job, triggered when @now was equal to 123, the parallel parts are no longer coordinated.

The desired behavior would be for all parallel parts of a job to sync at the same Perforce @now age.  Ideally this would be the @now age that initially triggered the job vis polling.

What is missing IMO is a built-in plugin feature/option that allows injecting/passing the @now changelist number that caused SCM polling to trigger down into the triggered job itself.  This way once the polling triggers a job, the @now age that the parallel job parts will actually sync to becomes a fixed value, not dependent on when agents become available for that job & depedendent on what was submitted since the polling queued the job.

It would be really helpful to have this fixed in the near future.

Hi Karl, could you check with product owner and/or developers whether there is a plan for fixing this resp. 65246?

Hi, in P4 plugin 1.14.3 release, we have added a new polling build filter - 'Polling latest change with pin' which when enabled, overrides the pin in the checkout step and polls to 'now'
Closing this