Hi newtopian - Please get me your pipeline code so I can test here. I'm wondering if disabling implicit sync may be a workaround.

Hi newtopian - Ignore that. Disabling implicit doe not change the behavior.

P4Jenkins will only build if there is something to sync. This is intended because 'pin' is normally used with a build label and you only need to rebuild if the build label has changed. You dont want a job building if the label has not been updated.

If I have two 'p4sync' steps in my pipeline script and only one of them is pinned the job will still build. From the polling log:

# Pinned at 4262 
P4: Polling with range: 4262,4262
... p4 changes -m20 //jenkins-master-PinPolledPipeline-Source-TEST/...@4262,4262 
... p4 repos -C 
P4: Polling no changes found.

# Not pinned
P4: Polling with range: 4274,now
... p4 changes -m20 //jenkins-master-PinPolledPipeline-Source-TEST-FILES/...@4274,now 
... p4 change -o 4276 
... found change: 4276

If after reading this you think its still not working as described please provide me more information about how you strcture everything. Are you syncing any new code in the job? Do you have multiple jobs or just steps? How are they connected to each other? Can you provide me the code?

If it is working as described the workaround would be to trigger the jobs from perforce instead of using polling.

Our pipeline resembles this pseudo seen below. The code is over-simplified and will likely not run as-is but it does give the essential of what is happening.

Basically here we need to ensure that all stages in the parallel to build at the same CL. By default each will sync to head at the time they start, since each gets assigned to a different agent it's quite possible one or more is put in the queue to wait for available resources. During that time it's very likely that new commits occur thus having them all sync to head creates a "staggered" build across multiple changes which is undesirable.

We used to do it in the way you suggests where one is not pinned, we struggled a bit to get the information across other stage but eventually made it work. however getting that changelist ID to propagate to other steps prooved to be challenging and it became much simpler to just fetch the change eagerly and pass it along the entire pipeline.

Now we have averything working just dandy and all other steps that need access to the changelist can do so naturally... except polling no longer work because of this side effect. And since there are no obvious alternative ways to sync through the plugin that would not generate this side effect we are basically stuck. Temporary solution was to change the polling with a cron and force trigger the job each 30 mins or so.

We experimented with perforce triggers and ran in another issue. Since the changelist is not (least wasnot at the time we tried) attached to the job being triggered we ran in situations where each commit woudl trigger a job but build system restrictions caused long queues. To make things worst when a job started it would pick head CL thus many jobs ended running at the same CL because of this which was very wasteful use of resources.

Last thing we had not tried yet was to simply do without the populate and run the p4 sync directly with a p4.run.

def pinnedchangelist = p4.run("changes", "-s", "submitted", "-m1", "${stream_location}/...")[0]['change']
pipeline{
  options{
    skipDefaultCheckout()
  }
  parallel{
    stages{
      stage('someplatform'){
	    agent('able-tagged'){}
        step('sync'){
          //see below for actual code
        }
		step('build'){
		  ///
		}
      }
      stage('someotherplatform'){
	    agent('able-tagged'){}
        step('sync'){
          //see below for actual code
        }
		step('build'){
		  ///
		}
      }
      stage('yetanother'){
	    agent('able-tagged'){}
        step('sync'){
          //see below for actual code
        }
		step('build'){
		  ///
		}
      }
    }
  }
}

the steps sync contains this code which does the actual sync

    def popImpl = [$class: "SyncOnlyImpl",
            force: needs_force_sync(config),
            have: true,
            modtime: true,
            parallel: null,
            pin: pinnedChangelist,
            quiet: false,
            revert: false]

    echo "==  Workspace preparation complete, performing a sync ${popImpl}"
    checkout([
            $class: 'PerforceScm',
            credential: "${config.p4_credential}".toString(),
            populate: popImpl,
            workspace: ws
            ])

Edit:
added skipDefaultCheckout() in option block at start of pipeline
reformat pipeline code

Hi newtopian - A quick aplogy. I owe you a response on this one but I'm still working on it. Will get back to you when I have something.

Hi p4karl & newtopian

Karl, for general context on how we are currently doing parallel builds & enforcing all parallel machines are synced to the same CL for a given build see this blog post from Riot Games, under the "Multiple Syncs in A Pipeline" section of the article:

https://technology.riotgames.com/news/using-perforce-complex-jenkins-pipeline

That's the core idea of what we're currently doing succesfully (with some tradeoffs and limitations that Eric is working on tackling with what he describes above).  Run a job with multiple seperate syncs occuring on parallel job parts & ensure they all sync to the same CL regardless of whether new CLs come into Perforce before some of the parallel parts begin (ex: waiting on an available agent).

Apologies for the tangent but it gives an idea of what we're aiming for in a broader sense.

Cheers.

I resolved the same problem by creating a separate polling pipeline, which just does the polling, and triggers the main pipeline with:

build(job: 'main_pipeline_job', wait: false)