-
Bug
-
Resolution: Fixed
-
Major
-
Docker Jenkins as master, Windows Server 2016 + Java 1.8.0 144 b0 and Gentoo as slaves
-
Powered by SuggestiMate
Client is created and sync of client appears in log but the files are not there. I need to log into node and issue p4 sync manually.
[JENKINS-48525] Perforce plugin doesn't sync
Also ran into this issue recently. William's workaround works for us at the moment. According to our admin the Perforce server has parallel sync on unless a client requests otherwise.
Windows 10 x86 (virtualized)
Jenkins Service runs as a local user
Job that that had failing p4 sync was run locally
1 macOS slave which we have observed no sync issues on
P4 Plugin 1.9.3
Jenkins 2.138.2
java.runtime.version 1.8.0_144-b01
Still an issue,
Jenkins 2.176.2
p4 plugin: 1.10.3
perforce server: Server version: P4D/LINUX26X86_64/2018.1/1673397 (2018/06/21)
simply changed the job to put false in the parallel sync and it worked
douglasawh or arkady_english - what was the resolution of your issue? We are seeing similar behavior so it could help us.
One of our teammates is seeing this again, although their automation stack is on an older version of the p4 plugin and p4d, but based on the p4java release notes it should have sufficient parallel sync support. They hit the issue where sync would run and report success in automation, however the files were not actually sync'd, and it wasn't resolved until they explicitly set the sync configuration to have parallel sync enabled, but with threads, minimum files, and minimum bytes all set to 0.
Environment information:
Running Jenkins 2.164.2 with P4 Plugin 1.7.4 (uses p4java 2017.2.1535715) and SCM API Plugin 2.6.3.
Client VMs (Jenkins agents) are syncing from a P4D Commit server on 2017.2.1650199.
P4V is not installed on the client VMs
Client VMs are an assortment of Windows Server 2016 (8-core, 32GB RAM) and Mac Mojave (4-core, 16GB RAM) VMs
Let me know if you need any more information
same here,
We setup an edge server dedicated to the build system just so we can enable parallel sync. We do not enable it on the main server because it causes p4.exe and p4v to behave as though it's frozen, The behavior on the lcients is that it will first list all files that will need to be synced, and then will actually transfer the bytes. P4.exe and p4V will show files flying by real fast then freese for the duration of the transfer. With the huge code base this can mean a frozed app for 30 min + generating alot of support and frustration.
On the build farm we can sync with the edge no problem, but activating the parallel sync will cause the files to be listed really fast and it will return immediately after as though listing the files was a signal to tell the operation is over. end result no files at all are transfeted. a normal (clean) sync takes almost 90 minutes, with aprallel activated it's down 15 seconds... except workspace is empty.
We just noticed the behaviour and it can be toggled on or off just by enabling parallel sync on the job:
this will sync the files but will take a really long time
parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'],
this will finish really quick successfully but no files are actually present on the workspace
parallel: [enable: true, minbytes: '1024', minfiles: '1', threads: '4'],
Both provide much the same file listing and operations when looking at the verbose files. Log sais file was refreshed or deleted or added... yet nothing appear with the parallel enabled.
Trying a parallel sync using p4.exe manually on the same edge server and same stream and it works, the problem seems unique to the java library.
Jenkins 2.263.1
P4 plugin 1.11.1
Server version: P4D/LINUX26X86_64/2020.1/1991450 (2020/07/31)
I wanted to comment on this bug as well, since it cost me about a full month of time and premature hair loss in trying to figure out why our build machines were suddenly not syncing from the depot. It turns out that the company who hosts and manages our Perforce depot (On Amazon ec2 / s3) had enabled parallel sync on our server without our knowledge, and ever since then, the issue prevented around 90% of our builds from syncing. I don't like thinking about how much time, money, and effort went into troubleshooting this. I can confirm that it ONLY affected Jenkins builds, but whether the root problem is in the P4Java library or in the Jenkins-specific code, I don't know.
In any case, I finally stumbled upon this Jira issue, and was able to "fix" the issue by editing every one of our 50+ Jenkinsfiles to enable parallel threading, per kroutley's comment above.
Now that I've said all that, I am happy to repro this problem and provide logs or other info that might help in fixing this issue, and will be watching this ticket for any follow up comments.
Perforce Server version: P4D/LINUX26X86_64/2020.2/2057778 (2020/12/18)
Jenkins versions tested: All versions since April 2021, LTS and otherwise
Jenkins p4-plugin versions tested: 1.11.3 thru current
Jenkins Build Machines are Windows 10 Pro x64 of various versions.
Lastly, one or more of these is what seems to have initiated the problem on the Perforce server (output from `p4 configure history` edited for clarity):
net.parallel.max changed from 'unset' to '4' on 2021/05/20 net.parallel.submit.threads changed from 'unset' to '4' on 2021/05/20 net.parallel.threads changed from 'unset' to '4' on 2021/05/20
A credentials bug has been found with case insensitive Perforce servers and P4Java (which sites under P4Jenkins) that silently stopped parrallel sync threads from syncing files. To workaround the problem use lowercase letters for the username in the Jenkins credential. For example if the Perforce user is call 'BuildUser' then specifying 'builduser' in the credential will allow parrallel sync threads to login correctly.
douglasawh I've opened ticket #00342618.