Provide a light weight check out functionality for perforce so that when initializing the job or running it will not check out whole code repo to master. When we hold a huge code base it take too much time to trigger the build jobs as it has to chekcout the whole repo on to master and configure from jenkinsfile and then it starts it's build process.
Must work for streams.
- is related to
-
JENKINS-46528 Entire stream synced to the script workspace "...@script".
-
- Closed
-
[JENKINS-45999] Provide light weight checkout functionality for perforce in pipeline jobs
I would be also very interested in knowing how to do a lightweight checkout on master node with streams. We have big repositories with lots of binary assets so doing a full checkout on master node is wasteful in terms of build time and disk utilization. We are also utilizing multibranch build system so this is especially bad for projects which have many active branches.
If there's a fix in progress, can you give any ETA when it would ship? Any way to circumvent this for now?
Other than that I have no complaints and I'm so happy we are finally able to utilize full power of Jenkins with Perforce. Keep up the good work!
Thanks, Antti
Antti Hirvonen
added a comment - - edited I would be also very interested in knowing how to do a lightweight checkout on master node with streams. We have big repositories with lots of binary assets so doing a full checkout on master node is wasteful in terms of build time and disk utilization. We are also utilizing multibranch build system so this is especially bad for projects which have many active branches.
If there's a fix in progress, can you give any ETA when it would ship? Any way to circumvent this for now?
Other than that I have no complaints and I'm so happy we are finally able to utilize full power of Jenkins with Perforce. Keep up the good work!
Thanks, Antti 
Paul Allen
added a comment - Hi Guys, I am looking to implement SCMFileSystem and related classes:
This will allow Jenkins to browse/navigate Perforce and find the individual files it needs. I may need to do additional work to resolve the specific problems in this and other issues, but it looks like the SCMFileSystem is the Jenkins way, although Perforce Views have been an alternative in the interim. I'm adding LightWeightCheckout to the roadmap, but doubt I'll have anything ready before the end of Jan, beginning of Feb.
For Multibranch streams, you could create a child stream which only shares the Jenkinsfile and associated scripts; unfortunately Virtual streams don't work in this situation, so you will need a full 'development' stream.
Paul Allen
added a comment - Hi Guys, I am looking to implement SCMFileSystem and related classes:
https://github.com/jenkinsci/scm-api-plugin/blob/master/src/main/java/jenkins/scm/api/SCMFileSystem.java
This will allow Jenkins to browse/navigate Perforce and find the individual files it needs. I may need to do additional work to resolve the specific problems in this and other issues, but it looks like the SCMFileSystem is the Jenkins way, although Perforce Views have been an alternative in the interim. I'm adding LightWeightCheckout to the roadmap, but doubt I'll have anything ready before the end of Jan, beginning of Feb.
For Multibranch streams, you could create a child stream which only shares the Jenkinsfile and associated scripts; unfortunately Virtual streams don't work in this situation, so you will need a full 'development' stream. Antti Hirvonen
added a comment - Thanks for the update. We'll be waiting for the fix. Code changed in jenkins
User: Paul Allen
Path:
src/main/java/org/jenkinsci/plugins/p4/client/NavigateHelper.java
src/main/java/org/jenkinsci/plugins/p4/client/TempClientHelper.java
src/main/java/org/jenkinsci/plugins/p4/scm/AbstractP4ScmSource.java
src/main/java/org/jenkinsci/plugins/p4/scm/BranchesScmSource.java
src/main/java/org/jenkinsci/plugins/p4/scm/P4SCMFile.java
src/main/java/org/jenkinsci/plugins/p4/scm/P4SCMFileSystem.java
src/test/java/org/jenkinsci/plugins/p4/scm/P4SCMFileSystemTest.java
http://jenkins-ci.org/commit/p4-plugin/3ef0aa574af14b320d0fd45847ca3ceb0f6fc3a4
Log:
Lightweight checkout support.
Implementation for scm-api classes SCMFile and SCMFileSystem, allowing
Jenkins to navigate Perforce within the scope of a workspace view.
Lightweight checkout uses a tempoary Perforce workspace to naviagete
and fetch the files. The client name and client view mapping will be
modified from a template name e.g. jenkins-${NODE_NAME}-${JOB_NAME} to
the tempoary name jenkinsTemp-UUID. Alternativly if a user as used
${P4_CLIENT} in the client mapping this will remain unchanged and will
be get expanded during the job run.
Paul Allen
added a comment - Ready for release.
Please try build 146 Antti Hirvonen
added a comment - Excellent, will definitely try this as soon as possible! Paul Allen
added a comment - Released in 1.8.4 
Russell Gallop
added a comment - I'm trying to understand what this means:
The client name and client view mapping will be
modified from a template name e.g. jenkins-${NODE_NAME}-${JOB_NAME} to
the temporary name jenkinsTemp-UUID. Alternativly if a user as used
${P4_CLIENT} in the client mapping this will remain unchanged and will
be get expanded during the job run.
I would like it to mean that if I specify my own "Workspace name" and then use "${P4_CLIENT}" in "View Mappings", then it will use my workspace name instead of jenkinsTemp-UUID. That doesn't appear to work this way. View Mapping remains unchanged but "Workspace name gets replaced.
This is annoying as it is creating (and not destroying) a large number of perforce workspaces which is bogging down our p4 server.
Is this supposed to allow user override of the workspace name, if so then I believe that it doesn't do that (on version 1.8.10)?
Russell Gallop
added a comment - I'm trying to understand what this means:
The client name and client view mapping will be
modified from a template name e.g. jenkins-${NODE_NAME}-${JOB_NAME} to
the temporary name jenkinsTemp-UUID. Alternativly if a user as used
${P4_CLIENT} in the client mapping this will remain unchanged and will
be get expanded during the job run.
I would like it to mean that if I specify my own "Workspace name" and then use "${P4_CLIENT}" in "View Mappings", then it will use my workspace name instead of jenkinsTemp-UUID. That doesn't appear to work this way. View Mapping remains unchanged but "Workspace name gets replaced.
This is annoying as it is creating (and not destroying) a large number of perforce workspaces which is bogging down our p4 server.
Is this supposed to allow user override of the workspace name, if so then I believe that it doesn't do that (on version 1.8.10)? Paul Allen
added a comment - Hi Russel - I have open an new issue to track the changes to the lightweight checkout.
Paul Allen
added a comment - Hi Russel - I have open an new issue to track the changes to the lightweight checkout.
JENKINS-54488 
re: https://issues.jenkins-ci.org/browse/JENKINS-45999?focusedCommentId=310943&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-310943 , specifically "I would recommend limiting the View so only the script (or scripts if required) are checked out and not all your source and assets."
How would this be applied when it comes to MultiBranch Pipeline jobs? I've got a stream depot with a checked in Jenkinsfile in each stream, which I of course can add whatever checkout step neccessary for the actual build, but I don't understand how I can still have Jenkins discover new streams and create subsequent jobs. Am I missing something?
Best,
D