Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47299

Multibranch pipelines with Jenkinsfile in big svn repositories is unusable


      We have a product stored in a normal SVN repo, we have trunk, branches, tags, etc. The size of each branch checkout is around a few gigabytes of diskspace. Not ideal, I know, but it's a separate problem. Now, when we create a multibranch pipeline job for this repository our jenkins master runs out of disk space after the first branch indexing. The reason is quite simple, the multibranch job creates one complete checkout for each branch, each requiring several gigabytes, which is completely unnecessary since it only needs to load the Jenkinsfile, the complete checkout is done on the slave later when doing the actual build.

      However, if we create a normal pipeline job for each branch (with loading Jenkinsfile from SVN), then we can specify checkout depth and limit the number of files to checkout, this option is not available for multbranch pipeline jobs.

      Now, I see three solutions to this:

      1. Add the ability to specify depth for multibranch pipeline jobs. This is probably the simplest solution, and the implementation should be similar to JENKINS-35227 (PR-189). The downside is that it won't work for projects where the Jenkinsfile isn't stored in the root (cause you will need to use infinite as depth).
      2. Always use checkout depth "files" when checking out the Jenkinsfile. (this will require some special handling when the Jenkins file is located in a sub folder, anyone who foresee a problem with this solution?
      3. Add proper support for lightweight retrieval of Jenkinsfile. This was introduced in JENKINS-33273, but support for subversion was never added. Anyone who knows if there is a blocker for implementing this or it is simply a matter of someone doing it?

      I'm willing to make a attempt on addressing this issue. However I would like some input from someone who knows the plugin well and can tell which one of the solutions is possible and ideal.

            Unassigned Unassigned
            jons Jon Sten
            17 Vote for this issue
            17 Start watching this issue