Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47299

Multibranch pipelines with Jenkinsfile in big svn repositories is unusable



    • Similar Issues:


      We have a product stored in a normal SVN repo, we have trunk, branches, tags, etc. The size of each branch checkout is around a few gigabytes of diskspace. Not ideal, I know, but it's a separate problem. Now, when we create a multibranch pipeline job for this repository our jenkins master runs out of disk space after the first branch indexing. The reason is quite simple, the multibranch job creates one complete checkout for each branch, each requiring several gigabytes, which is completely unnecessary since it only needs to load the Jenkinsfile, the complete checkout is done on the slave later when doing the actual build.

      However, if we create a normal pipeline job for each branch (with loading Jenkinsfile from SVN), then we can specify checkout depth and limit the number of files to checkout, this option is not available for multbranch pipeline jobs.

      Now, I see three solutions to this:

      1. Add the ability to specify depth for multibranch pipeline jobs. This is probably the simplest solution, and the implementation should be similar to JENKINS-35227 (PR-189). The downside is that it won't work for projects where the Jenkinsfile isn't stored in the root (cause you will need to use infinite as depth).
      2. Always use checkout depth "files" when checking out the Jenkinsfile. (this will require some special handling when the Jenkins file is located in a sub folder, anyone who foresee a problem with this solution?
      3. Add proper support for lightweight retrieval of Jenkinsfile. This was introduced in JENKINS-33273, but support for subversion was never added. Anyone who knows if there is a blocker for implementing this or it is simply a matter of someone doing it?

      I'm willing to make a attempt on addressing this issue. However I would like some input from someone who knows the plugin well and can tell which one of the solutions is possible and ideal.


          Issue Links


            jons Jon Sten created issue -
            jons Jon Sten made changes -
            Field Original Value New Value
            Attachment Multibranch svn configurations.png [ 39949 ]
            jons Jon Sten made changes -
            Attachment Pipeline svn configuration.png [ 39950 ]
            corentin_soen Corentin Soen made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            gbartels Graham Bartels made changes -
            Labels multibranch pipeline, svn
            gbartels Graham Bartels made changes -
            Labels multibranch pipeline, svn multibranch pipeline svn
            gbartels Graham Bartels made changes -
            Component/s pipeline [ 21692 ]
            ilmarinen Mathias Helminger made changes -
            Link This issue depends on JENKINS-48420 [ JENKINS-48420 ]
            vivek Vivek Pandey made changes -
            Labels multibranch pipeline svn multibranch pipeline svn triaged-2018-11
            jons Jon Sten made changes -
            Assignee Jon Sten [ jons ] Ivan Fernandez Calvo [ ifernandezcalvo ]
            ifernandezcalvo Ivan Fernandez Calvo made changes -
            Assignee Ivan Fernandez Calvo [ ifernandezcalvo ]


              Unassigned Unassigned
              jons Jon Sten
              17 Vote for this issue
              18 Start watching this issue