Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70870

Save libraries as JAR files rather than unpacked

    XMLWordPrintable

Details

    Description

      Historically Pipeline Groovy libraries have been stored in the build directory (and also, subsequently, in the optional cache directory) as directory trees with src/, vars/, and/or resources/ subdirectories. Since every build needs to keep a copy of the library (for use when restarting the controller, or using Replay or related functions such as Declarative stage restart), when large libraries are used a considerable portion of $JENKINS_HOME disk space can be taken up by library snapshots.

      It is possible to switch the storage format to a classpath-type JAR file (still using *.groovy source files). This has several advantages:

      • JARs can use DEFLATE compression, typically saving â…” of the disk space.
      • A library snapshot consumes only a single inode, as opposed to potentially dozens for an unpacked directory tree. On large controllers this can be significant—there are field reports of users running out of inodes.
      • The layout on disk and the code to access it are generally simpler. Only one URL need be added to the program's classpath even if both src/ and vars/ are used. (Really these are nearly interchangeable; vars/*.groovy are just classes in the default package that need a GlobalVariable binding.) Copying a library from cache is a single file copy. No separate *-name.txt is needed to keep a record of the original name of a library after a uniquified hash has been generated; the JAR manifest can serve this purpose.

      The performance impact at scale is unclear, though interactive testing showed no perceptible delay related to library access during the build. Only one JAR file need be opened, rather than one Groovy source file per class, which is generally a good thing (avoiding lots of small IOPS); on the other hand, the JAR file is likely kept open for the duration of the build, rather than source files being opened and then closed only during parsing. (The exact caching behavior is up to java.net.URLClassLoader.)

      Attachments

        Issue Links

          Activity

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: