Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61841

PathRemover methods can throw enormous CompositeIOExceptions

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • core
    • None
    • Jenkins 2.235

      Calls to Util.deleteRecursive and other methods that end up calling PathRemover.forceRemoveRecursive on large directories can end up throwing instances of CompositeIOException with a very large number of nested exceptions, leading to excessive memory usage.

      For example, I recently examined a heap dump where 40% of the ~12GB heap was occupied by 200MB instances of CompositeIOException, where each exception had ~4000 nested exceptions. In that case, the exceptions were being stored in Pipeline flow nodes via ErrorAction, so they were also being written to the file system, compounding the problem. The cause of the huge exceptions in that case was an attempt to delete a workspace with a large node_modules directory that Jenkins did not have permission to delete.

      I think it would make sense for PathRemover.forceDeleteRecursive to abort early after hitting 10 or so exceptions to avoid this kind of issue.

      Alternatively, we could keep all of the nested exception's messages, but drop the stack traces to reduce the memory footprint, but it seems very unlikely to me that additional exceptions beyond the first ~10 are going to provide any useful information that helps a user diagnose and fix the problem. Another drawback of this approach is that the stack traces for all of the exceptions would still be generated when the exceptions are created, which is a waste of CPU since we would null them out right after.

            dnusbaum Devin Nusbaum
            dnusbaum Devin Nusbaum
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: