[JEP-210] Optimize log handling in Pipeline and Durable Task

XMLWordPrintable

      Extracted from JENKINS-30896:

      The current design of LogActionImpl, using $id.log, was considered the minimum necessary for a working 1.0 release, not a serious implementation. It has a major problem: when there is a large amount of output, WorkflowRun.copyLogs must duplicate it all to log, doubling disk space requirements per build.

      It would be better to keep a single log file for the build. LogActionImpl should deprecated in favor of an implementation that simply stores a rangeset of offsets into that file. When parallel blocks are producing concurrent output, the single log file will be a bit jumbled (probably still human-readable in most cases), but the rangesets will keep track of what output came from where. The final output produced by WorkflowRun will still be processed to split at line boundaries, add in thread labels, etc. (TBD how and whether JENKINS-30777 could be supported in this mode.)

            Assignee:
            Jesse Glick
            Reporter:
            Jesse Glick
            Votes:
            35 Vote for this issue
            Watchers:
            83 Start watching this issue

              Created:
              Updated:
              Resolved: