create zip in parallel

XMLWordPrintable

      i use the zip step to create zips of large directories that have multiple gigabytes. Zipping is currently the slowest step in my pipeline and takes multiple minutes to complete. I did some research and found the `ParallelScatterZipCreator`. Tested the ParallelScatterZipCreator against the current implementation of the zip step and it took ~10 seconds for around 1.2gb of files vs ~60 seconds with the current logic.

       

      A 'parallel' setting on the current step or a separate 'zipParallel' step would imo be a good addition. If this feature is a good fit for this plugin, i would love to provide a PR for it.

            Assignee:
            rsandell
            Reporter:
            P
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: