Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-60907

Poor performance with jenkins.model.StandardArtifactManager.disableTrafficCompression=true

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • core
    • Jenkins 2.204.1
      Linux (Ubuntu 18.04)
      OpenJDK 8u242
    • Jenkins 2.220

      With the new jenkins.model.StandardArtifactManager.disableTrafficCompression property (JENKINS-26008 / PR #4205), I was expecting improved performances when archiving artifacts from slaves to master. Especially when there is no room for further compression (when the artifact is already a compressed archive).
      But my first tests with 2.204.1 and this option are disappointing. On my test setup:

      • with traffic compression (default setting), archiving a ~200 MB .tar.gz file consistently takes ~50 seconds (which is really not great already)
      • without traffic compression (the new option), it now takes ~240 seconds!

      I suspect this is because FilePath.TarCompression.NONE lacks a layer of buffering input/output streams, which is there in FilePath.TarCompression.GZIP:
      FilePath.java#L758

      • GZIPInputStream is a buffering InputStream (here with buffer size set to 8K)
      • GZIPOutputStream is a buffering OutputStream (with a default buffer size of 512), and there is also an additionnal BufferedOutputStream involved (with its default buffer size of 8K)

      I will submit a PR for FilePath.TarCompression.NONE to add the missing buffering layer, so that the difference with GZIP will really be only about compression.

          [JENKINS-60907] Poor performance with jenkins.model.StandardArtifactManager.disableTrafficCompression=true

          Thomas de Grenier de Latour created issue -
          Thomas de Grenier de Latour made changes -
          Description Original: With the new {{jenkins.model.StandardArtifactManager.disableTrafficCompression}} property (JENKINS-26008 / [PR #4205|https://github.com/jenkinsci/jenkins/pull/4205]), I was expecting improved performances when archiving artifacts from slaves to master. Especially when there is no room for further compression (when the artifact is already a compressed archive).
          But my first tests with 2.204.1 and this option are disappointing. On my test setup:
          - with traffic compression (default setting), archiving a ~200 MB .tar.gz file consistently takes ~50 seconds (which is really not great already)
          - without traffic compression (the new option), it now takes ~240 seconds!

          I suspect this is because {{FilePath.TarCompression.NONE}} lacks a layer of buffering input/output streams, which is there in {{FilePath.TarCompression.GZIP}}:
          [FilePath.java#L758|https://github.com/jenkinsci/jenkins/blob/jenkins-2.204.1/core/src/main/java/hudson/FilePath.java#L758]
           - {{GZIPInputStream}} is a buffering {{InputStream}} (here with buffer size set to 8K)
           - {{GZIPOutputStream}} is a buffering {{OutputStream}} (with a default buffer size of 512), and here there is also an addition {{BufferedOutputStream}} involved (with its default buffer size of 8K)

          I will submit a PR for {{FilePath.TarCompression.NONE}} to add the missing buffering layer, such that the difference with {{GZIP}} will really only be about compression.
          New: With the new {{jenkins.model.StandardArtifactManager.disableTrafficCompression}} property (JENKINS-26008 / [PR #4205|https://github.com/jenkinsci/jenkins/pull/4205]), I was expecting improved performances when archiving artifacts from slaves to master. Especially when there is no room for further compression (when the artifact is already a compressed archive).
           But my first tests with 2.204.1 and this option are disappointing. On my test setup:
           - with traffic compression (default setting), archiving a ~200 MB .tar.gz file consistently takes ~50 seconds (which is really not great already)
           - without traffic compression (the new option), it now takes ~240 seconds!

          I suspect this is because {{FilePath.TarCompression.NONE}} lacks a layer of buffering input/output streams, which is there in {{FilePath.TarCompression.GZIP}}:
           [FilePath.java#L758|https://github.com/jenkinsci/jenkins/blob/jenkins-2.204.1/core/src/main/java/hudson/FilePath.java#L758]
           - {{GZIPInputStream}} is a buffering {{InputStream}} (here with buffer size set to 8K)
           - {{GZIPOutputStream}} is a buffering {{OutputStream}} (with a default buffer size of 512), and there is also an additionnal {{BufferedOutputStream}} involved (with its default buffer size of 8K)

          I will submit a PR for {{FilePath.TarCompression.NONE}} to add the missing buffering layer, so that the difference with {{GZIP}} will really be only about compression.
          Thomas de Grenier de Latour made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Fixed but Unreleased [ 10203 ]
          Thomas de Grenier de Latour made changes -
          Labels New: lts-candidate performance
          Thomas de Grenier de Latour made changes -
          Released As New: https://jenkins.io/changelog/#v2.220
          Status Original: Fixed but Unreleased [ 10203 ] New: Resolved [ 5 ]
          Oleg Nenashev made changes -
          Released As Original: https://jenkins.io/changelog/#v2.220 New: Jenkins 2.220
          Thomas de Grenier de Latour made changes -
          Assignee New: Thomas de Grenier de Latour [ tom_gl ]
          Status Original: Resolved [ 5 ] New: Closed [ 6 ]
          Daniel Beck made changes -
          Labels Original: lts-candidate performance New: performance

            tom_gl Thomas de Grenier de Latour
            tom_gl Thomas de Grenier de Latour
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: