Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38640

Unstash painfully slow for large artifacts (due to compression?)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: workflow-api-plugin
    • Environment:
    • Similar Issues:

      Description

      When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom.

      node {
          deleteDir()
          stage "Stash"
          sh "dd if=/dev/zero of=data bs=1M count=10"
          stash name: 'build_outputs', includes: 'data'
          sh "date"
          node('lab') {
              deleteDir()
              stage "Unstash"
              unstash 'build_outputs'
              sh "ls -al"
          }
      }
      

      This looks similar to JENKINS-36914, in particular this comment:
      https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472

      I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM.

        Attachments

          Issue Links

            Activity

            jkoleszar John Koleszar created issue -
            andresrc Andres Rodriguez made changes -
            Field Original Value New Value
            Assignee Andres Rodriguez [ andresrc ]
            andresrc Andres Rodriguez made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            andresrc Andres Rodriguez made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            andresrc Andres Rodriguez made changes -
            Remote Link This issue links to "PR workflow-api-plugin#19 (Web Link)" [ 14913 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-36914 [ JENKINS-36914 ]
            jglick Jesse Glick made changes -
            Component/s workflow-api-plugin [ 21711 ]
            Component/s pipeline [ 21692 ]
            jglick Jesse Glick made changes -
            Labels gzip stash tar gzip performance stash tar
            jglick Jesse Glick made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            andresrc Andres Rodriguez made changes -
            Link This issue is related to JENKINS-38814 [ JENKINS-38814 ]
            olivergondza Oliver Gondža made changes -
            Description When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom.

            {{
            node {
                deleteDir()
                stage "Stash"
                sh "dd if=/dev/zero of=data bs=1M count=10"
                stash name: 'build_outputs', includes: 'data'
                sh "date"
                node('lab') {
                    deleteDir()
                    stage "Unstash"
                    unstash 'build_outputs'
                    sh "ls -al"
                }
            }
            }}

            This looks similar to JENKINS-36914, in particular this comment:
            https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472

            I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM.
            When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom.

            {noformat}
            node {
                deleteDir()
                stage "Stash"
                sh "dd if=/dev/zero of=data bs=1M count=10"
                stash name: 'build_outputs', includes: 'data'
                sh "date"
                node('lab') {
                    deleteDir()
                    stage "Unstash"
                    unstash 'build_outputs'
                    sh "ls -al"
                }
            }
            {noformat}

            This looks similar to JENKINS-36914, in particular this comment:
            https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472

            I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM.
            oleg_nenashev Oleg Nenashev made changes -
            Labels gzip performance stash tar 2.19.3-fixed gzip performance stash tar
            olivergondza Oliver Gondža made changes -
            Labels 2.19.3-fixed gzip performance stash tar 2.19.4-fixed gzip performance stash tar

              People

              Assignee:
              andresrc Andres Rodriguez
              Reporter:
              jkoleszar John Koleszar
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: