Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38640

Unstash painfully slow for large artifacts (due to compression?)

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • workflow-api-plugin

      When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom.

      node {
          deleteDir()
          stage "Stash"
          sh "dd if=/dev/zero of=data bs=1M count=10"
          stash name: 'build_outputs', includes: 'data'
          sh "date"
          node('lab') {
              deleteDir()
              stage "Unstash"
              unstash 'build_outputs'
              sh "ls -al"
          }
      }
      

      This looks similar to JENKINS-36914, in particular this comment:
      https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472

      I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM.

          [JENKINS-38640] Unstash painfully slow for large artifacts (due to compression?)

            andresrc Andres Rodriguez
            jkoleszar John Koleszar
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: