-
Bug
-
Resolution: Fixed
-
Minor
-
Master:
openjdk version "1.8.0_66-internal"
OpenJDK Runtime Environment (build 1.8.0_66-internal-b01)
OpenJDK 64-Bit Server VM (build 25.66-b01, mixed mode)
Slaves:
java version "1.7.0_111"
OpenJDK Runtime Environment (IcedTea 2.6.7) (7u111-2.6.7-0ubuntu0.14.04.3)
OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)
Master: openjdk version "1.8.0_66-internal" OpenJDK Runtime Environment (build 1.8.0_66-internal-b01) OpenJDK 64-Bit Server VM (build 25.66-b01, mixed mode) Slaves: java version "1.7.0_111" OpenJDK Runtime Environment (IcedTea 2.6.7) (7u111-2.6.7-0ubuntu0.14.04.3) OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)
When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom.
node { deleteDir() stage "Stash" sh "dd if=/dev/zero of=data bs=1M count=10" stash name: 'build_outputs', includes: 'data' sh "date" node('lab') { deleteDir() stage "Unstash" unstash 'build_outputs' sh "ls -al" } }
This looks similar to JENKINS-36914, in particular this comment:
https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472
I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM.
- is duplicated by
-
JENKINS-36914 stash step is excessively slow on ARM
-
- Closed
-
- is related to
-
JENKINS-38814 FilePath.untarFrom and unzipFrom should use a greedy RemoteInputStream
-
- Resolved
-
- links to
[JENKINS-38640] Unstash painfully slow for large artifacts (due to compression?)
Assignee | New: Andres Rodriguez [ andresrc ] |
Status | Original: Open [ 1 ] | New: In Progress [ 3 ] |
Status | Original: In Progress [ 3 ] | New: In Review [ 10005 ] |
Remote Link | New: This issue links to "PR workflow-api-plugin#19 (Web Link)" [ 14913 ] |
Link |
New:
This issue is duplicated by |
Component/s | New: workflow-api-plugin [ 21711 ] | |
Component/s | Original: pipeline [ 21692 ] |
Labels | Original: gzip stash tar | New: gzip performance stash tar |
Resolution | New: Fixed [ 1 ] | |
Status | Original: In Review [ 10005 ] | New: Resolved [ 5 ] |
External Workspace Manager (https://github.com/jenkinsci/external-workspace-manager-plugin/blob/master/README.md) could be a workaround. With this plugin you can share the entire workspace on filer instead of stashing particular files and getting overhead on the Jenkins side.