Details
-
Bug
-
Status: Resolved (View Workflow)
-
Minor
-
Resolution: Fixed
-
Master:
openjdk version "1.8.0_66-internal"
OpenJDK Runtime Environment (build 1.8.0_66-internal-b01)
OpenJDK 64-Bit Server VM (build 25.66-b01, mixed mode)
Slaves:
java version "1.7.0_111"
OpenJDK Runtime Environment (IcedTea 2.6.7) (7u111-2.6.7-0ubuntu0.14.04.3)
OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)
Master: openjdk version "1.8.0_66-internal" OpenJDK Runtime Environment (build 1.8.0_66-internal-b01) OpenJDK 64-Bit Server VM (build 25.66-b01, mixed mode) Slaves: java version "1.7.0_111" OpenJDK Runtime Environment (IcedTea 2.6.7) (7u111-2.6.7-0ubuntu0.14.04.3) OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)
Description
When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom.
node {
deleteDir()
stage "Stash"
sh "dd if=/dev/zero of=data bs=1M count=10"
stash name: 'build_outputs', includes: 'data'
sh "date"
node('lab') {
deleteDir()
stage "Unstash"
unstash 'build_outputs'
sh "ls -al"
}
}
This looks similar to JENKINS-36914, in particular this comment:
https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472
I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM.
Attachments
Issue Links
- is duplicated by
-
-
- Closed
-
- is related to
-
-
- Resolved
-
- links to
Activity
John Koleszar
created issue -
| Field | Original Value | New Value |
|---|---|---|
| Assignee | Andres Rodriguez [ andresrc ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | In Review [ 10005 ] |
| Remote Link | This issue links to "PR workflow-api-plugin#19 (Web Link)" [ 14913 ] |
| Link |
This issue is duplicated by |
| Component/s | workflow-api-plugin [ 21711 ] | |
| Component/s | pipeline [ 21692 ] |
| Labels | gzip stash tar | gzip performance stash tar |
| Resolution | Fixed [ 1 ] | |
| Status | In Review [ 10005 ] | Resolved [ 5 ] |
| Link |
This issue is related to |
| Description |
When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom. {{ node { deleteDir() stage "Stash" sh "dd if=/dev/zero of=data bs=1M count=10" stash name: 'build_outputs', includes: 'data' sh "date" node('lab') { deleteDir() stage "Unstash" unstash 'build_outputs' sh "ls -al" } } }} This looks similar to https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472 I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM. |
When sending a stash across the network, the data transfer can be significantly slower than the underlying network. This seems to be due to some slow on-the-fly compression happening during the transfer. Using the following script, the Unstash phase takes 1-2 seconds for 10MB of /dev/zero, and 49 seconds for 10MB of /dev/urandom. The Stash phase consistently takes 1-2 seconds. This makes me think there's a compression step, since /dev/zero should compress much faster than /dev/urandom. {noformat} node { deleteDir() stage "Stash" sh "dd if=/dev/zero of=data bs=1M count=10" stash name: 'build_outputs', includes: 'data' sh "date" node('lab') { deleteDir() stage "Unstash" unstash 'build_outputs' sh "ls -al" } } {noformat} This looks similar to https://issues.jenkins-ci.org/browse/JENKINS-36914?focusedCommentId=268472&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-268472 I'm not calling this a dupe of that because the original text refers to stash only, and this bug seems isolated to unstash. Also this is on x86, not ARM. |
| Labels | gzip performance stash tar | 2.19.3-fixed gzip performance stash tar |
| Labels | 2.19.3-fixed gzip performance stash tar | 2.19.4-fixed gzip performance stash tar |