-
Bug
-
Resolution: Fixed
-
Major
-
None
-
-
support-core:2.71
In case under $JENKINS_HOME/logs/tasks/ we have big files - I can easily reproduce this with 5 text files with around 50MB each of them - the support-core plugin takes a lot of minutes to generate the bundle.
To reproduce the issue:
1. We can have Anonymize support bundle contents disabled. To have this enabled, the performance issue is even worse, but we are concentrating the efforts when the anonymization is disabled.
2. Insert 5 text files with around 50Mb each of them under $JENKINS_HOME/logs/tasks/. The files must have this pattern
-rw-r--r-- 1 user staff 220688805 Dec 3 10:46 Workspace clean-up.log -rw-r--r-- 1 user staff 220688241 Dec 3 10:46 Workspace clean-up.log.1 -rw-r--r-- 1 user staff 220980327 Dec 3 10:46 Workspace clean-up.log.2 -rw-r--r-- 1 user staff 441376347 Dec 3 10:46 Workspace clean-up.log.3 -rw-r--r-- 1 user staff 441375743 Dec 3 10:46 Workspace clean-up.log.4 -rw-r--r-- 1 user staff 220687588 Dec 3 10:46 Workspace clean-up.log.5
3. Generate a support bundle by ONLY enabling Master Task Log Recorders to easily reproduce the issue and
The key seems to be that we are using FilterOutputStream.write when we should be using a BufferedOutputStream.write to improve the writing process.
In my case I have 1.7 Gb of data under $JENKINS_HOME/logs/tasks/ (6 files of 211 Mb each of them + rest of files are very small) and the generation takes 2 full CPUs of Mac Book Pro - so it also has high CPU consumption during this time.
We will see a stacktrace like the one below in the thread dumps when this issue happens.
"Handling POST /jenkins/support/generateAllBundles from 127.0.0.1 : qtp928487227-97" Id=97 Group=main RUNNABLE
at java.util.zip.Deflater.deflateBytes(Native Method)
at java.util.zip.Deflater.deflate(Deflater.java:442)
- locked java.util.zip.ZStreamRef@446ac8b3
at java.util.zip.Deflater.deflate(Deflater.java:364)
at org.apache.commons.compress.archivers.zip.StreamCompressor.deflate(StreamCompressor.java:262)
at org.apache.commons.compress.archivers.zip.StreamCompressor.deflateUntilInputIsNeeded(StreamCompressor.java:257)
at org.apache.commons.compress.archivers.zip.StreamCompressor.writeDeflated(StreamCompressor.java:238)
at org.apache.commons.compress.archivers.zip.StreamCompressor.write(StreamCompressor.java:205)
at org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.write(ZipArchiveOutputStream.java:926)
at org.apache.commons.compress.archivers.ArchiveOutputStream.write(ArchiveOutputStream.java:109)
at java.io.FilterOutputStream.write(FilterOutputStream.java:77)
at java.io.FilterOutputStream.write(FilterOutputStream.java:125)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2315)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2270)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2291)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2246)
at com.cloudbees.jenkins.support.api.BaseFileContent.writeTo(BaseFileContent.java:79)
at com.cloudbees.jenkins.support.api.FileContent.writeTo(FileContent.java:72)
at com.cloudbees.jenkins.support.SupportPlugin.writeBundle(SupportPlugin.java:351)
at com.cloudbees.jenkins.support.SupportPlugin.writeBundle(SupportPlugin.java:298)
at com.cloudbees.jenkins.support.SupportAction.prepareBundle(SupportAction.java:356)
at com.cloudbees.jenkins.support.SupportAction.doGenerateAllBundles(SupportAction.java:306)
- relates to
-
-
- Resolved
-