Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15382

Memory exhaustion parsing large test stdio from Surefire

      A user with some *-output.txt files pushing 70Mb received errors when publishing test results:

      [ERROR] FATAL ERROR 
      [INFO] ------------------------------------------------------------------------ 
      [INFO] Java heap space 
      [INFO] ------------------------------------------------------------------------ 
      [INFO] Trace 
      java.lang.OutOfMemoryError: Java heap space 
      	at java.util.Arrays.copyOf(Arrays.java:2882) 
      	at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) 
      	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515) 
      	at java.lang.StringBuffer.append(StringBuffer.java:306) 
      	at java.io.StringWriter.write(StringWriter.java:77) 
      	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1129) 
      	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1104) 
      	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1050) 
      	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1075) 
      	at org.apache.commons.io.IOUtils.toString(IOUtils.java:382) 
      	at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1103) 
      	at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1120) 
      	at hudson.tasks.junit.SuiteResult.<init>(SuiteResult.java:195) 
      	at hudson.tasks.junit.SuiteResult.parseSuite(SuiteResult.java:130) 
      	at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:115) 
      	at hudson.tasks.junit.TestResult.parse(TestResult.java:227) 
      	at hudson.tasks.junit.TestResult.parse(TestResult.java:163) 
      	at hudson.maven.reporters.SurefireArchiver.postExecute(SurefireArchiver.java:141) 
      	at hudson.maven.Maven2Builder.postExecute(Maven2Builder.java:155) 
      	at hudson.maven.MavenBuilder$Adapter.postExecute(MavenBuilder.java:310) 
      	at hudson.maven.agent.PluginManagerInterceptor$1MojoIntercepterImpl.callPost(PluginManagerInterceptor.java:170) 
      	at hudson.maven.agent.PluginManagerInterceptor.executeMojo(PluginManagerInterceptor.java:183) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348) 
      	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180) 
      	at org.apache.maven.lifecycle.LifecycleExecutorInterceptor.execute(LifecycleExecutorInterceptor.java:65) 
      	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328) 
      	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138) 
      	at org.apache.maven.cli.MavenCli.main(MavenCli.java:362) 
      [INFO] ------------------------------------------------------------------------ 
      [INFO] ...
      [INFO] Final Memory: 223M/1224M 
      [INFO] ------------------------------------------------------------------------
      

      Indeed it is silly to call FileUtils.readFileToString only to throw away the great majority of what was just read.

          [JENKINS-15382] Memory exhaustion parsing large test stdio from Surefire

          Jesse Glick created issue -

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/CaseResult.java
          core/src/main/java/hudson/tasks/junit/SuiteResult.java
          http://jenkins-ci.org/commit/jenkins/fe934aac007a20c43e803de61ef8b6cf28c4434f
          Log:
          [FIXED JENKINS-15382] Memory exhaustion parsing large test stdio from Surefire.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/tasks/junit/CaseResult.java core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/fe934aac007a20c43e803de61ef8b6cf28c4434f Log: [FIXED JENKINS-15382] Memory exhaustion parsing large test stdio from Surefire.
          SCM/JIRA link daemon made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

          dogfood added a comment -

          Integrated in jenkins_main_trunk #1982

          Result = SUCCESS

          dogfood added a comment - Integrated in jenkins_main_trunk #1982 Result = SUCCESS

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/CaseResult.java
          core/src/main/java/hudson/tasks/junit/SuiteResult.java
          http://jenkins-ci.org/commit/jenkins/4acc879bf56dc727838086c8c93816ad69222552
          Log:
          [FIXED JENKINS-15382] Memory exhaustion parsing large test stdio from Surefire.(cherry picked from commit fe934aac007a20c43e803de61ef8b6cf28c4434f)

          Conflicts:
          changelog.html

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/tasks/junit/CaseResult.java core/src/main/java/hudson/tasks/junit/SuiteResult.java http://jenkins-ci.org/commit/jenkins/4acc879bf56dc727838086c8c93816ad69222552 Log: [FIXED JENKINS-15382] Memory exhaustion parsing large test stdio from Surefire.(cherry picked from commit fe934aac007a20c43e803de61ef8b6cf28c4434f) Conflicts: changelog.html
          Jesse Glick made changes -
          Labels Original: maven memory performance New: 1.480.1-fixed maven memory performance

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/CaseResult.java
          core/src/main/java/hudson/tasks/junit/SuiteResult.java
          core/src/main/java/hudson/tasks/junit/TestResult.java
          core/src/main/java/hudson/util/TextFile.java
          test/src/test/groovy/hudson/util/TextFileTest.groovy
          test/src/test/resources/hudson/util/ascii.txt
          http://jenkins-ci.org/commit/jenkins/14d980c4377dc703df8db76f192de19246a0266e
          Log:
          [FIXED JENKINS-15382]

          Revisiting the fix. Avoid parsing the entire text by only looking at the tail portion.

          Compare: https://github.com/jenkinsci/jenkins/compare/0256f0a74e9b...14d980c4377d

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/src/main/java/hudson/tasks/junit/CaseResult.java core/src/main/java/hudson/tasks/junit/SuiteResult.java core/src/main/java/hudson/tasks/junit/TestResult.java core/src/main/java/hudson/util/TextFile.java test/src/test/groovy/hudson/util/TextFileTest.groovy test/src/test/resources/hudson/util/ascii.txt http://jenkins-ci.org/commit/jenkins/14d980c4377dc703df8db76f192de19246a0266e Log: [FIXED JENKINS-15382] Revisiting the fix. Avoid parsing the entire text by only looking at the tail portion. Compare: https://github.com/jenkinsci/jenkins/compare/0256f0a74e9b...14d980c4377d

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2680

          Result = UNSTABLE

          dogfood added a comment - Integrated in jenkins_main_trunk #2680 Result = UNSTABLE
          Nicolas De Loof made changes -
          Labels Original: 1.480.1-fixed maven memory performance New: 1.480.1-fixed lts-candidate maven memory performance

          marked as lts-candidated for last commit http://jenkins-ci.org/commit/jenkins/14d980c4377dc703df8db76f192de19246a0266e to revisit this fix

          Nicolas De Loof added a comment - marked as lts-candidated for last commit http://jenkins-ci.org/commit/jenkins/14d980c4377dc703df8db76f192de19246a0266e to revisit this fix

            Unassigned Unassigned
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: