Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13936

Fitnesse results parsing uses too much heap

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We are seeing OOME's in Jenkins when parsing large Fitnesse results files (35mb). For example:

      Reading results as US-ASCII from /.../fitnesse-result.xml
      Parsing results...
      java.lang.OutOfMemoryError: Java heap space
      at java.util.Arrays.copyOf(Arrays.java:2882)
      at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
      at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515)
      at java.lang.StringBuffer.append(StringBuffer.java:306)
      at com.sun.org.apache.xalan.internal.xsltc.runtime.StringValueHandler.characters(StringValueHandler.java:49)
      at com.sun.org.apache.xml.internal.utils.FastStringBuffer.sendSAXcharacters(FastStringBuffer.java:998)
      at com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.dispatchCharactersEvents(SAX2DTM2.java:3068)
      at com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1562)
      at com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.characters(DOMAdapter.java:330)
      at GregorSamsa.template$dot$2()
      at GregorSamsa.applyTemplates()
      at GregorSamsa.template$dot$0()
      at GregorSamsa.applyTemplates()
      at GregorSamsa.applyTemplates()
      at GregorSamsa.transform()
      at com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:603)
      at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:709)
      at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:313)
      at hudson.plugins.fitnesse.NativePageCountsParser.transformRawResults(NativePageCountsParser.java:25)
      at hudson.plugins.fitnesse.NativePageCountsParser.parse(NativePageCountsParser.java:17)
      at hudson.plugins.fitnesse.FitnesseResultsRecorder.getResults(FitnesseResultsRecorder.java:131)
      at hudson.plugins.fitnesse.FitnesseResultsRecorder.getResults(FitnesseResultsRecorder.java:109)
      at hudson.plugins.fitnesse.FitnesseResultsRecorder.perform(FitnesseResultsRecorder.java:73)
      at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
      at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:705)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:680)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:658)
      at hudson.model.Build$RunnerImpl.post2(Build.java:161)
      at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:627)
      at hudson.model.Run.run(Run.java:1400)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      Build step 'Publish Fitnesse results report' changed build result to FAILURE
      Build step 'Publish Fitnesse results report' marked build as failure

      I'm fairly certain this OOME is due to memory usage by this parsing since we don't see OOME at any other time, and there is plenty of memory usually available according to our monitoring.

        Attachments

          Activity

          recampbell Ryan Campbell created issue -
          recampbell Ryan Campbell made changes -
          Field Original Value New Value
          Description We are seeing OOME's in Jenkins when parsing large Fitnesse results files (35mb). For example:
          {quote}
          Reading results as US-ASCII from /.../fitnesse-result.xml
          Parsing results...
          java.lang.OutOfMemoryError: Java heap space
          at java.util.Arrays.copyOf(Arrays.java:2882)
          at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
          at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515)
          at java.lang.StringBuffer.append(StringBuffer.java:306)
          at com.sun.org.apache.xalan.internal.xsltc.runtime.StringValueHandler.characters(StringValueHandler.java:49)
          at com.sun.org.apache.xml.internal.utils.FastStringBuffer.sendSAXcharacters(FastStringBuffer.java:998)
          at com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.dispatchCharactersEvents(SAX2DTM2.java:3068)
          at com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1562)
          at com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.characters(DOMAdapter.java:330)
          at GregorSamsa.template$dot$2()
          at GregorSamsa.applyTemplates()
          at GregorSamsa.template$dot$0()
          at GregorSamsa.applyTemplates()
          at GregorSamsa.applyTemplates()
          at GregorSamsa.transform()
          at com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:603)
          at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:709)
          at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:313)
          at hudson.plugins.fitnesse.NativePageCountsParser.transformRawResults(NativePageCountsParser.java:25)
          at hudson.plugins.fitnesse.NativePageCountsParser.parse(NativePageCountsParser.java:17)
          at hudson.plugins.fitnesse.FitnesseResultsRecorder.getResults(FitnesseResultsRecorder.java:131)
          at hudson.plugins.fitnesse.FitnesseResultsRecorder.getResults(FitnesseResultsRecorder.java:109)
          at hudson.plugins.fitnesse.FitnesseResultsRecorder.perform(FitnesseResultsRecorder.java:73)
          at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:705)
          at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:680)
          at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:658)
          at hudson.model.Build$RunnerImpl.post2(Build.java:161)
          at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:627)
          at hudson.model.Run.run(Run.java:1400)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          Build step 'Publish Fitnesse results report' changed build result to FAILURE
          Build step 'Publish Fitnesse results report' marked build as failure
          {quote}

          Perhaps it would be more efficient to use a streaming parser like SAX?

          We are seeing OOME's in Jenkins when parsing large Fitnesse results files (35mb). For example:
          {quote}
          Reading results as US-ASCII from /.../fitnesse-result.xml
          Parsing results...
          java.lang.OutOfMemoryError: Java heap space
          at java.util.Arrays.copyOf(Arrays.java:2882)
          at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
          at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515)
          at java.lang.StringBuffer.append(StringBuffer.java:306)
          at com.sun.org.apache.xalan.internal.xsltc.runtime.StringValueHandler.characters(StringValueHandler.java:49)
          at com.sun.org.apache.xml.internal.utils.FastStringBuffer.sendSAXcharacters(FastStringBuffer.java:998)
          at com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.dispatchCharactersEvents(SAX2DTM2.java:3068)
          at com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1562)
          at com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.characters(DOMAdapter.java:330)
          at GregorSamsa.template$dot$2()
          at GregorSamsa.applyTemplates()
          at GregorSamsa.template$dot$0()
          at GregorSamsa.applyTemplates()
          at GregorSamsa.applyTemplates()
          at GregorSamsa.transform()
          at com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:603)
          at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:709)
          at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:313)
          at hudson.plugins.fitnesse.NativePageCountsParser.transformRawResults(NativePageCountsParser.java:25)
          at hudson.plugins.fitnesse.NativePageCountsParser.parse(NativePageCountsParser.java:17)
          at hudson.plugins.fitnesse.FitnesseResultsRecorder.getResults(FitnesseResultsRecorder.java:131)
          at hudson.plugins.fitnesse.FitnesseResultsRecorder.getResults(FitnesseResultsRecorder.java:109)
          at hudson.plugins.fitnesse.FitnesseResultsRecorder.perform(FitnesseResultsRecorder.java:73)
          at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
          at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:705)
          at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:680)
          at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:658)
          at hudson.model.Build$RunnerImpl.post2(Build.java:161)
          at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:627)
          at hudson.model.Run.run(Run.java:1400)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          Build step 'Publish Fitnesse results report' changed build result to FAILURE
          Build step 'Publish Fitnesse results report' marked build as failure
          {quote}

          I'm fairly certain this OOME is due to memory usage by this parsing since we don't see OOME at any other time, and there is plenty of memory usually available according to our monitoring.

          Hide
          stanio Stanimir Stamenkov added a comment -

          Probably related to JENKINS-15863.

          Show
          stanio Stanimir Stamenkov added a comment - Probably related to JENKINS-15863 .
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: antoine-aumjaud
          Path:
          src/main/java/hudson/plugins/fitnesse/CompoundFitnesseResults.java
          src/main/java/hudson/plugins/fitnesse/FitnesseResultsRecorder.java
          src/main/java/hudson/plugins/fitnesse/NativePageCounts.java
          src/main/java/hudson/plugins/fitnesse/NativePageCountsParser.java
          src/test/java/hudson/plugins/fitnesse/FitnesseResultsRecorderTest.java
          src/test/java/hudson/plugins/fitnesse/NativePageCountsParserTest.java
          src/test/java/hudson/plugins/fitnesse/NativePageCountsTest.java
          http://jenkins-ci.org/commit/fitnesse-plugin/7ea548eb733eea54fe20820ed3da33749e6c031e
          Log:
          JENKINS-15863JENKINS-13936 Write HTML content in file when XML is
          parsed.
          HTML contents of the result was saved in memory before to be written in
          files. To avoid OOM, I write the file when the XML si parsed.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: antoine-aumjaud Path: src/main/java/hudson/plugins/fitnesse/CompoundFitnesseResults.java src/main/java/hudson/plugins/fitnesse/FitnesseResultsRecorder.java src/main/java/hudson/plugins/fitnesse/NativePageCounts.java src/main/java/hudson/plugins/fitnesse/NativePageCountsParser.java src/test/java/hudson/plugins/fitnesse/FitnesseResultsRecorderTest.java src/test/java/hudson/plugins/fitnesse/NativePageCountsParserTest.java src/test/java/hudson/plugins/fitnesse/NativePageCountsTest.java http://jenkins-ci.org/commit/fitnesse-plugin/7ea548eb733eea54fe20820ed3da33749e6c031e Log: JENKINS-15863 JENKINS-13936 Write HTML content in file when XML is parsed. HTML contents of the result was saved in memory before to be written in files. To avoid OOM, I write the file when the XML si parsed.
          antoine_aumjaud Antoine Aumjaud made changes -
          Assignee Antoine Aumjaud [ antoine_aumjaud ]
          Hide
          antoine_aumjaud Antoine Aumjaud added a comment -

          fixed in version 1.10

          Show
          antoine_aumjaud Antoine Aumjaud added a comment - fixed in version 1.10
          antoine_aumjaud Antoine Aumjaud made changes -
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          antoine_aumjaud Antoine Aumjaud made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 144404 ] JNJira + In-Review [ 205892 ]

            People

            Assignee:
            antoine_aumjaud Antoine Aumjaud
            Reporter:
            recampbell Ryan Campbell
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: