Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56374

JUnitResultArchiver parsing XML in sort()->compare(), killing Jenkins

XMLWordPrintable

      Jenkins 2.150.1

      Jenkins dies with the JVM going into perpetual ergonomics GC. The JVM has a 4Gb heap. The biggest XML file is 180Mb. So that's not a fundamental limitation. Something dumb is going on.

      I think it might be repeatedly parsing the same XML file again and again, because when I dump jstack, I've seen a lot of different XML parsers there. This MIGHT be triggered by a @Parameterized junit test with 50,000 cases. In any case, this has made Jenkins totally unusable.

      It also dies when we have a single JUnit case with a 1Gb output XML, but we have worked around that. But we cannot reduce the size of our test suite just to keep Jenkins happy.

      Reported critical as this has stopped us from using Jenkins. Tempted to report blocker.

      Here is some stack.

      ```
      at com.thoughtworks.xstream.mapper.AnnotationMapper.processAnnotations(AnnotationMapper.java:180)
      at com.thoughtworks.xstream.mapper.AnnotationMapper.defaultImplementationOf(AnnotationMapper.java:141)
      at hudson.util.xstream.MapperDelegate.defaultImplementationOf(MapperDelegate.java:59)
      at com.thoughtworks.xstream.mapper.MapperWrapper.defaultImplementationOf(MapperWrapper.java:46)
      at hudson.util.RobustReflectionConverter.determineType(RobustReflectionConverter.java:475)
      at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:327)
      at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:270)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
      at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
      at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:71)
      at hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:85)
      at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:80)
      at hudson.util.RobustCollectionConverter.unmarshal(RobustCollectionConverter.java:76)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
      at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
      at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:393)
      at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:331)
      at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:270)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
      at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
      at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:71)
      at hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:85)
      at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:80)
      at hudson.util.RobustCollectionConverter.unmarshal(RobustCollectionConverter.java:76)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
      at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
      at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:393)
      at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:331)
      at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:270)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
      at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
      at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
      at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
      at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1189)
      at hudson.util.XStream2.unmarshal(XStream2.java:161)
      at hudson.util.XStream2.unmarshal(XStream2.java:132)
      at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1173)
      at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)
      at hudson.XmlFile.read(XmlFile.java:147)
      at hudson.tasks.junit.TestResultAction.load(TestResultAction.java:208)
      at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:145)

      • locked <0x0000000764f440a8> (a hudson.tasks.junit.TestResultAction)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:63)
        at hudson.tasks.test.AbstractTestResultAction.findCorrespondingResult(AbstractTestResultAction.java:252)
        at hudson.tasks.test.TestResult.getPreviousResult(TestResult.java:146)
        at hudson.tasks.junit.SuiteResult.getPreviousResult(SuiteResult.java:416)
        at hudson.tasks.junit.CaseResult.getPreviousResult(CaseResult.java:479)
        at hudson.tasks.junit.CaseResult.freeze(CaseResult.java:641)
        at hudson.tasks.junit.SuiteResult.freeze(SuiteResult.java:462)
        at hudson.tasks.junit.TestResult.freeze(TestResult.java:748)
        at hudson.tasks.junit.TestResultAction.load(TestResultAction.java:213)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:145)
        ...
        ...
        ...
        at hudson.tasks.junit.SuiteResult.freeze(SuiteResult.java:462)
        at hudson.tasks.junit.TestResult.freeze(TestResult.java:748)
        at hudson.tasks.junit.TestResultAction.load(TestResultAction.java:213)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:138)
      • locked <0x00000006ef085628> (a hudson.tasks.junit.TestResultAction)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:63)
        at hudson.tasks.test.AbstractTestResultAction.findCorrespondingResult(AbstractTestResultAction.java:252)
        at hudson.tasks.test.TestResult.getPreviousResult(TestResult.java:146)
        at hudson.tasks.junit.SuiteResult.getPreviousResult(SuiteResult.java:416)
        at hudson.tasks.junit.CaseResult.getPreviousResult(CaseResult.java:479)
        at hudson.tasks.junit.CaseResult.freeze(CaseResult.java:641)
        at hudson.tasks.junit.SuiteResult.freeze(SuiteResult.java:462)
        at hudson.tasks.junit.TestResult.freeze(TestResult.java:748)
        at hudson.tasks.junit.TestResultAction.load(TestResultAction.java:213)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:138)
      • locked <0x00000006e46233c0> (a hudson.tasks.junit.TestResultAction)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:63)
        at hudson.tasks.test.AbstractTestResultAction.findCorrespondingResult(AbstractTestResultAction.java:252)
        at hudson.tasks.test.TestResult.getPreviousResult(TestResult.java:146)
        at hudson.tasks.junit.SuiteResult.getPreviousResult(SuiteResult.java:416)
        at hudson.tasks.junit.CaseResult.getPreviousResult(CaseResult.java:479)
        at hudson.tasks.junit.CaseResult.freeze(CaseResult.java:641)
        at hudson.tasks.junit.SuiteResult.freeze(SuiteResult.java:462)
        at hudson.tasks.junit.TestResult.freeze(TestResult.java:748)
        at hudson.tasks.junit.TestResultAction.load(TestResultAction.java:213)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:138)
      • locked <0x00000006d144c000> (a hudson.tasks.junit.TestResultAction)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:63)
        at hudson.tasks.test.AbstractTestResultAction.findCorrespondingResult(AbstractTestResultAction.java:252)
        at hudson.tasks.test.TestResult.getPreviousResult(TestResult.java:146)
        at hudson.tasks.junit.SuiteResult.getPreviousResult(SuiteResult.java:416)
        at hudson.tasks.junit.CaseResult.getPreviousResult(CaseResult.java:479)
        at hudson.tasks.junit.CaseResult.freeze(CaseResult.java:641)
        at hudson.tasks.junit.SuiteResult.freeze(SuiteResult.java:462)
        at hudson.tasks.junit.TestResult.freeze(TestResult.java:748)
        at hudson.tasks.junit.TestResultAction.load(TestResultAction.java:213)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:145)
      • locked <0x00000006c3ce5168> (a hudson.tasks.junit.TestResultAction)
        at hudson.tasks.junit.TestResultAction.getResult(TestResultAction.java:63)
        at hudson.tasks.test.AbstractTestResultAction.findCorrespondingResult(AbstractTestResultAction.java:252)
        at hudson.tasks.test.TestResult.getPreviousResult(TestResult.java:146)
        at hudson.tasks.junit.SuiteResult.getPreviousResult(SuiteResult.java:416)
        at hudson.tasks.junit.CaseResult.getPreviousResult(CaseResult.java:479)
        at hudson.tasks.junit.CaseResult.getFailedSince(CaseResult.java:406)
        at hudson.tasks.junit.CaseResult.getAge(CaseResult.java:434)
        at hudson.tasks.junit.CaseResult$1.compare(CaseResult.java:754)
        at hudson.tasks.junit.CaseResult$1.compare(CaseResult.java:752)
        at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
        at java.util.TimSort.sort(TimSort.java:234)
        at java.util.Arrays.sort(Arrays.java:1512)
        at java.util.ArrayList.sort(ArrayList.java:1462)
        at java.util.Collections.sort(Collections.java:175)
        at hudson.tasks.junit.TestResult.freeze(TestResult.java:780)
        at hudson.tasks.junit.TestResultAction.setResult(TestResultAction.java:108)
      • locked <0x00000006d144c120> (a hudson.tasks.junit.TestResultAction)
        at hudson.tasks.junit.TestResultAction.<init>(TestResultAction.java:85)
        at hudson.tasks.junit.JUnitResultArchiver.parseAndAttach(JUnitResultArchiver.java:175)
      • locked <0x00000006c3d13a18> (a hudson.model.FreeStyleBuild)
        at hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:154)
        at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
        at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
        at hudson.model.Build$BuildExecution.post2(Build.java:186)
        at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
        at hudson.model.Run.execute(Run.java:1835)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:429)
        ```

            menonvarun Varun Menon
            shevek Shevek .
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: