Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13814

java.lang.OutOfMemoryError exception when getting the remote log

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • cvs-plugin
    • Jenkins 1.464 on Windows 7 64-bit, 12GB RAM; CVS Plug-in 2.4-SNAPSHOT

      We are getting a java.lang.OutOfMemoryError exception when getting the remote log.

      Started by an SCM change
      Building in workspace C:\x\workspace\project_name_here
      cvs checkout -r BRANCH_NAME_HERE -D 16 May 2012 23:27:38 -0700 -d path\projects path/projects
      cvs checkout: Updating path\projects
      cvs checkout: Updating path\projects/dir1
      cvs checkout: Updating path\projects/dir2/dirA
      ... (about 7,224 directories) ...
      cvs checkout: Updating path\projects/dirN
      cvs rlog: Logging path\projects
      ... (about 7,224 directories) ...
      cvs rlog: Logging path\projects/dirN
      FATAL: Java heap space
      java.lang.OutOfMemoryError: Java heap space
      at java.lang.StringCoding$StringDecoder.decode(Unknown Source)
      at java.lang.StringCoding.decode(Unknown Source)
      at java.lang.StringCoding.decode(Unknown Source)
      at java.lang.String.<init>(Unknown Source)
      at java.io.ByteArrayOutputStream.toString(Unknown Source)
      at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:540)
      at hudson.scm.CVSSCM.calculateChangeLog(CVSSCM.java:415)
      at hudson.scm.CVSSCM.checkout(CVSSCM.java:825)
      at hudson.model.AbstractProject.checkout(AbstractProject.java:1218)
      at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:586)
      at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:475)
      at hudson.model.Run.run(Run.java:1434)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:239)

      Some notes:

      • The path/ and path\ comes from the "Remote Name"/"Local Name" settings solution from Issue JENKINS-13264
      • We are using Jenkins 1.464 with the 2.4-SNAPSHOT version of the CVS plugin, but may not have been caused by this specific version
      • The arguments setting in the jenkins.xml file currently has -Xms1024m (anything larger and Jenkins service won't start)
      • The CVS module has many legacy projects spread out, and thus, contains more than 7,000 directories and many more files
      • Things had worked with the 1.6 plug-in & CVSNT client, but it also occasionally timed out getting the changelog, hence our testing of the 2.x plug-ins

          [JENKINS-13814] java.lang.OutOfMemoryError exception when getting the remote log

          James Gustafson created issue -

          Urban Novak added a comment -

          There are several ways to deal with this issue

          • disable changelog calculation
          • configure jenkins use 64-bit JVM with maximum heap size 2 GB or more
          • split CVS repository to several smaller ones

          Urban Novak added a comment - There are several ways to deal with this issue disable changelog calculation configure jenkins use 64-bit JVM with maximum heap size 2 GB or more split CVS repository to several smaller ones

          Michael Clarke added a comment -

          I'm not sure 2 of this options are viable - it's not acceptable to expect people to split CVS repositories and polling requires the same functions as changelog generation so preventing changelogs may not fix issues. For users on 32 bit systems, the other option isn't possible either meaning this issue needs fixed properly.

          Michael Clarke added a comment - I'm not sure 2 of this options are viable - it's not acceptable to expect people to split CVS repositories and polling requires the same functions as changelog generation so preventing changelogs may not fix issues. For users on 32 bit systems, the other option isn't possible either meaning this issue needs fixed properly.

          Urban Novak added a comment -

          I've encountered the same problem - it looks like there is a memory leak either in jenkins (1.465) or cvs plugin(2.4-SNAPSHOT (private-05/25/2012 14:52)). Restarting Jenkins fixed the problem, probably only temporarily.

          java.lang.OutOfMemoryError: Java heap space
          at java.lang.StringCoding$StringDecoder.decode(Unknown Source)
          at java.lang.StringCoding.decode(Unknown Source)
          at java.lang.StringCoding.decode(Unknown Source)
          at java.lang.String.<init>(Unknown Source)
          at java.io.ByteArrayOutputStream.toString(Unknown Source)
          at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:540)
          at hudson.scm.CVSSCM.calculateChangeLog(CVSSCM.java:415)
          at hudson.scm.CVSSCM.checkout(CVSSCM.java:825)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1218)
          at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:586)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:475)
          at hudson.model.Run.run(Run.java:1434)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:239)

          Urban Novak added a comment - I've encountered the same problem - it looks like there is a memory leak either in jenkins (1.465) or cvs plugin(2.4-SNAPSHOT (private-05/25/2012 14:52)). Restarting Jenkins fixed the problem, probably only temporarily. java.lang.OutOfMemoryError: Java heap space at java.lang.StringCoding$StringDecoder.decode(Unknown Source) at java.lang.StringCoding.decode(Unknown Source) at java.lang.StringCoding.decode(Unknown Source) at java.lang.String.<init>(Unknown Source) at java.io.ByteArrayOutputStream.toString(Unknown Source) at hudson.scm.CVSSCM.getRemoteLogForModule(CVSSCM.java:540) at hudson.scm.CVSSCM.calculateChangeLog(CVSSCM.java:415) at hudson.scm.CVSSCM.checkout(CVSSCM.java:825) at hudson.model.AbstractProject.checkout(AbstractProject.java:1218) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:586) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:475) at hudson.model.Run.run(Run.java:1434) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:239)

          Michael Clarke added a comment -

          The issue is being caused by the CVS rlog command not having any filters set so returning headers for every file within the module. This will affect any repository with a lot of files in it (both active files and those in the attic) and is most likely to result in an out of memory error at the point the byte array the log is created in is converted to a String). I'm going to be committing a change tonight that sets filters so should prevent this issue.

          Michael Clarke added a comment - The issue is being caused by the CVS rlog command not having any filters set so returning headers for every file within the module. This will affect any repository with a lot of files in it (both active files and those in the attic) and is most likely to result in an out of memory error at the point the byte array the log is created in is converted to a String). I'm going to be committing a change tonight that sets filters so should prevent this issue.

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: mc1arke
          Path:
          src/main/java/hudson/scm/CVSSCM.java
          src/main/java/hudson/scm/CvsChangeLogHelper.java
          src/test/java/hudson/scm/CvsChangeLogHelperTest.java
          http://jenkins-ci.org/commit/cvs-plugin/cd31307e33bc3749a71e5ad8d9f2a09cf3fd733d
          Log:
          [Fixed JENKINS-13227] change log and polling pick up branch changes properly
          [Fixed JENKINS-13814] change log applies filers to prevent OOM Exceptions

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: mc1arke Path: src/main/java/hudson/scm/CVSSCM.java src/main/java/hudson/scm/CvsChangeLogHelper.java src/test/java/hudson/scm/CvsChangeLogHelperTest.java http://jenkins-ci.org/commit/cvs-plugin/cd31307e33bc3749a71e5ad8d9f2a09cf3fd733d Log: [Fixed JENKINS-13227] change log and polling pick up branch changes properly [Fixed JENKINS-13814] change log applies filers to prevent OOM Exceptions

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: mc1arke
          Path:
          src/test/resources/hudson/scm/cvsRlogOutput2.txt
          http://jenkins-ci.org/commit/cvs-plugin/05e417c0c13367bda49761f9cab7618fdfba28ca
          Log:
          [Fixed JENKINS-13227] change log and polling pick up branch changes properly
          [Fixed JENKINS-13814] change log applies filers to prevent OOM Exceptions

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: mc1arke Path: src/test/resources/hudson/scm/cvsRlogOutput2.txt http://jenkins-ci.org/commit/cvs-plugin/05e417c0c13367bda49761f9cab7618fdfba28ca Log: [Fixed JENKINS-13227] change log and polling pick up branch changes properly [Fixed JENKINS-13814] change log applies filers to prevent OOM Exceptions
          Michael Clarke made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

          Michael Clarke added a comment -

          Fixed by re-applying previously set '-S' filter on rlog command

          Michael Clarke added a comment - Fixed by re-applying previously set '-S' filter on rlog command
          Michael Clarke made changes -
          Assignee New: Michael Clarke [ mc1arke ]
          Resolution New: Fixed [ 1 ]
          Status Original: In Progress [ 3 ] New: Resolved [ 5 ]

            mc1arke Michael Clarke
            jimg888 James Gustafson
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: