• 5.0.0-beta3

      After updating to the Task Scanner Plugin 4.52 and Warnings Plugin 4.63 I get an OutOfMemory Error when GitBlamer is used to create author and commit information. This change was introduced by JENKINS-6748.

      The repository is large but by no means too large:

       

      $ du -hs .
      668M .
      $ git count-objects --human-readable
      247 objects, 169.25 KiB
      $ git rev-list --all --count
      25881
      

      This is an example error thrown by the Task Scanner Plugin:

      [TASKS] Scanning folder '/my/workspace' for files matching the pattern '**/*.java' - excludes: 
      [TASKS] Found 4657 files to scan for tasks
      [TASKS] Found 1058 open tasks.
      <Git Blamer> Using GitBlamer to create author and commit information for all warnings.
      <Git Blamer> GIT_COMMIT=1a23b8068bf1226fd62085036db6bb30a0edea55, workspace=/my/workspace
       > git rev-parse 1a23b8068bf1226fd62085036db6bb30a0edea55^{commit} # timeout=10
      FATAL: Java heap space
      java.lang.OutOfMemoryError: Java heap space
       at org.eclipse.jgit.internal.storage.file.PackFile.read(PackFile.java:693)
       at org.eclipse.jgit.internal.storage.file.WindowCache.load(WindowCache.java:289)
       at org.eclipse.jgit.internal.storage.file.WindowCache.getOrLoad(WindowCache.java:368)
       at org.eclipse.jgit.internal.storage.file.WindowCache.get(WindowCache.java:179)
       at org.eclipse.jgit.internal.storage.file.WindowCursor.pin(WindowCursor.java:336)
       at org.eclipse.jgit.internal.storage.file.WindowCursor.copy(WindowCursor.java:237)
       at org.eclipse.jgit.internal.storage.file.PackFile.readFully(PackFile.java:592)
       at org.eclipse.jgit.internal.storage.file.PackFile.load(PackFile.java:767)
       at org.eclipse.jgit.internal.storage.file.PackFile.get(PackFile.java:273)
       at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:421)
       at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:390)
       at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:382)
       at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:154)
       at org.eclipse.jgit.treewalk.CanonicalTreeParser.reset(CanonicalTreeParser.java:214)
       at org.eclipse.jgit.treewalk.CanonicalTreeParser.createSubtreeIterator0(CanonicalTreeParser.java:248)
       at org.eclipse.jgit.treewalk.CanonicalTreeParser.createSubtreeIterator(CanonicalTreeParser.java:226)
       at org.eclipse.jgit.treewalk.CanonicalTreeParser.createSubtreeIterator(CanonicalTreeParser.java:70)
       at org.eclipse.jgit.treewalk.TreeWalk.enterSubtree(TreeWalk.java:1210)
       at org.eclipse.jgit.treewalk.TreeWalk.next(TreeWalk.java:828)
       at org.eclipse.jgit.blame.BlameGenerator.find(BlameGenerator.java:955)
       at org.eclipse.jgit.blame.BlameGenerator.processOne(BlameGenerator.java:603)
       at org.eclipse.jgit.blame.BlameGenerator.next(BlameGenerator.java:496)
       at org.eclipse.jgit.blame.BlameResult.computeAll(BlameResult.java:239)
       at org.eclipse.jgit.blame.BlameGenerator.computeBlameResult(BlameGenerator.java:450)
       at org.eclipse.jgit.api.BlameCommand.call(BlameCommand.java:230)
       at hudson.plugins.analysis.util.GitBlamer$BlameCallback.invoke(GitBlamer.java:130)
       at hudson.plugins.analysis.util.GitBlamer$BlameCallback.invoke(GitBlamer.java:104)
       at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:71)
       at hudson.plugins.analysis.util.GitBlamer.loadBlameResultsForFiles(GitBlamer.java:61)
       at hudson.plugins.analysis.util.GitBlamer.blame(GitBlamer.java:54)
       at hudson.plugins.analysis.util.AbstractBlamer$1.invoke(AbstractBlamer.java:62)

          [JENKINS-45750] GitBlamer causes OutOfMemoryError

          drulli The build is unfortunately on the master. I have a ton of builds on slaves, but they are all pipelines and the feature isn't enabled there so I'm not sure if I would see the same behavior there. The Jenkins JVM is started with -Xmx1024M and consumes around 400Mb according to melody monitoring.

          Where can I deactivate this feature globally or in the job configuration?

          Philipp Moeller added a comment - drulli The build is unfortunately on the master. I have a ton of builds on slaves, but they are all pipelines and the feature isn't enabled there so I'm not sure if I would see the same behavior there. The Jenkins JVM is started with -Xmx1024M and consumes around 400Mb according to melody monitoring. Where can I deactivate this feature globally or in the job configuration?

          Ulli Hafner added a comment -

          In the Jenkins global configuration.

          Ulli Hafner added a comment - In the Jenkins global configuration.

          OK, found it. Took a moment since I expected it to be somewhere under "Warnings" or "Tasks" but it is part of the base plugin.

          Anyway, how much heap space is this operations supposed to consume on a repo of this size?

          Philipp Moeller added a comment - OK, found it. Took a moment since I expected it to be somewhere under "Warnings" or "Tasks" but it is part of the base plugin. Anyway, how much heap space is this operations supposed to consume on a repo of this size?

          Matías Fernández added a comment - - edited

          Hello there,

          Same issue here and after updating to version 2.73 of Jenkins the error have more impact.

          I send data to know about the size of my repository.

          ➜  du -hs
          1.5G    .
          ➜   git count-objects --human-readable
          3786 objects, 17.36 MiB
          ➜   git rev-list --all --count
          19877

          The only way I could disable this was disabling PMD checks, I couldn't find if possible to disable this issue without disable those checks.

          Matías Fernández added a comment - - edited Hello there, Same issue here and after updating to version 2.73 of Jenkins the error have more impact. I send data to know about the size of my repository. ➜  du -hs 1.5G    . ➜   git count-objects --human-readable 3786 objects, 17.36 MiB ➜   git rev-list --all --count 19877 The only way I could disable this was disabling PMD checks, I couldn't find if possible to disable this issue without disable those checks.

          Ulli Hafner added a comment -

          Did you not find the option to disable (see above)? Why is this related to another Jenkins version?

          Ulli Hafner added a comment - Did you not find the option to disable (see above)? Why is this related to another Jenkins version?

          The version I have is 2.73, the option to disable isn't there at least.

          This bug can be reproduce in versions 2.60 and 2.73 for my information.

          Matías Fernández added a comment - The version I have is 2.73, the option to disable isn't there at least. This bug can be reproduce in versions 2.60 and 2.73 for my information.

          Ulli Hafner added a comment - - edited

          The option is from the plugin, so if you have the plugin enabled then you have the option. It is not related to Jenkins version.

          Ulli Hafner added a comment - - edited The option is from the plugin, so if you have the plugin enabled then you have the option. It is not related to Jenkins version.

          Hi Ulli, thanks for all your answers and now can understand the problem is in the plugin and not in the Jenkins version.

          I could found know the option you talk previously, i left where I could found this.

          Jenkins -> Manage Jenkins -> Configure System -> Static Code Analysis Plug-ins

          In there check "Omit Author and Commit Information".

          Thanks again for your help.

          Matías Fernández added a comment - Hi Ulli, thanks for all your answers and now can understand the problem is in the plugin and not in the Jenkins version. I could found know the option you talk previously, i left where I could found this. Jenkins -> Manage Jenkins -> Configure System -> Static Code Analysis Plug-ins In there check "Omit Author and Commit Information". Thanks again for your help.

          dirmeier added a comment -

          Hi drulli, we also have an OutOfMemery Exception but with another Stacktrace. (another issue?)

          ERROR: Build step failed with exception
          java.lang.OutOfMemoryError: GC overhead limit exceeded
          	at org.eclipse.jgit.internal.storage.file.PackFile.load(PackFile.java:878)
          Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to FIPS_Build6
          		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
          		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
          		at hudson.remoting.Channel.call(Channel.java:955)
          		at hudson.FilePath.act(FilePath.java:1071)
          		at hudson.FilePath.act(FilePath.java:1060)
          		at hudson.plugins.analysis.util.AbstractBlamer.invokeBlamer(AbstractBlamer.java:59)
          		at hudson.plugins.analysis.util.AbstractBlamer.blame(AbstractBlamer.java:45)
          		at hudson.plugins.analysis.core.HealthAwarePublisher.blame(HealthAwarePublisher.java:109)
          		at hudson.plugins.checkstyle.CheckStylePublisher.perform(CheckStylePublisher.java:81)
          		at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:69)
          		at hudson.plugins.analysis.core.HealthAwareRecorder.perform(HealthAwareRecorder.java:298)
          		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
          		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
          		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
          		at hudson.model.Build$BuildExecution.post2(Build.java:186)
          		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
          		at hudson.model.Run.execute(Run.java:1840)
          		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          		at hudson.model.ResourceController.execute(ResourceController.java:97)
          		at hudson.model.Executor.run(Executor.java:429)
          Caused: org.eclipse.jgit.errors.LargeObjectException$OutOfMemory: Out of memory loading unknown object
          	at org.eclipse.jgit.internal.storage.file.PackFile.load(PackFile.java:881)
          	at org.eclipse.jgit.internal.storage.file.PackFile.get(PackFile.java:273)
          	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:421)
          	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:390)
          	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:382)
          	at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:154)
          	at org.eclipse.jgit.treewalk.CanonicalTreeParser.reset(CanonicalTreeParser.java:214)
          	at org.eclipse.jgit.treewalk.TreeWalk.reset(TreeWalk.java:645)
          	at org.eclipse.jgit.blame.BlameGenerator.find(BlameGenerator.java:954)
          	at org.eclipse.jgit.blame.BlameGenerator.processOne(BlameGenerator.java:603)
          	at org.eclipse.jgit.blame.BlameGenerator.next(BlameGenerator.java:496)
          	at org.eclipse.jgit.blame.BlameResult.computeAll(BlameResult.java:239)
          	at org.eclipse.jgit.blame.BlameGenerator.computeBlameResult(BlameGenerator.java:450)
          	at org.eclipse.jgit.api.BlameCommand.call(BlameCommand.java:230)
          	at hudson.plugins.analysis.util.GitBlamer$BlameCallback.invoke(GitBlamer.java:130)
          	at hudson.plugins.analysis.util.GitBlamer$BlameCallback.invoke(GitBlamer.java:104)
          	at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
          	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:72)
          	at hudson.plugins.analysis.util.GitBlamer.loadBlameResultsForFiles(GitBlamer.java:61)
          	at hudson.plugins.analysis.util.GitBlamer.blame(GitBlamer.java:54)
          	at hudson.plugins.analysis.util.AbstractBlamer$1.invoke(AbstractBlamer.java:62)
          	at hudson.plugins.analysis.util.AbstractBlamer$1.invoke(AbstractBlamer.java:59)
          	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3085)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          	at hudson.remoting.Request$2.run(Request.java:369)
          	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          

          We still have a lot of warnings and in combination with a quite big repository. Built on a slave (as you can see in the stacktrace).

          I just disabled Git-Blamer and now it works again. I think this feature is not very useful for that many historical warnings. Maybe limit the blaming for new warnings only or up to a specifyable limit (about 100 warnings). Additionally it would be nice to have the disable-option in the job configuration instead of system-wide.

          Finally: we need to meet for a pizza

          dirmeier added a comment - Hi drulli , we also have an OutOfMemery Exception but with another Stacktrace. (another issue?) ERROR: Build step failed with exception java.lang.OutOfMemoryError: GC overhead limit exceeded at org.eclipse.jgit.internal.storage.file.PackFile.load(PackFile.java:878) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to FIPS_Build6 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:955) at hudson.FilePath.act(FilePath.java:1071) at hudson.FilePath.act(FilePath.java:1060) at hudson.plugins.analysis.util.AbstractBlamer.invokeBlamer(AbstractBlamer.java:59) at hudson.plugins.analysis.util.AbstractBlamer.blame(AbstractBlamer.java:45) at hudson.plugins.analysis.core.HealthAwarePublisher.blame(HealthAwarePublisher.java:109) at hudson.plugins.checkstyle.CheckStylePublisher.perform(CheckStylePublisher.java:81) at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:69) at hudson.plugins.analysis.core.HealthAwareRecorder.perform(HealthAwareRecorder.java:298) at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690) at hudson.model.Build$BuildExecution.post2(Build.java:186) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635) at hudson.model.Run.execute(Run.java:1840) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused: org.eclipse.jgit.errors.LargeObjectException$OutOfMemory: Out of memory loading unknown object at org.eclipse.jgit.internal.storage.file.PackFile.load(PackFile.java:881) at org.eclipse.jgit.internal.storage.file.PackFile.get(PackFile.java:273) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedObject(ObjectDirectory.java:421) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openPackedFromSelfOrAlternate(ObjectDirectory.java:390) at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:382) at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:154) at org.eclipse.jgit.treewalk.CanonicalTreeParser.reset(CanonicalTreeParser.java:214) at org.eclipse.jgit.treewalk.TreeWalk.reset(TreeWalk.java:645) at org.eclipse.jgit.blame.BlameGenerator.find(BlameGenerator.java:954) at org.eclipse.jgit.blame.BlameGenerator.processOne(BlameGenerator.java:603) at org.eclipse.jgit.blame.BlameGenerator.next(BlameGenerator.java:496) at org.eclipse.jgit.blame.BlameResult.computeAll(BlameResult.java:239) at org.eclipse.jgit.blame.BlameGenerator.computeBlameResult(BlameGenerator.java:450) at org.eclipse.jgit.api.BlameCommand.call(BlameCommand.java:230) at hudson.plugins.analysis.util.GitBlamer$BlameCallback.invoke(GitBlamer.java:130) at hudson.plugins.analysis.util.GitBlamer$BlameCallback.invoke(GitBlamer.java:104) at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:72) at hudson.plugins.analysis.util.GitBlamer.loadBlameResultsForFiles(GitBlamer.java:61) at hudson.plugins.analysis.util.GitBlamer.blame(GitBlamer.java:54) at hudson.plugins.analysis.util.AbstractBlamer$1.invoke(AbstractBlamer.java:62) at hudson.plugins.analysis.util.AbstractBlamer$1.invoke(AbstractBlamer.java:59) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3085) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) We still have a lot of warnings and in combination with a quite big repository. Built on a slave (as you can see in the stacktrace). I just disabled Git-Blamer and now it works again. I think this feature is not very useful for that many historical warnings. Maybe limit the blaming for new warnings only or up to a specifyable limit (about 100 warnings). Additionally it would be nice to have the disable-option in the job configuration instead of system-wide. Finally: we need to meet for a pizza

          Ulli Hafner added a comment -

          Should work better in the upcoming beta 2 of the warnings plugin. (Maybe upgrading the Git and Git client plugins to the latest betas will also help, since they now use EGit 5.1).

          Ulli Hafner added a comment - Should work better in the upcoming beta 2 of the warnings plugin. (Maybe upgrading the Git and Git client plugins to the latest betas will also help, since they now use EGit 5.1).

            drulli Ulli Hafner
            pmr Philipp Moeller
            Votes:
            5 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: