Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61251

Out of memory issues during builds

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Critical Critical
    • core

      Hi,

      we've started to get an Out of memory issues on our Jenkins instance last week. There was no notable change in the code (1 month ago, we've installed warnings-ng for pylint reports and Cobertura), but the problems started last week.

       

      Right now, we are seeing odd behavior with our builds and such that something is probably leaking memory and then GC utilizes a CPU for 100%. This is like one build looks like:
       
      Start: !1.png|thumbnail|
      During: !2.png|thumbnail|
      During: !3.png|thumbnail|
      After some time: !4.png|thumbnail|

      My guess is that the memory should be released, but something is holding it. I was trying to go through a your help page and tried to analyze a Heap Dump. The notable is Java.Util.HashMap class has almost all allocated bytes.

      I'm providing a 2 heap dumps, one during a build link (~2.93GB) and one after the Jenkins instance is unresponsible due to the Out Of Memory link (~3.57GB).

      They suggest that the problem is org.codehaus.groovy.ast.ClassNode, but to say the truth, I'm not a Java developer so I'm not getting what can be an issue.

      Also, I'm providing a /var/log/jenkins/jenkins.log which contains lots of warnings, which may help: link (~30MB)

      I've already spent a great of time debugging what is happening, but I can't get my head around.

      Could you take a look or point where the problem should be?

      Thanks
      Andrej

       

        1. threadCount.jpg
          threadCount.jpg
          161 kB
        2. fileDescriptors.jpg
          fileDescriptors.jpg
          169 kB
        3. 4.png
          4.png
          232 kB
        4. 3.png
          3.png
          243 kB
        5. 2.png
          2.png
          239 kB
        6. 1.png
          1.png
          244 kB

            Unassigned Unassigned
            astaruch Andrej Staruch
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: