• Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • core
    • None

      We're running Jenkins on an Ubuntu-12.10 virtual machine, with two quad-core CPUs and 16GB RAM.

      This instance has 66 slaves, and around 80 jobs.
      Our instance is pretty much on 100% on one CPU, and the thread count dump keeps going up.

      Attaching the graphs from the monitoring plug-in, and a thread dump
      Any help on debugging this would be appreciated

        1. cpu.png
          cpu.png
          22 kB
        2. cpu (1).png
          cpu (1).png
          25 kB
        3. threadCount.png
          threadCount.png
          25 kB
        4. threadCount (1).png
          threadCount (1).png
          28 kB
        5. Thread Dump
          1.04 MB

          [JENKINS-19346] 100% on one CPU

          evernat added a comment -

          It's perhaps not the real issue or not the only issue, but you have 4 threads running "deflateBytes", for org.kohsuke.stapler.compression.CompressionFilter.reportException.

          1) Do you still have threads running "deflateBytes" in a new thread dump?

          2) What version of Jenkins are you using? (Is it Jenkins v1.520 or later?)

          See JENKINS-14362 and JENKINS-17349

          evernat added a comment - It's perhaps not the real issue or not the only issue, but you have 4 threads running "deflateBytes", for org.kohsuke.stapler.compression.CompressionFilter.reportException. 1) Do you still have threads running "deflateBytes" in a new thread dump? 2) What version of Jenkins are you using? (Is it Jenkins v1.520 or later?) See JENKINS-14362 and JENKINS-17349

          Guy Rozendorn added a comment - - edited

          We're running Jenkins 1.519; we upgraded a few weeks ago to some later version (don't remember which) but it made things worse so we reverted back.

          and yes, I still see threads in deflateBytes

          Guy Rozendorn added a comment - - edited We're running Jenkins 1.519; we upgraded a few weeks ago to some later version (don't remember which) but it made things worse so we reverted back. and yes, I still see threads in deflateBytes

          Guy Rozendorn added a comment -

          we'll upgrade to the latest now and report back tomorrow

          Guy Rozendorn added a comment - we'll upgrade to the latest now and report back tomorrow

          Guy Rozendorn added a comment -

          we had to downgrade again, as we started hitting on other issues in the newer builds :-\

          Guy Rozendorn added a comment - we had to downgrade again, as we started hitting on other issues in the newer builds :-\

          Guy Rozendorn added a comment -

          I applied the workaround mentioned in JENKINS-14362, will report back tomorrow

          Guy Rozendorn added a comment - I applied the workaround mentioned in JENKINS-14362 , will report back tomorrow

          Guy Rozendorn added a comment -

          The workaround seems to help with the CPU usage, but not with the thread count, uploading new graphs

          Guy Rozendorn added a comment - The workaround seems to help with the CPU usage, but not with the thread count, uploading new graphs

          Oleg Nenashev added a comment -

          Marking as resolved/duplicated. The submitter confirmed that the fix for JENKINS-14362 resolves his issue

          Oleg Nenashev added a comment - Marking as resolved/duplicated. The submitter confirmed that the fix for JENKINS-14362 resolves his issue

            Unassigned Unassigned
            guyr Guy Rozendorn
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: