Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19628

Java out of memory bug when using Warnings plugin parsing self defined parser

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major
    • warnings-plugin
    • HP DL380 G7
      Xeon E5649 x 2
      32G memory

      CentOS 6.2 x86_64

      Our log file is over 20MB, when using a self defined parser to parsing it, it always shows out of memory exceptions even if I add the -Xmx4096m parameter.

      And I tried to split the log files into several small ones but it seems like warnings plugin collect all of then and parse them together. I think at least this procedure can be adjusted to parsing each file one by one and collect parsing result together in order to save memory.

          [JENKINS-19628] Java out of memory bug when using Warnings plugin parsing self defined parser

          Ulli Hafner added a comment -

          Can you please wrap code and expressions in {noformat} or {code} blocks (see Confluence help on macros).

          Ulli Hafner added a comment - Can you please wrap code and expressions in {noformat} or {code} blocks (see Confluence help on macros).

          Thomas Li added a comment -

          Done, please check, thks.

          Thomas Li added a comment - Done, please check, thks.

          Ulli Hafner added a comment -

          Ok, I now tried to reproduce the bug without success: I created your parser and let it scan the console log that contains all Java source files of Jenkins (5.716 KB). This works quite fast and had no problems so far.

          Maybe your command line argument is not really passed in your build. Can you have a look at the Jenkins process to see how much memory is allocated? Or there is not much memory left in your Jenkins when you start the Job... What does the Jenkins monitoring plugin report?

          Ulli Hafner added a comment - Ok, I now tried to reproduce the bug without success: I created your parser and let it scan the console log that contains all Java source files of Jenkins (5.716 KB). This works quite fast and had no problems so far. Maybe your command line argument is not really passed in your build. Can you have a look at the Jenkins process to see how much memory is allocated? Or there is not much memory left in your Jenkins when you start the Job... What does the Jenkins monitoring plugin report?

          Thomas Li added a comment -

          I mentioned in my original report that our log file is quite large, more than 20MB. The parser can process log file within 1MB quite well. I am using a server with 32GB memory and set java environment parameter -Xmx4096m. I don't know if I should enlarge the parameter?

          And there may be another problem as I mentioned. When jenkins process multiple log files it seems that it will process all the files together rather than one by one.

          Thomas Li added a comment - I mentioned in my original report that our log file is quite large, more than 20MB. The parser can process log file within 1MB quite well. I am using a server with 32GB memory and set java environment parameter -Xmx4096m. I don't know if I should enlarge the parameter? And there may be another problem as I mentioned. When jenkins process multiple log files it seems that it will process all the files together rather than one by one.

          Ulli Hafner added a comment -

          That does not change anything, even if the console log is 46.776 KB - and using the default vm settings. Can you please check if your Java process fills up the whole memory of 4GByte?. Is your build on a slave? Does your parser really not contain a newline character?

          Ulli Hafner added a comment - That does not change anything, even if the console log is 46.776 KB - and using the default vm settings. Can you please check if your Java process fills up the whole memory of 4GByte?. Is your build on a slave? Does your parser really not contain a newline character?

          Thomas Li added a comment -

          I mean MB not KB, 46.776KB is OK in my environment too.

          Thomas Li added a comment - I mean MB not KB, 46.776KB is OK in my environment too.

          Ulli Hafner added a comment -

          Sorry, I copied the number using my locale settings (German), the dot is used as separator between thousandths 46.776KB = 46,7MB (in German), in English the separators are used in the opposite way.

          Can you please check if your Java process fills up the whole memory of 4GByte?. Is your build on a slave? Does your parser really not contain a newline character?

          Ulli Hafner added a comment - Sorry, I copied the number using my locale settings (German), the dot is used as separator between thousandths 46.776KB = 46,7MB (in German), in English the separators are used in the opposite way. Can you please check if your Java process fills up the whole memory of 4GByte?. Is your build on a slave? Does your parser really not contain a newline character?

          Thomas Li added a comment -

          Ah, yes, the build is on a windows slave with 2GB memory. So it doesn't matter how many master's memory left. I tried add these parameters "-Xms512m -Xmx1024m" to the node's "Launch method" option. But it seems the same. I found that the java process cost more than 311MB memory when this job was running through windows task manager. How can I modify the slave's heap space?

          Thomas Li added a comment - Ah, yes, the build is on a windows slave with 2GB memory. So it doesn't matter how many master's memory left. I tried add these parameters "-Xms512m -Xmx1024m" to the node's "Launch method" option. But it seems the same. I found that the java process cost more than 311MB memory when this job was running through windows task manager. How can I modify the slave's heap space?

          Ulli Hafner added a comment -

          I don't know how to change that. How do you connect your slaves? ssh? jnlp?

          Ulli Hafner added a comment - I don't know how to change that. How do you connect your slaves? ssh? jnlp?

          Thomas Li added a comment -

          I've found the way to change slave's java memory heap. Just add "-Xmx512m" to '<arguments>' in jenkins-slave.xml which is located in slave's JENKINSROOT.
          And I connect my slave using jnlp.
          Thank you!

          Thomas Li added a comment - I've found the way to change slave's java memory heap. Just add "-Xmx512m" to '<arguments>' in jenkins-slave.xml which is located in slave's JENKINSROOT. And I connect my slave using jnlp. Thank you!

            drulli Ulli Hafner
            iaadoa Thomas Li
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: