• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • cppcheck-plugin
    • Latest Jenkins, Linux /redacted hostname/ 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux, SuSe most probably.

      CPPcheck might be taking over 3,823,295ms loading a 500-job project on account of, possibly, one erroneous file. Or something. Long story short, I used the JavaMelody monitoring in Jenkins and saw that cppCheckResult was taking a HELL of a long time with one method call, and other attempts to find other ways to do this or ways around the bug have given, at best, nothing.

      Handling GET /job/-------/cppcheckResult/graph : RequestHandlerThread24
      /job/-------/cppcheckResult/graph? GET
      3 823 295 0 0 com.thoughtworks.xstream.converters.reflection.FieldDictionary.fieldOrNull(FieldDictionary.java:113)

      It's a "replicatable" problem on our servers, but since it's internal stuff I can't link you to it.

          [JENKINS-17363] Ludicrously slow load time [with lazyloading]

          Jan Klass added a comment -

          We experience the same problem; very high loading times.
          The last time we worked around the issue by downgrading Jenkins and the CPPCheck plugin.

          I updated to Jenkins LTS again, and updated the CPPCheck plugin, and the issue occurs again.

          The long loading time occurs when the cppcheckResult/graph is requested as well.
          Requesting it directly there is no problem once downloaded; the not modified return HTTP status code works as expected.
          Same for the jobs page with the cppcheck result publishing, which contains this image. Once cached when the request does not lead to a retransmission of the image it works fine.

          When the image is seemingly generated one CPU core is used for 100 %. Other independant requests seem to be affected as well.

          So my guess is that

          • the graph image is not cached or the cache does not work/notice nothing changed
          • the image is regenerated in a very non performant way

          Jan Klass added a comment - We experience the same problem; very high loading times. The last time we worked around the issue by downgrading Jenkins and the CPPCheck plugin. I updated to Jenkins LTS again, and updated the CPPCheck plugin, and the issue occurs again. The long loading time occurs when the cppcheckResult/graph is requested as well. Requesting it directly there is no problem once downloaded; the not modified return HTTP status code works as expected. Same for the jobs page with the cppcheck result publishing, which contains this image. Once cached when the request does not lead to a retransmission of the image it works fine. When the image is seemingly generated one CPU core is used for 100 %. Other independant requests seem to be affected as well. So my guess is that the graph image is not cached or the cache does not work/notice nothing changed the image is regenerated in a very non performant way

          Jan Klass added a comment -

          The issue occurs when there are a lot of job builds, and only a few recent ones have data.

          Removing all builds from the job (in the file system) the graph seems to work fine again.
          I guess it is missing robustness - handling invalid data or job (older) builds with no (or outdated?) cppcheck data.

          Jan Klass added a comment - The issue occurs when there are a lot of job builds, and only a few recent ones have data. Removing all builds from the job (in the file system) the graph seems to work fine again. I guess it is missing robustness - handling invalid data or job (older) builds with no (or outdated?) cppcheck data.

          How many messages are logged in cppcheck ? check your build.xml for large sizes. In that case it is related to JENKINS-19437. See also https://groups.google.com/forum/#!topic/jenkinsci-users/z66arr7QN1M

          Ruben van Staveren added a comment - How many messages are logged in cppcheck ? check your build.xml for large sizes. In that case it is related to JENKINS-19437 . See also https://groups.google.com/forum/#!topic/jenkinsci-users/z66arr7QN1M

          Depending on how many messages are logged with cppcheck, it might be related to huge per run build.xml logs

          Ruben van Staveren added a comment - Depending on how many messages are logged with cppcheck, it might be related to huge per run build.xml logs

          Michal Turek added a comment -

          I will try to fix the issue the same as in SLOCCount's JENKINS-4769.

          Michal Turek added a comment - I will try to fix the issue the same as in SLOCCount's JENKINS-4769 .

          Michal Turek added a comment -

          Michal Turek added a comment - Fixed in https://github.com/mixalturek/cppcheck-plugin/commit/e9122560bfc6944395c4a72a628310516a90bf5e , pull request sent.

          Code changed in jenkins
          User: Michal Turek
          Path:
          pom.xml
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckBuildAction.java
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckPublisher.java
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckReport.java
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckStatistics.java
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSummary.java
          src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly
          src/main/resources/tabview/main.jelly
          http://jenkins-ci.org/commit/cppcheck-plugin/e9122560bfc6944395c4a72a628310516a90bf5e
          Log:
          JENKINS-17363 Ludicrously slow load time [with lazyloading]
          JENKINS-19437 Implement load on demand functionality in Cppcheck

          • All details from Cppcheck report is stored to build.xml which is wrong. The file quickly grows with many findings and causes performance and memory issues.
          • Storing of the details to external file cppcheck_details.xml and their lazy loading implemented to fix the issue. File build.xml now contains only small statitistics neccessary for trend graph and build summary.
          • There is no need to store CppcheckReport during the build at all, its content is fully dupplicated in CppcheckSourceContainer. CppcheckStatistics has nearly identical interface so it can be used instead of CppcheckReport most of time.
          • Incorrect uses of @SuppressWarnings("unused") removed to solve compiler warnings.
          • Cppcheck tool version is stored to build.xml together with CppcheckStatistics now. This fixes a bug in the details page, the information was expected in CppcheckReport but its field is transient so the information was never stored/loaded and displayed.
          • Content of tabview/main.jelly moved dirrectly to CppcheckResult/index.jelly, include removed. This was probably only a historical location.
          • New developer added to pom.xml.
          • Several TODOs unrelated to this commit added to the code, they will be fixed in future.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michal Turek Path: pom.xml src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckBuildAction.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckPublisher.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckReport.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckStatistics.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSummary.java src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly src/main/resources/tabview/main.jelly http://jenkins-ci.org/commit/cppcheck-plugin/e9122560bfc6944395c4a72a628310516a90bf5e Log: JENKINS-17363 Ludicrously slow load time [with lazyloading] JENKINS-19437 Implement load on demand functionality in Cppcheck All details from Cppcheck report is stored to build.xml which is wrong. The file quickly grows with many findings and causes performance and memory issues. Storing of the details to external file cppcheck_details.xml and their lazy loading implemented to fix the issue. File build.xml now contains only small statitistics neccessary for trend graph and build summary. There is no need to store CppcheckReport during the build at all, its content is fully dupplicated in CppcheckSourceContainer. CppcheckStatistics has nearly identical interface so it can be used instead of CppcheckReport most of time. Incorrect uses of @SuppressWarnings("unused") removed to solve compiler warnings. Cppcheck tool version is stored to build.xml together with CppcheckStatistics now. This fixes a bug in the details page, the information was expected in CppcheckReport but its field is transient so the information was never stored/loaded and displayed. Content of tabview/main.jelly moved dirrectly to CppcheckResult/index.jelly, include removed. This was probably only a historical location. New developer added to pom.xml. Several TODOs unrelated to this commit added to the code, they will be fixed in future.

          Code changed in jenkins
          User: Michal Turek
          Path:
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java
          src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java
          src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly
          src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/summary.jelly
          http://jenkins-ci.org/commit/cppcheck-plugin/5db2c791e4ceb93f5cafce525ddbde5047aa0b97
          Log:
          JENKINS-17363 Ludicrously slow load time [with lazyloading]
          JENKINS-19437 Implement load on demand functionality in Cppcheck

          • Lazy loaded details are stored in CppcheckResult member variable and never released by Java garbace collector since a reference to them exists forever. All build.xml files are loaded during Jenkins startup and never released, CppcheckResult objects are part of them. This is kind of a memory leak. The update will cause slight lower performance but releasing of the memory is much more important for long run tasks/daemons/servers.
          • Method lazyLoadSourceContainer() updated to return the data instead of their caching in a member variable.
          • If cached CppcheckSourceContainer object can't be loaded, an empty one will be created. Different constructor used to remove unnecessary exception catches. Recently added related if removed from CppcheckSourceContainer.
          • Condition moved from index.jelly to summary.jelly to be able to reuse the data and remove one unnecessary loading (performance).
          • Construction "key = +key" has no meaning, updated to "+key" to resolve a warning.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Michal Turek Path: src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckResult.java src/main/java/org/jenkinsci/plugins/cppcheck/CppcheckSourceContainer.java src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/index.jelly src/main/resources/org/jenkinsci/plugins/cppcheck/CppcheckResult/summary.jelly http://jenkins-ci.org/commit/cppcheck-plugin/5db2c791e4ceb93f5cafce525ddbde5047aa0b97 Log: JENKINS-17363 Ludicrously slow load time [with lazyloading] JENKINS-19437 Implement load on demand functionality in Cppcheck Lazy loaded details are stored in CppcheckResult member variable and never released by Java garbace collector since a reference to them exists forever. All build.xml files are loaded during Jenkins startup and never released, CppcheckResult objects are part of them. This is kind of a memory leak. The update will cause slight lower performance but releasing of the memory is much more important for long run tasks/daemons/servers. Method lazyLoadSourceContainer() updated to return the data instead of their caching in a member variable. If cached CppcheckSourceContainer object can't be loaded, an empty one will be created. Different constructor used to remove unnecessary exception catches. Recently added related if removed from CppcheckSourceContainer. Condition moved from index.jelly to summary.jelly to be able to reuse the data and remove one unnecessary loading (performance). Construction "key = + key" has no meaning, updated to " +key" to resolve a warning.

          Michal Turek added a comment -

          Fixed, will be released in version 1.15.

          Michal Turek added a comment - Fixed, will be released in version 1.15.

            mixalturek Michal Turek
            cybikbase Renaud Lepage
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: