Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55805

JavaDocParser is slow at parsing the log output

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • analysis-model
    • Plugin version 2.1.1
      Jenkins 2.150.2
    • warnings-ng 4.0.0, analysis-model-api 3.0.0

      First of all, the new plugin is great.

      Some days ago i noticed a major change in the build performance of my project (http://83.169.43.181/jenkins/job/HtmlUnit/). After some checking i found that this is because the parsing of the output takes really long.

      19:43:09 [Java] Sleeping for 5 seconds due to JENKINS-32191...
      19:43:14 [Java] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit')
      19:59:59 [Java] Attaching ResultAction with ID 'java' to run 'HtmlUnit #262'.
      19:59:59 [Java] Using reference build 'HtmlUnit #261' to compute new, fixed, and outstanding issues
      19:59:59 [Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
      19:59:59 [Java] No quality gates have been set - skipping
      19:59:59 [Java] Health report is disabled - skipping
      19:59:59 [Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)

      Yes the log is really long, but this was not a problem in the past. With a bit more testing i found out that the log contains two really long lines with javascript inside. Looks like this lines are killing the parser.
      Have attached the critical lines.

      This is a log output for a fast run (one without the js lines nothing else changed).

      22:37:19 [Java] Sleeping for 5 seconds due to JENKINS-32191...
      22:37:24 [Java] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit')
      22:37:24 [Java] Attaching ResultAction with ID 'java' to run 'HtmlUnit #263'.
      22:37:24 [Java] Using reference build 'HtmlUnit #262' to compute new, fixed, and outstanding issues
      22:37:24 [Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
      22:37:24 [Java] No quality gates have been set - skipping
      22:37:24 [Java] Health report is disabled - skipping
      22:37:24 [Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)

      If you like i can provide the whole log also.

        1. build_output.txt
          272 kB

          [JENKINS-55805] JavaDocParser is slow at parsing the log output

          Ronald Brill created issue -
          Ronald Brill made changes -
          Description Original: First of all, the new plugin is great.

          Some days ago i noticed a major change in the build performance of my project (http://83.169.43.181/jenkins/job/HtmlUnit/). After some checking i found that this is because the parsing of the output takes really long.

          19:43:09 [Java] Sleeping for 5 seconds due to JENKINS-32191...
          19:43:14 [Java] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit')
          19:59:59 [Java] Attaching ResultAction with ID 'java' to run 'HtmlUnit #262'.
          19:59:59 [Java] Using reference build 'HtmlUnit #261' to compute new, fixed, and outstanding issues
          19:59:59 [Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
          19:59:59 [Java] No quality gates have been set - skipping
          19:59:59 [Java] Health report is disabled - skipping
          19:59:59 [Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)

          Yes the log is really long, but this was not a problem in the past. With a bit more testing i found out that the log contains two really long lines with javascript inside. Looks like this lines are killing the parser.
          Have attached the critical lines.

          This is a log output for a fast run (one without the js lines nothing else changed).

          22:37:19 [Java] Sleeping for 5 seconds due to JENKINS-32191...
          22:37:24 [Java] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit')
          22:37:24 [Java] Attaching ResultAction with ID 'java' to run 'HtmlUnit #263'.
          22:37:24 [Java] Using reference build 'HtmlUnit #262' to compute new, fixed, and outstanding issues
          22:37:24 [Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
          22:37:24 [Java] No quality gates have been set - skipping
          22:37:24 [Java] Health report is disabled - skipping
          22:37:24 [Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)           		New: First of all, the new plugin is great.

          Some days ago i noticed a major change in the build performance of my project (http://83.169.43.181/jenkins/job/HtmlUnit/). After some checking i found that this is because the parsing of the output takes really long.

          19:43:09 [Java] Sleeping for 5 seconds due to JENKINS-32191...
          19:43:14 [Java] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit')
          19:59:59 [Java] Attaching ResultAction with ID 'java' to run 'HtmlUnit #262'.
          19:59:59 [Java] Using reference build 'HtmlUnit #261' to compute new, fixed, and outstanding issues
          19:59:59 [Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
          19:59:59 [Java] No quality gates have been set - skipping
          19:59:59 [Java] Health report is disabled - skipping
          19:59:59 [Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)

          Yes the log is really long, but this was not a problem in the past. With a bit more testing i found out that the log contains two really long lines with javascript inside. Looks like this lines are killing the parser.
          Have attached the critical lines.

          This is a log output for a fast run (one without the js lines nothing else changed).

          22:37:19 [Java] Sleeping for 5 seconds due to JENKINS-32191...
          22:37:24 [Java] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit')
          22:37:24 [Java] Attaching ResultAction with ID 'java' to run 'HtmlUnit #263'.
          22:37:24 [Java] Using reference build 'HtmlUnit #262' to compute new, fixed, and outstanding issues
          22:37:24 [Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
          22:37:24 [Java] No quality gates have been set - skipping
          22:37:24 [Java] Health report is disabled - skipping
          22:37:24 [Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)

          If you like i can provide the whole log also.

          Ulli Hafner added a comment -

          Did you choose the Java parser?

          Ulli Hafner added a comment - Did you choose the Java parser?

          Ronald Brill added a comment - - edited

          Thanks for looking at this.
          Yes, java and a bunch of PMD Spotbugs and all that stuff.
          In detail:

          • Java
          • Checkstyle
          • JavaDoc
          • PMD
          • CPD
          • Spotbugs
            All with the default settings.

          Ronald Brill added a comment - - edited Thanks for looking at this. Yes, java and a bunch of PMD Spotbugs and all that stuff. In detail: Java Checkstyle JavaDoc PMD CPD Spotbugs All with the default settings.

          Ronald Brill added a comment -

          Have removed the Java parser, not the JavaDoc parser is the blocker

          20:39:49 [JavaDoc] Sleeping for 5 seconds due to JENKINS-32191...
          20:39:54 [JavaDoc] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit')
          20:50:24 [JavaDoc] Attaching ResultAction with ID 'javadoc-warnings' to run 'HtmlUnit #268'.
          20:50:24 [JavaDoc] Using reference build 'HtmlUnit #267' to compute new, fixed, and outstanding issues
          20:50:24 [JavaDoc] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
          20:50:24 [JavaDoc] No quality gates have been set - skipping
          20:50:24 [JavaDoc] Health report is disabled - skipping
          20:50:24 [JavaDoc] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)

          Ronald Brill added a comment - Have removed the Java parser, not the JavaDoc parser is the blocker 20:39:49 [JavaDoc] Sleeping for 5 seconds due to JENKINS-32191 ... 20:39:54 [JavaDoc] Parsing console log (workspace: '/var/lib/jenkins/workspace/HtmlUnit') 20:50:24 [JavaDoc] Attaching ResultAction with ID 'javadoc-warnings' to run 'HtmlUnit #268'. 20:50:24 [JavaDoc] Using reference build 'HtmlUnit #267' to compute new, fixed, and outstanding issues 20:50:24 [JavaDoc] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0 20:50:24 [JavaDoc] No quality gates have been set - skipping 20:50:24 [JavaDoc] Health report is disabled - skipping 20:50:24 [JavaDoc] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)

          Ronald Brill added a comment -

          Maybe using the regex with the first e.g. 400 chars of a line to make sure this is a candidate for a deeper analysis is an option to make the parser faster.

          Ronald Brill added a comment - Maybe using the regex with the first e.g. 400 chars of a line to make sure this is a candidate for a deeper analysis is an option to make the parser faster.
          Ulli Hafner made changes -
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Fixed but Unreleased [ 10203 ]
          Ulli Hafner made changes -
          Released As New: warnings-ng 2.2.0 (and analysis model 2.1.1)
          Status Original: Fixed but Unreleased [ 10203 ] New: Resolved [ 5 ]

          Ronald Brill added a comment -

          Sorry but after updating to the latest version of the analysis and warnings ng plugin i still have the 10 minutes delay for parsing.

          Ronald Brill added a comment - Sorry but after updating to the latest version of the analysis and warnings ng plugin i still have the 10 minutes delay for parsing.
          Ronald Brill made changes -
          Resolution Original: Fixed [ 1 ]
          Status Original: Resolved [ 5 ] New: Reopened [ 4 ]

            Unassigned Unassigned
            rbri Ronald Brill
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: