Details
-
Improvement
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Fixed
-
None
-
Platform: PC, OS: Linux
Description
Some parsers are really slow for my system (4mb log file size). In total the
warnings plugin needs 1:45h to process the log with all parsers. Since I have
six different builds this will need more than 10h just for parsing the logs ...
Some tests with the unit tests showed that the MsBuild, gcc and erlang parsers
are really slow. Is there a way of to make the parsers used for a build
configurable or to speed up these parsers (e.g. the gnat parser and others need
less than a second to parse the log).
If the desired way is to speed up the parsers I can do some testing with
"better" regular expressions ...
Attachments
Activity
bergerbd
created issue -
bergerbd
added a comment - Created an attachment (id=443)
Sample patch for GCC parser I'll look at all these regexps if it is possible to switch to the mutli-line
mode as suggested in your patch. I haven't yet tried to test the parsers with a
large file...
If that does not help, I'll add an option to enable/configure the parsers in the
config screen.
bergerbd
added a comment - Created an attachment (id=444)
Activated multiline for erlang, gcc and msbuild
bergerbd
added a comment - Created an attachment (id=444)
Activated multiline for erlang, gcc and msbuild bergerbd
added a comment - The patch do not break any test case and speeds up the parsing (let's say 10s
total time for parsing my log).
The GnatParser is a little bit more severe now. (Less false positives in the
feature since it checks for the file endings (ads/adb))
Hope the patch helps. 
bergerbd
added a comment - The patch do not break any test case and speeds up the parsing (let's say 10s
total time for parsing my log).
The GnatParser is a little bit more severe now. (Less false positives in the
feature since it checks for the file endings (ads/adb))
Hope the patch helps. Thanks for adding the patches. I think in the long term I need to provide a
parser selection. (See issue
https://hudson.dev.java.net/issues/show_bug.cgi?id=2462).
| Field | Original Value | New Value |
|---|---|---|
| Status | Open [ 1 ] | In Progress [ 3 ] |
Code changed in hudson
User: : drulli
Path:
trunk/hudson/plugins/warnings/src/main/java/hudson/plugins/warnings/parser/ErlcParser.java
trunk/hudson/plugins/warnings/src/main/java/hudson/plugins/warnings/parser/GccParser.java
trunk/hudson/plugins/warnings/src/main/java/hudson/plugins/warnings/parser/GnatParser.java
trunk/hudson/plugins/warnings/src/main/java/hudson/plugins/warnings/parser/MsBuildParser.java
http://fisheye4.cenqua.com/changelog/hudson/?cs=13370
Log:
[FIXED JENKINS-2635] Speedup for several slow regexp parsers.
| Resolution | Fixed [ 1 ] | |
| Status | In Progress [ 3 ] | Resolved [ 5 ] |
| Status | Resolved [ 5 ] | Closed [ 6 ] |
| Workflow | JNJira [ 132708 ] | JNJira + In-Review [ 201611 ] |
I have made some additional tests with the GCC parser. It seems as if the single
line mode is the reason for the performance issue. It would be much faster if it
would be possible to specify the ^ and $ for the line based matcher (which is
not possible due to the special meaning of these symbols in the line mode). The
only possible solution is to switch to the multiline mode and specify the whole
line with ^ and $.
(timed on a Intel Core Duo @ 1.2GHz each core)
original gcc-parser:
original testcase -> 0.004 sec
my log -> killed after 60min
patched gcc-parser:
original test -> 0.006 sec
my log -> 1.24 sec