-
Improvement
-
Resolution: Fixed
-
Minor
-
None
-
Jenkins 1.480.3 running on Oracle Java version 1.7.0_03-b04 on 64-bit Linux
Warnings Plugin 4.32
Because the eclipse parser is known to be inefficient, it would be helpful to display a warning to the user to make sure they know what they are doing when they pick the eclipse parser. Something like:
Warning: you have selected the eclipse parser which is known to have performance problems. You probably want to use the javac parser instead (unless you really want to parse output from the eclipse compiler output).
Would be pretty helpful.
We stumbled upon this because we ran into a problem where the eclipse parser would cause our entire build system to come to a halt:
Some of our builds are getting "stuck" when running the Warnings plugin during the publish phase of the build. The builds affected are configured to use the "Javac (Eclipse)" parser and the "Javadoc" parser. The parsers are configured against the console log only.
When a build gets stuck running the Warnings plugin, any other build that also uses the Warnings plugin and is running at about the same time also gets stuck. Eventually, all of our slaves are consumed with builds stuck at the warnings plugin step and no further builds can be built.
We are able to kill all of the builds except the one build which triggered the problem. In order to kill the original build we end up restarting the entire Jenkins system.
We did some investigation and it appears that the Warnings plugin is getting stuck while using a regular expression to parse the console log. The thread dump of Jenkins shows that the Warnings plugin has a deep stack running a regular expression (see attached), and a quick peak at the system itself shows that the plugin is consuming 100% of one of our server's cpu cores.
It is clear from the thread stack and the build console log that the plugin is getting stuck when using the "Java (Eclipse)" plugin.
We know there have been problems in the past with parsing very large logs or files, however, the build console log is not very large (23kb).
We let the stuck build run for about 18 hours before we decided to kill it. We know the plugin was doing work (a) because the CPU core was pegged at 100%, and (b) the stack trace for the plugin worker thread would change periodically.
[JENKINS-17132] Show a warning when multi-line parser is selected
We can try to sanitize the log file so we can post it. (I work with jhurne)
Better would be tell us what you're looking for and we can post just that. Or if you have tests you'd like us to run (like a raw regex) we can run that.
It's also worth mentioning that we weren't sure why it was set to the Eclipse parser. We have switched that build to use the javac parser instead, because we're just doing a straight maven build. I'm not even sure how you can compile in Jenkins with Eclipse.
When you are using maven to compile your sources then I would recommend that you switch to the javac parser. It scans the console line by line, so the size of the log does not actually matter.
(Another option would be to checkout my sources and replace the contents of the eclipse.txt file and let the unit tests run)
We've been running with the javac parser for about a week now and we haven't seen the same issue come up again. So the problem is probably that the eclipse parser is very inefficient.
It would be nice, however if a warning could be displayed when configuring the plugin if the eclipse parser is selected (to help others avoid this mistake). It seems reasonable to assume (but correct me if I am wrong) that most Jenkins users are compiling with javac and not eclipse. Maybe a warning like the following could be displayed:
Warning: you have selected the eclipse parser which is known to have performance problems. You probably want to use the javac parser instead (unless you really want to parse output from the eclipse compiler output).
Changing the bug name to reflect this change in focus.
That might be a good idea. I made this issue more general as the doxygen parser also suffers from this behavior.
Code changed in jenkins
User: Ulli Hafner
Path:
src/main/java/io/jenkins/plugins/analysis/warnings/Doxygen.java
src/main/java/io/jenkins/plugins/analysis/warnings/DrMemory.java
src/main/java/io/jenkins/plugins/analysis/warnings/Eclipse.java
src/main/java/io/jenkins/plugins/analysis/warnings/GhsMulti.java
src/main/java/io/jenkins/plugins/analysis/warnings/GnuFortran.java
src/main/java/io/jenkins/plugins/analysis/warnings/NagFortran.java
src/main/java/io/jenkins/plugins/analysis/warnings/YuiCompressor.java
src/main/resources/io/jenkins/plugins/analysis/warnings/Messages.properties
http://jenkins-ci.org/commit/warnings-plugin/ec0337ca13dc5c1a90336bcce802e477c1125c34
Log:
[FIXED JENKINS-17132] Show a warning if slow multi-line parser is selected.
Can you please attach one of the logs that caused the problem?
The Eclipse Parser is known to be slow since it needs to parse the whole file in one step (multi line parsing). How do you invoke the Eclipse compiler?