Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27664

Job hanging, killing not possible

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • warnings-plugin
    • None

      I have a job that keeps hanging everytime it builds. The job ran fine in the past. No changes happend and suddenly it started hanging. ThradDump:

      Executor #0 for Slave_rlx-v121 : executing m_techlog_reporting/zk.mtr.build #945

"Executor #0 for Slave_rlx-v121 : executing m_techlog_reporting/zk.mtr.build #945" Id=79 Group=main RUNNABLE
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4692)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.match(Pattern.java:4683)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:3715)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Loop.matchInit(Pattern.java:4699)
	at java.util.regex.Pattern$Prolog.match(Pattern.java:4639)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$BranchConn.match(Pattern.java:4466)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Slice.match(Pattern.java:3870)
	at java.util.regex.Pattern$Branch.match(Pattern.java:4502)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4177)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$Branch.match(Pattern.java:4500)
	at java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
	at java.util.regex.Pattern$Curly.match(Pattern.java:4132)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$GroupTail.match(Pattern.java:4615)
	at java.util.regex.Pattern$BranchConn.match(Pattern.java:4466)
	at java.util.regex.Pattern$Slice.match(Pattern.java:3870)
	at java.util.regex.Pattern$Branch.match(Pattern.java:4502)
	at java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
	at java.util.regex.Pattern$Ques.match(Pattern.java:4079)
	at java.util.regex.Pattern$Start.match(Pattern.java:3408)
	at java.util.regex.Matcher.search(Matcher.java:1199)
	at java.util.regex.Matcher.find(Matcher.java:592)
	at hudson.plugins.warnings.parser.RegexpParser.findAnnotations(RegexpParser.java:86)
	at hudson.plugins.warnings.parser.RegexpDocumentParser.parse(RegexpDocumentParser.java:62)
	at hudson.plugins.warnings.parser.ParserRegistry.parse(ParserRegistry.java:280)
	at hudson.plugins.warnings.parser.ParserRegistry.parse(ParserRegistry.java:259)
	at hudson.plugins.warnings.WarningsPublisher.parseConsoleLog(WarningsPublisher.java:382)
	at hudson.plugins.warnings.WarningsPublisher.perform(WarningsPublisher.java:324)
	at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:152)
	at hudson.plugins.analysis.core.HealthAwareRecorder.perform(HealthAwareRecorder.java:347)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:734)
	at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:1046)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:683)
	at hudson.model.Run.execute(Run.java:1784)
	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:531)
	at hudson.model.ResourceController.execute(ResourceController.java:89)
	at hudson.model.Executor.run(Executor.java:240)

        1. eclipse-regex-explosion.txt
          2 kB

          [JENKINS-27664] Job hanging, killing not possible

          Kanstantsin Shautsou added a comment -

          And in the end it prints

          Parsing warnings in console log with parser Java Compiler (javac)
null

          Expected Java Warnings: 0 warnings.

          Kanstantsin Shautsou added a comment - And in the end it prints Parsing warnings in console log with parser Java Compiler (javac) null Expected Java Warnings: 0 warnings.

          Kanstantsin Shautsou added a comment -

          drulli is it possible to run this algorithm against log? I can download log to my laptop, but i don't know plugin internals atm

          Kanstantsin Shautsou added a comment - drulli is it possible to run this algorithm against log? I can download log to my laptop, but i don't know plugin internals atm

          Ulli Hafner added a comment -

          Yes, clone the plug-in and start JavacParserTest and replace the file issue14043.txt with your log file content.

          Ulli Hafner added a comment - Yes, clone the plug-in and start JavacParserTest and replace the file issue14043.txt with your log file content.

          Ulli Hafner added a comment -

          Or use the test ParserSpeed to see the speed of each parser. Here you need to fill all.txt with your log file content.

          Ulli Hafner added a comment - Or use the test ParserSpeed to see the speed of each parser. Here you need to fill all.txt with your log file content.

          Tobias Gruetzmacher added a comment -

          Here is a simple (20 lines!) sample file that throws the EclipseParser into a quite long loop (> 2 minutes on my system)... And the time grows very fast with the number of lines to parse...

          Tobias Gruetzmacher added a comment - Here is a simple (20 lines!) sample file that throws the EclipseParser into a quite long loop (> 2 minutes on my system)... And the time grows very fast with the number of lines to parse...

          Ulli Hafner added a comment -

          See https://issues.jenkins-ci.org/browse/JENKINS-27664?focusedCommentId=236828&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-236828.

          Ulli Hafner added a comment - See https://issues.jenkins-ci.org/browse/JENKINS-27664?focusedCommentId=236828&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-236828 .

          Tobias Gruetzmacher added a comment - - edited

          I think I found a problem in the EclipseParser RegEx, which is quite easy to fix:

          [^\\^]*

          matches newlines, creating a RegEx that has to scan the whole remaining file for each possible match up to this point. Changing this to:

          [^\\n\\^]*

          confines the match to the current line and fixes the quadratic runtime. Maybe add a test with timeout (

          @Test(timeout = 10000)

          ) which parses my attached testcase as a verification.

          Oh, and thanks for the awesome Jenkins plugin(s)

          (Shout out to https://regex101.com/ for helping me find this problem)

          PS: Quite "interesting" to see how the Eclipse Parser parses javac into totally wrong errors, but that is probably a new bug...
          PPS: Scratch that, I made an error. There are some other oddities in that regular expression (1-character character-groups, useless non-capture groups). It would also probably not hurt to restrict the search space for the "marker line" (line only with ^ chars) to maybe ~5 lines. I'll send a pull request this evening.

          Tobias Gruetzmacher added a comment - - edited I think I found a problem in the EclipseParser RegEx, which is quite easy to fix: [^\\^]* matches newlines, creating a RegEx that has to scan the whole remaining file for each possible match up to this point. Changing this to: [^\\n\\^]* confines the match to the current line and fixes the quadratic runtime. Maybe add a test with timeout ( @Test(timeout = 10000) ) which parses my attached testcase as a verification. Oh, and thanks for the awesome Jenkins plugin(s) (Shout out to https://regex101.com/ for helping me find this problem) PS: Quite "interesting" to see how the Eclipse Parser parses javac into totally wrong errors, but that is probably a new bug... PPS: Scratch that, I made an error. There are some other oddities in that regular expression (1-character character-groups, useless non-capture groups). It would also probably not hurt to restrict the search space for the "marker line" (line only with ^ chars) to maybe ~5 lines. I'll send a pull request this evening.

          Ulli Hafner added a comment -

          This fix looks promising! I'm looking forward for the PR

          Ulli Hafner added a comment - This fix looks promising! I'm looking forward for the PR

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Tobias Gruetzmacher
          Path:
          src/main/java/hudson/plugins/warnings/parser/EclipseParser.java
          src/test/java/hudson/plugins/warnings/parser/EclipseParserTest.java
          src/test/resources/hudson/plugins/warnings/parser/issue27664.txt
          http://jenkins-ci.org/commit/warnings-plugin/167f35f42c58b0a587a3def816ab9ea54792d84f
          Log:
          JENKINS-27664 Better Performance of EclipseParser.

          Tune the regular expression used in the EclipseParser to not backtrack
          through the whole log output.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Tobias Gruetzmacher Path: src/main/java/hudson/plugins/warnings/parser/EclipseParser.java src/test/java/hudson/plugins/warnings/parser/EclipseParserTest.java src/test/resources/hudson/plugins/warnings/parser/issue27664.txt http://jenkins-ci.org/commit/warnings-plugin/167f35f42c58b0a587a3def816ab9ea54792d84f Log: JENKINS-27664 Better Performance of EclipseParser. Tune the regular expression used in the EclipseParser to not backtrack through the whole log output.

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Ulli Hafner
          Path:
          src/main/java/hudson/plugins/warnings/parser/EclipseParser.java
          src/test/java/hudson/plugins/warnings/parser/EclipseParserTest.java
          src/test/resources/hudson/plugins/warnings/parser/issue27664.txt
          http://jenkins-ci.org/commit/warnings-plugin/c3f5088ba82fd59f7bf60fec15255b4ccf455832
          Log:
          Merge pull request #72 from TobiX/eclipse-parser-performance

          [FIXED JENKINS-27664] Improved performance of EclipseParser.

          Compare: https://github.com/jenkinsci/warnings-plugin/compare/6b50ca13c489...c3f5088ba82f

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ulli Hafner Path: src/main/java/hudson/plugins/warnings/parser/EclipseParser.java src/test/java/hudson/plugins/warnings/parser/EclipseParserTest.java src/test/resources/hudson/plugins/warnings/parser/issue27664.txt http://jenkins-ci.org/commit/warnings-plugin/c3f5088ba82fd59f7bf60fec15255b4ccf455832 Log: Merge pull request #72 from TobiX/eclipse-parser-performance [FIXED JENKINS-27664] Improved performance of EclipseParser. Compare: https://github.com/jenkinsci/warnings-plugin/compare/6b50ca13c489...c3f5088ba82f

            drulli Ulli Hafner
            ketevv Kevin Rancka
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: