Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35262

Warnings plugin - regex fails to detect pattern whereas it is correctly detected when declaring the regex in System Configuration page

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • warnings-plugin
    • warnings-plugin 4.52
      Jenkins 1.648

      Hello,

      I have an issue with a regex that does not trigger any warning when set as a console output parser in a job, but seems to be correctly formatted and working in the System Configuration page

      Here are all the details of my sample:
      I created the following sample regex in the System Configuration page:

      (start(?:(?!start)[\s\S])*?end   step: runtimedata.*)
      

      With the following Groovy script:

      import hudson.plugins.warnings.parser.Warning
      import hudson.plugins.analysis.util.model.Priority
      String type = "TEST"
      String category = "runtimedata"
      String errors = matcher.group(1)
      return new Warning("", 0, type, category, errors, Priority.HIGH);
      

      It is supposed to retrieve the lines in red in the following sample:
      start step: at 06/01/2016-12:05:42
      end step: jsruler at 06/01/2016-12:05:42
      start step: at 06/01/2016-12:05:42
      some details here
      on several lines
      end step: runtimedata at 06/01/2016-12:05:42

      start step: at 06/01/2016-12:05:42
      end step: dummynls at 06/01/2016-12:05:42

      This works correctly when I create the regex and put the exact same sample text in the "Example Log Message" field:

      One warning found
      file name: 
      line number: 0
      priority: High Priority
      category:  headerruler
      type: TEST
      message: start step: runtimed[...]data   at 06/01/2016-12:05:42
      

      FYI, I've also tested the regex in several online tools and it works.

      However, when I run a job with the exact same trace in the console output and set the corresponding regex to parse it, no warning is found.
      Please note that this behavior happens only with regex with the pattern (??!someString)

      I looked for answers in forums and such but I haven't been able find anyone else encountering this issue. Since I feel like it is specific to some regex patterns, it might be that nobody else tried this...

      I hope you can find the explanation and correct this, it is quite a pain for me to work around this! Feel free to request any additional information I might have forgotten.

      Best Regards,
      Tyoneb

          [JENKINS-35262] Warnings plugin - regex fails to detect pattern whereas it is correctly detected when declaring the regex in System Configuration page

          Code changed in jenkins
          User: Ulli Hafner
          Path:
          src/main/resources/hudson/plugins/warnings/GroovyParser/config.properties
          src/main/resources/hudson/plugins/warnings/GroovyParser/config_de.properties
          http://jenkins-ci.org/commit/warnings-plugin/a9c512cbc450a92f6bc0697d9e4cbfa238776057
          Log:
          [FIXED JENKINS-35262] Clarified help message.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ulli Hafner Path: src/main/resources/hudson/plugins/warnings/GroovyParser/config.properties src/main/resources/hudson/plugins/warnings/GroovyParser/config_de.properties http://jenkins-ci.org/commit/warnings-plugin/a9c512cbc450a92f6bc0697d9e4cbfa238776057 Log: [FIXED JENKINS-35262] Clarified help message.

          Hi Ulli,

          Thanks a lot for looking into this so quickly, and identifying the incorrect help message.

          However, I made some tests today and I have 2 feedbacks:

          1. 1. When I change my regex to add a \n character in it, it still does not find any warning. So my initial issue is not solved. Am I doing something wrong or should I reopen this ticket?
          (## start(?:(?!start)[\s\S])*?## end   step: headerruler.*\n)
          
          1. 2. There is still a discrepancy between the behavior of the regex parser when in the System Configuration page (which finds a Warning as is expected) and the regex parser in a job (which does not find any result). I agree that this issue is not as critical as the first one but it makes the analysis and convergence of a regex much harder... Plus it's very confusing for the end-user.

          Have you tried the sample I provided on your side?

          Best Regards,
          Tyoneb

          Benoît CHOMEL added a comment - Hi Ulli, Thanks a lot for looking into this so quickly, and identifying the incorrect help message. However, I made some tests today and I have 2 feedbacks: 1. When I change my regex to add a \n character in it, it still does not find any warning. So my initial issue is not solved. Am I doing something wrong or should I reopen this ticket? (## start(?:(?!start)[\s\S])*?## end step: headerruler.*\n) 2. There is still a discrepancy between the behavior of the regex parser when in the System Configuration page (which finds a Warning as is expected) and the regex parser in a job (which does not find any result). I agree that this issue is not as critical as the first one but it makes the analysis and convergence of a regex much harder... Plus it's very confusing for the end-user. Have you tried the sample I provided on your side? Best Regards, Tyoneb

          Unfortunately, my issue is not fixed by adding a \n in the regex. Please see my previous message for details.

          Benoît CHOMEL added a comment - Unfortunately, my issue is not fixed by adding a \n in the regex. Please see my previous message for details.

          Ulli Hafner added a comment -

          Can you please add a new comment with the three input elements (or attach them as files)? I tried to copy them using Copy&Paste into the three text boxes in the UI, however the example does not match in my case.

          Ulli Hafner added a comment - Can you please add a new comment with the three input elements (or attach them as files)? I tried to copy them using Copy&Paste into the three text boxes in the UI, however the example does not match in my case.

          Code changed in jenkins
          User: Ulli Hafner
          Path:
          src/test/java/hudson/plugins/warnings/GroovyParserTest.java
          http://jenkins-ci.org/commit/warnings-plugin/79fccda85c295fb2596baf31a105cca3727c1895
          Log:
          JENKINS-35262 Added testcase.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ulli Hafner Path: src/test/java/hudson/plugins/warnings/GroovyParserTest.java http://jenkins-ci.org/commit/warnings-plugin/79fccda85c295fb2596baf31a105cca3727c1895 Log: JENKINS-35262 Added testcase.

          Hum, this is shameful... I was trying to make a very easy sample for you to reproduce but I found that it worked correctly! So I went back to my initial sample and it works too...! I then edited my real regexes in my PPD env to add the \n and everything is okay. I'm baffled, I have no idea what I did wrong last time I tested it.
          I'm very sorry for having you spend time on a non-issue

          As non-blocking remaining issues I noticed:

          • In the System Configuration, if you enter your regex without the \n, it will still work as a multi-line parsing whereas it will not while using the parser in the logs.
          • In the System Configuration, when you enter a sample log message, it only finds 1 warning even if there are several. This might be a normal behavior as it is just a sample, but it behaves differently than when using the parser. My understanding is that the regex in the System Configuration does not run with the 'g' (global) modifier. If it is as designed, I think it should be written somewhere to avoid confusion.
            => In the end, if you can make the regex behave with the same modifiers in both context, I think that would really help the end-user
          • When displaying the details of a multi-line parsing (in a job), the result is displayed on a single line without interpreting the carriage return, probably because the HTML does not interpret it by default. FYI, I had the same problem when using Warnings regex results in the mail notifications and I used a <pre></pre> tag around the regex results.

          Anyway, sorry again, this ticket can be closed with your initial answer: \n is mandatory when working with multiple lines.

          Benoît CHOMEL added a comment - Hum, this is shameful... I was trying to make a very easy sample for you to reproduce but I found that it worked correctly! So I went back to my initial sample and it works too...! I then edited my real regexes in my PPD env to add the \n and everything is okay. I'm baffled, I have no idea what I did wrong last time I tested it. I'm very sorry for having you spend time on a non-issue As non-blocking remaining issues I noticed: In the System Configuration, if you enter your regex without the \n, it will still work as a multi-line parsing whereas it will not while using the parser in the logs. In the System Configuration, when you enter a sample log message, it only finds 1 warning even if there are several. This might be a normal behavior as it is just a sample, but it behaves differently than when using the parser. My understanding is that the regex in the System Configuration does not run with the 'g' (global) modifier. If it is as designed, I think it should be written somewhere to avoid confusion. => In the end, if you can make the regex behave with the same modifiers in both context, I think that would really help the end-user When displaying the details of a multi-line parsing (in a job), the result is displayed on a single line without interpreting the carriage return, probably because the HTML does not interpret it by default. FYI, I had the same problem when using Warnings regex results in the mail notifications and I used a <pre></pre> tag around the regex results. Anyway, sorry again, this ticket can be closed with your initial answer: \n is mandatory when working with multiple lines.

          Ulli Hafner added a comment -

          In the System Configuration, if you enter your regex without the \n, it will still work as a multi-line parsing whereas it will not while using the parser in the logs.

          Yes, this is not intuitive. I thought about adding a checkbox but finally used the current approach (since multiline scanning should be avoided due to performance reasons).

          In the System Configuration, when you enter a sample log message, it only finds 1 warning even if there are several. This might be a normal behavior as it is just a sample, but it behaves differently than when using the parser. My understanding is that the regex in the System Configuration does not run with the 'g' (global) modifier. If it is as designed, I think it should be written somewhere to avoid confusion.
          => In the end, if you can make the regex behave with the same modifiers in both context, I think that would really help the end-user

          I can change the message so that at least the number of matches is shown...

          When displaying the details of a multi-line parsing (in a job), the result is displayed on a single line without interpreting the carriage return, probably because the HTML does not interpret it by default. FYI, I had the same problem when using Warnings regex results in the mail notifications and I used a <pre></pre> tag around the regex results.

          I see.

          Ulli Hafner added a comment - In the System Configuration, if you enter your regex without the \n, it will still work as a multi-line parsing whereas it will not while using the parser in the logs. Yes, this is not intuitive. I thought about adding a checkbox but finally used the current approach (since multiline scanning should be avoided due to performance reasons). In the System Configuration, when you enter a sample log message, it only finds 1 warning even if there are several. This might be a normal behavior as it is just a sample, but it behaves differently than when using the parser. My understanding is that the regex in the System Configuration does not run with the 'g' (global) modifier. If it is as designed, I think it should be written somewhere to avoid confusion. => In the end, if you can make the regex behave with the same modifiers in both context, I think that would really help the end-user I can change the message so that at least the number of matches is shown... When displaying the details of a multi-line parsing (in a job), the result is displayed on a single line without interpreting the carriage return, probably because the HTML does not interpret it by default. FYI, I had the same problem when using Warnings regex results in the mail notifications and I used a <pre></pre> tag around the regex results. I see.

          Ulli Hafner added a comment - - edited

          But can you at least please paste the value of your three input fields now, then I can add these values into the test case. (Then the test is useful at least Or you can attach them as plain text files.

          Ulli Hafner added a comment - - edited But can you at least please paste the value of your three input fields now, then I can add these values into the test case. (Then the test is useful at least Or you can attach them as plain text files.

          Benoît CHOMEL added a comment - - edited

          Sure, please find my basic sample below. And thanks for the feedback regarding the other items.

          Sample regex:

          (make(?:(?!make)[\s\S])*?make-error:.*\n)
          

          Sample script:

          import hudson.plugins.warnings.parser.Warning
          import hudson.plugins.analysis.util.model.Priority
          String fileName = ""
          String type = "TEST"
          String category = "make-error"
          String errors = matcher.group(1)
          return new Warning(fileName, 0, type, category, errors, Priority.HIGH);
          

          Sample log messages:

          start build
          make: thisFile.js
          everything okay
          make: thisOtherFile.js
          error detail1: wrong character
          error detail2: encoding issue
          make-error: thisOtherFile.js: wrong encoding detected
          make: anotherFile.js
          make: yetAnotherFile.js
          end build
          

          This should only match the following section:

          make: thisOtherFile.js
          error detail1: wrong character
          error detail2: encoding issue
          make-error: thisOtherFile.js: wrong encoding detected
          

          Benoît CHOMEL added a comment - - edited Sure, please find my basic sample below. And thanks for the feedback regarding the other items. Sample regex: (make(?:(?!make)[\s\S])*?make-error:.*\n) Sample script: import hudson.plugins.warnings.parser.Warning import hudson.plugins.analysis.util.model.Priority String fileName = "" String type = "TEST" String category = "make-error" String errors = matcher.group(1) return new Warning(fileName, 0, type, category, errors, Priority.HIGH); Sample log messages: start build make: thisFile.js everything okay make: thisOtherFile.js error detail1: wrong character error detail2: encoding issue make-error: thisOtherFile.js: wrong encoding detected make: anotherFile.js make: yetAnotherFile.js end build This should only match the following section: make: thisOtherFile.js error detail1: wrong character error detail2: encoding issue make-error: thisOtherFile.js: wrong encoding detected

          Code changed in jenkins
          User: Ulli Hafner
          Path:
          src/test/java/hudson/plugins/warnings/GroovyParserTest.java
          http://jenkins-ci.org/commit/warnings-plugin/145b978989fd766f582fc3adc909bbb97a69a7a8
          Log:
          [FIXED JENKINS-35262] Update of testcase setup.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Ulli Hafner Path: src/test/java/hudson/plugins/warnings/GroovyParserTest.java http://jenkins-ci.org/commit/warnings-plugin/145b978989fd766f582fc3adc909bbb97a69a7a8 Log: [FIXED JENKINS-35262] Update of testcase setup.

            drulli Ulli Hafner
            tyoneb Benoît CHOMEL
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: