Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32866

Log Parser Plugin does not parse Pipeline console outputs

      Versions:

      • Jenkins: 1.644
      • Workflow plugins: 1.12
      • Log Parser Plugin: 2.0

      Issue:
      I have the following workflow job:

      node {
         step([$class: 'LogParserPublisher', parsingRulesPath: '/var/lib/jenkins/jenkins-rule-logparser', useProjectRule: false])
         echo 'Error: oh no'
      }
      

      and it produces the following console output:

      Started by user anonymous
      [Workflow] Allocate node : Start
      Running on master in /var/lib/jenkins/jobs/ntsvc-monitor-dev/workspace
      [Workflow] node {
      [Workflow] step
      [Workflow] echo
      Error: oh noes
      [Workflow] } //node
      [Workflow] Allocate node : End
      [Workflow] End of Workflow
      Finished: SUCCESS
      

      The log parser config (/var/lib/jenkins/jenkins-rule-logparser) is:

      error /Error/
      

      The parsed log view shows only:

      Started by user anonymous
      [Workflow] Allocate node : Start
      Running on master in /var/lib/jenkins/jobs/ntsvc-monitor-dev/workspace
      [Workflow] node {
      

      For a FreeStyle job, the same echo message and log parser rules, results in fully parsed log and the error message is detected/colored red. See attachments for views of parsed log screens.

      The Jenkins system log does not contain errors for the jobs or the Log Parse plugin... indeed, it contains success messages from the Log Parse plugin.

          [JENKINS-32866] Log Parser Plugin does not parse Pipeline console outputs

          Jeff Burke added a comment -

          Screen shots of parse working and not working.

          Jeff Burke added a comment - Screen shots of parse working and not working.

          Sorin Sbarnea added a comment -

          Is this working? It would be quite useful to enable it for pipelines, we are all moving to them now.

          Sorin Sbarnea added a comment - Is this working? It would be quite useful to enable it for pipelines, we are all moving to them now.

          IMHO this is the expected behavior. The log parser step only has access to the log to the point where it is called. If called in "finally" (as some sort of post build actions in the pipeline build) this works well:

          try {
            node {
                 echo 'Error: oh no'
            }
          } finally {
            node {
             step([$class: 'LogParserPublisher', parsingRulesPath: '/var/lib/jenkins/jenkins-rule-logparser', useProjectRule: false])
            }
          }
          

          In order to fail the build add "failBuildOnError: true" (and maybe you need another try/catch block around the log parser step to catch the error and set the build result to FAILURE explicitly (not tested)).

          Christoph Vogtländer added a comment - IMHO this is the expected behavior. The log parser step only has access to the log to the point where it is called. If called in "finally" (as some sort of post build actions in the pipeline build) this works well: try { node { echo 'Error: oh no' } } finally { node { step([$class: 'LogParserPublisher' , parsingRulesPath: '/ var /lib/jenkins/jenkins-rule-logparser' , useProjectRule: false ]) } } In order to fail the build add "failBuildOnError: true" (and maybe you need another try/catch block around the log parser step to catch the error and set the build result to FAILURE explicitly (not tested)).

          danniel hughes added a comment - - edited

          so, assuming the file of rules is {{/var/lib/jenkins/ParsingRuleFiles/Generic_Verify }}than to fail a build, it should be something like : 

           finally {
                  node('<same agent you run on >') {
                      try {
                               step([$class: 'LogParserPublisher', parsingRulesPath: '/var/lib/jenkins/ParsingRuleFiles/Generic_Verify', useProjectRule: false, *failBuildOnError: true, unstableOnWarning: true*])
                   } catch(all) {
                                print "ERROR: LogParserPublisher failed: \n" +all
                   }
                 }//node
          }//finally

          tested - and working , me very happy

          danniel hughes added a comment - - edited so, assuming the file of rules is {{ /var/lib/jenkins/ParsingRuleFiles/Generic_Verify }}than to fail a build, it should be something like :   finally {         node(' <same agent you run on > ') {             try {                      step( [$class: ' LogParserPublisher ', parsingRulesPath: ' /var/lib/jenkins/ParsingRuleFiles/Generic_Verify ', useProjectRule: false, * failBuildOnError : true , unstableOnWarning : true *] )          } catch(all) {                       print "ERROR: LogParserPublisher failed: \n" +all           }        }//node }//finally tested - and working , me very happy

          Lawrence Dubé added a comment - - edited

          How could I get to work if I have multiple stages, like the following and I want it to parse everything

          pipeline {
              agent { label 'testmachine'}
              stages {
                  stage('Level 1'){
                      steps {
                                bat '''
                                    call "%VS120COMNTOOLS%\\vsvars32.bat"
                                    devenv testlevel1.sln /build "Profile|x64"'''
                      }
                  }
                  stage('Level 2'){
                      steps {
                               bat '''
                                   call "%VS120COMNTOOLS%\\vsvars32.bat" 
                                   devenv testlevel2.sln /build "Profile|x64"'''
                      }
                 }
             }
          }

           

           

           

          Lawrence Dubé added a comment - - edited How could I get to work if I have multiple stages, like the following and I want it to parse everything pipeline { agent { label 'testmachine' } stages { stage( 'Level 1' ){ steps { bat ''' call "%VS120COMNTOOLS%\\vsvars32.bat" devenv testlevel1.sln /build "Profile|x64" ''' } } stage( 'Level 2' ){ steps { bat ''' call "%VS120COMNTOOLS%\\vsvars32.bat" devenv testlevel2.sln /build "Profile|x64" ''' } } } }      

          Pierre Marton added a comment -

          The given solution works for all stages, you just need to set the parser as the last stage.

          Pierre Marton added a comment - The given solution works for all stages, you just need to set the parser as the last stage.

          Jake D added a comment -

          Can I ask if this is tailored for Declarative Pipelines?

          Jake D added a comment - Can I ask if this is tailored for Declarative Pipelines?

          Pierre Marton added a comment -

          You can of course

          I don't see why it wouldn't work, just try.

          Pierre Marton added a comment - You can of course I don't see why it wouldn't work, just try.

          Martin Dan added a comment -

          Hi all!
          In my pipeline i have problem. Job failed in this stage(log-parser)

          pipeline {
          agent

          { label 'master' }

          triggers

          { pollSCM('* * * * *') }

          environment

          { ANSIBLE_CONFIG = "Jenkins-LTS/ansible.cfg" }

          ..........

          stage('Log-parser') {
          steps {
          logparser([$class: 'LogParserPublisher',
          parsingRulesPath: '/var/lib/jenkins/minimal-rules',
          useProjectRule: false,
          failBuildOnError: true,
          unstableOnWarning: false,
          Logparsergraphs: true ])

          error:
          java.lang.NoSuchMethodError: No such DSL method 'logparser' found among steps ....... and more string

          Martin Dan added a comment - Hi all! In my pipeline i have problem. Job failed in this stage(log-parser) pipeline { agent { label 'master' } triggers { pollSCM('* * * * *') } environment { ANSIBLE_CONFIG = "Jenkins-LTS/ansible.cfg" } .......... stage('Log-parser') { steps { logparser([$class: 'LogParserPublisher', parsingRulesPath: '/var/lib/jenkins/minimal-rules', useProjectRule: false, failBuildOnError: true, unstableOnWarning: false, Logparsergraphs: true ]) error: java.lang.NoSuchMethodError: No such DSL method 'logparser' found among steps ....... and more string

          Pierre Marton added a comment -

          Well what about the end of the stage?

          } catch(all)

          { print "ERROR: LogParserPublisher failed: \n" +all }

          }
          }

          Would be easier to help you if you put the all code in (at least the complete stage).

          Pierre Marton added a comment - Well what about the end of the stage? } catch(all) { print "ERROR: LogParserPublisher failed: \n" +all } } } Would be easier to help you if you put the all code in (at least the complete stage).

          Martin Dan added a comment -

          Hi! Thx for answer. her is cod

          pipeline

          { agent \{ label 'master'}

          triggers

          { pollSCM('* * * * *') }

          environment

          { ANSIBLE_CONFIG = "Jenkins-LTS/ansible.cfg" }

          stages{
          stage('Checkout') {
          steps

          { checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'Jenkins-LTS'], [$class: 'WipeWorkspace']], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'fd864bf0-5e33-4cf1-a130-67ac2c22bf17', url: 'https://github.com/MRDO5/jenkins-lts.git']]]) }

          }
          stage('Deploy') {
          parallel {
          stage('Syntax check') {
          steps {
          ansiColor('gnome-terminal')

          { ansiblePlaybook become: true, colorized: true, credentialsId: '7980492c-7fa3-41b6-9c8e-b44d3f7ce236', extras: '--syntax-check' , inventory: 'Jenkins-LTS/inventory', playbook: 'Jenkins-LTS/main.yml' }

          }
          }
          stage('Check provision for virtual machines') {
          steps {
          ansiColor('gnome-terminal')

          { ansiblePlaybook become: true, colorized: true, credentialsId: '7980492c-7fa3-41b6-9c8e-b44d3f7ce236', installation: 'ansible', inventory: 'Jenkins-LTS/inventory', playbook: 'Jenkins-LTS/main.yml' }

          }
          }
          }
          }
          stage('Log-parser') {
          steps

          { logparser([$class: 'LogParserPublisher', parsingRulesPath: '/var/lib/jenkins/minimal-rules', useProjectRule: false, failBuildOnError: true, unstableOnWarning: false, Logparsergraphs: true ]) }

          }
          }

           

          Martin Dan added a comment - Hi! Thx for answer. her is cod pipeline { agent \{ label 'master'} triggers { pollSCM('* * * * *') } environment { ANSIBLE_CONFIG = "Jenkins-LTS/ansible.cfg" } stages{ stage('Checkout') { steps { checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'Jenkins-LTS'], [$class: 'WipeWorkspace']], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'fd864bf0-5e33-4cf1-a130-67ac2c22bf17', url: 'https://github.com/MRDO5/jenkins-lts.git']]]) } } stage('Deploy') { parallel { stage('Syntax check') { steps { ansiColor('gnome-terminal') { ansiblePlaybook become: true, colorized: true, credentialsId: '7980492c-7fa3-41b6-9c8e-b44d3f7ce236', extras: '--syntax-check' , inventory: 'Jenkins-LTS/inventory', playbook: 'Jenkins-LTS/main.yml' } } } stage('Check provision for virtual machines') { steps { ansiColor('gnome-terminal') { ansiblePlaybook become: true, colorized: true, credentialsId: '7980492c-7fa3-41b6-9c8e-b44d3f7ce236', installation: 'ansible', inventory: 'Jenkins-LTS/inventory', playbook: 'Jenkins-LTS/main.yml' } } } } } stage('Log-parser') { steps { logparser([$class: 'LogParserPublisher', parsingRulesPath: '/var/lib/jenkins/minimal-rules', useProjectRule: false, failBuildOnError: true, unstableOnWarning: false, Logparsergraphs: true ]) } } }  

          Pierre Marton added a comment - - edited

          Well I'm using 2 different ways for pipeline code:

          in Jenkins job as last stage:

          stage("Parser") {
          node('master') {
          try 

          { step([$class: 'LogParserPublisher', parsingRulesPath: 'YOUR_PARSING_FILE', useProjectRule: false, failBuildOnError: true, unstableOnWarning: true]) }

          catch(all)

          { print "ERROR: LogParserPublisher failed: \n" +all }

          }
          }

          in Jenkinsfile:
          @Library('pipelib@master')
          import […]

          timestamps {
          try

          { // For finally LogParserPublisher […] }

          finally {
          node

          { step([$class: 'LogParserPublisher', parsingRulesPath: 'YOUR_PARSING_FILE', useProjectRule: false]) }

          }
          }

          The only difference with your code is that your parsing step is missing
          try {} / catch(all) {}
          or try {} / finally {}

          Also not sure how Ansible affects it as I'm not using it.

          Pierre Marton added a comment - - edited Well I'm using 2 different ways for pipeline code: in Jenkins job as last stage: stage("Parser") { node('master') { try  { step([$class: 'LogParserPublisher', parsingRulesPath: 'YOUR_PARSING_FILE', useProjectRule: false, failBuildOnError: true, unstableOnWarning: true]) } catch(all) { print "ERROR: LogParserPublisher failed: \n" +all } } } in Jenkinsfile: @Library('pipelib@master') import […] timestamps { try { // For finally LogParserPublisher […] } finally { node { step([$class: 'LogParserPublisher', parsingRulesPath: 'YOUR_PARSING_FILE', useProjectRule: false]) } } } The only difference with your code is that your parsing step is missing try {} / catch(all) {} or try {} / finally {} Also not sure how Ansible affects it as I'm not using it.

          Marc added a comment - - edited

          This issue and# JENKINS-35265 got in my way as well so I dived into it.

          Inspired by this blog I gave it a try.
          https://www.jenkins.io/blog/2016/05/25/update-plugin-for-pipeline/#build-actions-vs-project-actions

          The trend graph appeared once I replaced Publisher's getProjectAction with the implementation
          of the SimpleBuildStep.LastBuildAction interface to the LogParserAction.

          It works, but  as I have no experience with both Jenkins code base as java.

          mreinhardt  Please find the fix-showGraphs-with-pipeline.patch attached.

          Marc added a comment - - edited This issue and # JENKINS-35265 got in my way as well so I dived into it. Inspired by this blog I gave it a try. https://www.jenkins.io/blog/2016/05/25/update-plugin-for-pipeline/#build-actions-vs-project-actions The trend graph appeared once I replaced Publisher's getProjectAction with the implementation of the SimpleBuildStep.LastBuildAction interface to the LogParserAction. It works, but  as I have no experience with both Jenkins code base as java. mreinhardt   Please find the fix-showGraphs-with-pipeline.patch attached.

          thx. I published a new release with your fixes: https://github.com/jenkinsci/log-parser-plugin/releases/tag/v2.3.4

          Martin Reinhardt added a comment - thx. I published a new release with your fixes: https://github.com/jenkinsci/log-parser-plugin/releases/tag/v2.3.4

          Marc added a comment -

          mreinhardt  Thank you very much for this fast response.

          BTW: As far as I can see this also solved JENKINS-35265

          Marc added a comment - mreinhardt   Thank you very much for this fast response. BTW: As far as I can see this also solved JENKINS-35265

            mreinhardt Martin Reinhardt
            potatopankakes Jeff Burke
            Votes:
            13 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: