Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7490

console-parser should get the parsing rules directly from a textbox in the project config

    • Icon: New Feature New Feature
    • Resolution: Won't Do
    • Icon: Major Major
    • log-parser-plugin
    • None

      Current configuration option of log-parser has lots of issues:

      • hard to maintain
      • doesn't work with slaves
      • doesn't work if you do not have shell access to the hudson server
      • doesn't work if you want to transfer(copy) jobs from one server to another

      This is a request to include the parsing configuration inside the job configuration in a text box.

          [JENKINS-7490] console-parser should get the parsing rules directly from a textbox in the project config

          pjdarton added a comment -

          I'd agree with the above.
          Whilst the ability to share configuration between projects is a nice-to-have, the ability to store an individual project's configuration as that project's configuration is essential.
          i.e. One needs to have the option of having these parsing rules stored within the project configuration.
          By all means have the optional facility to use an externally-controlled file, but the default option should be to have the parsing rules stored on a per-project basis, as part of the project configuration, much like the build rules themselves.

          pjdarton added a comment - I'd agree with the above. Whilst the ability to share configuration between projects is a nice-to-have, the ability to store an individual project's configuration as that project's configuration is essential. i.e. One needs to have the option of having these parsing rules stored within the project configuration. By all means have the optional facility to use an externally-controlled file, but the default option should be to have the parsing rules stored on a per-project basis, as part of the project configuration, much like the build rules themselves.

          wbernard added a comment -

          I've also been struggling with the configuration of this plugin. In order to get it easily configurable I've made some custom modifications for my personal usage, which are quite similar to this requirement.
          I've attached my diff and a screenshot of what config looks like.

          To be even better and simpler to configure I think rules should be fully configurable through the GUI using a combobox for the severity and a textfield for the regular expression (validation of the regex would be the cherry on the cake).

          Any thoughts?

          Br,

          • William

          wbernard added a comment - I've also been struggling with the configuration of this plugin. In order to get it easily configurable I've made some custom modifications for my personal usage, which are quite similar to this requirement. I've attached my diff and a screenshot of what config looks like. To be even better and simpler to configure I think rules should be fully configurable through the GUI using a combobox for the severity and a textfield for the regular expression (validation of the regex would be the cherry on the cake). Any thoughts? Br, William

          jborghi added a comment -

          Hi William,
          I'd be concerned with the scaling of individual text and combo selection boxes for each entry. If there are only a few entries, it would work well. Our use case has probably 100 lines, so it is really only manageable in the current form with a single file.

          jborghi added a comment - Hi William, I'd be concerned with the scaling of individual text and combo selection boxes for each entry. If there are only a few entries, it would work well. Our use case has probably 100 lines, so it is really only manageable in the current form with a single file.

          I think a combination of single file and local to each job is OK. Especially, if Developers have write access to the job, but not to the main configuration.

          I also agree with jborgi, that it must have the option to use the central files. How about allowing import statements in the drop down box? This way you can mix and match, which means you have a central common parse file plus some additional project specific parsing files. It would even be better, if you can import more than one file.

          Another comment on top was, that it doesn't work with clients. My use case is building on windows and deploying on AIX into a WAS server. I don't want to store the files separately on the AIX boxes and then synchronize them every time. Therefore, the centrally stored config files need to be transferred from the master to the nodes when needed (the current version of the filter file when the job runs, not the version that was current when the job config was saved).

          peter_schuetze added a comment - I think a combination of single file and local to each job is OK. Especially, if Developers have write access to the job, but not to the main configuration. I also agree with jborgi, that it must have the option to use the central files. How about allowing import statements in the drop down box? This way you can mix and match, which means you have a central common parse file plus some additional project specific parsing files. It would even be better, if you can import more than one file. Another comment on top was, that it doesn't work with clients. My use case is building on windows and deploying on AIX into a WAS server. I don't want to store the files separately on the AIX boxes and then synchronize them every time. Therefore, the centrally stored config files need to be transferred from the master to the nodes when needed (the current version of the filter file when the job runs, not the version that was current when the job config was saved).

          Hi guys,
          I'd much rather have the parser file taken from the job's workspace. This will make the file show up in slave instances/nodes. It will also make it easy to copy jobs to other Jenkins instances.

          Although I was first thinking I could use one parser file for all my jobs, it turns out that the reality is that each job has its own set of rules. This is because of text lines coming out of a JUnit test or something where someone accidentally let out some text containing "error" or "warn" or "fail".

          For me, having the rules file in the workspace also means that it can be kept in SVN and the right version of the parsing rules will be used with each version of the source code.

          Does the filename get expanded for environment variables? Would it be possible to set up the path to the file as %WORKSPACE%\jobrules.parse

          John Muczynski added a comment - Hi guys, I'd much rather have the parser file taken from the job's workspace. This will make the file show up in slave instances/nodes. It will also make it easy to copy jobs to other Jenkins instances. Although I was first thinking I could use one parser file for all my jobs, it turns out that the reality is that each job has its own set of rules. This is because of text lines coming out of a JUnit test or something where someone accidentally let out some text containing "error" or "warn" or "fail". For me, having the rules file in the workspace also means that it can be kept in SVN and the right version of the parsing rules will be used with each version of the source code. Does the filename get expanded for environment variables? Would it be possible to set up the path to the file as %WORKSPACE%\jobrules.parse

          jborghi added a comment -

          Variables aren't expanded, but more importantly, Hudson needs to know if it should read the file from the slave or master. Currently reads from the master only.

          I was thinking to have the file read from the slave, via workspace root for relative paths, if it is a project specific configuration. For global configs, they will be read from the master. Seem ok?

          jborghi added a comment - Variables aren't expanded, but more importantly, Hudson needs to know if it should read the file from the slave or master. Currently reads from the master only. I was thinking to have the file read from the slave, via workspace root for relative paths, if it is a project specific configuration. For global configs, they will be read from the master. Seem ok?

          Thanks for asking. Yes it seems ok. I never thought of doing it that way. It sounds like something comprehensive that is sure to cover the needs of all.

          (I got log parsing running on my Jenkins slaves this afternoon. Almost all the rules are set up. After some 6-hour jobs run, I'll see if any more messages need to be suppressed. The web page for viewing the console log is impressive. Easy to use and intuitive.)

          I'm using a lot of svn:include properties, so handling data shared by several jobs is usually accomplished with an svn:include. If I could specify a workspace file for parsing rules, then I would use Ant concat to put together the rule file contents.

          I like how some of the other configuration sections report with a red message when Jenkins doesn't see the file at the workspace path the user specified. Is this hard to implement?

          John Muczynski added a comment - Thanks for asking. Yes it seems ok. I never thought of doing it that way. It sounds like something comprehensive that is sure to cover the needs of all. (I got log parsing running on my Jenkins slaves this afternoon. Almost all the rules are set up. After some 6-hour jobs run, I'll see if any more messages need to be suppressed. The web page for viewing the console log is impressive. Easy to use and intuitive.) I'm using a lot of svn:include properties, so handling data shared by several jobs is usually accomplished with an svn:include. If I could specify a workspace file for parsing rules, then I would use Ant concat to put together the rule file contents. I like how some of the other configuration sections report with a red message when Jenkins doesn't see the file at the workspace path the user specified. Is this hard to implement?

          sudalai kumar added a comment -

          hi

          sudalai kumar added a comment - hi

          Hi Sudalai,

          Nice to see you're working on this. How do you plan to implement it?

          Kind Regards,
          -John

          John Muczynski added a comment - Hi Sudalai, Nice to see you're working on this. How do you plan to implement it? Kind Regards, -John

          Waldek M added a comment - - edited

          Seems like Sudulai gave up .
          Wish I knew how to work with the plugins enough to help out here...

          Waldek M added a comment - - edited Seems like Sudulai gave up . Wish I knew how to work with the plugins enough to help out here...

            attune sudalai kumar
            ssbarnea Sorin Sbarnea
            Votes:
            33 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: