• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • analysis-model
    • None
    • beta8 (warnings-ng), beta17 (analysis-model-api)

      There are a number of things you can filter from reporting warnings on:

      • excludeCategory
      • pattern
        • Type: String
      • excludeFile
      • pattern
        • Type: String
      • excludeModule
      • pattern
        • Type: String
      • excludePackage
      • pattern
        • Type: String
      • excludeType
      • pattern
        • Type: String

      None of those appears to allow you to just specify a path/directory under which everything should be filtered, for the gcc4 format for example.

      I tried to use an Ant-style directory/*** but that was rejected as having one too many {}'s. I'm not sure that Module or Package will match an arbitrary path either though.

      It seems that this plugin is missing that ability.

          [JENKINS-54035] Gcc parser does not resolve absolute paths

          Brian J Murrell created issue -

          Ulli Hafner added a comment - - edited

          The excludeFile filter should work on the absolute path name. It expects a regular expression, e.g. .*/path/to/exclude/.*.

          Do all your warnings have an absolute path resolved? I think someone did report an issue with the gcc parser in the wiki: seems that only relative files are reported.

          Does the plugin report errors in the console log (or in the UI, you need to click on the exclamation mark icon).

          Ulli Hafner added a comment - - edited The excludeFile filter should work on the absolute path name. It expects a regular expression , e.g. .*/path/to/exclude/.* . Do all your warnings have an absolute path resolved? I think someone did report an issue with the gcc parser in the wiki: seems that only relative files are reported. Does the plugin report errors in the console log (or in the UI , you need to click on the exclamation mark icon).

          Ulli Hafner added a comment -

          If you have errors about not resolvable files it might help to use the parser GnuMakeGcc.

          Ulli Hafner added a comment - If you have errors about not resolvable files it might help to use the parser GnuMakeGcc .

          Ahhh. I didn't realise that it was REs since the docs said pattern which usually means _glob_s. I'll try using REs.

          If you have errors about not resolvable files it might help to use the parser GnuMakeGcc.

          Does this actually require that make be used to drive the gcc commands though?  We're not using make here.

          Brian J Murrell added a comment - Ahhh. I didn't realise that it was REs since the docs said pattern which usually means _glob_s. I'll try using REs. If you have errors about not resolvable files it might help to use the parser  GnuMakeGcc . Does this actually require that make be used to drive the gcc commands though?  We're not using make here.

          Brian J Murrell added a comment - - edited

          Surely the RE for your example should be:  .*\/path\/to\/exclude\/.*, yes? Or is / not considered special in whatever RE module you are using?

          Brian J Murrell added a comment - - edited Surely the RE for your example should be:  .*\/path\/to\/exclude\/.*, yes? Or is / not considered special in whatever RE module you are using?

          But that doesn't seem to be working either.  I have:

          filters: [excludeFile('.*\\/_build.external\\/.*')]
          

          But still seeing this when I click the exclamation mark:

          Can't resolve absolute paths for some files:- topology-linux.c- topology-linux.c- topology-linux.c- topology-linux.c- topology-linux.c  ... skipped logging of 761 additional errors ...Can't create fingerprints for some files:- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c  ... skipped logging of 767 additional errors ...Can't create blame requests for some affected files:- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c  ... skipped logging of 761 additional errors ...Git blame reported several errors:No blame results for request <@2/_build.external/mercury/Testing/util/test_atomic_queue.c - [32, 31]>.
          Git blame reported several errors:No blame results for request <_build.external/mercury/Testing/na/na_test.c - [517]>.
          Git blame reported several errors:No blame results for request <_build.external/pmix/src/mca/gds/gds.h - [320]>.
          Git blame reported several errors:No blame results for request <_build.external/mercury/src/util/mercury_atomic_queue.h - [158]>.
          Git blame reported several errors:No blame results for request <@2/_build.external/pmix/src/mca/gds/gds.h - [320]>.
          Git blame reported several errors:No blame results for request <@2/_build.external/mercury/Testing/na/na_test.c - [517]>.
          Git blame reported several errors:No blame results for request <@2/_build.external/mercury/src/util/mercury_atomic_queue.h - [158]>.
          Git blame reported several errors:No blame results for request <_build.external/mercury/Testing/driver/kwsys/ProcessUNIX.c - [1477]>.
          Git blame reported several errors:No blame results for request <@2/_build.external/mercury/Testing/driver/kwsys/ProcessUNIX.c - [1477]>.
          

          I would think I shouldn't be seeing those <@2/build.external/mercury _git blame issues if the filter was working correctly should i?

          On the other hand I also see:

          Information Messages
          Parsing console log (workspace: '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA')
          Post processing issues on 'wolf-53vm10' with encoding 'UTF-8'
          Resolving absolute file names for all issues
          -> 0 resolved, 766 unresolved, 14 already absolute
          Copying affected files to Jenkins' build folder /var/lib/jenkins/jobs/daos-stack-org/jobs/daos/branches/PR-13/builds/394
          -> 0 copied, 5 not in workspace, 176 not-found, 0 with I/O error
          Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
          -> resolved module names for 780 issues
          Resolving package names (or namespaces) by parsing the affected files
          -> resolved package names of 181 affected files
          Creating fingerprints for all affected code blocks to track issues over different builds
          -> created fingerprints for 8 issues
          Invoking Git blamer to create author and commit information for all affected files
          GIT_COMMIT env = 'HEAD'
          Git working tree = '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA'
          Git commit ID = 'a290947bd9eccd688aa6dbdd0654bd8e755a4aad'
          Job workspace = '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA'
          Created blame requests for 9 files - invoking Git blame on agent for each of the requests
          -> blamed authors of issues in 9 files
          Parsing console log (workspace: '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA')
          Applying 1 filters on the set of 780 issues (14 issues have been removed, 766 issues will be published)
          Using reference build 'daos-stack-org/daos/PR-13 #391' to compute new, fixed, and outstanding issues
          Issues delta (vs. reference build): outstanding: 766, new: 0, fixed: 14
          No quality gates have been set - skipping
          Health report is disabled - skipping

          and it does say 14 issues have been removed, but there should be many many more than 14.

          As an aside, is there any easier/quicker way to test different include/exclude filters against existing output rather than have to run the whole job again, complete with an actual compile, etc.?

          Brian J Murrell added a comment - But that doesn't seem to be working either.  I have: filters: [excludeFile('.*\\/_build.external\\/.*')] But still seeing this when I click the exclamation mark: Can't resolve absolute paths for some files:- topology-linux.c- topology-linux.c- topology-linux.c- topology-linux.c- topology-linux.c ... skipped logging of 761 additional errors ...Can't create fingerprints for some files:- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c- 'topology-linux.c', IO exception has been thrown: java.nio.file.NoSuchFileException: topology-linux.c ... skipped logging of 767 additional errors ...Can't create blame requests for some affected files:- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c- Skipping non-workspace file topology-linux.c ... skipped logging of 761 additional errors ...Git blame reported several errors:No blame results for request <@2/_build.external/mercury/Testing/util/test_atomic_queue.c - [32, 31]>. Git blame reported several errors:No blame results for request <_build.external/mercury/Testing/na/na_test.c - [517]>. Git blame reported several errors:No blame results for request <_build.external/pmix/src/mca/gds/gds.h - [320]>. Git blame reported several errors:No blame results for request <_build.external/mercury/src/util/mercury_atomic_queue.h - [158]>. Git blame reported several errors:No blame results for request <@2/_build.external/pmix/src/mca/gds/gds.h - [320]>. Git blame reported several errors:No blame results for request <@2/_build.external/mercury/Testing/na/na_test.c - [517]>. Git blame reported several errors:No blame results for request <@2/_build.external/mercury/src/util/mercury_atomic_queue.h - [158]>. Git blame reported several errors:No blame results for request <_build.external/mercury/Testing/driver/kwsys/ProcessUNIX.c - [1477]>. Git blame reported several errors:No blame results for request <@2/_build.external/mercury/Testing/driver/kwsys/ProcessUNIX.c - [1477]>. I would think I shouldn't be seeing those <@2/ build.external/mercury _ git blame issues if the filter was working correctly should i? On the other hand I also see: Information Messages Parsing console log (workspace: '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA') Post processing issues on 'wolf-53vm10' with encoding 'UTF-8' Resolving absolute file names for all issues -> 0 resolved, 766 unresolved, 14 already absolute Copying affected files to Jenkins' build folder /var/lib/jenkins/jobs/daos-stack-org/jobs/daos/branches/PR-13/builds/394 -> 0 copied, 5 not in workspace, 176 not-found, 0 with I/O error Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files) -> resolved module names for 780 issues Resolving package names (or namespaces) by parsing the affected files -> resolved package names of 181 affected files Creating fingerprints for all affected code blocks to track issues over different builds -> created fingerprints for 8 issues Invoking Git blamer to create author and commit information for all affected files GIT_COMMIT env = 'HEAD' Git working tree = '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA' Git commit ID = 'a290947bd9eccd688aa6dbdd0654bd8e755a4aad' Job workspace = '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA' Created blame requests for 9 files - invoking Git blame on agent for each of the requests -> blamed authors of issues in 9 files Parsing console log (workspace: '/var/lib/jenkins/workspace/daos-stack-org_daos_PR-13-XSYVVAKVRRCIRZBUBQLEPYMVXQHYAQV52HHPQR55EURLM5NXUHMA') Applying 1 filters on the set of 780 issues (14 issues have been removed, 766 issues will be published) Using reference build 'daos-stack-org/daos/PR-13 #391' to compute new, fixed, and outstanding issues Issues delta (vs. reference build): outstanding: 766, new: 0, fixed: 14 No quality gates have been set - skipping Health report is disabled - skipping and it does say 14 issues have been removed, but there should be many many more than 14. As an aside, is there any easier/quicker way to test different include/exclude filters against existing output rather than have to run the whole job again, complete with an actual compile, etc.?

          I've tried GnuMakeGcc since in fact the files I want to exclude in in projects under the excluded subdirectory built with make.

          I have also added a second filter to match non-fully-qualified path names in addition to the fully-qualified-path filter I already had:

          filters: [excludeFile('.*\\/_build\\.external\\/.*'),
                    excludeFile('_build\\.external\\/.*')] 

          Neither of those has eliminated the bulk of those 700+ compiler warnings that I want to exclude.

          Brian J Murrell added a comment - I've tried  GnuMakeGcc since in fact the files I want to exclude in in projects under the excluded subdirectory built with make . I have also added a second filter to match non-fully-qualified path names in addition to the fully-qualified-path filter I already had: filters: [excludeFile('.*\\/_build\\.external\\/.*'), excludeFile('_build\\.external\\/.*')] Neither of those has eliminated the bulk of those 700+ compiler warnings that I want to exclude.

          Ulli Hafner added a comment - - edited

          Currently the filter is invoked after all preprocessing steps. I.e. after absolute paths have been detected, etc. Maybe it makes sense to move this step to the beginning (or after the absolute path resolving).

          From the log I see that almost every warning has no absolute path (0 resolved, 766 unresolved, 14 already absolute). I.e. the parser seems not to work as expected.

          What makes me wonder: the log says: applying 1 filter, but in your example you have 2.
          But anyway, I think you need to change the regular expression so that it matches the entire file name, not only a portion. I.e. in your example the .* is missing in the front. See the new test that shows that the filter works.

          Maybe it makes sense to add an example text box in the UI of the snippet generator so you can experiment with the filter regexp.

          Should I change the name of the 'pattern' box to 'regular expression'? I can clarify as well that the whole property text must match. Java regular expressions are a little bit different since they match the entire string. I'll check if there is something available that matches only a part of the string.

          Ulli Hafner added a comment - - edited Currently the filter is invoked after all preprocessing steps. I.e. after absolute paths have been detected, etc. Maybe it makes sense to move this step to the beginning (or after the absolute path resolving). From the log I see that almost every warning has no absolute path (0 resolved, 766 unresolved, 14 already absolute). I.e. the parser seems not to work as expected. What makes me wonder: the log says: applying 1 filter, but in your example you have 2. But anyway, I think you need to change the regular expression so that it matches the entire file name, not only a portion. I.e. in your example the .* is missing in the front. See the new test that shows that the filter works. Maybe it makes sense to add an example text box in the UI of the snippet generator so you can experiment with the filter regexp. Should I change the name of the 'pattern' box to 'regular expression'? I can clarify as well that the whole property text must match. Java regular expressions are a little bit different since they match the entire string. I'll check if there is something available that matches only a part of the string.

          From the log I see that almost every warning has no absolute path (0 resolved, 766 unresolved, 14 already absolute). I.e. the parser seems not to work as expected.

          If filenames are always supposed to be absolute, then, yes, I would agree that the parser is not quite working correctly. I'm not sure how it's supposed to calculate absolute pathnames when gcc output looks like:

          [Build on CentOS 7] keyval_lex.c: In function ‘pmix_util_keyval_yylex’:
          [Build on CentOS 7] keyval_lex.c:947:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
          [Build on CentOS 7]     for ( yyl = 0; yyl < pmix_util_keyval_yyleng; ++yyl )
          

          I think you need to change the regular expression so that it matches the entire file name, not only a portion. I.e. in your example the .* is missing in the front

          I believe had subsequently added that. as noted in this comment. If those expressions are not covering both relative and absolute, let me know what you think needs to be different about them.

          Should I change the name of the 'pattern' box to 'regular expression'?

          IMHO, yes. Maybe qualify it as a Java Regular Expression if they are different and for bonus points, point the Java Regular Expression text at a reference for what they are if you can.

          Brian J Murrell added a comment - From the log I see that almost every warning has no absolute path (0 resolved, 766 unresolved, 14 already absolute). I.e. the parser seems not to work as expected. If filenames are always supposed to be absolute, then, yes, I would agree that the parser is not quite working correctly. I'm not sure how it's supposed to calculate absolute pathnames when gcc output looks like: [Build on CentOS 7] keyval_lex.c: In function ‘pmix_util_keyval_yylex’: [Build on CentOS 7] keyval_lex.c:947:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] [Build on CentOS 7] for ( yyl = 0; yyl < pmix_util_keyval_yyleng; ++yyl ) I think you need to change the regular expression so that it matches the entire file name, not only a portion. I.e. in your example the .* is missing in the front I believe had subsequently added that. as noted in this comment. If those expressions are not covering both relative and absolute, let me know what you think needs to be different about them. Should I change the name of the 'pattern' box to 'regular expression'? IMHO, yes. Maybe qualify it as a Java Regular Expression if they are different and for bonus points, point the Java Regular Expression text at a reference for what they are if you can.

          Can we make any further progress here at getting the exclude functionality to work?  Is there any additional information I can provide?

          Brian J Murrell added a comment - Can we make any further progress here at getting the exclude functionality to work?  Is there any additional information I can provide?

            drulli Ulli Hafner
            brianjmurrell Brian J Murrell
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: