[JENKINS-56612] Message filter does not work for multiple-line messages

Type: Bug
Resolution: Fixed
Priority: Major
Component/s: analysis-model
Labels:
None
Environment:
Jenkins: 2.168
warnings-ng-plugin: 4.0.0

Similar Issues:
Powered by SuggestiMate

Show
Released As:
5.0.0 (analysis-model and warnings-ng)

Pipeline:

recordIssues enabledForFailure: true, filters: [
  excludeFile('.*qrc_icons\\.cpp.*'),
  excludeMessage('.*QVTKWidget.*'),
  excludeMessage('.*tmpnam.*')],
  tools: [gcc4(name: 'GCC-GUI', id: 'gcc4-gui', pattern: 'build/build*.log')],
  unstableTotalAll: 1

With warnings-ng-plugin 3.0.3:

Error Messages

Can't resolve absolute paths for some files:
- createSurface.cpp
- Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h

Information Messages

Searching for all files in '/var/jenkins/workspace/ufz_ogs_PR-2429' that match the pattern 'build/build*.log'
-> found 2 files
Successfully parsed file /var/jenkins/workspace/ufz_ogs_PR-2429/build/build1.log
-> found 2 issues (skipped 5 duplicates)
Successfully parsed file /var/jenkins/workspace/ufz_ogs_PR-2429/build/build2.log
-> found 2 issues (skipped 6 duplicates)
Post processing issues on 'envinf11w' with source code encoding 'UTF-8'
Resolving absolute file names for all issues in workspace '/var/jenkins/workspace/ufz_ogs_PR-2429'
-> 0 resolved, 2 unresolved, 0 already resolved
Copying affected files to Jenkins' build folder '/datadrive/jenkins-home/jobs/ufz/jobs/ogs/branches/PR-2429/builds/2/files-with-issues'
-> 0 copied, 0 not in workspace, 2 not-found, 0 with I/O error
Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
-> resolved module names for 2 issues
Resolving package names (or namespaces) by parsing the affected files
-> resolved package names of 2 affected files
Applying 3 filters on the set of 2 issues (2 issues have been removed, 0 issues will be published)
Creating fingerprints for all affected code blocks to track issues over different builds
-> created fingerprints for 0 issues
Invoking Git blamer to create author and commit information for all affected files
GIT_COMMIT env = 'HEAD'
Git working tree = '/var/jenkins/workspace/ufz_ogs_PR-2429'
Git commit ID = 'ecd82eb50254ae2401a98c518f7305fccf7b2454'
Job workspace = '/data3/var/jenkins/workspace/ufz_ogs_PR-2429'
Created no blame requests - Git blame will be skipped
-> blamed authors of issues in 0 files
Using reference build 'ufz/ogs/PR-2429 #1' to compute new, fixed, and outstanding issues
Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
Evaluating quality gates
-> PASSED - Total number of issues (any severity): 0 - Quality QualityGate: 1
-> All quality gates have been passed
Health report is disabled - skipping

With warnings-ng-plugin 4.0.0:

Error Messages

Can't resolve absolute paths for some files:
- createSurface.cpp
- Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
Can't create fingerprints for some files:
- 'Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h', IO exception has been thrown: java.nio.file.NoSuchFileException: Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- 'Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h', IO exception has been thrown: java.nio.file.NoSuchFileException: Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- 'Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h', IO exception has been thrown: java.nio.file.NoSuchFileException: Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- 'Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h', IO exception has been thrown: java.nio.file.NoSuchFileException: Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- 'Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h', IO exception has been thrown: java.nio.file.NoSuchFileException: Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- 'Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h', IO exception has been thrown: java.nio.file.NoSuchFileException: Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
Can't create blame requests for some affected files:
- Skipping non-workspace file Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- Skipping non-workspace file Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- Skipping non-workspace file Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- Skipping non-workspace file Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- Skipping non-workspace file Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h
- Skipping non-workspace file Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h

Information Messages

Searching for all files in '/var/jenkins/workspace/ufz_ogs_PR-2429' that match the pattern 'build/build*.log'
-> found 2 files
Successfully parsed file /var/jenkins/workspace/ufz_ogs_PR-2429/build/build1.log
-> found 7 issues (skipped 0 duplicates)
Successfully parsed file /var/jenkins/workspace/ufz_ogs_PR-2429/build/build2.log
-> found 7 issues (skipped 1 duplicate)
Post processing issues on 'envinf56' with source code encoding 'UTF-8'
Resolving absolute file names for all issues in workspace '/var/jenkins/workspace/ufz_ogs_PR-2429'
-> 0 resolved, 2 unresolved, 0 already resolved
Copying affected files to Jenkins' build folder '/datadrive/jenkins-home/jobs/ufz/jobs/ogs/branches/PR-2429/builds/3/files-with-issues'
-> 0 copied, 0 not in workspace, 2 not-found, 0 with I/O error
Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
-> resolved module names for 7 issues
Resolving package names (or namespaces) by parsing the affected files
-> resolved package names of 2 affected files
Applying 3 filters on the set of 7 issues (1 issues have been removed, 6 issues will be published)
Creating fingerprints for all affected code blocks to track issues over different builds
-> created fingerprints for 0 issues
Invoking Git blamer to create author and commit information for all affected files
GIT_COMMIT env = 'HEAD'
Git working tree = '/var/jenkins/workspace/ufz_ogs_PR-2429'
Git commit ID = '84c4b4764737e38408db0eba5c0d97ba289702f1'
Job workspace = '/var/jenkins/workspace/ufz_ogs_PR-2429'
Created no blame requests - Git blame will be skipped
-> blamed authors of issues in 0 files
Using reference build 'ufz/ogs/PR-2429 #1' to compute new, fixed, and outstanding issues
Issues delta (vs. reference build): outstanding: 0, new: 6, fixed: 0
Evaluating quality gates
-> WARNING - Total number of issues (any severity): 6 - Quality QualityGate: 1
-> Some quality gates have been missed: overall result is WARNING
Health report is disabled - skipping

It is a CMake-based build and the file ui_VisualizationWidgetBase.h is located in the build-directory (it is auto-generated by Qt uic-compiler).

Lars Bilke added a comment - 2019-03-19 08:46 - edited

In 4.0.0 the excludeMessage('.*QVTKWidget.*') which should filter warnings coming from ui_VisualizationWidgetBase.h does not work. In 3.0.3 it is fine.

Lars Bilke added a comment - 2019-03-19 08:46 - edited In 4.0.0 the excludeMessage('.* QVTKWidget. *') which should filter warnings coming from ui_VisualizationWidgetBase.h does not work. In 3.0.3 it is fine.

Ulli Hafner added a comment - 2019-03-19 08:55

Ok, then the actual problem is that in 4.0 the message filter is not working? Why don't you filter by file name (ui_VisualizationWidgetBase.h)?

What exactly is the warning message? (Maybe it helps if you also provide the log lines that are the basis for the warning.

Ulli Hafner added a comment - 2019-03-19 08:55 Ok, then the actual problem is that in 4.0 the message filter is not working? Why don't you filter by file name (ui_VisualizationWidgetBase.h)? What exactly is the warning message? (Maybe it helps if you also provide the log lines that are the basis for the warning.

Lars Bilke added a comment - 2019-03-19 09:07

Here is a warning message (full log):

15:16:42  [521/891] Building CXX object Applications/DataExplorer/CMakeFiles/DataExplorer.dir/DataExplorer_autogen/mocs_compilation.cpp.o
15:16:42  In file included from ../Applications/DataExplorer/VtkVis/VisualizationWidget.h:18:0,
15:16:42                   from Applications/DataExplorer/DataExplorer_autogen/include/ui_mainwindow.h:29,
15:16:42                   from Applications/DataExplorer/DataExplorer_autogen/EWIEGA46WW/../../../../../Applications/DataExplorer/mainwindow.h:18,
15:16:42                   from Applications/DataExplorer/DataExplorer_autogen/EWIEGA46WW/moc_mainwindow.cpp:9,
15:16:42                   from Applications/DataExplorer/DataExplorer_autogen/mocs_compilation.cpp:2:
15:16:42  Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h: In member function 'void Ui_VisualizationWidgetBase::setupUi(QWidget*)':
15:16:42  Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h:263:59: warning: 'QVTKWidget::QVTKWidget(QWidget*, Qt::WindowFlags)' is deprecated [-Wdeprecated-declarations]
15:16:42           vtkWidget = new QVTKWidget(VisualizationWidgetBase);

This specific message filter is not working maybe it is because it cannot generate the fingerprint of the specific file. Yes filter by name would work but this would also filter other warnings from this file and with 3.0.3 the message filter worked in this case. The build logs for the two examples above are exactly the same, please note that 4.0.0 also does not detect duplicates in this case.

Lars Bilke added a comment - 2019-03-19 09:07 Here is a warning message ( full log ): 15:16:42 [521/891] Building CXX object Applications/DataExplorer/CMakeFiles/DataExplorer.dir/DataExplorer_autogen/mocs_compilation.cpp.o 15:16:42 In file included from ../Applications/DataExplorer/VtkVis/VisualizationWidget.h:18:0, 15:16:42 from Applications/DataExplorer/DataExplorer_autogen/include/ui_mainwindow.h:29, 15:16:42 from Applications/DataExplorer/DataExplorer_autogen/EWIEGA46WW/../../../../../Applications/DataExplorer/mainwindow.h:18, 15:16:42 from Applications/DataExplorer/DataExplorer_autogen/EWIEGA46WW/moc_mainwindow.cpp:9, 15:16:42 from Applications/DataExplorer/DataExplorer_autogen/mocs_compilation.cpp:2: 15:16:42 Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h: In member function 'void Ui_VisualizationWidgetBase::setupUi(QWidget*)': 15:16:42 Applications/DataExplorer/VtkVis/VtkVis_autogen/include/ui_VisualizationWidgetBase.h:263:59: warning: 'QVTKWidget::QVTKWidget(QWidget*, Qt::WindowFlags)' is deprecated [-Wdeprecated-declarations] 15:16:42 vtkWidget = new QVTKWidget(VisualizationWidgetBase); This specific message filter is not working maybe it is because it cannot generate the fingerprint of the specific file. Yes filter by name would work but this would also filter other warnings from this file and with 3.0.3 the message filter worked in this case. The build logs for the two examples above are exactly the same, please note that 4.0.0 also does not detect duplicates in this case.

Ulli Hafner added a comment - 2019-03-19 10:19

Fixed with commit/44e3b6d. The problem was that now the parser reads multi-line messages.

Ulli Hafner added a comment - 2019-03-19 10:19 Fixed with commit/44e3b6d . The problem was that now the parser reads multi-line messages.

Ulli Hafner added a comment - 2019-03-19 10:21 - edited

I wonder if we should add another issue for your logs: the details view button in the issues table should also show the messages in multiple lines. The message is currently separated by newline which does nothing in HTML.

Ulli Hafner added a comment - 2019-03-19 10:21 - edited I wonder if we should add another issue for your logs: the details view button in the issues table should also show the messages in multiple lines. The message is currently separated by newline which does nothing in HTML.

Ulli Hafner added a comment - 2019-03-19 10:24

And another tip/question: Normally the gcc parser detects the absolute path by combining the path of the current work directory and the relative path of the warning. In your case there is no current work directory printed. We currently have support for make and ninja messages. Would it be possible to add those messages to your build?

Ulli Hafner added a comment - 2019-03-19 10:24 And another tip/question: Normally the gcc parser detects the absolute path by combining the path of the current work directory and the relative path of the warning. In your case there is no current work directory printed. We currently have support for make and ninja messages. Would it be possible to add those messages to your build?

Lars Bilke added a comment - 2019-03-19 10:32

Thanks a lot for the quick fix!

What do you mean with "add those messages to your build"? The log is already generated with ninja although we build via cmake --build:

#!/bin/bash set -eo pipefail cd build cmake --build . --config Release --target package -j 4 2>&1 | tee build1.log

Lars Bilke added a comment - 2019-03-19 10:32 Thanks a lot for the quick fix! What do you mean with "add those messages to your build"? The log is already generated with ninja although we build via cmake --build: #!/bin/bash set -eo pipefail cd build cmake --build . --config Release --target package -j 4 2>&1 | tee build1.log

Ulli Hafner added a comment - 2019-03-19 10:59

I don't know how to invoke ninja, I'm a Java guy I just added support for ninja path detection: https://github.com/jenkinsci/analysis-model/blob/master/src/main/java/edu/hm/hafner/analysis/LookaheadParser.java#L28

Is there an option in ninja to get those log messages in your build as well?

Ulli Hafner added a comment - 2019-03-19 10:59 I don't know how to invoke ninja, I'm a Java guy I just added support for ninja path detection: https://github.com/jenkinsci/analysis-model/blob/master/src/main/java/edu/hm/hafner/analysis/LookaheadParser.java#L28 Is there an option in ninja to get those log messages in your build as well?

Lars Bilke added a comment - 2019-03-19 11:34

Ah I see. But this is output from CMake not Ninja. CMake configures the build and generates e.g. Makefiles or Ninja build files. CMake-based projects are often built out-of-source (i.e. having a build-directory separate from the source directory) so with your implementation you detect the build-directory (not the usage of Ninja).

Yep, I can try to get the log message into the build log. Would it be enough to have it in one of several build files? I.e. adding a file which contains that line to pattern: 'build/build*.log'? Because what I do is like this:

$ mkdir build; cd build
$ cmake -G Ninja ../sources
  ...
  -- Build files have been written to ...
$ ninja

Every command is piped to a log file.

Lars Bilke added a comment - 2019-03-19 11:34 Ah I see. But this is output from CMake not Ninja. CMake configures the build and generates e.g. Makefiles or Ninja build files. CMake-based projects are often built out-of-source (i.e. having a build-directory separate from the source directory) so with your implementation you detect the build-directory (not the usage of Ninja). Yep, I can try to get the log message into the build log. Would it be enough to have it in one of several build files? I.e. adding a file which contains that line to pattern: 'build/build*.log' ? Because what I do is like this: $ mkdir build; cd build $ cmake -G Ninja ../sources ... -- Build files have been written to ... $ ninja Every command is piped to a log file.

Ulli Hafner added a comment - 2019-03-19 11:56

This should work, here is the example that I am using in my test: https://github.com/jenkinsci/analysis-model/blob/master/src/test/resources/edu/hm/hafner/analysis/parser/issue56020.ninja.log

Ulli Hafner added a comment - 2019-03-19 11:56 This should work, here is the example that I am using in my test: https://github.com/jenkinsci/analysis-model/blob/master/src/test/resources/edu/hm/hafner/analysis/parser/issue56020.ninja.log

Assignee:: Ulli Hafner

Reporter:: Lars Bilke

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Created:: 2019-03-19 07:31

Updated:: 2019-05-08 22:19

Resolved:: 2019-03-19 10:19

Jenkins

Details

Description

Attachments

Activity

Collapse comment: Lars Bilke added a comment - 2019-03-19 08:46, Edited by Lars Bilke - 2019-03-19 08:47

Expand comment: Lars Bilke added a comment - 2019-03-19 08:46, Edited by Lars Bilke - 2019-03-19 08:47

Collapse comment: Ulli Hafner added a comment - 2019-03-19 08:55

Expand comment: Ulli Hafner added a comment - 2019-03-19 08:55

Collapse comment: Lars Bilke added a comment - 2019-03-19 09:07

Expand comment: Lars Bilke added a comment - 2019-03-19 09:07

Collapse comment: Ulli Hafner added a comment - 2019-03-19 10:19

Expand comment: Ulli Hafner added a comment - 2019-03-19 10:19

Collapse comment: Ulli Hafner added a comment - 2019-03-19 10:21, Edited by Ulli Hafner - 2019-03-19 10:21

Expand comment: Ulli Hafner added a comment - 2019-03-19 10:21, Edited by Ulli Hafner - 2019-03-19 10:21

Collapse comment: Ulli Hafner added a comment - 2019-03-19 10:24

Expand comment: Ulli Hafner added a comment - 2019-03-19 10:24

Collapse comment: Lars Bilke added a comment - 2019-03-19 10:32

Expand comment: Lars Bilke added a comment - 2019-03-19 10:32

Collapse comment: Ulli Hafner added a comment - 2019-03-19 10:59

Expand comment: Ulli Hafner added a comment - 2019-03-19 10:59

Collapse comment: Lars Bilke added a comment - 2019-03-19 11:34

Expand comment: Lars Bilke added a comment - 2019-03-19 11:34

Collapse comment: Ulli Hafner added a comment - 2019-03-19 11:56

Expand comment: Ulli Hafner added a comment - 2019-03-19 11:56

People

Dates