-
New Feature
-
Resolution: Unresolved
-
Minor
-
None
When using CTest for running tests with valgrind (the memcheck mode)
ctest -T memcheck
It produces a differently formatted test report DynamicAnalysis.xml. This report is ignored by the current xunit test report plugin (No errors, the tests don't show up).
It still is found by the plugin
[xUnit] [INFO] - [CTest-Version N/A (default)] - 2 test report file(s) were found with the pattern 'build-tests/Testing/**/Test.xml,build-tests/Testing/**/DynamicAnalysis.xml' relative to '/home/builduser/workspace/Misc/cmake-build-test' for the testing framework 'CTest-Version N/A (default)'.
See attached example
[JENKINS-37844] Add support for Dynamic Analysis (memcheck with valgrind) reports created by CTest
Linus Mellberg
added a comment - Not sure I remember exactly. Are now handling this in other ways (not using CTests memorycheck and instead setting up valgrind manually and using xunit-valgrind tool).
Seems like I might be wrong about that there are two files generated. Seems like the `ctest -T Memcheck` command generates only the DynamicAnalysis.xml report, and `ctest -T Test` generates Test.xml. Don't what the normal workflow would be. I think I would be fine with having two TestTypes configured and thus duplication. Something like:
<xunit plugin="xunit@1.102"> <types> <CTestType> <pattern>build/Testing/**/Test.xml</pattern> <skipNoTestFiles>false</skipNoTestFiles> <failIfNotNew>true</failIfNotNew> <deleteOutputFiles>true</deleteOutputFiles> <stopProcessingIfError>true</stopProcessingIfError> </CTestType> <CTestType> <!-- Memcheck, wouldn't currently work since the DynamicAnalysis.xml isn't handled --> <pattern>build/Testing/**/DynamicAnalysis.xml</pattern> <skipNoTestFiles>false</skipNoTestFiles> <failIfNotNew>true</failIfNotNew> <deleteOutputFiles>true</deleteOutputFiles> <stopProcessingIfError>true</stopProcessingIfError> </CTestType> <!-- What I'm currently doing <ValgrindJunitHudsonTestType> <pattern>build/**/*memcheck.xml</pattern> <skipNoTestFiles>true</skipNoTestFiles> <failIfNotNew>false</failIfNotNew> <deleteOutputFiles>true</deleteOutputFiles> <stopProcessingIfError>true</stopProcessingIfError> </ValgrindJunitHudsonTestType> --> </types> <extraConfiguration> <testTimeMargin>3000</testTimeMargin> </extraConfiguration> </xunit>
I want to be able to only run the memorychecker for some tests and not for others. So I run both the `ctest -T Memcheck` and `ctest -T Test` with some filtering on each command, but a more sane scenario might be to run one or the other.
Linus Mellberg
added a comment - Not sure I remember exactly. Are now handling this in other ways (not using CTests memorycheck and instead setting up valgrind manually and using xunit-valgrind tool).
Seems like I might be wrong about that there are two files generated. Seems like the `ctest -T Memcheck` command generates only the DynamicAnalysis.xml report, and `ctest -T Test` generates Test.xml . Don't what the normal workflow would be. I think I would be fine with having two TestTypes configured and thus duplication. Something like:
<xunit plugin= "xunit@1.102" >
<types>
<CTestType>
<pattern> build/Testing/**/Test.xml </pattern>
<skipNoTestFiles> false </skipNoTestFiles>
<failIfNotNew> true </failIfNotNew>
<deleteOutputFiles> true </deleteOutputFiles>
<stopProcessingIfError> true </stopProcessingIfError>
</CTestType>
<CTestType> <!-- Memcheck, wouldn 't currently work since the DynamicAnalysis.xml isn' t handled -->
<pattern> build/Testing/**/DynamicAnalysis.xml </pattern>
<skipNoTestFiles> false </skipNoTestFiles>
<failIfNotNew> true </failIfNotNew>
<deleteOutputFiles> true </deleteOutputFiles>
<stopProcessingIfError> true </stopProcessingIfError>
</CTestType>
<!-- What I'm currently doing
<ValgrindJunitHudsonTestType>
<pattern> build/**/*memcheck.xml </pattern>
<skipNoTestFiles> true </skipNoTestFiles>
<failIfNotNew> false </failIfNotNew>
<deleteOutputFiles> true </deleteOutputFiles>
<stopProcessingIfError> true </stopProcessingIfError>
</ValgrindJunitHudsonTestType>
-->
</types>
<extraConfiguration>
<testTimeMargin> 3000 </testTimeMargin>
</extraConfiguration>
</xunit>
I want to be able to only run the memorychecker for some tests and not for others. So I run both the `ctest -T Memcheck` and `ctest -T Test` with some filtering on each command, but a more sane scenario might be to run one or the other. 
Cristian
added a comment - FWIW I would say the problem is ctest, not this plugin. https://gitlab.kitware.com/cmake/cmake/-/issues/19003#note_666994
Cristian
added a comment - FWIW I would say the problem is ctest, not this plugin. https://gitlab.kitware.com/cmake/cmake/-/issues/19003#note_666994 
What do you expect here? There are two report files. I can not merge the result toghether. xUnit strategy for each source report use XSL to produce an equivalent junit report. If I got two file two report will be in output so you will have test doubled.