Status: Closed (View Workflow)
If a test fails then CTest returns a non-zero return code (8 in my configuration) and jenkins build is stopped.
This makes the build fail even if higher unstable and fail thresholds are selected.
An option to select the return code to be ignored will work the best.
Yes, adding an option to ignore the exit status would do the job in my case.
On the other side, it can be that CTest returns a different error code if there is another kind of error (wrong parameters, etc.). That kind of errors shouldn't be ignored. Unfortunately the CTest documentation does not specifies the error codes.
I have reported the issue to CTest: https://gitlab.kitware.com/cmake/cmake/issues/16151
Please try https://jenkins.ci.cloudbees.com/job/plugins/job/cmakebuilder-plugin/org.jenkins-ci.plugins$cmakebuilder/102/artifact/org.jenkins-ci.plugins/cmakebuilder/2.4.3-SNAPSHOT/cmakebuilder-2.4.3-SNAPSHOT.hpi and provide some feedback.
My unittests are unreliable here, since ctest v. 188.8.131.52 seems to always exit with success so they never fail for this issue .
And please have a look at the online help and send suggestions, as I am not a native speaker.
Unfortunately it didn't work out as expected. Actually even that the Plugin Manager shows that "CMake plugin 2.4.3-SNAPSHOT (private-e95ede2f-jenkins)" is installed, I couldn't find any option to disable the checking of the exit code. Currently I'm using Jenkins 2.5 installed on Windows 2008 Server.
The relevant section in the "Console Output":
The following tests FAILED: 72 - testFoo (Failed) 74 - testBar (Failed) 75 - testBaz (Failed) 116 - testQux (SEGFAULT) Errors while running CTest Build step 'CMake/CPack/CTest execution' changed build result to FAILURE
You will find a new text box labeled Ignore exit codes in the CMake/CPack/CTest build step now, which should satisfy your needs.
Today I got the chance to test this feature in version 2.4.3 and it works as expected. Thank you!
Would it be sufficient to add a boolean option that allows to ignore the exit status of ctest it it does not indicate success?