-
Bug
-
Resolution: Not A Defect
-
Major
-
None
In perl test suites, it's common to do a SKIPALL on certain test files when you for instance determine that you're on the wrong OS, or if long tests are not enabled.
<testsuites> <testsuite failures="0" errors="0" tests="0" name="00_devel_smoke_binaries_t"> <system-out><![CDATA[1..0 # SKIP Long-running tests disabled unless SLOW_TESTS environment variable set. ]]></system-out> <system-err></system-err> </testsuite> </testsuites>
When the TAP parser hits these, it declares the build a failure. If the well formed XML is declaring 0 tests, how is this a failure condition?
[JENKINS-16326] TAP Parser can't handle SKIP_ALL
Todd Rinaldo
added a comment - Isn't the XML what's actually parsed? In which case, wouldn't this be what's parsed?
<testsuite failures="0" errors="0" tests="0" name="00_devel_smoke_binaries_t">
Todd Rinaldo
added a comment - Isn't the XML what's actually parsed? In which case, wouldn't this be what's parsed?
<testsuite failures= "0"
errors= "0"
tests= "0"
name= "00_devel_smoke_binaries_t" >
Not really, the plug-in looks for plan/text files containing TAP streams like the following:
1..2
ok 1 - All is fine...
not ok 2 - Some message # comments
I'll update the Wiki this week. Will try to make more clear what type of file you need to have to use the plug-in 
Good! in that case that's easier. Here's what the output looks like in that case:
$>prove -wlvm t/00_devel_smoke_pl.t t/00_devel_smoke_pl.t .. 1..0 # SKIP Long-running tests disabled unless SLOW_TESTS environment variable set. skipped: Long-running tests disabled unless SLOW_TESTS environment variable set. Files=1, Tests=0, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.04 cusr 0.00 csys = 0.07 CPU) Result: NOTESTS
Todd Rinaldo
added a comment - - edited Good! in that case that's easier. Here's what the output looks like in that case:
$>prove -wlvm t/00_devel_smoke_pl.t
t/00_devel_smoke_pl.t ..
1..0 # SKIP Long -running tests disabled unless SLOW_TESTS environment variable set.
skipped: Long -running tests disabled unless SLOW_TESTS environment variable set.
Files=1, Tests=0, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.04 cusr 0.00 csys = 0.07 CPU)
Result: NOTESTS
Hi there,
First I created a simple freestyle job, with a shell build step (touch archive).
Then I downloaded XML-Simple-2.20, copied the t/ folder to my job workspace and executed in another shell build step: prove t/*.t --archive archive/.
Finally, added TAP to my project, with archive/*.t as pattern.
The result archive/7_SaxStuff.t contains a SKIP_ALL: 1..0 # SKIP no XML::SAX
It is interpreted neither as failure or as passed, it's simply treated as a skipped test
Can you run your tests with prove and archive the results to an existing directory and check what happens, please?
Thank you in advance, -B
Marking as not a defect. If we find that it's not possible to interpret SKIP_ALL with the plug-in I'll change the bug status TY
Hello Todd,
Let me see if I understand it right. The following text:
1..0 # SKIP Long-running tests disabled unless SLOW_TESTS environment variable set.
is not interpreted as a skip all plan, right? We can fix that in tap4j (tap plug-in uses tap4j for parsing TAP files).
/B