-
Bug
-
Resolution: Fixed
-
Major
-
centos 6
A .tap file containing a NOTESTS result (from eg a skip_all declaration in a .t file) such as
/opt/performance/scripts/t/dbcoststest1.t .. skipped: No Relevant Tests as No DB Costs for GetSmoidTrackV11New
Files=1, Tests=0, 1 wallclock secs ( 0.01 usr 0.02 sys + 0.54 cusr 0.04 csys = 0.61 CPU)
Result: NOTESTS
results in the following errors when running a plan from Jenkins:
Parsing TAP test result [/var/lib/jenkins/jobs/Performance-Server-Trunk/builds/2012-09-26_16-06-21/tap/dbcostsGetSmoidTrackV11New.tap].
org.tap4j.parser.ParserException: Error parsing TAP Stream: Missing TAP Plan.
at org.tap4j.parser.Tap13YamlParser.parseFile(Tap13YamlParser.java:587)
at org.tap4j.plugin.TapParser.parse(TapParser.java:94)
at org.tap4j.plugin.TapPublisher.loadResults(TapPublisher.java:240)
at org.tap4j.plugin.TapPublisher.perform(TapPublisher.java:182)
at hudson.tasks.BuildStepMonitor$2.perform(BuildStepMonitor.java:27)
at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:705)
at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:680)
at hudson.maven.MavenModuleSetBuild$RunnerImpl.post2(MavenModuleSetBuild.java:996)
at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:627)
at hudson.model.Run.run(Run.java:1438)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:479)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:238)
Caused by: org.tap4j.parser.ParserException: Missing TAP Plan.
at org.tap4j.parser.Tap13YamlParser.checkTAPPlanIsSet(Tap13YamlParser.java:353)
at org.tap4j.parser.Tap13YamlParser.postProcess(Tap13YamlParser.java:736)
at org.tap4j.parser.Tap13YamlParser.parseFile(Tap13YamlParser.java:585)
... 12 more
- is blocking
-
-
- Closed
-
[JENKINS-15322] NOTESTS in TAP response gives parse error and stack trace from plugin
Alex Newnham
added a comment - JENKINS-16325 seems to be concerned with both
- skipping certain tests
- appending TAP output from iterating thru multiple .t test files and output to one .tap file.
Using TAP plugin 1.9, I have not managed to get the stack trace when I only skip certain tests but I can duplicate the stack trace from the plugin when I append tests to one .tap file. I am not sure if multiple tests in one .tap file breaks the TAP spec (for a particular TAP version or perhaps all TAP versions).
So although the stack trace from this issue looks the same as the trace in JENKINS-16325, the reasons for the traces differ. However, the author of JENKINS-16325 mentions in his comments on that issue that he is still affected by JENKINS-16326 which does pretty much duplicate this issue.
My tests have been with the simplest TAP files possible, I may have missed some complexity, and I didn't use prove.
Hope this is useful!
Alex
Alex Newnham
added a comment - JENKINS-16325 seems to be concerned with both
skipping certain tests
appending TAP output from iterating thru multiple .t test files and output to one .tap file.
Using TAP plugin 1.9, I have not managed to get the stack trace when I only skip certain tests but I can duplicate the stack trace from the plugin when I append tests to one .tap file. I am not sure if multiple tests in one .tap file breaks the TAP spec (for a particular TAP version or perhaps all TAP versions).
So although the stack trace from this issue looks the same as the trace in JENKINS-16325 , the reasons for the traces differ. However, the author of JENKINS-16325 mentions in his comments on that issue that he is still affected by JENKINS-16326 which does pretty much duplicate this issue.
My tests have been with the simplest TAP files possible, I may have missed some complexity, and I didn't use prove .
Hope this is useful!
Alex Hi @Alex,
IN JENKINS-16325 a user suggested:
prove *.t --archive tap/
That worked fine for me. One TAP Strem (in this case, a TAP file) cannot, or at least should not, contain multiple TAP files. In TAP specification a blank line in a file is ignored, as well as any non-TAP related token. So we would have to write a special parser to handle multiple TAP files within a single file.
I would prefer avoid that if possible.
Let me know if that works for you too.
Thanks! /B
Alex Newnham
added a comment - I can't see a use case for multiple TAP streams within a single file. None of my TAP is in this format so it's not a feature I am looking for.
thanks
Alex
Alex Newnham
added a comment - I can't see a use case for multiple TAP streams within a single file. None of my TAP is in this format so it's not a feature I am looking for.
thanks
Alex Alex Newnham
added a comment - Update: Excellent news! http://testanything.org/ is back up and running.
Alex Newnham
added a comment - Update: Excellent news! http://testanything.org/ is back up and running. Yay! There are some broken links, and seems like they had to restore an old backup. I'll add tap4j information and examples there again.
As a side note, I believe I've fixed this issue, but as I had to fix tap4j parser, I'll use a few more hours to test it.
I'll update the Wiki with information on how to use the plug-in with prove --archive and an example with skip_all.
The link to the bug fixed in tap4j: https://github.com/tupilabs/tap4j/issues/13
Wiki updated. Created the section "Running Perl tests with prove" with an example of how one can run prove and archive its output. The procedure is the same even if you use NOTESTS in TAP.
You can follow the instructions there, but there's another bug which causes the plug-in to not list tests which plan is marked as skip all. The remaining work will continue there, please feel free to comment there.
Both issues will be released together in 2.0.
Hi Alex,
Do you know if this issue is related to
JENKINS-16325, please?If so, maybe I can link both issues as related here so you would get update notifications from both.
Thanks, Bruno