-
Bug
-
Resolution: Fixed
-
Major
-
xUnit 2.0.3
Jenkins ver. 2.126
I have created a simple pipeline with two parallel steps.
In each step I create a valid JUnit xml file and use xUnit to analyse it.
During the run xunit can analyse one of the JUnit xml file, but for the other one it only logs out the message: WARNING: All test reports are empty.
Attached pipeline.log log.txt
Example pipeline:
timestamps { node { deleteDir() sleep 1 parallel( dateiEins: { stage('generate xml file 1'){ dir('file1') { dir('test-output') { dir('junit') { writeFile file: 'TEST-de.edict.services.portaltests.SlotSessionTest_ES.xml', text: '''<?xml version="1.0" encoding="UTF-8"?> <!-- Generated by de.edict.listener.JUnitReportReporterBase --> <testsuite hostname="hhbld03t" name="de.edict.services.portaltests.SlotSessionTest--ES" tests="11" failures="0" timestamp="26 Jun 2018 13:51:32 GMT" time="265.564" errors="1"> <testcase name="testExpireSlotSessionWhileGameStartLossCondition--ES" time="17.388" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testConsistencyOfSlotsessionAndAccountDataAfterHeartbeat--ES" time="59.843" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testSlotSessionWinning--ES" time="21.539" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testExpireSlotSessionWhileGameStartTimeCondition--ES" time="17.597" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testSlotSessionLimitAmountExceeded--ES" time="13.583" classname="de.edict.services.portaltests.SlotSessionTest"> <error type="java.lang.AssertionError" message="Expecting message type SLOT_LOSS_LIMIT_EXCEEDED in Response. Expected: is "SLOT_LOSS_LIMIT_EXCEEDED" but: was "ACCOUNT_EMPTY""> <![CDATA[java.lang.AssertionError: Expecting message type SLOT_LOSS_LIMIT_EXCEEDED in Response. Expected: is "SLOT_LOSS_LIMIT_EXCEEDED" but: was "ACCOUNT_EMPTY" at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) at de.edict.services.portaltests.SlotSessionTest.testSlotSessionLimitAmountExceeded(SlotSessionTest.java:288) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at org.testng.TestRunner.privateRun(TestRunner.java:767) at org.testng.TestRunner.run(TestRunner.java:617) at org.testng.SuiteRunner.runTest(SuiteRunner.java:348) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305) at org.testng.SuiteRunner.run(SuiteRunner.java:254) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224) at org.testng.TestNG.runSuitesLocally(TestNG.java:1149) at org.testng.TestNG.run(TestNG.java:1057) at org.testng.TestNG.privateMain(TestNG.java:1364) at org.testng.TestNG.main(TestNG.java:1333) ]]> </error> </testcase> <!-- testSlotSessionLimitAmountExceeded-ES --> <testcase name="testConsistencyOfSlotsessionAndAccountData--ES" time="18.456" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testCreateSlotSessionWithGameStart--ES" time="2.897" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testExpireSlotSessionWhileHeartbeat--ES" time="35.558" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testCheckPortalRestService--ES" time="28.069" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testCheckBORestService--ES" time="17.425" classname="de.edict.services.portaltests.SlotSessionTest"/> <testcase name="testLossLimitWarning--ES" time="33.209" classname="de.edict.services.portaltests.SlotSessionTest"/> </testsuite> <!-- de.edict.services.portaltests.SlotSessionTest-ES --> ''' sh 'ls -l' xunit testTimeMargin: '3000', thresholdMode: 1, thresholds: [failed(failureNewThreshold: '0', failureThreshold: '0'), skipped()], tools: [JUnit(deleteOutputFiles: false, failIfNotNew: false, pattern: 'TEST-*.xml', skipNoTestFiles: false, stopProcessingIfError: true)] } } } } }, dateiZwei: { stage('generate xml file 2'){ dir('file2') { dir('test-output') { dir('junit') { writeFile file: 'TEST-de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests--IoM_default.xml', text: '''<?xml version="1.0" encoding="UTF-8"?> <!-- Generated by de.edict.listener.JUnitReportReporterBase --> <testsuite hostname="hhbld01t" name="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests--IoM_default" tests="15" failures="1" timestamp="26 Jun 2018 15:41:26 GMT" time="55.391" errors="0"> <testcase name="testCreateBonusOfferWithCountryRestrictions--IoM_default" time="12.167" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testFirstPayInWith50EURWithVoucherCode--IoM_default" time="1.454" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testCreateBonusOfferTrigger999--IoM_default" time="2.114" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testEveryPayInDuringThePeriodWithVisa--IoM_default" time="26.713" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"> <failure type="com.googlecode.sardine.impl.SardineException" message="Unexpected response"> <![CDATA[com.googlecode.sardine.impl.SardineException: Unexpected response at com.googlecode.sardine.impl.handler.ValidatingResponseHandler.validateResponse(ValidatingResponseHandler.java:49) at com.googlecode.sardine.impl.handler.VoidResponseHandler.handleResponse(VoidResponseHandler.java:35) at com.googlecode.sardine.impl.SardineImpl.get(SardineImpl.java:478) at com.googlecode.sardine.impl.SardineImpl.get(SardineImpl.java:458) at de.edict.eoc.test.utils.TextHelper.connectWikiWebdav(TextHelper.java:75) at de.edict.eoc.test.bonus.BonusOfferTestHelper.checkDefaultStakerestrictionsAgainstWiki(BonusOfferTestHelper.java:339) at de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests.testEveryPayInDuringThePeriodWithVisa(BonusOfferDepositTests.java:178) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ]]> </failure> </testcase> <!-- testEveryPayInDuringThePeriodWithVisa-IoM_default --> <testcase name="TryToCreateBonusWithoutCorrectPermission--IoM_default" time="0.610" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testBonusOfferWithSpecialCharacter--IoM_default" time="3.118" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testFirstPayinWithOneHourDuration--IoM_default" time="0.693" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testBonusOfferWithSpecialCharacterMinus--IoM_default" time="0.501" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testFirstPayinWithoutStartAndEnddate--IoM_default" time="0.542" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testCreateBonusOfferWithCountryAndPaymentRestrictions--IoM_default" time="2.136" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testEveryfifthPayInWithVisa--IoM_default" time="0.675" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testEveryPayInDuringThePeriod--IoM_default" time="1.956" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testFirstPayInWith50EUR--IoM_default" time="0.706" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testCreateBonusOfferSartTodayTop5Restrictions--IoM_default" time="1.055" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> <testcase name="testCreateBonusOfferAllMax--IoM_default" time="0.951" classname="de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests"/> </testsuite> <!-- de.edict.services.backofficetests.bonusoffer.BonusOfferDepositTests-IoM_default --> ''' sh 'ls -l' xunit testTimeMargin: '3000', thresholdMode: 1, thresholds: [failed(failureNewThreshold: '0', failureThreshold: '0'), skipped()], tools: [JUnit(deleteOutputFiles: false, failIfNotNew: false, pattern: "TEST-*.xml", skipNoTestFiles: false, stopProcessingIfError: true)] } } } } } ) } }
I would like to raise this problem to major - at least: This bug has been introduced with the upgrade from 2.0.2 to 2.0.3 and it's not only cosmetical.
It also means that xUnit don't fail build jobs when it finds failures in the jUnit resultfiles - meaning that continuous deployments will deploy because they're not stopped by xUnit.
I can add that even though xUnit warns that all test reports are empty, they're parsed correctly and can be seen in the usual Jenkins testresult view for the job.