Details
-
Type:
Bug
-
Status: Open (View Workflow)
-
Priority:
Major
-
Resolution: Unresolved
-
Component/s: nunit-plugin
-
Labels:None
-
Environment:nunit-plugin 0.21, Jenkins 2.83 on a Windows Server 2008 R2
-
Similar Issues:
Description
Using the latest nunit plugin on latest (not LTS) stable Jenkins leads to a wrong displayed test case summary.
In our example the non-parameterized test cases which are mixed in one test class with parameterized test cases are referenced to a wrong class.
In our example the parameterized test cases are referenced to the package "Base.Tests" and the non-parameterized test cases are referenced to the package "Base" which is wrong. They are in the package "Base.Tests". You can see this in the screenshots, every test which is displayed in "Base" is wrong and belongs to "Base.Tests"
This is our test result output xml.
<test-suite type="TestFixture" name="CryptUtilsTests" executed="True" result="Success" success="True" time="0.308" asserts="0"> <results> <test-suite type="ParameterizedTest" name="ComputerSha256HashAndCompareWithExpectedResult" executed="True" result="Success" success="True" time="0.010" asserts="0"> <results> <test-case name="Base.Tests.CryptUtilsTests.ComputerSha256HashAndCompareWithExpectedResult("pass","D74FF0BDE50BE8F1")" executed="True" result="Success" success="True" time="0.005" asserts="1" /> <test-case name="Base.Tests.CryptUtilsTests.ComputerSha256HashAndCompareWithExpectedResult("Horst","1FC1E73359247428B9855DFD")" executed="True" result="Success" success="True" time="0.000" asserts="1" /> <test-case name="Base.Tests.CryptUtilsTests.ComputerSha256HashAndCompareWithExpectedResult("Hugo","D70B489BB97D74C8E69A60EA62DC8FFD")" executed="True" result="Success" success="True" time="0.001" asserts="1" /> <test-case name="Base.Tests.CryptUtilsTests.ComputerSha256HashAndCompareWithExpectedResult("GeheimesPasswort!","2600F32BA82835F62D46B7")" executed="True" result="Success" success="True" time="0.000" asserts="1" /> </results> </test-suite> <test-case name="Base.Tests.CryptUtilsTests.ComputesSaltAndPasswordHashCorrectly" executed="True" result="Success" success="True" time="0.024" asserts="4" /> <test-case name="Base.Tests.CryptUtilsTests.ComputesSaltAndPasswordHashFromDifferentTasksCorrectly" executed="True" result="Success" success="True" time="0.135" asserts="1" /> <test-case name="Base.Tests.CryptUtilsTests.ComputesSaltAndPasswordHashFromDifferentThreadsCorrectly" executed="True" result="Success" success="True" time="0.095" asserts="1" /> <test-case name="Base.Tests.CryptUtilsTests.EncyrptAndDecryptWindows" executed="True" result="Success" success="True" time="0.034" asserts="15" /> </results> </test-suite>
Attachments
Activity
Can't test it with different version because it's our production server. We just discovered it with the version I have mentioned.
Daniel Isenmann
added a comment - Can't test it with different version because it's our production server. We just discovered it with the version I have mentioned. That isn't even valid XML though (what you pasted). There are quoted strings inside of attributes. When I try and run this through the plugin locally, it just dies on trying to process the XML itself.
Changed the XML accordingly. Seems that the quotes were insert during copy, they are now masked correctly.
Daniel Isenmann
added a comment - Changed the XML accordingly. Seems that the quotes were insert during copy, they are now masked correctly. This doesn't look like valid nunit3 output, is it nunit3 or nunit2? The <results> nodes are not part of the NUnit3 XML specifications. https://github.com/nunit/docs/wiki/Test-Result-XML-Format
We are running nunit2
Daniel Isenmann
added a comment - We are running nunit2 
Did this behave differently with a different version of Jenkins, or are you just saying it doesn't match things up in general?