-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Jenkins ver. 1.585, Testlink 1.9.13, Google Chrome 44.0.2403.107(64bit)
I have a configured the Testlink plugin on Jenkins. After I run my TestNG test cases, I proceed to update the results in Testlink.
The test cases are displayed correctly, but all of them are updated as "Passed"
TestLink build ID: 282
TestLink build name: build_2015-07-27_09-40-21
Total of 4 tests. Where 4 (+4) passed, 0 failed, 0 were blocked and 0 were not executed.
List of test cases and execution result status
Test case ID Test case external ID Version Name Test project ID Execution status
87555 SOL-10503 1 Add Provider Credential Policy 67109 Passed
87557 SOL-10504 1 Assign Prov Credential Policy to Provider Admin 67109 Passed
87559 SOL-10505 1 Add Customer Credential Policy 67109 Passed
87561 SOL-10506 1 Assign Cust Credential Policy to Customer Admin 67109 Passed
In my test cases TestNG result, 2 of the test cases have failed.
TestNG Results
Total Tests: 4 (+4)
Failed Tests: 2 (+2)
com.company.abc.def.auto.applicationtest.test.GenericTest.testAdd
com.company.abc.def.auto.applicationtest.test.GenericTest.testAdd
Skipped Tests: 0 (±0)
Failed Configurations: 0 (±0)
Skipped Configurations: 0 (±0)
I have a suspicion it might be related to my Test result seeking strategy definition :
I use :
TestNG class name
Include Pattern -> output/testng-results.xml
Key Custom Field -> TL-Jenkins
I am using : TestLink 1.9.13 (Stormbringer)
Jenkins ver. 1.585
Testlink Plugin 3.10
[JENKINS-29716] Incorrect Execution Results when using TestNG class name results seeking strategy
Arne Wohletz
added a comment - I have a similar problem.
TestNG plugin states that 1 from 1 TC passed, but the TestLink plugin results show a 'Not run'.
The testng-results.xml file is OK, shows that the TC passed. Somehow Testlink does not confirm this result.
I also use a
'TestNG class name' result seeking strategy
'Include pattern' = target/**/testng-results.xml
and a 'Publish TestNG results' post-build-action:
TestNG XML report pattern = target/**/testng-results.xml
I am currently learning from this tutorial: https://wiki.jenkins-ci.org/download/attachments/753702/jenkins.pdf
TestLink v1.9.14 (Padawan)
TestLink Plugin v3.12
TestNG Results Plugin v1.13
Jenkins v1.651.3
Can somebody help?
Could it be that the TestLink plugin wants to retrieve the test results before those are written out in the xml file, setting it to 'not run' as a default?
Arne Wohletz
added a comment - I have a similar problem.
TestNG plugin states that 1 from 1 TC passed, but the TestLink plugin results show a 'Not run'.
The testng-results.xml file is OK, shows that the TC passed. Somehow Testlink does not confirm this result.
I also use a
'TestNG class name' result seeking strategy
'Include pattern' = target/**/testng-results.xml
and a 'Publish TestNG results' post-build-action:
TestNG XML report pattern = target/**/testng-results.xml
I am currently learning from this tutorial: https://wiki.jenkins-ci.org/download/attachments/753702/jenkins.pdf
TestLink v1.9.14 (Padawan)
TestLink Plugin v3.12
TestNG Results Plugin v1.13
Jenkins v1.651.3
Can somebody help?
Could it be that the TestLink plugin wants to retrieve the test results before those are written out in the xml file, setting it to 'not run' as a default? > Could it be that the TestLink plugin wants to retrieve the test results before those are written out in the xml file, setting it to 'not run' as a default?
By default, for each test case retrieved for the given test plan, the status is set to not run.
Then it will use the strategy you chose to search for test results.
For JUnit, it uses Jenkins' JUNit parser.
For TAP, it uses tap4j.
And for TestNG it uses a custom testng-parser - see https://github.com/tupilabs/testng-parser
So it would require investigation to define whether there's something wrong with the parser, if the configuration is correct, of if it's a bug or incompatibility in the Java and/or PHP layer.
As for the tutorial, I want to move it under www.gitbook.com, so that we can use Markdown for source, and get pull requests and issues. Then I would update it to the latest version of Jenkins, libraries and TestLink. But unfortunately I don't have the resources for doing it right now (mainly time, but also $ 
)
Arne Wohletz
added a comment - Hello Bruno,
thanks for for quick response.
I found the solution which works for me - the class name that is used in the example is actually 'jenkins.plugins.testlink.examples.TestCurrentTime' whereas in TestLink I used 'hudson.plugins.testlink.examples.TestCurrentTime' as it said in the tutorial PDF (page 15).
Now it works perfectly!
Maybe that also helps voss qa.
Arne Wohletz
added a comment - Hello Bruno,
thanks for for quick response.
I found the solution which works for me - the class name that is used in the example is actually 'jenkins.plugins.testlink.examples.TestCurrentTime' whereas in TestLink I used 'hudson.plugins.testlink.examples.TestCurrentTime' as it said in the tutorial PDF (page 15).
Now it works perfectly!
Maybe that also helps voss qa . 
Anybody able to assist with this?