[JENKINS-13821] When test uses DataProvider it mark in TestLink only by last result - Jenkins Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Component/s: testlink-plugin
Labels:
None
Environment:
win7, Jenkins ver. 1.452, TestLink 1.9.3, Jenkins TestLink Plugin 3.1.2

Similar Issues:

Show

Description

I have such test

@Test(dataProvider = "validateFieldBase", dataProviderClass = CampaignData.class)
public void validateNameField(String campaignName)

{ ... }

and such DataProvider for it

@DataProvider
public static Object[][] validateFieldBase() {
Object[][] word = {

{"<script language='JavaScript'> alert('Hello') </script>"}

{"ASDGFFLKLKLKlddvddsghdsh"}

{"12344557688008853"}

{":!@#$%^&*()_."}

{"ASDASDasdas-123123211212&^%&^%"}

,
};
return(word);
}

And I have test cause in TestLink mapped by Test Result Seeking Strategies - TestNG Method Name.

Problem: test cause in TestLink marks failed or passed by the last DataProvider result (in example by value "ASDASDasdas-123123211212&^%&%")

Expected result: if all DataProvider values passed than test cause in Testlink should mark as passed,
if even one DataProvider value is failed than test cause in Testlink should mark as failed.

may be its not a bug, but I think it will be a really nice feature.

P.S. : I tried to use successPercentage=100 but it's uncertainly for me when and for what it should be used.

Attachments

Activity

Ascending order - Click to sort in descending order

Denis Veselovskiy created issue - 2012-05-18 08:28

Bruno P. Kinoshita added a comment - 2012-06-17 13:39

Removing testng from components

Bruno P. Kinoshita added a comment - 2012-06-17 13:39 Removing testng from components

Bruno P. Kinoshita made changes - 2012-06-17 13:39

Field	Original Value	New Value
Component/s	testng [ 15801 ]

Bruno P. Kinoshita added a comment - 2012-06-17 13:40

Makes sense to me Let me see what can I do. Maybe we will need a new result seeking strategy.

Bruno P. Kinoshita added a comment - 2012-06-17 13:40 Makes sense to me Let me see what can I do. Maybe we will need a new result seeking strategy.

Bruno P. Kinoshita added a comment - 2012-06-17 13:44

I will cut a release now, and will work on this during the next week Denis. Sorry for the wait.

Bruno P. Kinoshita added a comment - 2012-06-17 13:44 I will cut a release now, and will work on this during the next week Denis. Sorry for the wait.

Denis Veselovskiy added a comment - 2012-06-18 07:35 - edited

Thanks )
I need this feature so much that I started to write my own testng-results.xml parser. And parser runs in @AfterSuite changing testng-results.xml so it marks in TestLink better ))
But I hope you will do this feature as well

Denis Veselovskiy added a comment - 2012-06-18 07:35 - edited Thanks ) I need this feature so much that I started to write my own testng-results.xml parser. And parser runs in @AfterSuite changing testng-results.xml so it marks in TestLink better )) But I hope you will do this feature as well

Denis Veselovskiy added a comment - 2012-07-16 12:25

Is any progress ?

Denis Veselovskiy added a comment - 2012-07-16 12:25 Is any progress ?

Bruno P. Kinoshita added a comment - 2012-07-26 01:38

Hi Denis,

I'm working on other issues at moment, sorry for the delay. But I plan to cut a new release of testlink-plugin before August. Could you share your testng-results.xml parser, please? I could use it as requirement for the new code

Thanks

Bruno P. Kinoshita added a comment - 2012-07-26 01:38 Hi Denis, I'm working on other issues at moment, sorry for the delay. But I plan to cut a new release of testlink-plugin before August. Could you share your testng-results.xml parser, please? I could use it as requirement for the new code Thanks

Bruno P. Kinoshita made changes - 2012-11-25 05:16

Status

Open [ 1 ]

In Progress [ 3 ]

SCM/JIRA link daemon added a comment - 2013-04-17 02:43

Code changed in jenkins
User: Bruno P. Kinoshita
Path:
pom.xml
src/main/java/hudson/plugins/testlink/TestLinkBuilderDescriptor.java
src/main/java/hudson/plugins/testlink/result/AbstractTestNGResultSeeker.java
src/main/java/hudson/plugins/testlink/result/TestNGClassNameResultSeeker.java
src/main/java/hudson/plugins/testlink/result/TestNGMethodNameDataProviderNameResultSeeker.java
src/main/java/hudson/plugins/testlink/result/TestNGMethodNameResultSeeker.java
src/main/java/hudson/plugins/testlink/result/TestNGSuiteNameResultSeeker.java
src/main/java/hudson/plugins/testlink/testng/Class.java
src/main/java/hudson/plugins/testlink/testng/ParserException.java
src/main/java/hudson/plugins/testlink/testng/Suite.java
src/main/java/hudson/plugins/testlink/testng/Test.java
src/main/java/hudson/plugins/testlink/testng/TestMethod.java
src/main/java/hudson/plugins/testlink/testng/TestNGParser.java
src/main/java/hudson/plugins/testlink/testng/TestNGXmlHandler.java
src/test/java/hudson/plugins/testlink/parser/TestTestNGParser.java
src/test/java/hudson/plugins/testlink/result/issue11894/TestIssue11894.java
src/test/java/hudson/plugins/testlink/util/TestParserException.java
http://jenkins-ci.org/commit/testlink-plugin/69ea27429dcb0c280b04c033de38d2595fb819d5
Log:
[FIXED JENKINS-13821] Replaced TestNG classes by testng-parser, and fixed the data providers behaviour.

Compare: https://github.com/jenkinsci/testlink-plugin/compare/a1f6cb12788f...69ea27429dcb

SCM/JIRA link daemon added a comment - 2013-04-17 02:43 Code changed in jenkins User: Bruno P. Kinoshita Path: pom.xml src/main/java/hudson/plugins/testlink/TestLinkBuilderDescriptor.java src/main/java/hudson/plugins/testlink/result/AbstractTestNGResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGClassNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGMethodNameDataProviderNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGMethodNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGSuiteNameResultSeeker.java src/main/java/hudson/plugins/testlink/testng/Class.java src/main/java/hudson/plugins/testlink/testng/ParserException.java src/main/java/hudson/plugins/testlink/testng/Suite.java src/main/java/hudson/plugins/testlink/testng/Test.java src/main/java/hudson/plugins/testlink/testng/TestMethod.java src/main/java/hudson/plugins/testlink/testng/TestNGParser.java src/main/java/hudson/plugins/testlink/testng/TestNGXmlHandler.java src/test/java/hudson/plugins/testlink/parser/TestTestNGParser.java src/test/java/hudson/plugins/testlink/result/issue11894/TestIssue11894.java src/test/java/hudson/plugins/testlink/util/TestParserException.java http://jenkins-ci.org/commit/testlink-plugin/69ea27429dcb0c280b04c033de38d2595fb819d5 Log: [FIXED JENKINS-13821] Replaced TestNG classes by testng-parser, and fixed the data providers behaviour. Compare: https://github.com/jenkinsci/testlink-plugin/compare/a1f6cb12788f...69ea27429dcb

SCM/JIRA link daemon made changes - 2013-04-17 02:43

Resolution		Fixed [ 1 ]
Status	In Progress [ 3 ]	Resolved [ 5 ]

Bruno P. Kinoshita added a comment - 2013-04-17 02:44

Fixed in testng-parser (added to the project as dependency too).

https://github.com/tupilabs/testng-parser/issues/1

Bruno P. Kinoshita added a comment - 2013-04-17 02:44 Fixed in testng-parser (added to the project as dependency too). https://github.com/tupilabs/testng-parser/issues/1

Bruno P. Kinoshita added a comment - 2013-04-17 03:05

Did few tests and everything seems to be working, no regression in other parts of the system. But we did change the old behaviour. This will be included in the next release.

My intention was to release it last Sunday, but I couldn't. Will try to cut a new release tomorrow or Thursday night. Feel free to ping me if I don't release it and take too long to provide any feedback.

All the best, Bruno

Bruno P. Kinoshita added a comment - 2013-04-17 03:05 Did few tests and everything seems to be working, no regression in other parts of the system. But we did change the old behaviour. This will be included in the next release. My intention was to release it last Sunday, but I couldn't. Will try to cut a new release tomorrow or Thursday night. Feel free to ping me if I don't release it and take too long to provide any feedback. All the best, Bruno

Bruno P. Kinoshita added a comment - 2013-04-17 22:05

Fixed in 3.4

Bruno P. Kinoshita added a comment - 2013-04-17 22:05 Fixed in 3.4

Bruno P. Kinoshita made changes - 2013-04-17 22:05

Status

Resolved [ 5 ]

Closed [ 6 ]

R. Tyler Croy made changes - 2016-07-25 23:59

Workflow

JNJira [ 144289 ]

JNJira + In-Review [ 205865 ]

People

Assignee:: Bruno P. Kinoshita

Reporter:: Denis Veselovskiy

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 2012-05-18 08:28

Updated:: 2013-04-17 22:05

Resolved:: 2013-04-17 02:43