-
New Feature
-
Resolution: Unresolved
-
Minor
-
None
I'm using Selenium with Java-Api V1.9.6-1 for automated testing. Used Testlink-Version is 1.9.7.
In the custom_config.inc.php file we changed $tlCfg->results in the following way:
$tlCfg->results['status_code'] = array (
"failed_h" => 'h',
"failed_m" => 'm',
"failed_l" => 'l',
"blocked" => 'b',
"passed" => 'p',
"not_run" => 'n',
"not_available" => 'x',
"unknown" => 'u',
"all" => 'a'
);
Now I am not able to write failed testcase results via api to the database, because of the enumeration object used for the execution that only allows FAILED for the result. Is there any way to write the custom result status via api to the testlink-database?
So after modifying
testlink/lib/api/xmlrpc/v1
**
- An array containing strings for valid statuses
- Will be initialized using user configuration via config_get()
*/
public $statusCode;
public $codeStatus;
/**
- Constructor sets up the IXR_Server and db connection
*/
public function __construct($callbacks = array())
{
$this->dbObj = new database(DB_TYPE);
$this->dbObj->db->SetFetchMode(ADODB_FETCH_ASSOC);
$this->_connectToDB();
$this->tcaseMgr=new testcase($this->dbObj);
$this->tprojectMgr=new testproject($this->dbObj);
$this->tplanMgr = new testplan($this->dbObj);
$this->tplanMetricsMgr = new tlTestPlanMetrics($this->dbObj);
$this->reqSpecMgr=new requirement_spec_mgr($this->dbObj);
$this->reqMgr=new requirement_mgr($this->dbObj);
$this->tables = $this->tcaseMgr->getDBTables();
$resultsCfg = config_get('results');
foreach($resultsCfg['status_label_for_exec_ui'] as $key => $label )
if( isset($this->statusCode['not_run']) )
{ unset($this->statusCode['not_run']); }$this->codeStatus=array_flip($this->statusCode);
$this->initMethodYellowPages();
$this->methods += $callbacks;
$this->IXR_Server($this->methods);
}
You will then need to have a mean to define custom result status from the plugin, e.g. only execute "failed_h" => 'h', "failed_m" => 'm', "failed_l" => 'l' test cases execution status while also able to write back these custom results
[JENKINS-19369] Testlink "test error" results not selectable
Postponed due to the need to implement new feature in TestLink.
Andrew Sim
added a comment - Sure i agreed this is a testlink internal api issue, i 've open an issue in testlink Mantis tracker regarding the custom results via api issue, i also notice it also affecting testlink internal test report via its direct URL link using api also.
Andrew Sim
added a comment - Sure i agreed this is a testlink internal api issue, i 've open an issue in testlink Mantis tracker regarding the custom results via api issue, i also notice it also affecting testlink internal test report via its direct URL link using api also. >i also notice it also affecting testlink internal test report via its direct URL link using api also.
Nice catch. Let's work on this later when the issue is resolved in TestLink.
Thanks Andrew
Andrew Sim
added a comment - $tlCfg->results['status_label_for_exec_ui'] = array (
"my_status" => 'x'
);
I 've tried to configure custom "status" via testlink, so via testlink api "getLastExecutionResult" will return the custom status and write back to testlink via "reportTCResult" with the custom "status"
So in Jenkins , there must be a way user can key in the keyword for the custom "status" for jenkins to retrieve this status from testlink or update the result with this status as well
Andrew Sim
added a comment - $tlCfg->results ['status_label_for_exec_ui'] = array (
"my_status" => 'x'
);
I 've tried to configure custom "status" via testlink, so via testlink api "getLastExecutionResult" will return the custom status and write back to testlink via "reportTCResult" with the custom "status"
So in Jenkins , there must be a way user can key in the keyword for the custom "status" for jenkins to retrieve this status from testlink or update the result with this status as well Andrew Sim
added a comment - you may refer to testlink mantis for the reported issue 0005883 Andrew Sim
added a comment - I don’t think changing testlink Jenkins plugin can fulfill this requirement.
1. All the 4 Execution Status “Not Run”, “Passed”, “Failed”, “Blocked” are from ExecutionStatus class, in Jenkins plugin whaich are currently now the only selectable execution status that can be put on Jenkins. But this ExecutionStatus class itself is not in testilnk Jenkins plugin source file. It’s in testlink-java-api file, which is included by maven, the following is from pom.xml. There’s no source file.
<dependency>
<groupId>br.eti.kinoshita</groupId>
<artifactId>testlink-java-api</artifactId>
<version>1.9.7-0</version>
</dependency>
2. In TestLinkBuilder class, method perform is the method which is called when a Jenkins job is executed, it eventually calls this method to get result from testlink server.
final TestCase[] testCases = this.api.getTestCasesForTestPlan(getTestPlan().getId(), null, null, null, null, null, null, executionStatus, // execute status
ExecutionType.AUTOMATED, Boolean.TRUE, TestCaseDetails.FULL);
This method getTestCasesForTestPlan is from TestLinkAPI class, which is from testlink-java-api, which we don’t have source file.
What we can do with this testlink Jenkins plugin source file is to put “Test Error” in the list, no more. To fulfill your other requirement, we need to analyze and modify testlink-java-api source files. That means more source files to analyze and work on.
It would be helpful, if someone is able to provide the source for the Testlink java api in this case
Andrew Sim
added a comment - I don’t think changing testlink Jenkins plugin can fulfill this requirement.
1. All the 4 Execution Status “Not Run”, “Passed”, “Failed”, “Blocked” are from ExecutionStatus class, in Jenkins plugin whaich are currently now the only selectable execution status that can be put on Jenkins. But this ExecutionStatus class itself is not in testilnk Jenkins plugin source file. It’s in testlink-java-api file, which is included by maven, the following is from pom.xml. There’s no source file.
<dependency>
<groupId>br.eti.kinoshita</groupId>
<artifactId>testlink-java-api</artifactId>
<version>1.9.7-0</version>
</dependency>
2. In TestLinkBuilder class, method perform is the method which is called when a Jenkins job is executed, it eventually calls this method to get result from testlink server.
final TestCase[] testCases = this.api.getTestCasesForTestPlan(getTestPlan().getId(), null, null, null, null, null, null, executionStatus, // execute status
ExecutionType.AUTOMATED, Boolean.TRUE, TestCaseDetails.FULL);
This method getTestCasesForTestPlan is from TestLinkAPI class, which is from testlink-java-api, which we don’t have source file.
What we can do with this testlink Jenkins plugin source file is to put “Test Error” in the list, no more. To fulfill your other requirement, we need to analyze and modify testlink-java-api source files. That means more source files to analyze and work on.
It would be helpful, if someone is able to provide the source for the Testlink java api in this case Hi @Andrew, I'm the maintainer of testlink-java-api. Perhaps I can help here? I've recently started sifting the issues to prepare a new release. I will re-read this issue again later, but 1.9.7 changed some internal functions (some called by getTestCasesForTestPlan() XML-RPC API method) that caused lots of bugs in Jenkins TestLink Plug-in :o/
I'll be working on a new version in the next days, let's see if we can get it fixed.
Hi Andrew!
Thanks for reporting the issue! See comments below :o)
> I'm using Selenium with Java-Api V1.9.6-1 for automated testing. Used Testlink-Version is 1.9.7.
>So after modifying
>testlink/lib/api/xmlrpc/v1
This issue must be first addressed in TestLink XML-RPC API.
>You will then need to have a mean to define custom result status from the plugin, e.g. only execute "failed_h" => 'h', "failed_m" => 'm', "failed_l" => 'l' test cases execution status while also able to write back these custom results
Sure thing. It will be a great new feature to be added to the plug-in.
I'm stopping the work on this issue until someone take care to fill in issues in TestLink, and those issues get fixed there. Then I can either merge a pull request or work on this feature in a future development cycle.
Thanks
Bruno