Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13821

When test uses DataProvider it mark in TestLink only by last result

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • testlink-plugin
    • None
    • win7, Jenkins ver. 1.452, TestLink 1.9.3, Jenkins TestLink Plugin 3.1.2

    Description

      I have such test

      @Test(dataProvider = "validateFieldBase", dataProviderClass = CampaignData.class)
      public void validateNameField(String campaignName)

      { ... }

      and such DataProvider for it

      @DataProvider
      public static Object[][] validateFieldBase() {
      Object[][] word = {

      {"<script language='JavaScript'> alert('Hello') </script>"}

      ,

      {"ASDGFFLKLKLKlddvddsghdsh"}

      ,

      {"12344557688008853"}

      ,

      {":!@#$%^&*()_."}

      ,

      {"ASDASDasdas-123123211212&^%&^%"}

      ,
      };
      return(word);
      }

      And I have test cause in TestLink mapped by Test Result Seeking Strategies - TestNG Method Name.

      Problem: test cause in TestLink marks failed or passed by the last DataProvider result (in example by value "ASDASDasdas-123123211212&%&%")

      Expected result: if all DataProvider values passed than test cause in Testlink should mark as passed,
      if even one DataProvider value is failed than test cause in Testlink should mark as failed.

      may be its not a bug, but I think it will be a really nice feature.

      P.S. : I tried to use successPercentage=100 but it's uncertainly for me when and for what it should be used.

      Attachments

        Activity

          Ascending order - Click to sort in descending order
          View 5 older comments
          kinow Bruno P. Kinoshita added a comment -

          Hi Denis,

          I'm working on other issues at moment, sorry for the delay. But I plan to cut a new release of testlink-plugin before August. Could you share your testng-results.xml parser, please? I could use it as requirement for the new code

          Thanks

          kinow Bruno P. Kinoshita added a comment - Hi Denis, I'm working on other issues at moment, sorry for the delay. But I plan to cut a new release of testlink-plugin before August. Could you share your testng-results.xml parser, please? I could use it as requirement for the new code Thanks
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Bruno P. Kinoshita
          Path:
          pom.xml
          src/main/java/hudson/plugins/testlink/TestLinkBuilderDescriptor.java
          src/main/java/hudson/plugins/testlink/result/AbstractTestNGResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGClassNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGMethodNameDataProviderNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGMethodNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGSuiteNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/testng/Class.java
          src/main/java/hudson/plugins/testlink/testng/ParserException.java
          src/main/java/hudson/plugins/testlink/testng/Suite.java
          src/main/java/hudson/plugins/testlink/testng/Test.java
          src/main/java/hudson/plugins/testlink/testng/TestMethod.java
          src/main/java/hudson/plugins/testlink/testng/TestNGParser.java
          src/main/java/hudson/plugins/testlink/testng/TestNGXmlHandler.java
          src/test/java/hudson/plugins/testlink/parser/TestTestNGParser.java
          src/test/java/hudson/plugins/testlink/result/issue11894/TestIssue11894.java
          src/test/java/hudson/plugins/testlink/util/TestParserException.java
          http://jenkins-ci.org/commit/testlink-plugin/69ea27429dcb0c280b04c033de38d2595fb819d5
          Log:
          [FIXED JENKINS-13821] Replaced TestNG classes by testng-parser, and fixed the data providers behaviour.

          Compare: https://github.com/jenkinsci/testlink-plugin/compare/a1f6cb12788f...69ea27429dcb

          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Bruno P. Kinoshita Path: pom.xml src/main/java/hudson/plugins/testlink/TestLinkBuilderDescriptor.java src/main/java/hudson/plugins/testlink/result/AbstractTestNGResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGClassNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGMethodNameDataProviderNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGMethodNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGSuiteNameResultSeeker.java src/main/java/hudson/plugins/testlink/testng/Class.java src/main/java/hudson/plugins/testlink/testng/ParserException.java src/main/java/hudson/plugins/testlink/testng/Suite.java src/main/java/hudson/plugins/testlink/testng/Test.java src/main/java/hudson/plugins/testlink/testng/TestMethod.java src/main/java/hudson/plugins/testlink/testng/TestNGParser.java src/main/java/hudson/plugins/testlink/testng/TestNGXmlHandler.java src/test/java/hudson/plugins/testlink/parser/TestTestNGParser.java src/test/java/hudson/plugins/testlink/result/issue11894/TestIssue11894.java src/test/java/hudson/plugins/testlink/util/TestParserException.java http://jenkins-ci.org/commit/testlink-plugin/69ea27429dcb0c280b04c033de38d2595fb819d5 Log: [FIXED JENKINS-13821] Replaced TestNG classes by testng-parser, and fixed the data providers behaviour. Compare: https://github.com/jenkinsci/testlink-plugin/compare/a1f6cb12788f...69ea27429dcb
          kinow Bruno P. Kinoshita added a comment -

          Fixed in testng-parser (added to the project as dependency too).

          https://github.com/tupilabs/testng-parser/issues/1

          kinow Bruno P. Kinoshita added a comment - Fixed in testng-parser (added to the project as dependency too). https://github.com/tupilabs/testng-parser/issues/1
          kinow Bruno P. Kinoshita added a comment -

          Did few tests and everything seems to be working, no regression in other parts of the system. But we did change the old behaviour. This will be included in the next release.

          My intention was to release it last Sunday, but I couldn't. Will try to cut a new release tomorrow or Thursday night. Feel free to ping me if I don't release it and take too long to provide any feedback.

          All the best, Bruno

          kinow Bruno P. Kinoshita added a comment - Did few tests and everything seems to be working, no regression in other parts of the system. But we did change the old behaviour. This will be included in the next release. My intention was to release it last Sunday, but I couldn't. Will try to cut a new release tomorrow or Thursday night. Feel free to ping me if I don't release it and take too long to provide any feedback. All the best, Bruno
          kinow Bruno P. Kinoshita added a comment -

          Fixed in 3.4

          kinow Bruno P. Kinoshita added a comment - Fixed in 3.4

          People

            kinow Bruno P. Kinoshita
            kasheylm Denis Veselovskiy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: