Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13821

When test uses DataProvider it mark in TestLink only by last result

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • testlink-plugin
    • None
    • win7, Jenkins ver. 1.452, TestLink 1.9.3, Jenkins TestLink Plugin 3.1.2

    Description

      I have such test

      @Test(dataProvider = "validateFieldBase", dataProviderClass = CampaignData.class)
      public void validateNameField(String campaignName)

      { ... }

      and such DataProvider for it

      @DataProvider
      public static Object[][] validateFieldBase() {
      Object[][] word = {

      {"<script language='JavaScript'> alert('Hello') </script>"}

      ,

      {"ASDGFFLKLKLKlddvddsghdsh"}

      ,

      {"12344557688008853"}

      ,

      {":!@#$%^&*()_."}

      ,

      {"ASDASDasdas-123123211212&^%&^%"}

      ,
      };
      return(word);
      }

      And I have test cause in TestLink mapped by Test Result Seeking Strategies - TestNG Method Name.

      Problem: test cause in TestLink marks failed or passed by the last DataProvider result (in example by value "ASDASDasdas-123123211212&%&%")

      Expected result: if all DataProvider values passed than test cause in Testlink should mark as passed,
      if even one DataProvider value is failed than test cause in Testlink should mark as failed.

      may be its not a bug, but I think it will be a really nice feature.

      P.S. : I tried to use successPercentage=100 but it's uncertainly for me when and for what it should be used.

      Attachments

        Activity

          Removing testng from components

          kinow Bruno P. Kinoshita added a comment - Removing testng from components

          Makes sense to me Let me see what can I do. Maybe we will need a new result seeking strategy.

          kinow Bruno P. Kinoshita added a comment - Makes sense to me Let me see what can I do. Maybe we will need a new result seeking strategy.

          I will cut a release now, and will work on this during the next week Denis. Sorry for the wait.

          kinow Bruno P. Kinoshita added a comment - I will cut a release now, and will work on this during the next week Denis. Sorry for the wait.
          kasheylm Denis Veselovskiy added a comment - - edited

          Thanks )
          I need this feature so much that I started to write my own testng-results.xml parser. And parser runs in @AfterSuite changing testng-results.xml so it marks in TestLink better ))
          But I hope you will do this feature as well

          kasheylm Denis Veselovskiy added a comment - - edited Thanks ) I need this feature so much that I started to write my own testng-results.xml parser. And parser runs in @AfterSuite changing testng-results.xml so it marks in TestLink better )) But I hope you will do this feature as well

          Is any progress ?

          kasheylm Denis Veselovskiy added a comment - Is any progress ?

          Hi Denis,

          I'm working on other issues at moment, sorry for the delay. But I plan to cut a new release of testlink-plugin before August. Could you share your testng-results.xml parser, please? I could use it as requirement for the new code

          Thanks

          kinow Bruno P. Kinoshita added a comment - Hi Denis, I'm working on other issues at moment, sorry for the delay. But I plan to cut a new release of testlink-plugin before August. Could you share your testng-results.xml parser, please? I could use it as requirement for the new code Thanks

          Code changed in jenkins
          User: Bruno P. Kinoshita
          Path:
          pom.xml
          src/main/java/hudson/plugins/testlink/TestLinkBuilderDescriptor.java
          src/main/java/hudson/plugins/testlink/result/AbstractTestNGResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGClassNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGMethodNameDataProviderNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGMethodNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/result/TestNGSuiteNameResultSeeker.java
          src/main/java/hudson/plugins/testlink/testng/Class.java
          src/main/java/hudson/plugins/testlink/testng/ParserException.java
          src/main/java/hudson/plugins/testlink/testng/Suite.java
          src/main/java/hudson/plugins/testlink/testng/Test.java
          src/main/java/hudson/plugins/testlink/testng/TestMethod.java
          src/main/java/hudson/plugins/testlink/testng/TestNGParser.java
          src/main/java/hudson/plugins/testlink/testng/TestNGXmlHandler.java
          src/test/java/hudson/plugins/testlink/parser/TestTestNGParser.java
          src/test/java/hudson/plugins/testlink/result/issue11894/TestIssue11894.java
          src/test/java/hudson/plugins/testlink/util/TestParserException.java
          http://jenkins-ci.org/commit/testlink-plugin/69ea27429dcb0c280b04c033de38d2595fb819d5
          Log:
          [FIXED JENKINS-13821] Replaced TestNG classes by testng-parser, and fixed the data providers behaviour.

          Compare: https://github.com/jenkinsci/testlink-plugin/compare/a1f6cb12788f...69ea27429dcb

          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Bruno P. Kinoshita Path: pom.xml src/main/java/hudson/plugins/testlink/TestLinkBuilderDescriptor.java src/main/java/hudson/plugins/testlink/result/AbstractTestNGResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGClassNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGMethodNameDataProviderNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGMethodNameResultSeeker.java src/main/java/hudson/plugins/testlink/result/TestNGSuiteNameResultSeeker.java src/main/java/hudson/plugins/testlink/testng/Class.java src/main/java/hudson/plugins/testlink/testng/ParserException.java src/main/java/hudson/plugins/testlink/testng/Suite.java src/main/java/hudson/plugins/testlink/testng/Test.java src/main/java/hudson/plugins/testlink/testng/TestMethod.java src/main/java/hudson/plugins/testlink/testng/TestNGParser.java src/main/java/hudson/plugins/testlink/testng/TestNGXmlHandler.java src/test/java/hudson/plugins/testlink/parser/TestTestNGParser.java src/test/java/hudson/plugins/testlink/result/issue11894/TestIssue11894.java src/test/java/hudson/plugins/testlink/util/TestParserException.java http://jenkins-ci.org/commit/testlink-plugin/69ea27429dcb0c280b04c033de38d2595fb819d5 Log: [FIXED JENKINS-13821] Replaced TestNG classes by testng-parser, and fixed the data providers behaviour. Compare: https://github.com/jenkinsci/testlink-plugin/compare/a1f6cb12788f...69ea27429dcb

          Fixed in testng-parser (added to the project as dependency too).

          https://github.com/tupilabs/testng-parser/issues/1

          kinow Bruno P. Kinoshita added a comment - Fixed in testng-parser (added to the project as dependency too). https://github.com/tupilabs/testng-parser/issues/1

          Did few tests and everything seems to be working, no regression in other parts of the system. But we did change the old behaviour. This will be included in the next release.

          My intention was to release it last Sunday, but I couldn't. Will try to cut a new release tomorrow or Thursday night. Feel free to ping me if I don't release it and take too long to provide any feedback.

          All the best, Bruno

          kinow Bruno P. Kinoshita added a comment - Did few tests and everything seems to be working, no regression in other parts of the system. But we did change the old behaviour. This will be included in the next release. My intention was to release it last Sunday, but I couldn't. Will try to cut a new release tomorrow or Thursday night. Feel free to ping me if I don't release it and take too long to provide any feedback. All the best, Bruno

          Fixed in 3.4

          kinow Bruno P. Kinoshita added a comment - Fixed in 3.4

          People

            kinow Bruno P. Kinoshita
            kasheylm Denis Veselovskiy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: