There is a problem while retrieving results for testng when the testng XML file contains the constructions <suite-file path="./suite1.xml"/> with multiple suites.
      Here are the details.

      At first, this is the example that works OK:
      <suite name="TAM TestSuite">
      <test name="MRCT1100"><classes><class name="com.ingenico.tam.testsuite.merchants.MRCT1101"></class></classes></test>
      <test name="SCNWZD1100"><classes><class name="com.ingenico.tam.testsuite.scenariowizard.SCNWZD1100"></class></classes></test>
      </suite>
      Here there are two tests, the first one is successful, the second one fails. In this format everything is OK, see the screenshot jenkins-ok.png.
      I attach also the file testng-results.ok for this case.

      Next, I do the same tests java with the same confuguration of testlink and plugin. I replace only the testng XML file by three files (attached): All.xml, TAMMerchants.xml, TAMScenarioWizard.xml.
      The tests are the same, but they are organized in separate files:
      <suite name="TAM TestSuite" parallel="false" thread-count="1" preserve-order="true">
      <suite-files>
      <suite-file path="./TAMMerchants.xml"/>
      <suite-file path="./TAMScenarioWizard.xml"/>
      </suite-files>
      </suite>
      In this case both tests are executed by testng, but the problem appears when the plugin retrieves the result, see the screenshot jenkins-ko.PNG attached.
      The test from TAMScenarioWizard.xml is Not Run.
      However it was executed, see the file testng-results.ko attached.

      In TestLink also:

      • for the first case the test MRCT1100 is successful, and SCNWZD1100 failes
      • in the second case the test MRCT1100 is successful, and SCNWZD1100 Not Run

      It seems that there is a problem when processing testng-results.xml: in first case it contains only one suite ,and it works, and in the seconde case there are three suite tags, and only one result is retrieved.

      I join also the file config.xml

        1. All.xml
          0.3 kB
        2. config.xml
          4 kB
        3. jenkins-ko.PNG
          jenkins-ko.PNG
          25 kB
        4. jenkins-ok.PNG
          jenkins-ok.PNG
          24 kB
        5. ko62.xml
          18 kB
        6. ok63.xml
          21 kB
        7. TAMMerchants.xml
          0.8 kB
        8. TAMScenarioWizard.xml
          1.0 kB
        9. testng-results.ko
          3 kB
        10. testng-results.ok
          4 kB

          [JENKINS-26000] Invalid processing of suite-files in testng

          A B added a comment - - edited

          The file ko62.xml is the XML RPC exchange between the plugin and testlink for build 62 where the second test is not run

          A B added a comment - - edited The file ko62.xml is the XML RPC exchange between the plugin and testlink for build 62 where the second test is not run

          A B added a comment -

          The file ok63.xml is the XML RPC exchange between the plugin and testlink for build 63 where the second test is present

          A B added a comment - The file ok63.xml is the XML RPC exchange between the plugin and testlink for build 63 where the second test is present

          A B added a comment -

          A added two more files: ko62.xml and ok63.xml containing the XML RPC messages exchanged between the plugin and testlink.
          The main difference is at the end:

          • ko62.xml: the plugin sends back only one test case result
          • ok63.xml: the plugin sends back two test case results

          the builds 62 and 63 use the same set of test cases, but for 62 they are structured into suites, and in 63 they are in the same suite.

          A B added a comment - A added two more files: ko62.xml and ok63.xml containing the XML RPC messages exchanged between the plugin and testlink. The main difference is at the end: ko62.xml: the plugin sends back only one test case result ok63.xml: the plugin sends back two test case results the builds 62 and 63 use the same set of test cases, but for 62 they are structured into suites, and in 63 they are in the same suite.

          Hello.

          I have exactly the same problem described below
          Jenkins Version : 1.613
          Plugin Version : 3.10

          A correction time is it planned?

          aurelien dubois added a comment - Hello. I have exactly the same problem described below Jenkins Version : 1.613 Plugin Version : 3.10 A correction time is it planned?

            kinow Bruno P. Kinoshita
            ab_ing A B
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: