Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26000

Invalid processing of suite-files in testng

XMLWordPrintable

      There is a problem while retrieving results for testng when the testng XML file contains the constructions <suite-file path="./suite1.xml"/> with multiple suites.
      Here are the details.

      At first, this is the example that works OK:
      <suite name="TAM TestSuite">
      <test name="MRCT1100"><classes><class name="com.ingenico.tam.testsuite.merchants.MRCT1101"></class></classes></test>
      <test name="SCNWZD1100"><classes><class name="com.ingenico.tam.testsuite.scenariowizard.SCNWZD1100"></class></classes></test>
      </suite>
      Here there are two tests, the first one is successful, the second one fails. In this format everything is OK, see the screenshot jenkins-ok.png.
      I attach also the file testng-results.ok for this case.

      Next, I do the same tests java with the same confuguration of testlink and plugin. I replace only the testng XML file by three files (attached): All.xml, TAMMerchants.xml, TAMScenarioWizard.xml.
      The tests are the same, but they are organized in separate files:
      <suite name="TAM TestSuite" parallel="false" thread-count="1" preserve-order="true">
      <suite-files>
      <suite-file path="./TAMMerchants.xml"/>
      <suite-file path="./TAMScenarioWizard.xml"/>
      </suite-files>
      </suite>
      In this case both tests are executed by testng, but the problem appears when the plugin retrieves the result, see the screenshot jenkins-ko.PNG attached.
      The test from TAMScenarioWizard.xml is Not Run.
      However it was executed, see the file testng-results.ko attached.

      In TestLink also:

      • for the first case the test MRCT1100 is successful, and SCNWZD1100 failes
      • in the second case the test MRCT1100 is successful, and SCNWZD1100 Not Run

      It seems that there is a problem when processing testng-results.xml: in first case it contains only one suite ,and it works, and in the seconde case there are three suite tags, and only one result is retrieved.

      I join also the file config.xml

        1. All.xml
          0.3 kB
        2. config.xml
          4 kB
        3. jenkins-ko.PNG
          jenkins-ko.PNG
          25 kB
        4. jenkins-ok.PNG
          jenkins-ok.PNG
          24 kB
        5. ko62.xml
          18 kB
        6. ok63.xml
          21 kB
        7. TAMMerchants.xml
          0.8 kB
        8. TAMScenarioWizard.xml
          1.0 kB
        9. testng-results.ko
          3 kB
        10. testng-results.ok
          4 kB

            kinow Bruno P. Kinoshita
            ab_ing A B
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: