Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-17941

Parse errors with Git's TAP test suite

    XMLWordPrintable

Details

    Description

      I'm running tests on Git for Windows that I compile myself. To do so I'm using the following command line insode the git/t directory:

      $ prove -j 5 -a test-results.tar.gz ./t[0-9]*.sh

      Please find the generated archive attached.

      I'm later unpacking the archive and using this iin the TAP plugin's "Test results" field:

      test-results/*.sh

      This results in the following page:

      https://qa.nest-initiative.org/view/msysGit/job/msysgit-mingwGitDevEnv-test/lastCompletedBuild/tapResults/

      As you can see, for all failed (sub-)tests there are lines starting with "#" in the table instead of the number of the failing test and the other columns. As a result, the summary "8641 tests, 8204 ok, 1 not ok, 1 skipped, 0 Bail Out!." at the top of the page is wrong. I've manually extracted the correct summary to this page in the "The following test(s) failed" section:

      https://qa.nest-initiative.org/view/msysGit/job/msysgit-mingwGitDevEnv-test/lastCompletedBuild/

      As you can see, there are way more tests failing than the 1 test that the TAP plugin lists in "TAP Test Result (1 failure / ±0)" on the same page.

      Attachments

        Issue Links

          Activity

            > It was Tap Test Result, and you had to click on a Click "here" link to go to the page. Now there's a link (mimicking the behaviour of the other test result), but the name was changed to Extended TAP Test Results. What do you think?

            Thanks, I think that's much clearer than before. But still, for my personal taste, only having what is now called "TAP Extended Test Results" would be enough, and you could get rid of generating the "TAP Test Results" page at all. I like that colorful table much better than that other expandable / collapsible table

            sschuberth Sebastian Schuberth added a comment - > It was Tap Test Result, and you had to click on a Click "here" link to go to the page. Now there's a link (mimicking the behaviour of the other test result), but the name was changed to Extended TAP Test Results. What do you think? Thanks, I think that's much clearer than before. But still, for my personal taste, only having what is now called "TAP Extended Test Results" would be enough, and you could get rid of generating the "TAP Test Results" page at all. I like that colorful table much better than that other expandable / collapsible table

            > Let me know if you need anything else, feel free to re-open it later if I missed anything, or report new issues.

            Thanks a lot for your responsiveness and all the fixes! I'll test your improvements as soon as your plugin has a new release that I can install via Jenkins' update mechanism (I do not have admin access on that Jenkins so I need to wait for an "official" update of your plugin). I'll file new tickets in case I find more issues because this one is already getting a bit crowded

            Thanks again!

            sschuberth Sebastian Schuberth added a comment - > Let me know if you need anything else, feel free to re-open it later if I missed anything, or report new issues. Thanks a lot for your responsiveness and all the fixes! I'll test your improvements as soon as your plugin has a new release that I can install via Jenkins' update mechanism (I do not have admin access on that Jenkins so I need to wait for an "official" update of your plugin). I'll file new tickets in case I find more issues because this one is already getting a bit crowded Thanks again!

            No problem, keep posting interesting issues, and we'll make TAP plug-in even more interesting :o)

            I'll cut a new release tonight from home, but I'm afraid some issues will take longer to fix. So I'll work on Jenkins plug-ins till next Friday, where we can cut another release of tap-plugin, if needed. Release early, release often.

            Cheers

            kinow Bruno P. Kinoshita added a comment - No problem, keep posting interesting issues, and we'll make TAP plug-in even more interesting :o) I'll cut a new release tonight from home, but I'm afraid some issues will take longer to fix. So I'll work on Jenkins plug-ins till next Friday, where we can cut another release of tap-plugin, if needed. Release early, release often. Cheers

            > Release early, release often.

            Exactly Your efforts are highly appreciated!

            sschuberth Sebastian Schuberth added a comment - > Release early, release often. Exactly Your efforts are highly appreciated!

            Fixed in 1.11 (should be available through the update site in the next minute/hours)

            kinow Bruno P. Kinoshita added a comment - Fixed in 1.11 (should be available through the update site in the next minute/hours)

            People

              kinow Bruno P. Kinoshita
              sschuberth Sebastian Schuberth
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: