• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • maven-plugin
    • None

      The current implementation of Hudson's Maven support needlessly requires the use of maven-surefire-plugin for executing tests. If an alternative test runner plugin is used, Hudson's maven-plugin will not recognize test results, meaning that UNSTABLE builds are wrongly considered to be FAILED builds, and also that build status does not include test result reports.

      This behavior is due to the fact that the SurefireArchiver class includes conditional logic that insists that the test runner plugin is Surefire. If this conditional check is removed, then test results are correctly found and interpreted, and unstable builds are no longer considered failed builds.

      I've attached a simple patch that demonstrates this. Even though the contents of the patch may not be the correct fix, it at least illustrates the nature of the problem.

          [JENKINS-8334] maven-plugin insists on Surefire to run tests

          jvoegele created issue -

          timeck added a comment -

          to be sure, using a different other than surefire for running tests is likely a very odd exception, but we are indeed doing that at the moment. What would be nice is way to disable the check via config, or a way to allow customization of the MojoInfo inspection so we could cover our plugin too.

          timeck added a comment - to be sure, using a different other than surefire for running tests is likely a very odd exception, but we are indeed doing that at the moment. What would be nice is way to disable the check via config, or a way to allow customization of the MojoInfo inspection so we could cover our plugin too.

          kutzi added a comment -

          What kind of plugin are you using for the tests?
          There are already additional plugins supported (maven-junit-plugin, flexmojos-maven-plugin), maybe we can support your plugin, too.

          kutzi added a comment - What kind of plugin are you using for the tests? There are already additional plugins supported (maven-junit-plugin, flexmojos-maven-plugin), maybe we can support your plugin, too.

          timeck added a comment -

          our plugin is really just a big hack and just used internally in our projects. It isn't likely ever going to become another "standard" test runner or anything.

          We're essentially subclassing the standard surefire one so we can do some programmatic adjustments to the test classpath. I'd love to find another way to accomplish this but this is the only way we've found to run our special test setups effectively

          timeck added a comment - our plugin is really just a big hack and just used internally in our projects. It isn't likely ever going to become another "standard" test runner or anything. We're essentially subclassing the standard surefire one so we can do some programmatic adjustments to the test classpath. I'd love to find another way to accomplish this but this is the only way we've found to run our special test setups effectively

          hhuynh added a comment -

          I'm with timeck. If you could add our plugin into the list of recognized test plugins for now, that could work also:

          <groupId>org.terracotta.maven.plugins</groupId>
          <artifactId>toolkit-resolver-plugin</artifactId>

          hhuynh added a comment - I'm with timeck. If you could add our plugin into the list of recognized test plugins for now, that could work also: <groupId>org.terracotta.maven.plugins</groupId> <artifactId>toolkit-resolver-plugin</artifactId>

          kutzi added a comment -

          I don't think it's the right way to add every in-house plugin to the list of recognized plugins.
          Also, I don't want to remove the check altogether, as I suppose it was added with a purpose (which I don't know)

          Maybe there are some custom jenkins plugins which could help you? Have you looked for them? I'm thinking about something like the xUnit plugin. Not sure if it can help in your case.

          Sorry, that I cannot do more fore your.

          kutzi added a comment - I don't think it's the right way to add every in-house plugin to the list of recognized plugins. Also, I don't want to remove the check altogether, as I suppose it was added with a purpose (which I don't know) Maybe there are some custom jenkins plugins which could help you? Have you looked for them? I'm thinking about something like the xUnit plugin. Not sure if it can help in your case. Sorry, that I cannot do more fore your.

          hhuynh added a comment -

          There was no comment as to why the check needed in the code but it seems like the author was making sure he's parsing JUnit reports format.

          I understand your point but if there's away to disable the check or a configuration to add a test plugin to the recognized plugin list, it could be a compromise

          hhuynh added a comment - There was no comment as to why the check needed in the code but it seems like the author was making sure he's parsing JUnit reports format. I understand your point but if there's away to disable the check or a configuration to add a test plugin to the recognized plugin list, it could be a compromise

          hhuynh added a comment - - edited

          Could you please add this option to SurefireArchiver.postExecute() bypass the check?

          -        if (!isSurefireTest(mojo)) return true;
          +        if (!Boolean.getBoolean("jenkins.ignore.surefire.check")) {
          +    	      if (!isSurefireTest(mojo)) return true;
          +        }
          

          hhuynh added a comment - - edited Could you please add this option to SurefireArchiver.postExecute() bypass the check? - if (!isSurefireTest(mojo)) return true ; + if (! Boolean .getBoolean( "jenkins.ignore.surefire.check" )) { + if (!isSurefireTest(mojo)) return true ; + }

          This is super important for supporting alternate JVM languages in maven builds. The xUnit plugin plugs into the free-style build, so it doesn't help if your project is configured as a maven build.

          Why not just check for junit xml files after all the plugins have executed?

          Alexander Bertram added a comment - This is super important for supporting alternate JVM languages in maven builds. The xUnit plugin plugs into the free-style build, so it doesn't help if your project is configured as a maven build. Why not just check for junit xml files after all the plugins have executed?
          kutzi made changes -
          Assignee New: kutzi [ kutzi ]

            kutzi kutzi
            jvoegele jvoegele
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: