Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7970

If all JUnit tests are skipped, the build fails

      When all JUnit tests are skipped, the JUnit result archiver considers the build to have failed, since pass count and fail count are both zero.

      I believe that the attached patch would fix the problem, though I haven't succeeded to build it and test it.

          [JENKINS-7970] If all JUnit tests are skipped, the build fails

          Richard Bondi added a comment -

          Can't this just be made an option, a checkbox to optionally mark a build as not failed if all tests were skipped?

          Here's an example use case where the project should not fail if there are no tests.

          I have to run integration tests in project A against an unreliable customer's system; I use xslt in a separate project B to filter out only project A tests that failed due to that customer system, by marking all other tests as skipped. When there are no tests after filtering, project B contains no tests, indicating merely that the customer's system behaved for once. Currently Jenkins says such builds are failures; obviously in this use case the opposite is true.

          Richard Bondi added a comment - Can't this just be made an option, a checkbox to optionally mark a build as not failed if all tests were skipped? Here's an example use case where the project should not fail if there are no tests. I have to run integration tests in project A against an unreliable customer's system; I use xslt in a separate project B to filter out only project A tests that failed due to that customer system, by marking all other tests as skipped. When there are no tests after filtering, project B contains no tests, indicating merely that the customer's system behaved for once. Currently Jenkins says such builds are failures; obviously in this use case the opposite is true.

          Dave Hunt added a comment -

          Any update on this issue? The patch looks simple enough, and seems like the right thing to do. I'm also open to the suggestion of a checkbox (preferably in the main configuration) to allow 100% skip to be considered a pass if there is still reluctance to merge the patch as it is.

          Dave Hunt added a comment - Any update on this issue? The patch looks simple enough, and seems like the right thing to do. I'm also open to the suggestion of a checkbox (preferably in the main configuration) to allow 100% skip to be considered a pass if there is still reluctance to merge the patch as it is.

          Todd Edwards added a comment -

          I would also like to see this patch merged.

          Todd Edwards added a comment - I would also like to see this patch merged.

          kutzi added a comment -

          Agree, it's absolutely reasonable to expect the build to only fail if there are really no test results - i.e. also no skipped tests.

          kutzi added a comment - Agree, it's absolutely reasonable to expect the build to only fail if there are really no test results - i.e. also no skipped tests.

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java
          core/src/main/java/hudson/tasks/junit/TestResult.java
          http://jenkins-ci.org/commit/jenkins/9bc7e21dc0bd679e7b6506f264079630493237d4
          Log:
          [FIXED JENKINS-7970] JUnit result archiver should only fail builds if there are really no results - i.e. also no skipped tests.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: changelog.html core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java core/src/main/java/hudson/tasks/junit/TestResult.java http://jenkins-ci.org/commit/jenkins/9bc7e21dc0bd679e7b6506f264079630493237d4 Log: [FIXED JENKINS-7970] JUnit result archiver should only fail builds if there are really no results - i.e. also no skipped tests.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2427
          [FIXED JENKINS-7970] JUnit result archiver should only fail builds if there are really no results - i.e. also no skipped tests. (Revision 9bc7e21dc0bd679e7b6506f264079630493237d4)

          Result = SUCCESS
          Christoph Kutzinski : 9bc7e21dc0bd679e7b6506f264079630493237d4
          Files :

          • changelog.html
          • core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java
          • core/src/main/java/hudson/tasks/junit/TestResult.java

          dogfood added a comment - Integrated in jenkins_main_trunk #2427 [FIXED JENKINS-7970] JUnit result archiver should only fail builds if there are really no results - i.e. also no skipped tests. (Revision 9bc7e21dc0bd679e7b6506f264079630493237d4) Result = SUCCESS Christoph Kutzinski : 9bc7e21dc0bd679e7b6506f264079630493237d4 Files : changelog.html core/src/main/java/hudson/tasks/junit/JUnitResultArchiver.java core/src/main/java/hudson/tasks/junit/TestResult.java

          Todd Edwards added a comment -

          Thank you so much!

          Todd Edwards added a comment - Thank you so much!

          Joel Beaudoin added a comment -

          Any estimate as to when this will make it into a release package? I don't see it in the changelog yet.

          Joel Beaudoin added a comment - Any estimate as to when this will make it into a release package? I don't see it in the changelog yet.

          kutzi added a comment - - edited

          Will be in 1.511. Look under 'upcoming changes': http://jenkins-ci.org/changelog

          kutzi added a comment - - edited Will be in 1.511. Look under 'upcoming changes': http://jenkins-ci.org/changelog

          Joel Beaudoin added a comment -

          Thanks mate. I missed the 'upcoming changes' link.

          Joel Beaudoin added a comment - Thanks mate. I missed the 'upcoming changes' link.

            kutzi kutzi
            mhenoch mhenoch
            Votes:
            4 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: