-
Bug
-
Resolution: Fixed
-
Critical
-
None
-
Hudson 1.351, Maven 2.2.1, Surefire 2.5, TestNG 5.9
Jenkins 1.420
Occasionally we have some test which lets Surefire fail hard which Hudson doesn't recognize. I.e. the tests are not executed, but the build result is still success.
E.g. we had a test which failed with a NoSuchFieldException in the constructor of the test class:
Running TestSuite
log4j:WARN No appenders could be found for logger (...).
log4j:WARN Please initialize the log4j system properly.
org.apache.maven.surefire.booter.SurefireExecutionException:
Cannot instantiate class XyzTest; nested exception is org.testng.TestNGException:
Cannot instantiate class XyzTest
org.testng.TestNGException:
Cannot instantiate class XyzTest
at org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:35)
at org.testng.internal.ClassHelper.createInstance(ClassHelper.java:330)
at org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:62)
at org.testng.internal.ClassImpl.getInstances(ClassImpl.java:81)
at org.testng.internal.TestNGClassFinder.<init>(TestNGClassFinder.java:114)
at org.testng.TestRunner.initMethods(TestRunner.java:289)
at org.testng.TestRunner.init(TestRunner.java:235)
at org.testng.TestRunner.init(TestRunner.java:197)
at org.testng.TestRunner.<init>(TestRunner.java:141)
at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:488)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:250)
at org.testng.SuiteRunner.run(SuiteRunner.java:204)
at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:877)
at org.testng.TestNG.runSuitesLocally(TestNG.java:842)
at org.testng.TestNG.run(TestNG.java:751)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.testng.internal.ObjectFactoryImpl.newInstance(ObjectFactoryImpl.java:26)
... 23 more
Caused by: java.lang.NoSuchFieldException: cipherService
at java.lang.Class.getDeclaredField(Class.java:1882)
at XyzTest.<init>(XyzTest.java:91)
... 28 more
[ERROR] There are test failures.
- is duplicated by
-
JENKINS-9770 Maven 2/3 job not failing when tests cannot be run
-
- Resolved
-
[JENKINS-6700] Failure in test class constructor or @Before method is not reported (was: Maven plugin doesn't set build to unstable on SurefireExecutionException)
Link |
New:
This issue duplicates |
Link |
New:
This issue is related to |
Component/s | New: junit [ 15499 ] | |
Component/s | Original: maven2 [ 15487 ] | |
Environment | Original: Hudson 1.351, Maven 2.2.1, Surefire 2.5, TestNG 5.9 |
New:
Hudson 1.351, Maven 2.2.1, Surefire 2.5, TestNG 5.9 Jenkins 1.420 |
Summary | Original: Maven plugin doesn't set build to unstable on SurefireExecutionException | New: Failure in test class constructor or @Before method is not reported (was: Maven plugin doesn't set build to unstable on SurefireExecutionException) |
Assignee | New: kutzi [ kutzi ] |
Resolution | New: Fixed [ 1 ] | |
Status | Original: Open [ 1 ] | New: Resolved [ 5 ] |
This problem is broader than the problem description indicates. It's nothing specifically to do with Maven. Here's my version of the problem:
We're running some very standard Java JUnit4 tests, and using the "Publish JUnit test result report" option. I only just noticed we have failing tests that are never reported in the build results.
The problem is with some tests that have a @Before method, which gets an exception. The JUnit runner writes an XML report file in the expected place, with the error information, but it appears that Jenkins does not do anything useful with it. The XML file looks like:
I presume that the absence of any <testcase classname="blah" name="blah" time="1.249" /> means that the report file is simply ignored.
I'm somewhat concerned that we have had tests failing for some time, with no indication.