-
Improvement
-
Resolution: Fixed
-
Blocker
-
None
-
-
1119.va_a_5e9068da_d7
The JUnit Result Archiver will fail the build if the test result files that match the filemask do not have a very recent modification time. The error message it prints is this:
"Test reports were found but none of them are new. Did tests run?"
This can be inconvenient in several settings; notably, when the clocks on the master and slave fall out of sync, or when the test results were generated by some process that isn't part of the build itself – maybe they were scp'd from somewhere else, for instance. Or for testing the JUnitResultArchiver itself.
In my opinion, the behavior should be changed such that check-for-recent-modifications should be off by default. Additionally, the user should be able to specify a "recent" threshold via a setting.
- duplicates
-
JENKINS-5393 Jenkins falsely fails builds due to aggressive up-to-date check on junit artifacts
-
- Open
-
- is duplicated by
-
-
- Resolved
-
- is related to
-
-
- Closed
-
-
-
- In Progress
-
- links to
[JENKINS-6268] Make checking for modification time of junit results be configurable and off by default
bshine
created issue -
NetAppBlueDevil
added a comment - I just ran into this because we are using network storage and the storage system time skewed earlier than the server time and the JUnit output files ended up dated earlier than the build. This just happened after a year of running with no issue, so it took a while to track down.
I'm not sure I really understand the use case for skipping files based on time anyway. In my case, this is just a single master build which starts with a clean, so all the files generated by the build are good to process. The system clock issues if they occur are not really relevant for me.
Looking at TestResult.java, this issue is almost resolved in the logic. There is a flag for ignoring the timestamp issues, it needs to be elevated to a project configurable setting. Currently it is defined in that class as
private static final boolean checkTimestamps = true; // TODO: change to System.getProperty
NetAppBlueDevil
added a comment - I just ran into this because we are using network storage and the storage system time skewed earlier than the server time and the JUnit output files ended up dated earlier than the build. This just happened after a year of running with no issue, so it took a while to track down.
I'm not sure I really understand the use case for skipping files based on time anyway. In my case, this is just a single master build which starts with a clean, so all the files generated by the build are good to process. The system clock issues if they occur are not really relevant for me.
Looking at TestResult.java, this issue is almost resolved in the logic. There is a flag for ignoring the timestamp issues, it needs to be elevated to a project configurable setting. Currently it is defined in that class as
private static final boolean checkTimestamps = true; // TODO: change to System.getProperty 
ebann
added a comment - I have many Test jobs and one job that build up the whole result.
As some jobs may take a long time to end, the test-ender job refuses to publish junit results.
18:12:32 Test reports were found but none of them are new. Did tests run?
18:12:32 For example, /tmp/jenkins_slave/workspace/TESTS_ENDER/TEST-Statistic.AllTests.xml is 16 hr old
18:12:32
18:12:32 Build step 'Publier le rapport des résultats des tests JUnit' changed build result to FAILURE
ebann
added a comment - I have many Test jobs and one job that build up the whole result.
As some jobs may take a long time to end, the test-ender job refuses to publish junit results.
18:12:32 Test reports were found but none of them are new. Did tests run?
18:12:32 For example, /tmp/jenkins_slave/workspace/TESTS_ENDER/TEST-Statistic.AllTests.xml is 16 hr old
18:12:32
18:12:32 Build step 'Publier le rapport des résultats des tests JUnit' changed build result to FAILURE 
kutzi
made changes -
| Link | New: This issue duplicates JENKINS-5393 [ JENKINS-5393 ] |
Stanislav Tyurikov
added a comment - I have multi-module maven project, which rebuilds only changed modules. I want to display test results for old modules too. But this issue is a blocker for this.
Stanislav Tyurikov
added a comment - I have multi-module maven project, which rebuilds only changed modules. I want to display test results for old modules too. But this issue is a blocker for this. Stanislav Tyurikov
made changes -
| Priority | Original: Major [ 3 ] | New: Blocker [ 1 ] |
Unreliable clocks on build machines and on file systems which support build machines seem like they will create many more problems than these few that you've detected.
If you have reliable clocks and are still seeing the issue, then you could consider the work around of adding a build step which updates the time stamp of the result files. The Linux command "touch" will update the time stamp of one or more files and similar commands are available for Windows and other platforms.
Radim Kubacki
added a comment - More valid use-case then shifted clocks across build machines is a Gradle build. If I commit a change that has no impact on tests than Gradle skips their execution. But Jenkins tries to be smarter and marks the build as failed. See a longer version with workarounds at http://www.practicalgradle.org/blog/2011/06/incremental-tests-with-jenkins/
Sounds more like a bug than RFE to me.
Radim Kubacki
added a comment - More valid use-case then shifted clocks across build machines is a Gradle build. If I commit a change that has no impact on tests than Gradle skips their execution. But Jenkins tries to be smarter and marks the build as failed. See a longer version with workarounds at http://www.practicalgradle.org/blog/2011/06/incremental-tests-with-jenkins/
Sounds more like a bug than RFE to me. 
Matthias Heil
added a comment - Seems to be that this kind of feature is not wanted in the core, and that users should switch to the xunit plugin to publish junit results.
Matthias Heil
added a comment - Seems to be that this kind of feature is not wanted in the core, and that users should switch to the xunit plugin to publish junit results. 
I have to use Hudson on a virtual machine on which clock behaves in the following way (less than a second between successive date invocations):
[pcwindci] /etc/sysconfig > date
Wed Nov 17 09:37:23 CET 2010
[pcwindci] /etc/sysconfig > date
Wed Nov 17 09:34:46 CET 2010
[pcwindci] /etc/sysconfig > date
Wed Nov 17 09:37:23 CET 2010
This makes using Hudson impossible in my environment. I know my setup and would like to completely disable the check for modifications.