I just ran into this because we are using network storage and the storage system time skewed earlier than the server time and the JUnit output files ended up dated earlier than the build. This just happened after a year of running with no issue, so it took a while to track down.
I'm not sure I really understand the use case for skipping files based on time anyway. In my case, this is just a single master build which starts with a clean, so all the files generated by the build are good to process. The system clock issues if they occur are not really relevant for me.
Looking at TestResult.java, this issue is almost resolved in the logic. There is a flag for ignoring the timestamp issues, it needs to be elevated to a project configurable setting. Currently it is defined in that class as
private static final boolean checkTimestamps = true; // TODO: change to System.getProperty