After a report is published there is a java process holding a handle to the report configuration file. Thus this file can't be cleaned up by Jenkins (either though the workspace wipe process, or the Workspace Cleanup plugin This has unpredictable and confusing results, as the next time a build fails, a report is still generated using the previous successful build configuration file.

Perhaps related to JENKINS-19994, but deletion always seems to fail specifically on the summary report configuration file.