Jenkins doesn't even delete project workspaces when projects are deleted, so deleting any other caches would be surprising behavior.
This would require fairly complex behavior anyway, as nodes could be offline when the project gets deleted (and there would be the expectation that it'd get deleted the next time the slave is online). Pretty big can of worms right there for questionable benefit.
FWIW all of this could be implemented in the plugin through extension points that already exist AFAICT, so requires no core improvements. (I just don't think this would be useful.)
Use a real artifacts repository like Artifactory, and a client which does artifact caching, like Gradle or Maven.