After conversations with jglick and oleg_nenashev regarding JENKINS-51306 we have agreed on that a good solution for ATH and PCT runs based on a BOM is to force a sane dependency graph in your BOM file. This graph should be enough to run the specified tests without any further plugin modification. In this situation makes sense to modify the ATH to have an "essentials mode" where WithPlugins just validates that the war file under tests is able to execute the test but it does not install/update or modify existing plugins in any way. If the validation fails, that is there is a requirement on a plugin not already installed or installed in a lower version than the one needed by the test, the test should fail or optionally be skipped with the proper reason