We have a multi-phase legacy job for product testing, where one phase has several sequential steps to set up a deployment, then another runs the tests against it, another analyzes the results etc.

The sequential set-up phase in a recent build failed to install the product image, and as that sub-job in the phase had settings to fail the phase and kill other builds, other steps such as configuring that deployment indeed did not happen.

However, even though this phase overall had a setting to continue into the next phase if the build is "Successful or Unstable, but not Failed", tests commenced against the incorrectly prepared product.

The expected outcome was for the MultiPhase job to fail and not do any other steps right there, when it decided the installation was broken.

We are now reconfiguring such jobs to require a "Successful" outcome (and hope it will behave better), but still suspect something can be wrong in the plugin. Or our setup and use of it, somehow...