Thanks for the info - I have my setup similar to yours (with CloudFormation, JasC), but until now, the Jenkins Docker Image I create also used to work on my local env (my mac) - but now this unfortunately is not the case anymore and its harder to do some first local testing of the image.
The most important part for me right now is your first part "Listing credentials" - this effectively hinders my workflow right now.
I don't know the internals of the credential providers, but from a users point of view: I have a job configured e.g. a pipeline and the only thing to reference credentials are by a string-id. For me as a pipeline admin, I don't care which provider actually holds the credentials. If it is not found by any of the providers, it should throw an exception. So there is no way for me to say that I want this credentials retrieved rather from AWS and not from the internal jenkins provider. Saying this, I don't understand why a provider should throw an exception if it can't resolve credentials for a given ID (unless the credentials framework swallows the exceptions and interprets it as a "NotFound" to hand over to the next provider installed until one can resolve it).