When adding a cloud it says: "AWS IAM Access Key used to connect to ECS. If not specified, implicit authentication mechanisms are used (IAM roles...)"

This does not actually work.

My instance has the correct policy and is confirmed from the cli:

$ aws --region us-east-1 ecs list-clusters
{
    "clusterArns": [
        "arn:aws:ecs:us-east-1:XXXXXXXXXXXX:cluster/jenkins"
    ]
}

Looking through the github repos it appears that only the EC2 Cloud plugin actually uses InstanceProfileCredentialsProvider. (https://github.com/search?q=org%3Ajenkinsci+InstanceProfileCredentialsProvider&type=Code)