The button should be "Uninstall" not "Yes"
https://github.com/jenkinsci/jenkins/blob/e3afbdbf3c5f6c4d09ade1dcb5508e6f3feb901e/core/src/main/resources/hudson/PluginWrapper/uninstall.properties#L1