For some reason it is not possible to uninstall the windows-slaves plugin.
This plugin was detached from the core in 2013, and many old plugins implicitly depend on it (Jenkins assumes they might need something from a detached plugin).
thanks Oleg Nenashev - that makes sense. I think also now the message is a bit more clear in the UI of why the plugin can not be removed. So this ticket can be closed.