Renaming an agent does not persist across restarts, and jenkins loads back the old agent as well as the new agent. The root issue seems to be that the directory under ${JENKINS_HOME}/nodes/<agent> is left untouched on a rename, and is left alongside the one created for the new agent. 

Steps to reproduce:

1. Create a new agent
2. Configure -> Rename the agent and click Save
3. Restart Jenkins to see the old agent appear

This caused several issues where the old and new agents were trying to connect to the same host, causing build issues

This is reproducible on several jenkins LTS versions (2.107, 2.138, 2.150), so I'm not sure when it was introduced.