There's a serious performance issue in the code cleaning up duplicate Tool Installer, in hudson.tools.DownloadFromUrlInstaller$DescriptorImpl.reduce(). On our hosted environment (DEV@cloud), it takes multiple minutes to complete and consumes a full CPU.
Here is a stack trace of where the code is stuck at:
2017-02-16_14:43:24.25621 "Download metadata thread" #136500 daemon prio=5 os_prio=0 tid=0x00000000059e9800 nid=0x676a runnable [0x00007f7687dfc000]
2017-02-16_14:43:24.25621 java.lang.Thread.State: RUNNABLE
2017-02-16_14:43:24.25622 at hudson.tools.DownloadFromUrlInstaller$DescriptorImpl.reduce(DownloadFromUrlInstaller.java:184)
2017-02-16_14:43:24.25624 at hudson.tools.DownloadFromUrlInstaller$DescriptorImpl.access$100(DownloadFromUrlInstaller.java:123)
2017-02-16_14:43:24.25624 at hudson.tools.DownloadFromUrlInstaller$DescriptorImpl$1.reduce(DownloadFromUrlInstaller.java:140)
2017-02-16_14:43:24.25625 at hudson.model.DownloadService$Downloadable.updateNow(DownloadService.java:428)
The problem was introduced in https://github.com/jenkinsci/jenkins/commit/80a6a9e7cbc1f2c7ecbf7c1161c55dae8c0233e0
Coming up with a patch.