Hello,
I spent some time investigating this issue. It seems to me that Jenkins itself is built considering that you set a directory in the tool installation path box, that's why you get this warning.
However, when you run a job, many plugins do not append the executable name to the path therefore you have to enter the full path to make it work.
For instance, MSTest and FxCop runner plugin have the same issue. If you set a directory, it won't work because the exe file name is not included.
We could correct these plugins so that they behave like they should : you would have to set the directory where the tool is installed in the administration and then, the plugin will itself append the executable name at building stage. However, we should probably keep supporting the current behavior : including the executable name in the tool configuration screen for backward compatibility.
I don't have much experience in Jenkins plugin development and lifecycle so I'd like to have the opinion of more experienced people about what should be done here.
The same happens with MSBuild configuration