No, I'm not manipulating the build programmatically. I switched a build from a fixed version to gradlew like shown in the two screenshots attached. After the change, the gradleName property is still set in the config file. The current logic in Gradle.java is still trying to use the tool version, although it should use the wrapper and the gradle version that comes with it. See the attached jenkins-24682.zip file for a minimal example that fails with the current code and is fixed in my pull request.
From my point of view the plugin should not consider the installed tools at all if gradle wrapper is selected. If created a pull request here: https://github.com/jenkinsci/gradle-plugin/pull/23