Hosh I do not know if the proposed withGitCredentials pipeline step would help in your case. I've not yet found a case where the original claims in this bug report could be verified. As far as I can tell, the git plugin uses command line git to fetch and checkout content from the remote repository. As far as I can tell, it does that with speed that is comparable to git clone.
Since you're using Pipeline, you probably don't want "Wipe out repository and force clone". That same operation can be done from the pipeline itself with a pipeline task. Moving that into the pipeline definition places one more part of the job definition inside source control.
You probably also do not want branch specifier as "$BRANCH" because that means the change history in the job will not be usable. The change history shows the changes from one build to the next, but if you build a different branch on job n-1 than is built on job n, then change log is not very useful. In most cases that I've seen, it is better to use a multibranch pipeline and allow Jenkins to create and destroy jobs as branches are created and destroyed in the repository.
If your git provider is GitHub, Bitbucket, GitLab, or Gitea, then you should probably use the plugin that is specific to those implementations, rather than the general purpose "Git" provider that you've selected as your SCM provider. The Git SCM provider does not know that GitHub, Bitbucket, GitLab, and Gitea all provide REST APIs that can make some git operations (like polling and reading the Jenkinsfile) much faster.