The exclamation mark is not added by me. I guess it is done by the SVNRepository class used by the SVN Tagging code.
As per documentation, to access SVN for specific revisions, it is needed to provide url in that format.
Another observation is that, the random failures happen only when deletion of the existing tag is performed. When this deletion happens, the repository revision goes up by one revision and then the tagging fails.
Is it needed that an SVN update is done within the working copy before attempting a copyClient.doCopy operation within the tagging plugin?