Even if smtp server info is removed from the mailer-plugin configuration it "remembers", the old configuration and still sends emails.

I ended up having to put in a bad smtp server name in order to get it to stop sending emails.

NOTE: It would be good to have a simple, enable, disable check box.