Throttle Concurrent Builds plugin doesn't appear to work

I just installed this today to try and avoid some issues I have with locks & latches. I have some multi-configuration jobs. I want to allow multiple jobs to run at the same time, but only 1 job of a given category to run per node. This sounds exactly like what the plugin was written for. So I went into my configuration and created a category and set Maximum Total Concurrent Builds to 0 and Maximum Concurrent Builds Per Node to 1. I then went to all of my jobs and assigned them to this category and checked the radio button for throttle as part of one or more categories.

I then manually started two jobs from the same category to see if one waited for the other. It didn't work. Both jobs started on the same node. I also tried settings Maximum Concurrent Builds Per Node in the job itself to 1 and that didn't help either.

What am I missing about this configuration?