There is a serious issue with the Gerrit trigger plugin and its "retrigger" functionality.

Problem

This issue causes the plugin to get confused about how many and which builds it started for a particular changeset/patchset combination.

This behaviour leads to inconsistent verification results in Gerrit, because it fails to wait for all tests to complete.

Replication instructions

1. Start a blank, vanilla Jenkins server of an arbitrary version on your local host (the jobs below assume that a BASH shell is available).
2. Install the gerrit-trigger plugin (arbitrary version, as all are affected)
3. Increase the executor count of the "master" node to at least 6.
4. Restart the server and create a connection to a Gerrit service.
5. Shut down the server and extract the two jobs (Test_1 and Test_2) into the "jobs" directory of the Jenkins server.
   • Test_1 will always fail, Test_2 will always succeed. Both will create a lock-file named "LOCKFILE-${JOB_NAME}-${BUILD_NUMBER}" in "${JENKINS_HOME}/workspace/Central" and wait for it to be deleted, before succeeding or failing.
6. (Optional) Alter the Test_1 and Test_2 Gerrit Trigger settings to listen to a specific repository. By default, both repo and branch are set to "**".
   

After this set-up is done, upload a patchset to the Gerrit service that you configured Jenkins to listen to.

This will cause a build of "Test_1" and "Test_2" to be started.
Now, while they are running (remember that they will wait until you delete their lock-file), go into the "manual trigger" interface:
http://[jenkins]/gerrit_manual_trigger/?

Search for the changeset and issue a retrigger. This will cause two additional builds to spawn, one for "Test_1" and one for "Test_2".

Now that 4 tests are running, the correct behaviour of the Gerrit trigger plugin is to ignore the first two runs that have been started by the "Patchset Created" event.

Instead, it should wait until BOTH of the retriggered builds have finished and then upload a "Verified -1" message to Gerrit.

To test this, first delete the two original lock files:

${JENKINS_HOME}/workspace/Central/LOCKFILE-Test_1-1
${JENKINS_HOME}/workspace/Central/LOCKFILE-Test_2-1

This tests the first half of the problem. This works fine. After both have finished, the Gerrit Trigger plugin will still wait for the other two jobs to finish.

Now, to expose the bug, ONLY delete the lockfile for the re-triggered TEST_2 job:
${JENKINS_HOME}/workspace/Central/LOCKFILE-Test_2-2

This will let Test_2 finish successfully (as Test_2 always succeeds).
The bug that happens now, is that the trigger plugin uploads a "Verified +1" message, DESPITE "Test_1 #2" not yet being finished.

If you then also let "Test_1" finish, it uploads yet another verification, this time "Verified -1".

This behaviour is absolutely wrong, as it sends a verification state without waiting for all jobs to complete, thus leading to inconsistent results in Gerrit.

Example output in Gerrit, when error occurs

Here's an excerpt of how this behaviour looks like in Gerrit:

##############################################################################
Jenkins
Patch Set 1: -Verified
Build Started Test_2/1/ (1/2)
-------------------------------------------------------
Jenkins
Patch Set 1:
Build Started Test_1/1/ (2/2)
-------------------------------------------------------
Jenkins
Patch Set 1:
Build Started Test_2/2/ (2/2)
-------------------------------------------------------
Jenkins
Patch Set 1:
Build Started Test_1/2/ (2/2)
-------------------------------------------------------
Jenkins
Patch Set 1: Verified+1
Build Successful 
Test_2/2/ : SUCCESS
-------------------------------------------------------
Jenkins
Patch Set 1: Verified-1
Build Failed 
Test_1/2/ : FAILURE
##############################################################################

You can see two bugs outline above at work here:

1.) The plugin gets confused when counting the number of jobs started
2.) The plugin does not wait for (Test_1 #2) after (Test_2 #2) has finished.

[EDIT: Improved text formatting. Now the report is more than just a wall of text]