A joining build blocked waiting for an upstream build to finish will not run if upstream build is also in the queue.

Setup 4 projects all set with both the block on upstream and downstream checkboxes set.

Project A: trigger project, build step, sleep 60 secs. triggers B1 & B2
Project B1: build project, build step, sleep 60 secs. triggers C
Project B2: build project, build step, sleep 120 secs. triggers C
Project C: join project, build step, sleep 60 secs.

In this case we would expect that triggering A would cause one run of each A,B1,B2 & C.

This worked fine as long as the start job was not called multiple times.
However if A is triggered multiple times the builds A,B1 & B2 will be run multiple times and C will be run only once.

This seems to be due to the fact that if A is triggered while either A or B1 are running, C will not run until all of the triggers of A,B1,B2 have been completed.

I have looked at the code and found the problem, as described below.

The queue class calls the isBuildBlocked() implemented in the Abstract project [1] to see if it can do a build of a particular project that is in the queue.

This implementation gets the list of unblocked tasks that are in the queue and the list of up or down stream projects, and then checks to see if any of the these are in the unblocked task list. [2]&[3]

When the A project is called multiple times while the B1,B2 jobs are run, it blocks because the downstream jobs are building and are returned in the unblocked task list.[3]

When the B1,B2 jobs trigger the C build, it will block until all of the build jobs are finished as the running build jobs are returned in the unblocked task list. [2]

When all of the build jobs finish the queue processes all of the waiting items again in the order added [4], this time the A project gets run as there are no items in the unblocked list as the C project still has the state of blocked (as it has not been updated).
See description and comments at the top of the file [5] on how a task moves between the states.

This seems to be the direct result of fixing JENKINS-8929

Tracking further we can see that Job C will be run with different amounts of upstream projects depending on when Job A is triggered.

So the time frame for these multiples occuring is
Case A
If there is a trigger between the A job starting and B1 finishing
Another build set will be run adding to the join that is queued.

Case B
If there is a trigger between the first of the build jobs finishing and the join job running
The build sets will be run adding to the next join that is queued.
This time frame can vary a lot depending on the if the slave has the needed file available or if it needs to copy these to the workspace, seen when the builds move between different slaves.

This is an offshoot of looking at the issue, JENKINS-14108 which was trying to join projects using the parametrized trigger, we removed this and replaced it with a normal downstream build trigger.

[1] https://github.com/jenkinsci/jenkins/blob/jenkins-1.454/core/src/main/java/hudson/model/AbstractProject.java#L1012
[2] https://github.com/jenkinsci/jenkins/blob/jenkins-1.454/core/src/main/java/hudson/model/AbstractProject.java#L1097 downstream
[3] https://github.com/jenkinsci/jenkins/blob/jenkins-1.454/core/src/main/java/hudson/model/AbstractProject.java#L1114 upstream
[4] https://github.com/jenkinsci/jenkins/blob/jenkins-1.454/core/src/main/java/hudson/model/Queue.java#L896
[5] https://github.com/jenkinsci/jenkins/blob/jenkins-1.454/core/src/main/java/hudson/model/Queue.java#L130