Summary

When sparsing out the matrix in any randomized way, builds are kept forever, disregarding the job's Build Discarder settings.  All sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  

Rationale

I need to run random subsets of a lot of possible combinations because running all combinations every time would require a lot of resources.  I suspect that this is not an unusual scenario.

Steps to reproduce:

Create a matrix job:

• Add some axis combinations (more than 2 total combinations).
• Set build discarder to keep 3 builds (for example)
• Add a String parameter named MATRIX_SPARSE_FACTOR to control sparse execution
• Set a Combination Filter to something like this:
  (new java.util.Random()).nextInt(MATRIX_SPARSE_FACTOR.toInteger()) == 0
• Run this job several times (more than the number of builds to keep) with a parameter value >1 (eg 2). 
• Old builds are not discarded, ever, growing infinitely.
• Run the job 3+ times with a parameter value of 1
• Old builds are discarded once there are 3 "complete" builds (all combinations).

Note: the same issue is seen when using the Matrix Groovy Execution Strategy Plugin with a Groovy Execution Strategy that returns random results (changes between executions).

Screenshot of the tested configuration:

Correct Behavior:

When using a non-random matrix combination filter string instead, things work, for example:

index%3!=0 

 This produces the same execution (one of three combinations execute), but old builds with this configuration are correctly discarded.  However, this is not random, so successive builds run the same combinations.

Screenshot

This is what a job configured to keep 5 builds looks like: (it has >100 builds right now)

Additional Troubleshooting

Experimenting with consistent sparse matrix (same subsets chosen in each build), with a build discarder set to keep 3 builds:

index%(MATRIX_SPARSE_FACTOR.toInteger()) == 0

• 10 builds with all combinations:
       3 are kept.
• 10 builds with all combinations followed by 10 builds with every 3rd combination:
       1 kept from first 10, and 3 from the second.
• 10 "all", then 10 "1/3", then 10 "1/2":
       1 kept from "all", 1 kept from "1/3", 3 kept from "1/2"
• 10 "all", then 10 "1/3", then 10 "1/2", then 10 "1/4":
       1 kept from "all", 1 kept from "1/3", 1 kept from "1/2", 3 kept from "1/4"

In other words, at least one is always kept from each different combination, and the build discarder only applies to the repetition of the latest variant, unless it is "run all", in which case all prior builds are finally removed once we have N successive builds with "all".