[JENKINS-49350] Matrix Job Combination Filter causes builds to be kept forever if it's random

Type: Bug
Resolution: Unresolved
Priority: Minor
Component/s: matrix-project-plugin
Labels:
None
Environment:
Jenkins 2.98
Matrix Plugin: 1.12
Matrix Groovy Execution Strategy Plugin: 1.0.7

Similar Issues:
Powered by SuggestiMate

Show

Summary

When sparsing out the matrix in any randomized way, builds are kept forever, disregarding the job's Build Discarder settings. All sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.

Rationale

I need to run random subsets of a lot of possible combinations because running all combinations every time would require a lot of resources. I suspect that this is not an unusual scenario.

Steps to reproduce:

Create a matrix job:

Add some axis combinations (more than 2 total combinations).
Set build discarder to keep 3 builds (for example)
Add a String parameter named MATRIX_SPARSE_FACTOR to control sparse execution
Set a Combination Filter to something like this:
(new java.util.Random()).nextInt(MATRIX_SPARSE_FACTOR.toInteger()) == 0
Run this job several times (more than the number of builds to keep) with a parameter value >1 (eg 2).
Old builds are not discarded, ever, growing infinitely.
Run the job 3+ times with a parameter value of 1
Old builds are discarded once there are 3 "complete" builds (all combinations).

Note: the same issue is seen when using the Matrix Groovy Execution Strategy Plugin with a Groovy Execution Strategy that returns random results (changes between executions).

Screenshot of the tested configuration:

Correct Behavior:

When using a non-random matrix combination filter string instead, things work, for example:

index%3!=0

This produces the same execution (one of three combinations execute), but old builds with this configuration are correctly discarded. However, this is not random, so successive builds run the same combinations.

Screenshot

This is what a job configured to keep 5 builds looks like: (it has >100 builds right now)

Additional Troubleshooting

Experimenting with consistent sparse matrix (same subsets chosen in each build), with a build discarder set to keep 3 builds:

index%(MATRIX_SPARSE_FACTOR.toInteger()) == 0

10 builds with all combinations:
3 are kept.
10 builds with all combinations followed by 10 builds with every 3rd combination:
1 kept from first 10, and 3 from the second.
10 "all", then 10 "1/3", then 10 "1/2":
1 kept from "all", 1 kept from "1/3", 3 kept from "1/2"
10 "all", then 10 "1/3", then 10 "1/2", then 10 "1/4":
1 kept from "all", 1 kept from "1/3", 1 kept from "1/2", 3 kept from "1/4"

In other words, at least one is always kept from each different combination, and the build discarder only applies to the repetition of the latest variant, unless it is "run all", in which case all prior builds are finally removed once we have N successive builds with "all".

- - Sort By Name
  - Sort By Date
  - Ascending
  - Descending
  - Thumbnails
  - List
  - Download All

Screenshot_2018-03-07_08-24-33.png
79 kB
2018-03-07 15:24
build-keep-forever.png
69 kB
2018-02-02 20:13

Alexander Komarov created issue - 2018-02-02 20:02

Alexander Komarov made changes - 2018-02-02 20:03

Description

Original: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any number would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:

{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:

{code:java}
index%3!=0 {code}

This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:

{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.

h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

New: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:

{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:

{code:java}
index%3!=0 {code}

This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:

{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.

h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

Alexander Komarov made changes - 2018-02-02 20:09

Description

Original: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:

{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:

{code:java}
index%3!=0 {code}

This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:

{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.

h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

New: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the default Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:

{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:

{code:java}
index%3!=0 {code}

This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:

{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.

h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

Alexander Komarov made changes - 2018-02-02 20:09

Description

Original: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the default Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:

{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:

{code:java}
index%3!=0 {code}

This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:

{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.

h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

New: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:

{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:

{code:java}
index%3!=0 {code}

This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:

{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.

h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

Alexander Komarov made changes - 2018-02-02 20:10

Description

Original: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:

{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:

{code:java}
index%3!=0 {code}

This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:

{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.

h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

New: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:
{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:
{code:java}
index%3!=0 {code}
This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:
{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.
h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

Alexander Komarov made changes - 2018-02-02 20:13

Attachment

New: build-keep-forever.png [ 41323 ]

Alexander Komarov made changes - 2018-02-02 20:14

Description

Original: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:
{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:
{code:java}
index%3!=0 {code}
This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:
{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.
h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

New: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:
{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:
{code:java}
index%3!=0 {code}
This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:
{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.
h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

h2. Screenshot

This is what a job configured to keep 5 builds looks like: (it has >100 builds right now)

!build-keep-forever.png!

Alexander Komarov made changes - 2018-03-07 14:56

Assignee

Original: Kohsuke Kawaguchi [ kohsuke ]

Alexander Komarov made changes - 2018-03-07 14:59

Description

Original: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:
{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:
{code:java}
index%3!=0 {code}
This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:
{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation.
h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion.

h2. Screenshot

This is what a job configured to keep 5 builds looks like: (it has >100 builds right now)

!build-keep-forever.png!

New: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:
{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:
{code:java}
index%3!=0 {code}
This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:
{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation. In some cases, I needed to run *N* successive builds that exercise every combination before the old builds got discarded, where *N* is the number of builds to keep configured for the job.
h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion. Sometimes running several of these is required.

h2. Screenshot

This is what a job configured to keep 5 builds looks like: (it has >100 builds right now)

!build-keep-forever.png!

Alexander Komarov made changes - 2018-03-07 15:13

Summary

Original: Matrix groovy combination filter causes builds to be kept forever

New: Matrix Job Combination Filter causes builds to be kept forever if it's random

Alexander Komarov made changes - 2018-03-07 15:23

Description

Original: _Note: this issue applies to the Matrix Groovy Execution Strategy Plugin, but I didn't find a component name for it so it's assigned to matrix plugin._

When sparsing out the matrix using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin], builds are *kept forever*, disregarding the job's Build Discarder settings, unless all combinations run in one build. Any sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.  This issue does not arrise when using the built-in Matrix Combination Filter with the Classic strategy.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (any subset would work)
* Use a *groovy execution strategy* for the matrix combination filter that picks a random subset each time.

Groovy code for the execution strategy:
{code:java}
def result = [:]
java.util.Collections.shuffle(combinations)
combinations.eachWithIndex{combo, idx ->
  if (idx % 3 == 0) {
    result[idx] = [combo]
  }
}
println "Selected ${result}"
result
{code}
This will cause a random 1/3 of the combinations to run each time.

Run this job several times (more than the number of builds to keep). *Old builds are not discarded,* ever.
h2. Correct Behavior:

When using a normal matrix combination filter string instead, things work, for example:
{code:java}
index%3!=0 {code}
This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

When instead the filter is random, builds are still discarded, although not consistently:
{code:java}
(new java.util.Random()).nextInt(2) == 0
{code}
Most likely they are discarded when the combinations ran in the prior build are completely covered by the sum total of newer builds. However, the builds are not marked with "Keep Forever", so I believe that this is a different situation. In some cases, I needed to run *N* successive builds that exercise every combination before the old builds got discarded, where *N* is the number of builds to keep configured for the job.
h2. Workaround

For the matrix execution strategy, triggering one build with all combinations enabled (either by temporarily reconfiguring the job or by making the execution strategy parameterized) will cause the excess old builds to get discarded upon completion. Sometimes running several of these is required.

h2. Screenshot

This is what a job configured to keep 5 builds looks like: (it has >100 builds right now)

!build-keep-forever.png!

New: h2. Summary

When sparsing out the matrix in any randomized way, *builds are kept forever*, disregarding the job's Build Discarder settings. All sparse builds are marked "Keep This Build Forever" because newer builds depend on older builds.
h2. Rationale

I need to run random subsets of a lot of possible combinations because running all combinations every time would require a lot of resources. I suspect that this is not an unusual scenario.
h2. Steps to reproduce:

Create a matrix job:
* Add some axis combinations (more than 2 total combinations).
* Set build discarder to keep 3 builds (for example)
* Add a String parameter named MATRIX_SPARSE_FACTOR to control sparse execution
* Set a Combination Filter to something like this:
*(new java.util.Random()).nextInt(MATRIX_SPARSE_FACTOR.toInteger()) == 0*
* Run this job several times (more than the number of builds to keep) with a parameter value >1 (eg 2).
* *Old builds are not discarded,* ever, growing infinitely.
* Run the job 3+ times with a parameter value of 1
* Old builds are discarded once there are 3 "complete" builds (all combinations).

Note: the same issue is seen when using the [Matrix Groovy Execution Strategy Plugin|https://wiki.jenkins.io/display/JENKINS/Matrix+Groovy+Execution+Strategy+Plugin] with a Groovy Execution Strategy that returns random results (changes between executions).
h2. Correct Behavior:

When using a non-random matrix combination filter string instead, things work, for example:
{code:java}
index%3!=0 {code}
This produces the same execution (one of three combinations execute), but old builds with this configuration are *correctly discarded*. However, this is not random, so successive builds run the same combinations.

h2.
h2. Screenshot

This is what a job configured to keep 5 builds looks like: (it has >100 builds right now)

!build-keep-forever.png!

Assignee:: Unassigned

Reporter:: Alexander Komarov

Votes:: 1 Vote for this issue

Watchers:: 2 Start watching this issue

Created:: 2018-02-02 20:02

Updated:: 2018-03-07 15:42

Jenkins

Details

Description

Summary

Rationale

Steps to reproduce:

Correct Behavior:

Screenshot

Additional Troubleshooting

Attachments

Attachments

Activity

People

Dates