-
Bug
-
Resolution: Fixed
-
Critical
-
None
-
Jenkins 1.397, heavy-job 1.0, slave-squatter trunk head (1.2-SNAPSHOT 10/07/2010)
RHEL 6 (java-1.6.0-openjdk 1.6.0_17)
We have been running the slave-squatter and heavy-jobs plugins since they were released under Hudson 1.377 without issue. Recently, we upgraded to Jenkins 1.397 and have started to run into situations where large numbers of executor threads die due to an exception thrown within hudson.model.queue.MappingWorksheet.
This behavior has been observed before in JENKINS-7667.
I believe the simple cause of the exception is that the load predictor routine in MappingWorksheet can come up with a maximum load that exceeds the number of available executors. When the code attempts to reduce the apparent executor pool (here), the underlying list throws the exception.
The simple solution is to clamp minIdle to 0 if it is less than 0.
That solution will prevent the exception that is killing executors (and is probably a good idea to implement if for no other reason than the load predictor is an extension point, and we cannot guarantee that implementers will always return <= the number of executors!). However, this does not address the underlying logical issue in the current load predictors (i.e., why is the predicted load greater than the number of executors???).
- is related to
-
-
- Open
-
[JENKINS-8882] Slave-Squatter + Heavy-Job plugin causes many executor threads to die
Code changed in jenkins
User: jsiirola
Path:
core/src/main/java/hudson/model/queue/MappingWorksheet.java
http://jenkins-ci.org/commit/jenkins/da4ddde7564a15f6e8d9ee83f8499b7369dbe24b
Log:
[FIXED JENKINS-8882].
Clamp the lower bound of the predicted minimum number of idle executors
at 0 to prevent an IndexOutOfBoundsException.
Compare: https://github.com/jenkinsci/jenkins/compare/a3354b1...da4ddde
Integrated in 
jenkins_main_trunk #887
[FIXED JENKINS-8882].
Kohsuke Kawaguchi : da4ddde7564a15f6e8d9ee83f8499b7369dbe24b
Files :
- core/src/main/java/hudson/model/queue/MappingWorksheet.java
Code changed in jenkins
User: jsiirola
Path:
core/src/main/java/hudson/model/queue/MappingWorksheet.java
http://jenkins-ci.org/commit/jenkins/8a2209fe73c5b5eef88bf2e1cffc5d3141060032
Log:
[FIXED JENKINS-8882].
Clamp the lower bound of the predicted minimum number of idle executors
at 0 to prevent an IndexOutOfBoundsException.
(cherry picked from commit da4ddde7564a15f6e8d9ee83f8499b7369dbe24b)
Code changed in jenkins
User: jsiirola
Path:
core/src/main/java/hudson/model/queue/MappingWorksheet.java
http://jenkins-ci.org/commit/jenkins/da4ddde7564a15f6e8d9ee83f8499b7369dbe24b
Log:
[FIXED JENKINS-8882].
Clamp the lower bound of the predicted minimum number of idle executors
at 0 to prevent an IndexOutOfBoundsException.
Code changed in jenkins
User: jsiirola
Path:
core/src/main/java/hudson/model/queue/MappingWorksheet.java
http://jenkins-ci.org/commit/jenkins/da4ddde7564a15f6e8d9ee83f8499b7369dbe24b
Log:
[FIXED JENKINS-8882].
Clamp the lower bound of the predicted minimum number of idle executors
at 0 to prevent an IndexOutOfBoundsException.
Behavior reported here was observed in 7667.