Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26739

ISE from AbstractLazyLoadRunMap.proposeNewNumber for concurrent matrix builds

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • matrix-project-plugin
    • master on Linux - 2-3 Linux slaves with 10 executors - 1 windows slave with one executor. Problem only shows up on linux slaves where we have the highest load - up to 30 parallel jobs

      Thread has died

      java.lang.IllegalStateException: cannot create a build with number 8558 since that (or higher) is already in use among [8099, 8312, 8317, 8318, 8319, 8320, 8321, 8322, 8323, 8326, 8328, 8329, 8330, 8331, 8333, 8335, 8336, 8338, 8340, 8341, 8348, 8351, 8355, 8358, 8360, 8361, 8362, 8363, 8370, 8371, 8380, 8381, 8386, 8387, 8394, 8397, 8398, 8399, 8400, 8401, 8402, 8403, 8404, 8405, 8406, 8407, 8408, 8409, 8418, 8419, 8420, 8421, 8422, 8423, 8424, 8426, 8428, 8435, 8436, 8440, 8441, 8442, 8484, 8487, 8488, 8489, 8490, 8491, 8492, 8493, 8495, 8497, 8498, 8499, 8500, 8501, 8508, 8512, 8513, 8514, 8515, 8522, 8523, 8524, 8526, 8527, 8528, 8529, 8530, 8531, 8535, 8536, 8537, 8545, 8546, 8549, 8550, 8552, 8554, 8555, 8556, 8557, 8560, 8563]
      at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361)
      at hudson.model.RunMap.put(RunMap.java:189)
      at hudson.matrix.MatrixConfiguration.newBuild(MatrixConfiguration.java:284)
      at hudson.matrix.MatrixConfiguration.newBuild(MatrixConfiguration.java:74)
      at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1205)
      at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
      at hudson.model.Executor.run(Executor.java:213)

      more info

          [JENKINS-26739] ISE from AbstractLazyLoadRunMap.proposeNewNumber for concurrent matrix builds

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3995
          [FIXED JENKINS-26739] Suppress monotonicity assertion for MatrixRun. (Revision 5ada903554a493fcfbc8e23c5c8d02bc50c17845)

          Result = SUCCESS
          jesse glick : 5ada903554a493fcfbc8e23c5c8d02bc50c17845
          Files :

          • core/src/main/java/hudson/model/RunMap.java

          dogfood added a comment - Integrated in jenkins_main_trunk #3995 [FIXED JENKINS-26739] Suppress monotonicity assertion for MatrixRun. (Revision 5ada903554a493fcfbc8e23c5c8d02bc50c17845) Result = SUCCESS jesse glick : 5ada903554a493fcfbc8e23c5c8d02bc50c17845 Files : core/src/main/java/hudson/model/RunMap.java

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3997
          JENKINS-26739 Backporting #1592 to rc. (Revision 6ee6fc3b2f7a9bceca1b04c4af5e6f1b024e2ab4)

          Result = SUCCESS
          jesse glick : 6ee6fc3b2f7a9bceca1b04c4af5e6f1b024e2ab4
          Files :

          • changelog.html
          • core/src/main/java/hudson/model/RunMap.java

          dogfood added a comment - Integrated in jenkins_main_trunk #3997 JENKINS-26739 Backporting #1592 to rc. (Revision 6ee6fc3b2f7a9bceca1b04c4af5e6f1b024e2ab4) Result = SUCCESS jesse glick : 6ee6fc3b2f7a9bceca1b04c4af5e6f1b024e2ab4 Files : changelog.html core/src/main/java/hudson/model/RunMap.java

          I have not seen this issue anymore since we upgraded to 1.600 - many thanks!

          Thomas Müller added a comment - I have not seen this issue anymore since we upgraded to 1.600 - many thanks!

          Greg BOUGEARD added a comment -

          Hi, we currently have this issue with 1.605 and 1.606 version.

          Greg BOUGEARD added a comment - Hi, we currently have this issue with 1.605 and 1.606 version.

          Jesse Glick added a comment -

          gbougeard really with the same stack trace? Seems more likely that you are seeing some other issue with a similar but distinct symptom. File it separately, with steps to reproduce if at all possible (otherwise at least a log file, support bundle from the Support Core plugin, etc.), blocking JENKINS-24380.

          Jesse Glick added a comment - gbougeard really with the same stack trace? Seems more likely that you are seeing some other issue with a similar but distinct symptom. File it separately, with steps to reproduce if at all possible (otherwise at least a log file, support bundle from the Support Core plugin, etc.), blocking JENKINS-24380 .

          Greg BOUGEARD added a comment - - edited

          Do you think it's a different issue?

            Mar 25, 2015 12:57:18 PM SEVERE hudson.model.Executor run
            Unexpected executor death
            java.lang.IllegalStateException: /var/lib/jenkins/jobs/service-mysql-migrations_master/configurations/axis-BASE_TAG/prod/builds/218 already existed; will not overwite with service-mysql-migrations_master/BASE_TAG=prod #218
          	at hudson.model.RunMap.put(RunMap.java:187)
          	at hudson.matrix.MatrixConfiguration.newBuild(MatrixConfiguration.java:284)
          	at hudson.matrix.MatrixConfiguration.newBuild(MatrixConfiguration.java:74)
          	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1205)
          	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
          

          Greg BOUGEARD added a comment - - edited Do you think it's a different issue? Mar 25, 2015 12:57:18 PM SEVERE hudson.model.Executor run Unexpected executor death java.lang.IllegalStateException: / var /lib/jenkins/jobs/service-mysql-migrations_master/configurations/axis-BASE_TAG/prod/builds/218 already existed; will not overwite with service-mysql-migrations_master/BASE_TAG=prod #218 at hudson.model.RunMap.put(RunMap.java:187) at hudson.matrix.MatrixConfiguration.newBuild(MatrixConfiguration.java:284) at hudson.matrix.MatrixConfiguration.newBuild(MatrixConfiguration.java:74) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1205) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)

          Jesse Glick added a comment -

          gbougeard Correct, that is unrelated, and already tracked as JENKINS-26582 (currently with no known way to reproduce).

          Jesse Glick added a comment - gbougeard Correct, that is unrelated, and already tracked as JENKINS-26582 (currently with no known way to reproduce).

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/test/java/hudson/matrix/MatrixProjectTest.java
          http://jenkins-ci.org/commit/matrix-project-plugin/66592112c2984ecd379cf6ff0ab8e1eed68dfa60
          Log:
          JENKINS-26739 testConcurrentBuild should pass again as of 1.602.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/test/java/hudson/matrix/MatrixProjectTest.java http://jenkins-ci.org/commit/matrix-project-plugin/66592112c2984ecd379cf6ff0ab8e1eed68dfa60 Log: JENKINS-26739 testConcurrentBuild should pass again as of 1.602.

          Dan Ssajner added a comment -

          Reopening this issue. I'm seeing this exception and a lot of dead executors on jenkins 1.622.

          java.lang.IllegalStateException: cannot create a build with number 2322 since that (or higher) is already in use among [2261, 2262, 2263, 2264, 2265, 2266, 2267, 2268, 2269, 2270, 2271, 2272, 2273, 2274, 2275, 2276, 2277, 2278, 2279, 2280, 2281, 2282, 2283, 2284, 2285, 2286, 2287, 2288, 2289, 2290, 2291, 2292, 2293, 2294, 2295, 2296, 2297, 2298, 2299, 2300, 2301, 2302, 2303, 2304, 2305, 2306, 2307, 2308, 2309, 2310, 2457, 2458, 2459, 2460, 2461, 2462, 2463, 2464, 2465, 2466, 2467, 2468, 2469, 2470, 2471, 2472, 2473, 2474, 2475, 2476, 2477, 2478, 2479, 2480, 2481, 2482, 2483, 2484, 2485, 2486, 2487, 2488, 2489, 2490, 2491, 2492, 2493, 2494, 2495, 2496, 2497, 2498, 2499, 2500, 2501, 2502, 2503, 2504, 2505, 2506]
          at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361)
          at hudson.model.RunMap.put(RunMap.java:192)
          at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
          at hudson.model.AbstractProject.newBuild(AbstractProject.java:1019)
          at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1218)
          at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
          at hudson.model.Executor$1.call(Executor.java:335)
          at hudson.model.Executor$1.call(Executor.java:317)
          at hudson.model.Queue._withLock(Queue.java:1345)
          at hudson.model.Queue.withLock(Queue.java:1210)
          at hudson.model.Executor.run(Executor.java:317)

          Dan Ssajner added a comment - Reopening this issue. I'm seeing this exception and a lot of dead executors on jenkins 1.622. java.lang.IllegalStateException: cannot create a build with number 2322 since that (or higher) is already in use among [2261, 2262, 2263, 2264, 2265, 2266, 2267, 2268, 2269, 2270, 2271, 2272, 2273, 2274, 2275, 2276, 2277, 2278, 2279, 2280, 2281, 2282, 2283, 2284, 2285, 2286, 2287, 2288, 2289, 2290, 2291, 2292, 2293, 2294, 2295, 2296, 2297, 2298, 2299, 2300, 2301, 2302, 2303, 2304, 2305, 2306, 2307, 2308, 2309, 2310, 2457, 2458, 2459, 2460, 2461, 2462, 2463, 2464, 2465, 2466, 2467, 2468, 2469, 2470, 2471, 2472, 2473, 2474, 2475, 2476, 2477, 2478, 2479, 2480, 2481, 2482, 2483, 2484, 2485, 2486, 2487, 2488, 2489, 2490, 2491, 2492, 2493, 2494, 2495, 2496, 2497, 2498, 2499, 2500, 2501, 2502, 2503, 2504, 2505, 2506] at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361) at hudson.model.RunMap.put(RunMap.java:192) at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178) at hudson.model.AbstractProject.newBuild(AbstractProject.java:1019) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1218) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144) at hudson.model.Executor$1.call(Executor.java:335) at hudson.model.Executor$1.call(Executor.java:317) at hudson.model.Queue._withLock(Queue.java:1345) at hudson.model.Queue.withLock(Queue.java:1210) at hudson.model.Executor.run(Executor.java:317)

          Daniel Beck added a comment -

          This issue is specifically for Matrix Projects. Your problem is with a different project type. When filing a new issue (assuming none exists yet), please try to determine why Jenkins would assign a build number much lower than the highest existing build number.

          Daniel Beck added a comment - This issue is specifically for Matrix Projects. Your problem is with a different project type. When filing a new issue (assuming none exists yet), please try to determine why Jenkins would assign a build number much lower than the highest existing build number.

            jglick Jesse Glick
            deepdiver Thomas Müller
            Votes:
            3 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: