Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22529

IndexOutOfBoundsException in executor threads

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core
    • Jenkins Core - 1.509.4
      RHEL6-based Server
      ~400 executors on 80 slave nodes

      Seems that there is a concurrency issue in queue handlers.
      The issue sometimes appears in jenkins-1.509.4 .

      java.lang.IndexOutOfBoundsException: Index: 62, Size: 62
      at java.util.ArrayList.rangeCheck(ArrayList.java:635)
      at java.util.ArrayList.get(ArrayList.java:411)
      at hudson.model.queue.MappingWorksheet$ReadOnlyList.get(MappingWorksheet.java:102)
      at hudson.model.queue.MappingWorksheet$ExecutorChunk.execute(MappingWorksheet.java:150)
      at hudson.model.queue.MappingWorksheet$ExecutorChunk.access$000(MappingWorksheet.java:110)
      at hudson.model.queue.MappingWorksheet$Mapping.execute(MappingWorksheet.java:298)
      at hudson.model.Queue.maintain(Queue.java:1048)
      at hudson.model.Queue.pop(Queue.java:866)
      at hudson.model.Executor.grabJob(Executor.java:285)
      at hudson.model.Executor.run(Executor.java:206)

          [JENKINS-22529] IndexOutOfBoundsException in executor threads

          Oleg Nenashev added a comment -

          Unfortunately, I still have to care about it.

          The last valuable change in hudson.model.queue.MappingWorksheet was in 1.520, but the most of code (including referenced classes) have not been changed since 2011. I suppose the issue is still active
          https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/queue/MappingWorksheet.java

          BTW, it makes sense to somehow reproduce it in unit tests

          Oleg Nenashev added a comment - Unfortunately, I still have to care about it. The last valuable change in hudson.model.queue.MappingWorksheet was in 1.520, but the most of code (including referenced classes) have not been changed since 2011. I suppose the issue is still active https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/queue/MappingWorksheet.java BTW, it makes sense to somehow reproduce it in unit tests

          Daniel Beck added a comment -

          Just meant that this won't get much love here, given it's out of community support, unless you can show it's happening in newer Jenkins versions.

          If it's possible to set up a Jenkins using specific steps and reproducible experience the problem, providing these instructions alone would help already IMO.

          Daniel Beck added a comment - Just meant that this won't get much love here, given it's out of community support, unless you can show it's happening in newer Jenkins versions. If it's possible to set up a Jenkins using specific steps and reproducible experience the problem, providing these instructions alone would help already IMO.

          Oleg Nenashev added a comment -

          I've re-assigned the issue to myself.
          When I finish other high-priority assignment (seems to be "never ever"), I'll try to create a reproducible scenario for the issue

          Oleg Nenashev added a comment - I've re-assigned the issue to myself. When I finish other high-priority assignment (seems to be "never ever"), I'll try to create a reproducible scenario for the issue

          Frederik Deweerdt added a comment - - edited

          Hi,

          I'm seeing this with an even older version: 1.480.3. The trace is slightly difference, but similar:
          java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
          at java.util.ArrayList.rangeCheck(ArrayList.java:571)
          at java.util.ArrayList.get(ArrayList.java:349)
          at hudson.model.queue.MappingWorksheet$ReadOnlyList.get(MappingWorksheet.java:102)
          at hudson.model.queue.MappingWorksheet$ExecutorChunk.execute(MappingWorksheet.java:150)
          at hudson.model.queue.MappingWorksheet$ExecutorChunk.access$000(MappingWorksheet.java:110)
          at hudson.model.queue.MappingWorksheet$Mapping.execute(MappingWorksheet.java:298)
          at hudson.model.Queue.maintain(Queue.java:1045)
          at hudson.model.Queue.pop(Queue.java:863)
          at hudson.model.Executor.grabJob(Executor.java:285)
          at hudson.model.Executor.run(Executor.java:206)
          more info

          In particular, the MappingWorksheet bits are the same.

          This issue https://issues.jenkins-ci.org/browse/JENKINS-19799 seems to be exactly the same as well, and I agree with the assement made there: if isAvailable is false, there's nothing preventing 'e' to go out of bounds in ExecutorChunk.execute().

          Attaching a patch against lastest git

          Frederik Deweerdt added a comment - - edited Hi, I'm seeing this with an even older version: 1.480.3. The trace is slightly difference, but similar: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 at java.util.ArrayList.rangeCheck(ArrayList.java:571) at java.util.ArrayList.get(ArrayList.java:349) at hudson.model.queue.MappingWorksheet$ReadOnlyList.get(MappingWorksheet.java:102) at hudson.model.queue.MappingWorksheet$ExecutorChunk.execute(MappingWorksheet.java:150) at hudson.model.queue.MappingWorksheet$ExecutorChunk.access$000(MappingWorksheet.java:110) at hudson.model.queue.MappingWorksheet$Mapping.execute(MappingWorksheet.java:298) at hudson.model.Queue.maintain(Queue.java:1045) at hudson.model.Queue.pop(Queue.java:863) at hudson.model.Executor.grabJob(Executor.java:285) at hudson.model.Executor.run(Executor.java:206) more info In particular, the MappingWorksheet bits are the same. This issue https://issues.jenkins-ci.org/browse/JENKINS-19799 seems to be exactly the same as well, and I agree with the assement made there: if isAvailable is false, there's nothing preventing 'e' to go out of bounds in ExecutorChunk.execute(). Attaching a patch against lastest git

          Make sure e doesn't go out of bounds.

          Frederik Deweerdt added a comment - Make sure e doesn't go out of bounds.

          Daniel Beck added a comment -

          Frederik: Please consider submitting a pull request on Github.

          Daniel Beck added a comment - Frederik: Please consider submitting a pull request on Github.

          Frederik Deweerdt added a comment - Done: https://github.com/jenkinsci/jenkins/pull/1300 Thanks, Frederik

          Oleg Nenashev added a comment -

          I'm working on a new PR for the issue

          Oleg Nenashev added a comment - I'm working on a new PR for the issue

          Aaron Curley added a comment -

          Ugh, sorry about the accidental reassignment!

          Aaron Curley added a comment - Ugh, sorry about the accidental reassignment!

          Oleg Nenashev added a comment -

          aaron312 feel free to take it you want. I will unlikely get to it anytime soon

          Oleg Nenashev added a comment - aaron312 feel free to take it you want. I will unlikely get to it anytime soon

            Unassigned Unassigned
            oleg_nenashev Oleg Nenashev
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: