Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6528

request handler threads get stuck in WeakHashMap

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Major
    • Resolution: Fixed
    • core
    • None

    Description

      RequestHandlerThreads seem to get stuck in parsing test results. We have multiple times seen in the thread dump traces like this:

      "Handling GET /hudson/job/.../190/ : RequestHandlerThread10" Id=328 Group=main RUNNABLE
      at java.util.WeakHashMap.get(WeakHashMap.java:355)
      at hudson.tasks.test.TestObject.uniquifyName(TestObject.java:335)

      • locked hudson.tasks.junit.CaseResult@3f7e9c00
        at hudson.tasks.junit.CaseResult.getSafeName(CaseResult.java:223)

      "Handling GET /hudson/job/.../lastCompletedBuild/testReport/ : RequestHandlerThread29" Id=18760 Group=main RUNNABLE
      at java.util.WeakHashMap.get(WeakHashMap.java:355)
      at hudson.tasks.test.TestObject.uniquifyName(TestObject.java:335)

      • locked hudson.tasks.junit.CaseResult@592d41ae
        at hudson.tasks.junit.CaseResult.getSafeName(CaseResult.java:223)

      It looks a lot like e.g. https://issues.apache.org/jira/browse/BEANUTILS-318

      With my limited Java skills it would appear like the static map UNIQUIFIED_NAMES could be accessed by multiple threads simultaneously through different instances of TestObjects.

      Attachments

        Activity

          Ascending order - Click to sort in descending order
          timotm TimoTM created issue -
          TimoTM TimoTM added a comment -

          Also when this happens, there are as many threads busy looping as threadDump shows threads stuck in WeakHashMap.get().

          We made a modification to our Hudson installation like described in that apache bug report (i.e. wrap the WeakHashMap inside Collection.Synchronized) and haven't yet seen this reoccurring, and everything else seems to work normally.

          TimoTM TimoTM added a comment - Also when this happens, there are as many threads busy looping as threadDump shows threads stuck in WeakHashMap.get(). We made a modification to our Hudson installation like described in that apache bug report (i.e. wrap the WeakHashMap inside Collection.Synchronized) and haven't yet seen this reoccurring, and everything else seems to work normally.
          TimoTM TimoTM added a comment -

          Patch to fix the issue

          TimoTM TimoTM added a comment - Patch to fix the issue
          timotm TimoTM made changes -
          Field Original Value New Value
          Attachment 6528.patch [ 19696 ]
          evernat evernat added a comment -

          @TimoTM
          Your analysis was good, and it was fixed some time ago by
          https://github.com/jenkinsci/jenkins/commit/4ffe3c7b4c4b8bd7a3034b74f5ec617175f8396a#core/src/main/java/hudson/tasks/test/TestObject.java
          or
          https://github.com/jenkinsci/jenkins/commit/734e7e01306e1696655e619d8902a7e6e98b4315#core/src/main/java/hudson/tasks/test/TestObject.java

          evernat evernat added a comment - @TimoTM Your analysis was good, and it was fixed some time ago by https://github.com/jenkinsci/jenkins/commit/4ffe3c7b4c4b8bd7a3034b74f5ec617175f8396a#core/src/main/java/hudson/tasks/test/TestObject.java or https://github.com/jenkinsci/jenkins/commit/734e7e01306e1696655e619d8902a7e6e98b4315#core/src/main/java/hudson/tasks/test/TestObject.java
          evernat evernat made changes -
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 136623 ] JNJira + In-Review [ 187241 ]

          People

            Unassigned Unassigned
            timotm TimoTM
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: