Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-58382

Build is stuck due to blocked thread

XMLWordPrintable

    • v1.0.39

      There is a build which is stuck on intial stage.

      I saw in monitoring screen the below message:

      Threads on 50aa853439c2@XXXXXX: Number = 110, Maximum = 526, Total started = 464,563
      Warning, the following threads are deadlocked : Handling POST************/214/kill from XXXXX : qtp544724190-466789, jenkins.util.Timer 9, OkHttp https://*******.com:XXXX/..., OkHttp https://*******.com:XXXX/...
       
      I tried to kill that thread but it's not getting deleted.
      Then I went to the threadDump log and saw:

      jenkins.util.Timer 9

      "jenkins.util.Timer 9" Id=71 Group=main BLOCKED on java.lang.String@39c5d884 owned by "OkHttp https://*******.com:XXXX/..." Id=139290 at io.fabric8.jenkins.openshiftsync.JenkinsUtils.triggerJob(JenkinsUtils.java:422) - blocked on java.lang.String@39c5d884 at io.fabric8.jenkins.openshiftsync.BuildWatcher.addEventToJenkinsJobRun(BuildWatcher.java:345) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.handleBuildList(JenkinsUtils.java:815) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.maybeScheduleNext(JenkinsUtils.java:717) at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.onDeleted(BuildSyncRunListener.java:184) at hudson.model.listeners.RunListener.fireDeleted(RunListener.java:269) at hudson.model.Run.delete(Run.java:1573) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.deleteRun(JenkinsUtils.java:567) - locked java.lang.Class@b802391 at io.fabric8.jenkins.openshiftsync.BuildWatcher.reconcileRunsAndBuilds(BuildWatcher.java:494) at io.fabric8.jenkins.openshiftsync.BuildWatcher.access$100(BuildWatcher.java:65) at io.fabric8.jenkins.openshiftsync.BuildWatcher$1.doRun(BuildWatcher.java:148) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@28c61284
       
       
      so I guess it's something with the openshift sync plugin. can you suggest what is the next step I should try to do?

        1. Thread Dump.docx
          71 kB

          [JENKINS-58382] Build is stuck due to blocked thread

          Ariel M created issue -
          Ariel M made changes -
          Attachment New: Thread Dump.docx [ 47950 ]
          Gabe Montero made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          Gabe Montero made changes -
          Released As New: v1.0.39
          Resolution New: Fixed [ 1 ]
          Status Original: In Progress [ 3 ] New: Resolved [ 5 ]
          Bismaya Mohapatra made changes -
          Description Original: There is a build which is stuck on intial stage.

          I saw in monitoring screen the below message:

          *Threads on 50aa853439c2@172.17.0.4:* Number = 110, Maximum = 526, Total started = 464,563
          Warning, the following threads are deadlocked : Handling POST /job/oc-cd-drop22/job/oc-cd-drop22-oc-cd-ocp/214/kill from 10.75.36.119 : qtp{color:#de350b}544724190-466789, jenkins.util.Timer [#9]{color}, OkHttp https://ilocpde07-master.corp.amdocs.com:8443/..., OkHttp [https://ilocpde07-master.corp.amdocs.com:8443/...]
           
          I tried to kill that thread but it's not getting deleted.
          Then I went to the threadDump log and saw:
          h2. jenkins.util.Timer [#9]
          "jenkins.util.Timer [#9]" Id=71 Group=main BLOCKED on java.lang.String@39c5d884 owned by "OkHttp https://ilocpde07-master.corp.amdocs.com:8443/..." Id=139290{color:#de350b} at io.fabric8.jenkins.openshiftsync{color}.JenkinsUtils.triggerJob(JenkinsUtils.java:422) - blocked on java.lang.String@39c5d884 at io.fabric8.jenkins.openshiftsync.BuildWatcher.addEventToJenkinsJobRun(BuildWatcher.java:345) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.handleBuildList(JenkinsUtils.java:815) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.maybeScheduleNext(JenkinsUtils.java:717) at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.onDeleted(BuildSyncRunListener.java:184) at hudson.model.listeners.RunListener.fireDeleted(RunListener.java:269) at hudson.model.Run.delete(Run.java:1573) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.deleteRun(JenkinsUtils.java:567) - locked java.lang.Class@b802391 at io.fabric8.jenkins.openshiftsync.BuildWatcher.reconcileRunsAndBuilds(BuildWatcher.java:494) at io.fabric8.jenkins.openshiftsync.BuildWatcher.access$100(BuildWatcher.java:65) at io.fabric8.jenkins.openshiftsync.BuildWatcher$1.doRun(BuildWatcher.java:148) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@28c61284
           
           
          so I guess it's something with the openshift sync plugin. can you suggest what is the next step I should try to do?           		New: There is a build which is stuck on intial stage.

          I saw in monitoring screen the below message:

          *Threads on 50aa853439c2@XXXXXX:* Number = 110, Maximum = 526, Total started = 464,563
           Warning, the following threads are deadlocked : Handling POST************/214/kill from XXXXX : qtp{color:#de350b}544724190-466789, jenkins.util.Timer [#9]{color}, OkHttp [https://*******.com:XXXX/|https://ilocpde07-master.corp.amdocs.com:8443/]..., OkHttp [https://*******.com:XXXX/...|https://ilocpde07-master.corp.amdocs.com:8443/...]
            
           I tried to kill that thread but it's not getting deleted.
           Then I went to the threadDump log and saw:
          h2. jenkins.util.Timer [#9]

          "jenkins.util.Timer [#9]" Id=71 Group=main BLOCKED on java.lang.String@39c5d884 owned by "OkHttp [https://*******.com:XXXX/|https://ilocpde07-master.corp.amdocs.com:8443/]..." Id=139290{color:#de350b} at io.fabric8.jenkins.openshiftsync{color}.JenkinsUtils.triggerJob(JenkinsUtils.java:422) - blocked on java.lang.String@39c5d884 at io.fabric8.jenkins.openshiftsync.BuildWatcher.addEventToJenkinsJobRun(BuildWatcher.java:345) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.handleBuildList(JenkinsUtils.java:815) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.maybeScheduleNext(JenkinsUtils.java:717) at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.onDeleted(BuildSyncRunListener.java:184) at hudson.model.listeners.RunListener.fireDeleted(RunListener.java:269) at hudson.model.Run.delete(Run.java:1573) at io.fabric8.jenkins.openshiftsync.JenkinsUtils.deleteRun(JenkinsUtils.java:567) - locked java.lang.Class@b802391 at io.fabric8.jenkins.openshiftsync.BuildWatcher.reconcileRunsAndBuilds(BuildWatcher.java:494) at io.fabric8.jenkins.openshiftsync.BuildWatcher.access$100(BuildWatcher.java:65) at io.fabric8.jenkins.openshiftsync.BuildWatcher$1.doRun(BuildWatcher.java:148) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@28c61284
            
            
           so I guess it's something with the openshift sync plugin. can you suggest what is the next step I should try to do?

            gmontero Gabe Montero
            arielma Ariel M
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: