Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44784

Branch indexing blocked by workspace cleanup hung on bad connection

XMLWordPrintable

      Branch indexing was observed to hang in several threads:

      "Executor #-1 for master : executing BranchIndexing[...] / waiting for ... state=TIMED_WAITING ...
    - waiting on <...> (a hudson.remoting.UserRequest)
    - locked <...> (a hudson.remoting.UserRequest)
    at java.lang.Object.wait(Native Method)
    at hudson.remoting.Request.call(Request.java:147)
    at hudson.remoting.Channel.call(Channel.java:829)
    at hudson.FilePath.act(FilePath.java:985)
    at hudson.FilePath.act(FilePath.java:974)
    at hudson.FilePath.isDirectory(FilePath.java:1507)
    at jenkins.branch.WorkspaceLocatorImpl$Deleter.cleanUp(WorkspaceLocatorImpl.java:133)
    at jenkins.branch.WorkspaceLocatorImpl$Deleter.onDeleted(WorkspaceLocatorImpl.java:125)
    at hudson.model.listeners.ItemListener$4.apply(ItemListener.java:205)
    at hudson.model.listeners.ItemListener$4.apply(ItemListener.java:203)
    at hudson.model.listeners.ItemListener.forAll(ItemListener.java:167)
    at hudson.model.listeners.ItemListener.fireOnDeleted(ItemListener.java:203)
    at com.cloudbees.hudson.plugins.folder.AbstractFolder.onDeleted(AbstractFolder.java:1195)
    at com.cloudbees.hudson.plugins.folder.AbstractFolder.onDeleted(AbstractFolder.java:149)
    at hudson.model.AbstractItem.delete(AbstractItem.java:575)
    at hudson.model.Job.delete(Job.java:683)
    at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277)
    at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:162)
    at jenkins.branch.MultiBranchProject$BranchIndexing.run(MultiBranchProject.java:967)
    at hudson.model.ResourceController.execute(ResourceController.java:98)
    at hudson.model.Executor.run(Executor.java:405)

      Apparently the agent connection had gone sour, which may indicate a bug in Remoting. Regardless, I have observed cases where, say, FilePath.isDirectory just hangs indefinitely for no reason I have been able to diagnose or reproduce. The Timeout utility in Pipeline tries to work around this for specific cases, introduced for JENKINS-32986. The fix of JENKINS-34564 should be amended to perform workspace cleanup asynchronously; there is no reason to block the thread calling Job.delete (usually branch indexing; for dead branches, could potentially be an HTTP request).

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: