Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63082

Deadlock launching remote agent

    XMLWordPrintable

Details

    • 2.253

    Description

      We are seeing the bug originally described in JENKINS-62181. Remote agents hang on launch. This is using a supposedly patched release.

      In our experience, we are seeing a correct launch of this agent the first time after Jenkins is rebooted. Then, after the agent times out (in-demand delay 1, idle delay 5) and goes down, it cannot be restarting, it deadlocks on launch.

      It hangs here:

      <===[JENKINS REMOTING CAPACITY]===>channel started
      Remoting version: 4.3
      This is a Unix agent

      I don't know how to put in a test for Java deadlocks. Please advise. This is blocking usage.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-62181 Remoting deadlock when launching remote agent

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-63458 Agent deadlock associated with SlaveInitializer, part III

          • Blocker - Blocks development and/or testing work, production could not run.
          • Resolved
          links to

          Web Link jenkins #4886

          Activity

            Ascending order - Click to sort in descending order
            View 10 older comments
            jpfeuffer Julianus Pfeuffer added a comment -

            Yes, it seems to work now. At first, I still had some hiccups (on far fewer slaves) since there were still some (hung up) java processes running on the slaves but after a restart of those, all of them are up again.

            jpfeuffer Julianus Pfeuffer added a comment - Yes, it seems to work now. At first, I still had some hiccups (on far fewer slaves) since there were still some (hung up) java processes running on the slaves but after a restart of those, all of them are up again.
            vegar_andersen Vegar Andersen added a comment -

            I have upgraded to version 2.253 and we are still experiencing problems with this exact issue. After upgrading and restarting more than half of our nodes deadlock in the manner described above. I have not been able to identify a procedure that will bring them up again dependably. Is anyone else having better luck at this?

            vegar_andersen Vegar Andersen added a comment - I have upgraded to version 2.253 and we are still experiencing problems with this exact issue. After upgrading and restarting more than half of our nodes deadlock in the manner described above. I have not been able to identify a procedure that will bring them up again dependably. Is anyone else having better luck at this?
            jglick Jesse Glick added a comment -

            vegar_andersen please attach a thread dump.

            jglick Jesse Glick added a comment - vegar_andersen please attach a thread dump.
            f5k Fredrik de Vibe added a comment - - edited

            jglick I just attached one, it's from the same Jenkins instance that vegar_andersen refers to.

            f5k Fredrik de Vibe added a comment - - edited jglick I just attached one, it's from the same Jenkins instance that vegar_andersen refers to.
            jglick Jesse Glick added a comment -

            f5k vegar_andersen thanks, I have filed your deadlock as JENKINS-63458.

            jglick Jesse Glick added a comment - f5k vegar_andersen thanks, I have filed your deadlock as JENKINS-63458 .

            People

              jglick Jesse Glick
              gehlhaar Dan Gehlhaar
              Votes:
              3 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: