Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-76064

PowerShell build step never starts and hangs indefinitely

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • durable-task-plugin
    • None
    • Jenkins 2.492.3
      Azure VM Agents 1013.v7a_2a_cd831714
      Durable Task Plugin v587.v84b_877235b_45

      We will see something like the following in the console output:

      [build] $ powershell.exe -NoProfile -NonInteractive -ExecutionPolicy Bypass -File C:\Users\user\AppData\Local\Temp\jenkins14472050587877702380.ps1

      But the step will never start.

      When we login/RDP to the windows agent and try to run the following, a console window opens and hangs:

      powershell.exe -NoProfile -NonInteractive

      And once this happens on one Jenkins agent, it seems to immediately spread to all other agents - so they all begin hanging on any PowerShell build step (even in the middle of existing ones), no matter how old each agent is.  This does not appear to be an agent imaging issue, as most agents have been successfully processing PowerShell build steps for hours before this happens.  And it does not appear to be a specific build step issue, as once this happens all PowerShell build steps cease to function.  And it does not appear to be an agent resource issue, as we have no problem connecting remotely to the agent and inspecting disk, CPU, and memory and they are plentiful.

      We saw that a flaw in the Durable Task Plugin caused this or something very similar in the past.  So we just upgraded our Durable Task Plugin from v587.v84b_877235b_45 to v595.ve87b_f1318d67, but that did not fix the issue. 

      The only mitigation we've found so far is to take the agent offline, abort the build, restart the agent, bring it back online, retry the build.  This also fixes PowerShell hanging when we login and try to run a PowerShell session.  We have hundreds of PowerShell build steps run every day, so this is not a viable short-term mitigation strategy.

            Unassigned Unassigned
            bboss Bret
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: