Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-71281

Powershell with returnStdout hangs when using Invoke-Command

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • Jenkins 2.387.2
      PowerShell plugin (powershell): 2.0
      Pipeline: Nodes and Processes (workflow-durable-task-step): 1241.v1a_63e465f943
      Windows Server 2019 Standard version 1809
      PS version : 7.2.1

      I have a weird behavior when using the command pwsh with returnStdout set to true and the powershell cmdlet Invoke-Command.
      Consider the following pipeline :

      pipeline {
          agent { label 'JenkinsAgentWithPowershell7' }
          options {
              timestamps()
              timeout(time: 2, unit: 'MINUTES')
          }
          parameters {
              string(name: 'amountLoop', defaultValue: '80', description: 'Integer loop Amount')
          }
          stages {
              stage("Stage Test") {
                  steps {
                      script {
                          try {
                              def resultPwsh = pwsh(
                                  returnStdout: true,
                                  returnStatus: false,
                                  script: '''
                                      $start = Get-Date
                                      $data = $null
                                      $amountLoop = \'''' + amountLoop + '''\'
      
                                      [scriptblock]$scriptblock = {
                                          $int_max_value = $using:amountLoop
      
                                          Write-Host "In invoke [elapsed time=$((Get-Date) - $Using:start)]"
      
                                          $array_value = @()
                                          (1..$int_max_value) | Foreach-Object {
                                              $array_value += "guinea_pig_$($_)"
                                          }
      
                                          $array_value | Foreach-Object {
                                              Write-Host "[Foreach-Object] value = $($_) [elapsed time=$((Get-Date) - $Using:start)]"
                                          }
                                          foreach($value in $array_value){
                                              Write-Host "[foreach] value = $($value) [elapsed time=$((Get-Date) - $Using:start)]"
                                          }
      
                                          $response = $array_value | ConvertTo-Json -Compress
                                          return $response
                                      }
                                      Write-Host "call invoke command [elapsed time=$((Get-Date) - $start)]"
                                      $splatting_invoke_command = @{
                                          ConfigurationName = 'Powershell.7'
                                          ComputerName      = "$( [System.Net.Dns]::GetHostByName($env:computerName).HostName )"
                                          ScriptBlock       = $scriptblock
                                      }
                                      $data = Invoke-Command @splatting_invoke_command
                                      Write-Host "data from invoke = $($data) [elapsed time=$((Get-Date) - $start)]"
                                      
                                      return $data
                                  '''
                              )
      
                              echo "resultPwsh=${resultPwsh}"
                          }
                          catch (err) {
                              error "Error catched : ${err.toString()}"
                          }
                      }
                  }
              }
          }
      }
      

       

      If I execute the powershell script in a powershell console it works fine.
      When I launch the pipeline if the amount of loop is low (example 5) the script works fine.

       

      09:13:37  Timeout set to expire in 2 min 0 sec
      [Pipeline] {
      [Pipeline] stage
      [Pipeline] { (Stage Test)
      [Pipeline] script
      [Pipeline] {
      [Pipeline] pwsh
      09:13:38  call invoke command [elapsed time=00:00:00.0010786]
      09:13:39  In invoke [elapsed time=00:00:00.7243390]
      09:13:39  [Foreach-Object] value = guinea_pig_1 [elapsed time=00:00:00.7550209]
      09:13:39  [Foreach-Object] value = guinea_pig_2 [elapsed time=00:00:00.7564009]
      09:13:39  [Foreach-Object] value = guinea_pig_3 [elapsed time=00:00:00.7573834]
      09:13:39  [Foreach-Object] value = guinea_pig_4 [elapsed time=00:00:00.7582481]
      09:13:39  [Foreach-Object] value = guinea_pig_5 [elapsed time=00:00:00.7590925]
      09:13:39  [foreach] value = guinea_pig_1 [elapsed time=00:00:00.7602465]
      09:13:39  [foreach] value = guinea_pig_2 [elapsed time=00:00:00.7611029]
      09:13:39  [foreach] value = guinea_pig_3 [elapsed time=00:00:00.7619235]
      09:13:39  [foreach] value = guinea_pig_4 [elapsed time=00:00:00.7627388]
      09:13:39  [foreach] value = guinea_pig_5 [elapsed time=00:00:00.7635645]
      09:13:39  data from invoke = ["guinea_pig_1","guinea_pig_2","guinea_pig_3","guinea_pig_4","guinea_pig_5"] [elapsed time=00:00:00.8128529]
      [Pipeline] echo
      09:13:39  resultRunAs=["guinea_pig_1","guinea_pig_2","guinea_pig_3","guinea_pig_4","guinea_pig_5"]
      09:13:39  
      [Pipeline] }
      [Pipeline] // script
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] }
      [Pipeline] // timestamps
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      Finished: SUCCESS 

      But if I increase the amount of loop (example 80) the script hangs and timeout.

       

      09:14:37  Timeout set to expire in 2 min 0 sec
      [Pipeline] {
      [Pipeline] stage
      [Pipeline] { (Stage Test)
      [Pipeline] script
      [Pipeline] {
      [Pipeline] pwsh
      09:14:38  call invoke command [elapsed time=00:00:00.0009059]
      09:14:39  In invoke [elapsed time=00:00:00.7209058]
      09:14:39  [Foreach-Object] value = guinea_pig_1 [elapsed time=00:00:00.7636166]
      09:14:39  [Foreach-Object] value = guinea_pig_2 [elapsed time=00:00:00.7654486]
      09:14:39  [Foreach-Object] value = guinea_pig_3 [elapsed time=00:00:00.7668519]
      09:14:39  [Foreach-Object] value = guinea_pig_4 [elapsed time=00:00:00.7681719]
      09:14:39  [Foreach-Object] value = guinea_pig_5 [elapsed time=00:00:00.7694217]
      09:14:39  [Foreach-Object] value = guinea_pig_6 [elapsed time=00:00:00.7706422]
      09:14:39  [Foreach-Object] value = guinea_pig_7 [elapsed time=00:00:00.7718780]
      09:14:39  [Foreach-Object] value = guinea_pig_8 [elapsed time=00:00:00.7731238]
      09:14:39  [Foreach-Object] value = guinea_pig_9 [elapsed time=00:00:00.7743461]
      09:14:39  [Foreach-Object] value = guinea_pig_10 [elapsed time=00:00:00.7755632]
      09:14:39  [Foreach-Object] value = guinea_pig_11 [elapsed time=00:00:00.7767536]
      09:14:39  [Foreach-Object] value = guinea_pig_12 [elapsed time=00:00:00.7778299]
      09:14:39  [Foreach-Object] value = guinea_pig_13 [elapsed time=00:00:00.7789158]
      09:14:39  [Foreach-Object] value = guinea_pig_14 [elapsed time=00:00:00.7800974]
      09:14:39  [Foreach-Object] value = guinea_pig_15 [elapsed time=00:00:00.7812100]
      09:14:39  [Foreach-Object] value = guinea_pig_16 [elapsed time=00:00:00.7823295]
      09:14:39  [Foreach-Object] value = guinea_pig_17 [elapsed time=00:00:00.7835177]
      09:14:39  [Foreach-Object] value = guinea_pig_18 [elapsed time=00:00:00.7845542]
      09:14:39  [Foreach-Object] value = guinea_pig_19 [elapsed time=00:00:00.7856747]
      09:14:39  [Foreach-Object] value = guinea_pig_20 [elapsed time=00:00:00.7867954]
      09:14:39  [Foreach-Object] value = guinea_pig_21 [elapsed time=00:00:00.8130105]
      09:14:39  [Foreach-Object] value = guinea_pig_22 [elapsed time=00:00:00.8145213]
      09:14:39  [Foreach-Object] value = guinea_pig_23 [elapsed time=00:00:00.8157142]
      09:14:39  [Foreach-Object] value = guinea_pig_24 [elapsed time=00:00:00.8167274]
      09:14:39  [Foreach-Object] value = guinea_pig_25 [elapsed time=00:00:00.8177801]
      09:14:39  [Foreach-Object] value = guinea_pig_26 [elapsed time=00:00:00.8187570]
      09:14:39  [Foreach-Object] value = guinea_pig_27 [elapsed time=00:00:00.8196644]
      09:14:39  [Foreach-Object] value = guinea_pig_28 [elapsed time=00:00:00.8206429]
      09:14:39  [Foreach-Object] value = guinea_pig_29 [elapsed time=00:00:00.8216685]
      09:14:39  [Foreach-Object] value = guinea_pig_30 [elapsed time=00:00:00.8226360]
      09:14:39  [Foreach-Object] value = guinea_pig_31 [elapsed time=00:00:00.8236376]
      09:14:39  [Foreach-Object] value = guinea_pig_32 [elapsed time=00:00:00.8246294]
      09:14:39  [Foreach-Object] value = guinea_pig_33 [elapsed time=00:00:00.8255654]
      09:14:39  [Foreach-Object] value = guinea_pig_34 [elapsed time=00:00:00.8264886]
      09:14:39  [Foreach-Object] value = guinea_pig_35 [elapsed time=00:00:00.8275127]
      09:14:39  [Foreach-Object] value = guinea_pig_36 [elapsed time=00:00:00.8288818]
      09:14:39  [Foreach-Object] value = guinea_pig_37 [elapsed time=00:00:00.8299226]
      09:14:39  [Foreach-Object] value = guinea_pig_38 [elapsed time=00:00:00.8310028]
      09:14:39  [Foreach-Object] value = guinea_pig_39 [elapsed time=00:00:00.8321239]
      09:14:39  [Foreach-Object] value = guinea_pig_40 [elapsed time=00:00:00.8331686]
      09:14:39  [Foreach-Object] value = guinea_pig_41 [elapsed time=00:00:00.8341904]
      09:14:39  [Foreach-Object] value = guinea_pig_42 [elapsed time=00:00:00.8352166]
      09:14:39  [Foreach-Object] value = guinea_pig_43 [elapsed time=00:00:00.8362682]
      09:14:39  [Foreach-Object] value = guinea_pig_44 [elapsed time=00:00:00.8372438]
      09:14:39  [Foreach-Object] value = guinea_pig_45 [elapsed time=00:00:00.8434064]
      09:14:39  [Foreach-Object] value = guinea_pig_46 [elapsed time=00:00:00.8444876]
      09:14:39  [Foreach-Object] value = guinea_pig_47 [elapsed time=00:00:00.8455167]
      09:14:39  [Foreach-Object] value = guinea_pig_48 [elapsed time=00:00:00.8464736]
      09:14:39  [Foreach-Object] value = guinea_pig_49 [elapsed time=00:00:00.8474714]
      09:14:39  [Foreach-Object] value = guinea_pig_50 [elapsed time=00:00:00.8484266]
      09:14:39  [Foreach-Object] value = guinea_pig_51 [elapsed time=00:00:00.8493614]
      09:14:39  [Foreach-Object] value = guinea_pig_52 [elapsed time=00:00:00.8502628]
      09:14:39  [Foreach-Object] value = guinea_pig_53 [elapsed time=00:00:00.8512897]
      09:14:39  [Foreach-Object] value = guinea_pig_54 [elapsed time=00:00:00.8522533]
      09:14:39  [Foreach-Object] value = guinea_pig_55 [elapsed time=00:00:00.8531861]
      09:14:39  [Foreach-Object] value = guinea_pig_56 [elapsed time=00:00:00.8542015]
      09:14:39  [Foreach-Object] value = guinea_pig_57 [elapsed time=00:00:00.8550948]
      09:16:37  Cancelling nested steps due to timeout
      09:16:37  Sending interrupt signal to process
      09:16:57  After 20s process did not stop
      [Pipeline] error
      [Pipeline] }
      [Pipeline] // script
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] }
      [Pipeline] // timestamps
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      ERROR: Error catched : org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
      Finished: FAILURE 

       

      If I set the pwsh parameter returnStdout to false, then the script works fine whatever the amount of loop.

            Unassigned Unassigned
            scivray47 Sylvain
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: