Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-67036

Jenkins Powershell wrapper breaks valid Powershell scripts

    XMLWordPrintable

Details

    Description

      The following script fails when run in Jenkins:

      node("windows") {
        powershell('''\
          function Invoke-NativeCommand {
            # Do something useful here
          }
      
          function New-GitInvoker([string]$Path) {
            return {
              Invoke-NativeCommand git `-C $Path @args
            }.GetNewClosure()
          }
      		
          $git = New-GitInvoker 'testRepoPath'
          & $git fetch `-p ssh://someUrl
          '''.stripIndent()
        )
      }
      

      The error is:

      powershell.exe : Invoke-NativeCommand : The term 'Invoke-NativeCommand' is not recognized as the name of a cmdlet, function, script
      
      At D:\Jenkins\workspace\<job_name>@tmp\durable-38d496fa\powershellWrapper.ps1:3 char:1
      + & powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -Comm ...
      + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          + CategoryInfo          : NotSpecified: (Invoke-NativeCo...nction, script :String) [], RemoteException
          + FullyQualifiedErrorId : NativeCommandError
      file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
      
      At D:\Jenkins\workspace\<job_name>@tmp\durable-38d496fa\powershellScript.ps1:7 char:3
      +         Invoke-NativeCommand git `-C $Path @args
      +         ~~~~~~~~~~~~~~~~~~~~
          + CategoryInfo          : ObjectNotFound: (Invoke-NativeCommand:String) [], ParentContainsErrorRecordException
          + FullyQualifiedErrorId : CommandNotFoundException
      

      When run directly using PowerShell, the same script runs to completion without error.  Perhaps the wrapper messes up closures?

      A workaround exists - simply write the powershell script to a file and run it directly using powershell in the "bat" pipeline command:

      bat 'powershell -NoProfile -File myscript.ps1'

      Jenkins 2.289.1
      durable-task plugin version 1.37
      Pipeline: Nodes and Processes 2.39

      Attachments

        Activity

          I guess you're using Windows PowerShell 5.1. Do you get the same error with PowerShell 7?

          kon Kalle Niemitalo added a comment - I guess you're using Windows PowerShell 5.1. Do you get the same error with PowerShell 7?

          It seems the problem is that GetNewClosure makes a closure over variables but not over functions, and when powershellWrapper.ps1 executes powershellScript.ps1, the script block that attempts to run Invoke-NativeCommand is executed in a scope where Invoke-NativeCommand is not defined. The following should work:

              function New-GitInvoker([string]$Path) {
                $inc = Get-Command Invoke-NativeCommand
                return {
                  & $inc git `-C $Path @args
                }.GetNewClosure()
              }
          

          But the real fix would be to change lines 254, 276, and 282 of PowershellScript.java to use the dot sourcing operator (.) instead of the call operator (&), so that everything is evaluated in the same scope.

          kon Kalle Niemitalo added a comment - It seems the problem is that GetNewClosure makes a closure over variables but not over functions, and when powershellWrapper.ps1 executes powershellScript.ps1, the script block that attempts to run Invoke-NativeCommand is executed in a scope where Invoke-NativeCommand is not defined. The following should work: function New-GitInvoker([string]$Path) { $inc = Get-Command Invoke-NativeCommand return { & $inc git `-C $Path @args }.GetNewClosure() } But the real fix would be to change lines 254 , 276 , and 282 of PowershellScript.java to use the dot sourcing operator (.) instead of the call operator (&), so that everything is evaluated in the same scope.

          GetNewClosure makes a closure over variables but not over functions

          Specifically,  ScriptBlock.GetNewClosure calls PSModuleInfo.CaptureLocals, which only copies variables and not functions.

          kon Kalle Niemitalo added a comment - GetNewClosure makes a closure over variables but not over functions Specifically,  ScriptBlock.GetNewClosure calls PSModuleInfo.CaptureLocals , which only copies variables and not functions.

          I didn't find a PowerShell bug report for this behavior of GetNewClosure, but this 2019-03-26 comment on issue 6745 is a bit related.

          kon Kalle Niemitalo added a comment - I didn't find a PowerShell bug report for this behavior of GetNewClosure, but this 2019-03-26 comment on issue 6745 is a bit related.
          timpatt Tim Patterson added a comment - - edited

          kon - Thanks for such a complete and timely explanation! We're still running Powershell v5.1. It turns out that I've actually come across this kind of "capture" issue before when trying to create a lambda that refers to an object's $this field (which also isn't captured by GetNewClosure). I tried to use your work-around and it largely worked but I still hit another strange error so I think I'll stick with my hack solution for now; I've been using Powershell semi-regularly for a couple of years now and I still don't understand how it works?!! I suspect your "dot sourcing" solution should solve this problem, though... Thanks again!

          timpatt Tim Patterson added a comment - - edited kon - Thanks for such a complete and timely explanation! We're still running Powershell v5.1. It turns out that I've actually come across this kind of "capture" issue before when trying to create a lambda that refers to an object's $this field (which also isn't captured by GetNewClosure ). I tried to use your work-around and it largely worked but I still hit another strange error so I think I'll stick with my hack solution for now; I've been using Powershell semi-regularly for a couple of years now and I still don't understand how it works?!! I suspect your "dot sourcing" solution should solve this problem, though... Thanks again!

          People

            Unassigned Unassigned
            timpatt Tim Patterson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: