Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34581

Please add support for Pipelines to the Powershell plugin

      I would like to use the powershell plugin from within pipelines, please add support for the pipelines to the powershell plugin.

      This is especially important as we approach Jenkins 2.0 and its move towards pipelines.

          [JENKINS-34581] Please add support for Pipelines to the Powershell plugin

          Andrew Bayer added a comment -

          It'd be really great to get real Pipeline integration, a la sh and bat - i.e., a DurableTaskStep implementation for Powershell.

          Andrew Bayer added a comment - It'd be really great to get real Pipeline integration, a la sh and bat - i.e., a DurableTaskStep implementation for Powershell.

          Jesse Glick added a comment -

          For now you should use the bat step with powershell.exe. Making Powershell be a SimpleBuildStep is a bad idea.

          Jesse Glick added a comment - For now you should use the bat step with powershell.exe . Making Powershell be a SimpleBuildStep is a bad idea.

          Clint Chapman added a comment -

          jglick, Bad idea because?

          Clint Chapman added a comment - jglick , Bad idea because?

          Jesse Glick added a comment -

          Because it would not be durable.

          The choices are:

          • use JENKINS-37011 to provide something that looks for purposes of Snippet Generator and script syntax like a regular step, but which really just calls bat with some wrapper code
          • implement a first-class FileMonitoringTask that bypasses *.bat entirely

          Jesse Glick added a comment - Because it would not be durable. The choices are: use JENKINS-37011 to provide something that looks for purposes of Snippet Generator and script syntax like a regular step, but which really just calls bat with some wrapper code implement a first-class FileMonitoringTask that bypasses *.bat entirely

          Jesse Glick added a comment -

          I would vote for the second choice. Probably more robust. Might be easier to implement something like JENKINS-25053 in that mode too.

          Jesse Glick added a comment - I would vote for the second choice. Probably more robust. Might be easier to implement something like JENKINS-25053 in that mode too.

          Andrzej Korczynski added a comment - - edited

          jglick in the meantime, do you know a there a better way of handling powershell inside bat step ? so that when powershell script fails (for example it run some tests and they fail), the bat fails too and the build/pipeline fails ? Right now jenkins thinks the bat step went successfully and we need to do some tricks along the lines of http://jenkins-ci.361315.n4.nabble.com/Fail-the-build-if-the-Powershell-script-fails-td2164982.html

          I also vote for native PS support inside pipelines.

          Andrzej Korczynski added a comment - - edited jglick in the meantime, do you know a there a better way of handling powershell inside bat step ? so that when powershell script fails (for example it run some tests and they fail), the bat fails too and the build/pipeline fails ? Right now jenkins thinks the bat step went successfully and we need to do some tricks along the lines of http://jenkins-ci.361315.n4.nabble.com/Fail-the-build-if-the-Powershell-script-fails-td2164982.html I also vote for native PS support inside pipelines.

          Jesse Glick added a comment -

          I am the last person to ask for advice on Windows shell scripting matters. Try variants outside Jenkins until you find something that works.

          Jesse Glick added a comment - I am the last person to ask for advice on Windows shell scripting matters. Try variants outside Jenkins until you find something that works.

          Joel Reed added a comment -

          I'm eagerly following this issue as PowerShell is big part of our Jenkins use. The "implement a first-class FileMonitoringTask that bypasses *.bat entirely" mentioned above by Jesse is something I am eager and interested in evaluating/experiencing/testing. However as we are currently limited to Windows traditional command execution (and its binary feedback) via the bat mechanism I can speak to gotchas as it pertains to powershell.exe and success/failure behaviors. Both for Andrzej's concern above and others who may wind their way here. I've been using the bat step for a little while now and the traditional freestyle PowerShell plugin for some time.

          The biggest thing to keep in mind are PowerShell's idea of a terminating errors and non-terminating errors. Without extra effort PowerShell non-terminating errors are not going to cause Jenkins nor the Windows command execution that marries the two to be recognized as a "failure". Understanding your code and making sure that you generate terminating errors where needed to get that "failure" surfaced in Jenkins is critical. Additionally knowing when and how the PowerShell.exe interpreter registers the failure or success of native exe's vs cmdlets.

          Check the following URLs for clarity on that. Bottom line the "-ErrorAction Stop" builtin parameter to cmdlets is your friend here, along with try/catch/finally. And also making sure you understand "$LastExitCode vs $?", the builtin PowerShell variables and how you exit your script in the context of Jenkins.

          https://blogs.technet.microsoft.com/heyscriptingguy/2015/09/16/understanding-non-terminating-errors-in-powershell/
          http://stackoverflow.com/questions/10666035/difference-between-and-lastexitcode-in-powershell

          Joel Reed added a comment - I'm eagerly following this issue as PowerShell is big part of our Jenkins use. The "implement a first-class FileMonitoringTask that bypasses *.bat entirely" mentioned above by Jesse is something I am eager and interested in evaluating/experiencing/testing. However as we are currently limited to Windows traditional command execution (and its binary feedback) via the bat mechanism I can speak to gotchas as it pertains to powershell.exe and success/failure behaviors. Both for Andrzej's concern above and others who may wind their way here. I've been using the bat step for a little while now and the traditional freestyle PowerShell plugin for some time. The biggest thing to keep in mind are PowerShell's idea of a terminating errors and non-terminating errors. Without extra effort PowerShell non-terminating errors are not going to cause Jenkins nor the Windows command execution that marries the two to be recognized as a "failure". Understanding your code and making sure that you generate terminating errors where needed to get that "failure" surfaced in Jenkins is critical. Additionally knowing when and how the PowerShell.exe interpreter registers the failure or success of native exe's vs cmdlets. Check the following URLs for clarity on that. Bottom line the "-ErrorAction Stop" builtin parameter to cmdlets is your friend here, along with try/catch/finally. And also making sure you understand "$LastExitCode vs $?", the builtin PowerShell variables and how you exit your script in the context of Jenkins. https://blogs.technet.microsoft.com/heyscriptingguy/2015/09/16/understanding-non-terminating-errors-in-powershell/ http://stackoverflow.com/questions/10666035/difference-between-and-lastexitcode-in-powershell

          Kevin Browder added a comment -

          It does seem useful to have first class support for this given how limiting bat can be, however I think the following works in the normal "bat" step in the meantime:

          bat '''
          <# : BATCH
          @echo off & setlocal
          powershell -noprofile -NoLogo "iex (${%~f0} | out-string)"
          exit /b %errorlevel%
          : END BATCH #>
          Write-Output "Hello World"
          exit 2
          '''
          

          Super ugly but functional.
          Stolen from http://stackoverflow.com/a/33065387/2242609

          Kevin Browder added a comment - It does seem useful to have first class support for this given how limiting bat can be, however I think the following works in the normal "bat" step in the meantime: bat ''' <# : BATCH @echo off & setlocal powershell -noprofile -NoLogo "iex (${%~f0} | out-string)" exit /b %errorlevel% : END BATCH #> Write-Output "Hello World" exit 2 ''' Super ugly but functional. Stolen from http://stackoverflow.com/a/33065387/2242609

          Joel Reed added a comment -

          That is an interesting approach. What I've settled on is the following simple basic Jenkinfile example:

          node {
              checkout scm
          
              stage ('Build Module'){
                  bat "powershell.exe -NonInteractive -ExecutionPolicy Bypass -Command \"& '.\\01_build.ps1'\""
              }
             stage ('Test Module'){
                  bat "powershell.exe -NonInteractive -ExecutionPolicy Bypass -Command \"& '.\\02_test.ps1'\""
              }
             stage ('Deploy Module'){
                  bat "powershell.exe -NonInteractive -ExecutionPolicy Bypass -Command \"& '.\\03_deploy.ps1'\""
              }
          }
          

          There's a lot of escaping going on but I think it most closely resembles what the PowerShell freestyle plugin is doing and how it is executing the script code. It also keeps the "code" out of my Jenkinsfile, which I prefer to be more static. If my various .ps1 files are properly handling terminating and non-terminating errors appropriately then I'm getting the results I expect to see in my various stages. Jenkins then reflects that via build results or Blue Ocean or what not. I have this successfully working and supporting a number of traditional CI/CD PowerShell module build/test/deploy pipelines and a number of other more unique non-traditional pipelines that are doing more workflow and less CI/CD style things. Bottom line if my scripts fail they show fail and when they work the pipeline keeps going.

          Joel Reed added a comment - That is an interesting approach. What I've settled on is the following simple basic Jenkinfile example: node { checkout scm stage ( 'Build Module' ){ bat "powershell.exe -NonInteractive -ExecutionPolicy Bypass -Command \" & '.\\01_build.ps1' \"" } stage ( 'Test Module' ){ bat "powershell.exe -NonInteractive -ExecutionPolicy Bypass -Command \" & '.\\02_test.ps1' \"" } stage ( 'Deploy Module' ){ bat "powershell.exe -NonInteractive -ExecutionPolicy Bypass -Command \" & '.\\03_deploy.ps1' \"" } } There's a lot of escaping going on but I think it most closely resembles what the PowerShell freestyle plugin is doing and how it is executing the script code. It also keeps the "code" out of my Jenkinsfile, which I prefer to be more static. If my various .ps1 files are properly handling terminating and non-terminating errors appropriately then I'm getting the results I expect to see in my various stages. Jenkins then reflects that via build results or Blue Ocean or what not. I have this successfully working and supporting a number of traditional CI/CD PowerShell module build/test/deploy pipelines and a number of other more unique non-traditional pipelines that are doing more workflow and less CI/CD style things. Bottom line if my scripts fail they show fail and when they work the pipeline keeps going.

          Yogesh Bhole added a comment -

          Indeed you need to use lot of escaping (see example below) to execute simple statements. Adding PowerShell support without use of bat as workaround could save lot of time and efforts.

          bat "powershell Copy-Item -path ${mypath}\\Deployment\\Debug\\Server -Destination \\\\myserver\\d\$\\SQLServer -Force -Recurse"
          

          Yogesh Bhole added a comment - Indeed you need to use lot of escaping (see example below) to execute simple statements. Adding PowerShell support without use of bat as workaround could save lot of time and efforts. bat "powershell Copy-Item -path ${mypath}\\Deployment\\Debug\\Server -Destination \\\\myserver\\d\$\\SQLServer -Force -Recurse"

          Jesse Glick added a comment -

          kbrowder as an interim workaround, you could create a repo on github.com with vars/powershell.groovy so that people could

          @Library('github.com/kbrowser/powershell') import _
          powershell '''
          Write-Output "Hello World"
          exit 2
          '''
          

          Jesse Glick added a comment - kbrowder as an interim workaround, you could create a repo on github.com with vars/powershell.groovy so that people could @Library( 'github.com/kbrowser/powershell' ) import _ powershell ''' Write-Output "Hello World" exit 2 '''

          jglick What are the contents of vars/powershell.groovy? Where can I find that library?

          Marley Kudiabor added a comment - jglick What are the contents of vars/powershell.groovy? Where can I find that library?

          Jesse Glick added a comment -

          It does not exist, I was suggesting one be created.

          Jesse Glick added a comment - It does not exist, I was suggesting one be created.

          Do we have plan to support powershell in Jenkins pipeline in upcoming releases, or we need to rely on bat as suggested in this thread

          Vijay Tripathi added a comment - Do we have plan to support powershell in Jenkins pipeline in upcoming releases, or we need to rely on bat as suggested in this thread

          Jesse Glick added a comment -

          There is no immediate plan.

          Jesse Glick added a comment - There is no immediate plan.

          I have added powershell support to the durable task plugin, and I am currently testing it. I will respond to this thread once I am ready to submit a PR.

          Gabriel Loewen added a comment - I have added powershell support to the durable task plugin, and I am currently testing it. I will respond to this thread once I am ready to submit a PR.

          My changes are in the following repositories:

          https://github.com/gabloe/durable-task-plugin

          https://github.com/gabloe/workflow-durable-task-step-plugin

          All unit tests are passing during build, and I've tested it in Jenkins using some example pipeline code (See attached screenshots).

           

          Gabriel Loewen added a comment - My changes are in the following repositories: https://github.com/gabloe/durable-task-plugin https://github.com/gabloe/workflow-durable-task-step-plugin All unit tests are passing during build, and I've tested it in Jenkins using some example pipeline code (See attached screenshots).  

          gabloe Will these changes be rolled into the next version or will we need to pull your change and build it ourselves?

          Marley Kudiabor added a comment - gabloe Will these changes be rolled into the next version or will we need to pull your change and build it ourselves?

          Yogesh Bhole added a comment -

          Thanks gabloe for the support. Hoping powershell support will be available in Jenkins pipeline soon.

          Yogesh Bhole added a comment - Thanks gabloe for the support. Hoping powershell support will be available in Jenkins pipeline soon.

          @Marley Kudiabor - Please don't build out of my repo as the plugin may have changes made during the review process. This needs to be fully vetted and accepted into the maintained repository. So you should wait for the official support.

          I have submitted 2 pull requests for review:
          https://github.com/jenkinsci/workflow-durable-task-step-plugin/pull/35
          https://github.com/jenkinsci/durable-task-plugin/pull/39

          Thanks,
          Gabriel

          Gabriel Loewen added a comment - @Marley Kudiabor - Please don't build out of my repo as the plugin may have changes made during the review process. This needs to be fully vetted and accepted into the maintained repository. So you should wait for the official support. I have submitted 2 pull requests for review: https://github.com/jenkinsci/workflow-durable-task-step-plugin/pull/35 https://github.com/jenkinsci/durable-task-plugin/pull/39 Thanks, Gabriel

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/org/jenkinsci/plugins/durabletask/PowershellScript.java
          src/main/resources/org/jenkinsci/plugins/durabletask/Messages.properties
          src/main/resources/org/jenkinsci/plugins/durabletask/PowershellScript/config.jelly
          src/main/resources/org/jenkinsci/plugins/durabletask/PowershellScript/help-script.html
          src/test/java/org/jenkinsci/plugins/durabletask/PowershellScriptTest.java
          http://jenkins-ci.org/commit/durable-task-plugin/2a6b8ec0d93dd28537976517e217ce27c01a1bd8
          Log:
          Merge pull request #39 from gabloe/master

          [FIXED JENKINS-34581] Adding Powershell support

          Compare: https://github.com/jenkinsci/durable-task-plugin/compare/774f2ad4e48c...2a6b8ec0d93d

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/durabletask/PowershellScript.java src/main/resources/org/jenkinsci/plugins/durabletask/Messages.properties src/main/resources/org/jenkinsci/plugins/durabletask/PowershellScript/config.jelly src/main/resources/org/jenkinsci/plugins/durabletask/PowershellScript/help-script.html src/test/java/org/jenkinsci/plugins/durabletask/PowershellScriptTest.java http://jenkins-ci.org/commit/durable-task-plugin/2a6b8ec0d93dd28537976517e217ce27c01a1bd8 Log: Merge pull request #39 from gabloe/master [FIXED JENKINS-34581] Adding Powershell support Compare: https://github.com/jenkinsci/durable-task-plugin/compare/774f2ad4e48c...2a6b8ec0d93d

          What plugins, and what versions of plugins, will need to be updated to receive this?
          Just the durable task plugin and it's dependencies?

          Marley Kudiabor added a comment - What plugins, and what versions of plugins, will need to be updated to receive this? Just the durable task plugin and it's dependencies?

          Jesse Glick added a comment -

          Pipeline: Nodes and Processes suffices.

          Jesse Glick added a comment - Pipeline: Nodes and Processes suffices.

          Brad Wehmeier added a comment -

          In my environment, executing this on a slave agent does not execute the PowerShell commands in the workspace directory. (Linux Master, Windows Slave).

          Brad Wehmeier added a comment - In my environment, executing this on a slave agent does not execute the PowerShell commands in the workspace directory. (Linux Master, Windows Slave).

          Jesse Glick added a comment -

          bradleywehmeier if you have discovered any bugs, please file separate issues (with a link to this one) containing complete, minimal steps to reproduce from scratch (and ideally also a pull request correcting the behavior).

          Jesse Glick added a comment - bradleywehmeier if you have discovered any bugs, please file separate issues (with a link to this one) containing complete, minimal steps to reproduce from scratch (and ideally also a pull request correcting the behavior).

          Gabriel Loewen added a comment - - edited

          bradleywehmeier Which directory does your command get executed in? I tested in my env, which is also a Linux Master with a Windows Slave and I see the following.  I expect the commands to be executed under C:\Program Files (x86)\Jenkins\workspace\TestWS.

          Observations

          Input:

          Get the working directory from a PowerShell step as well as a Batch script step: 

          node {
               powershell '$(pwd).Path'
               bat 'echo %cd%'
           }
          

          Output:

          [Pipeline] {
          [Pipeline] powershell
          [TestWS] Running PowerShell script
          C:\Program Files (x86)\Jenkins\workspace\TestWS
          [Pipeline] bat
          [TestWS] Running batch script
          
          C:\Program Files (x86)\Jenkins\workspace\TestWS>echo C:\Program Files (x86)\Jenkins\workspace\TestWS 
          C:\Program Files (x86)\Jenkins\workspace\TestWS
          [Pipeline] }
          [Pipeline] // node
          [Pipeline] End of Pipeline
          Finished: SUCCESS
          

           

          Can you share what your results look like?

          Gabriel Loewen added a comment - - edited bradleywehmeier Which directory does your command get executed in? I tested in my env, which is also a Linux Master with a Windows Slave and I see the following.  I expect the commands to be executed under C:\Program Files (x86)\Jenkins\workspace\TestWS. Observations Input: Get the working directory from a PowerShell step as well as a Batch script step:  node {     powershell '$(pwd).Path'     bat 'echo %cd%' } Output: [Pipeline] { [Pipeline] powershell [TestWS] Running PowerShell script C:\Program Files (x86)\Jenkins\workspace\TestWS [Pipeline] bat [TestWS] Running batch script C:\Program Files (x86)\Jenkins\workspace\TestWS>echo C:\Program Files (x86)\Jenkins\workspace\TestWS C:\Program Files (x86)\Jenkins\workspace\TestWS [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS   Can you share what your results look like?

          Brad Wehmeier added a comment -

          gabloe, I determined the difference was because it was loading the PowerShell Profile for the user the build agent was executing as. I'll spin this off to another issue at this point (following Jesse's guidance). It will probably be an improvement to optionally set the -NoProfile flag when calling the PowerShell executable.

          Brad Wehmeier added a comment - gabloe , I determined the difference was because it was loading the PowerShell Profile for the user the build agent was executing as. I'll spin this off to another issue at this point (following Jesse's guidance). It will probably be an improvement to optionally set the -NoProfile  flag when calling the PowerShell executable.

          Bharat Bhatia added a comment -

          gabloparm I am using Write-Host in my powershell script but dont see any output in console. Do you know if there is any other way to produce console output.

          Bharat Bhatia added a comment - gabloparm I am using Write-Host in my powershell script but dont see any output in console. Do you know if there is any other way to produce console output.

          Write-Host does not produce any output that can be consumed (e.g. written to the log). You can simply replace all of your calls to Write-Host with calls to Write-Output.

          Gabriel Loewen added a comment - Write-Host does not produce any output that can be consumed (e.g. written to the log). You can simply replace all of your calls to Write-Host with calls to Write-Output.

          Gabriel Loewen added a comment - Here's a few things to read to understand why this is the behavior of Write-Host: http://www.jsnover.com/blog/2013/12/07/write-host-considered-harmful/ http://m.windowsitpro.com/windows/write-output-or-write-host-powershell

          gabloe Running powershell in the Jenkins Pipeline is hardcoded to use no profile. The powershell plugin defaults to the opposite. Can this be made configurable?

          sebastian slutzky added a comment - gabloe Running powershell in the Jenkins Pipeline is hardcoded to use no profile . The powershell plugin defaults to the opposite. Can this be made configurable?

            gabloe Gabriel Loewen
            stuartwhelan Stuart Whelan
            Votes:
            40 Vote for this issue
            Watchers:
            51 Start watching this issue

              Created:
              Updated:
              Resolved: