Currently, diffing files requires using OS specific implementations (sh, bat).

      This has the potential of triggering security approval, which can be an inconvenience.

      Proposing add new `diff` utility step, to allow OS agnostic use in PIpeline jobs.

          [JENKINS-37492] Propose adding `diff` utility step

          Patrick Wolf added a comment -

          owood why would a sh or bat trigger a security approval? Script security is based on sandbox groovy execution. Anything run in a sh or bat step is executed on the build agent, not the master, and this will work on the files locally on the build agent. After JENKINS-26055 is complete we could build a 'diff' step that is OS agnostic and runs on the build agents but doing all of this work on the master seems inefficient.

          Patrick Wolf added a comment - owood why would a sh or bat trigger a security approval? Script security is based on sandbox groovy execution. Anything run in a sh or bat step is executed on the build agent, not the master, and this will work on the files locally on the build agent. After JENKINS-26055 is complete we could build a 'diff' step that is OS agnostic and runs on the build agents but doing all of this work on the master seems inefficient.

          Ron Dagostino added a comment -

          Actually, I think the security issue is that it is not possible to use the Java File API without having to approve a method that – according to the script approval UI – opens a security vulnerability.

          Ron Dagostino added a comment - Actually, I think the security issue is that it is not possible to use the Java File API without having to approve a method that – according to the script approval UI – opens a security vulnerability.

          Patrick Wolf added a comment -

          Steps defined in the Global Library or in a Plugin aren't run in the Pipeline Sandbox. Plugins can do pretty much anything and only Jenkins admins can define functions in the Global library. Admins are intrinsically trusted in Jenkins so these functions are not sandboxed.

          Someone could create a new plugin that implements a diff step using the Java File API will no script security issues.

          Alternatively, if someone can define a Pipeline step to do this in a Global Library it could be shared here. https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples

          Patrick Wolf added a comment - Steps defined in the Global Library or in a Plugin aren't run in the Pipeline Sandbox. Plugins can do pretty much anything and only Jenkins admins can define functions in the Global library. Admins are intrinsically trusted in Jenkins so these functions are not sandboxed. Someone could create a new plugin that implements a diff step using the Java File API will no script security issues. Alternatively, if someone can define a Pipeline step to do this in a Global Library it could be shared here. https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples

          Ron Dagostino added a comment -

          I just added vars/getFile.groovy to my Jenkins global library as follows:

          String call(name) {
          new java.io.File(name)
          }
          this

          Then I tried to invoke it from a Jenkinsfile (e.g. "echo getFile('/tmp').getAbsolutePath()") and received the error "org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String"
          with this stack trace:

          org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String
          at org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist.rejectNew(StaticWhitelist.java:185)
          at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onNewInstance(SandboxInterceptor.java:130)
          at org.kohsuke.groovy.sandbox.impl.Checker$3.call(Checker.java:191)
          at org.kohsuke.groovy.sandbox.impl.Checker.checkedConstructor(Checker.java:188)
          at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.constructorCall(SandboxInvoker.java:19)
          etc...

          When I look at the script approval UI I see "new java.io.File java.lang.String Approving this signature may introduce a security vulnerability! You are advised to deny it."

          Am I missing something? It seems to me that the sandbox applies to the global library plugin at https://github.com/jenkinsci/workflow-cps-global-lib-plugin.

          Ron Dagostino added a comment - I just added vars/getFile.groovy to my Jenkins global library as follows: String call(name) { new java.io.File(name) } this Then I tried to invoke it from a Jenkinsfile (e.g. "echo getFile('/tmp').getAbsolutePath()") and received the error "org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String" with this stack trace: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String at org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist.rejectNew(StaticWhitelist.java:185) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onNewInstance(SandboxInterceptor.java:130) at org.kohsuke.groovy.sandbox.impl.Checker$3.call(Checker.java:191) at org.kohsuke.groovy.sandbox.impl.Checker.checkedConstructor(Checker.java:188) at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.constructorCall(SandboxInvoker.java:19) etc... When I look at the script approval UI I see "new java.io.File java.lang.String Approving this signature may introduce a security vulnerability! You are advised to deny it." Am I missing something? It seems to me that the sandbox applies to the global library plugin at https://github.com/jenkinsci/workflow-cps-global-lib-plugin .

          Patrick Wolf added a comment -

          https://issues.jenkins-ci.org/browse/JENKINS-34650

          Make sure you are running a current update of Pipeline so this change is included. It was a recent change by Kohsuke.

          In your example, it seems that you are still reference the File.getAbsolutePath() call in your Jenkinsfile. Rather, what happens if you change your global script to the specific call:

          String call(name)

          { new java.io.File(name).getAbsolutePath() }

          this

          Or you could pass in the name of the file as first param and the action to return on the second param. As long as you are referencing a File object in your Jenkinsfile directly you will get this sandbox warning.

          Patrick Wolf added a comment - https://issues.jenkins-ci.org/browse/JENKINS-34650 Make sure you are running a current update of Pipeline so this change is included. It was a recent change by Kohsuke. In your example, it seems that you are still reference the File.getAbsolutePath() call in your Jenkinsfile. Rather, what happens if you change your global script to the specific call: String call(name) { new java.io.File(name).getAbsolutePath() } this Or you could pass in the name of the file as first param and the action to return on the second param. As long as you are referencing a File object in your Jenkinsfile directly you will get this sandbox warning.

          Ron Dagostino added a comment -

          Thanks. The below script does not work for me (I still get "new java.io.File java.lang.String Approving this signature may introduce a security vulnerability! You are advised to deny it."), and I am not running the latest workflow plugin, so it looks like I will have to upgrade to a newer core to get the fix from https://issues.jenkins-ci.org/browse/JENKINS-34650. I agree that adding a diff script is a relatively easy thing to do when that fix is applied, and the script could be shared at https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples as you suggest.

          String call (String path) {
          getFilePathNonCPS(path)
          }
          @NonCPS
          String getFilePathNonCPS(String path) {
          new java.io.File(path).getAbsolutePath()
          }
          this

          Ron Dagostino added a comment - Thanks. The below script does not work for me (I still get "new java.io.File java.lang.String Approving this signature may introduce a security vulnerability! You are advised to deny it."), and I am not running the latest workflow plugin, so it looks like I will have to upgrade to a newer core to get the fix from https://issues.jenkins-ci.org/browse/JENKINS-34650 . I agree that adding a diff script is a relatively easy thing to do when that fix is applied, and the script could be shared at https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples as you suggest. String call (String path) { getFilePathNonCPS(path) } @NonCPS String getFilePathNonCPS(String path) { new java.io.File(path).getAbsolutePath() } this

          Patrick Wolf added a comment -

          I apologize. I didn't think about the context before rndgstn. What you are trying to do won't work anyway.

          A java.io.File works on the local filesystem but most times you will want to diff two files in your workspace. Pipeline executes on the master in a flyweight executor so java.io.File has a context of the master - hence the security exception.

          What you want is this: http://javadoc.jenkins-ci.org/hudson/FilePath.html

          The best working example would be the readFile step: https://github.com/jenkinsci/workflow-basic-steps-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/steps/ReadFileStep.java

          This takes a hudson.FilePath and streams it. If you wanted to do a diff it would be very similar but take two file parameters and compare them.

          Patrick Wolf added a comment - I apologize. I didn't think about the context before rndgstn . What you are trying to do won't work anyway. A java.io.File works on the local filesystem but most times you will want to diff two files in your workspace. Pipeline executes on the master in a flyweight executor so java.io.File has a context of the master - hence the security exception. What you want is this: http://javadoc.jenkins-ci.org/hudson/FilePath.html The best working example would be the readFile step: https://github.com/jenkinsci/workflow-basic-steps-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/steps/ReadFileStep.java This takes a hudson.FilePath and streams it. If you wanted to do a diff it would be very similar but take two file parameters and compare them.

          Michael Neale added a comment - - edited

          I understand the platform agnostic angle, but "This has the potential of triggering security approval, which can be an inconvenience." - I don't see how this is the case if you are doing the diff inside sh or bat?

          In terms of a diff step - I assume you mean it takes 3 params: 2 input files, and an output file? or would it take 2 strings and produce a string? or some combination?

          Michael Neale added a comment - - edited I understand the platform agnostic angle, but "This has the potential of triggering security approval, which can be an inconvenience." - I don't see how this is the case if you are doing the diff inside sh or bat? In terms of a diff step - I assume you mean it takes 3 params: 2 input files, and an output file? or would it take 2 strings and produce a string? or some combination?

          Ron Dagostino added a comment -

          Thanks, hrmpw, for the FilePath vs. FIle info. I have also confirmed that the shared global library code is trusted implicitly in recent versions as per https://issues.jenkins-ci.org/browse/JENKINS-34650, so yes, it would be possible to simply write the script and share it at https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples. I will do so if I end up writing it.

          michaelneale, I am personally interested in a simple "are two files the same?" step, not full diff functionality.

          Ron Dagostino added a comment - Thanks, hrmpw , for the FilePath vs. FIle info. I have also confirmed that the shared global library code is trusted implicitly in recent versions as per https://issues.jenkins-ci.org/browse/JENKINS-34650 , so yes, it would be possible to simply write the script and share it at https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples . I will do so if I end up writing it. michaelneale , I am personally interested in a simple "are two files the same?" step, not full diff functionality.

          Michael Neale added a comment -

          rndgstn right - well that would be far easier to do as a step I think, or global var lib.

          Michael Neale added a comment - rndgstn right - well that would be far easier to do as a step I think, or global var lib.

            rsandell rsandell
            owood Owen Wood
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: