• workflow-cps 2.75

      Hello,

      Following the introduction of a new warning log message in version 2.7.1 of the Pipeline:Groovy plugin, we are now receiving types of the following warning in builds using the Artifactory Plugin:

      expected to call org.jfrog.hudson.pipeline.common.types.ArtifactoryServer.upload but wound up catching artifactoryUpload
      

      The pipeline API supported by the Artifactory plugin creates objects, on which methods can be invoked.
      For example:

      def server = Artifactory.server 'my-server'
      server.upload ...
      

      Notice that the 'server' object allows executing the 'artifactoryUpload' step.
      This OOP style is very convenient and allows reusing code and configuration. This approach received very positive feedbacks since it was introduced.
      Currently however, this is achieved by having the 'server' instance store the CpsScript, so that it can be used to invoke the step (by using 'cpsScript.invokeMethod').

      The Artifactory pipeline API is available for a few years already and has become extremely popular.
      We would appreciate your advice on how to change the plugin in a way that will both satisfy the CPS guidelines without breaking its current functionality and structure.

      Alternatively, the usage of cpsScript.invokeMethod could be added as an exception to the cps mismatch report, similar to the proposed addition in this PR.

      Thank you

          [JENKINS-58643] CPS Mismatches on cpsScript.InvokeMethod

          For what it's worth, the  Templating Engine Plugin  had almost the exact same issue where objects created and injected into the binding were throwing CPS Mismatch warnings.

          we're refactoring anything placed in the binding that's going to be invoking methods to be CPS transformed before being placed injected.  

          After doing this - the only CpsMismatch warnings we got were around metaprogramming examples such as using InvokerHelper to invoke methods, or by taking advantage of groovy's methodMissing functionality.

          those use cases are being addressed in https://github.com/cloudbees/groovy-cps/pull/99

          Steven Terrana added a comment - For what it's worth, the  Templating Engine Plugin   had almost the exact same issue where objects created and injected into the binding were throwing CPS Mismatch warnings. we're refactoring anything placed in the binding that's going to be invoking methods to be CPS transformed before being placed injected.   After doing this - the only CpsMismatch warnings we got were around metaprogramming examples such as using InvokerHelper to invoke methods, or by taking advantage of groovy's methodMissing functionality. those use cases are being addressed in  https://github.com/cloudbees/groovy-cps/pull/99

          Thanks you sterrana - we're not sure we can implement a similar solution in the Artifactory Plugin.

          dnusbaum - In our case, the method name used in the pipeline script (for example, "upload"), is different from the name returned by DescriptorImpl.getFunctionName() ("artifactoryUpload").  Can you please recommend a solution?

          Thanks

          Eyal Ben Moshe added a comment - Thanks you sterrana  - we're not sure we can implement a similar solution in the Artifactory Plugin. dnusbaum - In our case, the method name used in the pipeline script (for example, "upload"), is different from the name returned by DescriptorImpl.getFunctionName() ("artifactoryUpload").  Can you please recommend a solution? Thanks

          Devin Nusbaum added a comment -

          eyalbe Hmm, looks like your code is rather unique, you directly call CpsScript.invokeMethod inside of ArtifactoryServer.upload. This kind of metaprogramming is opaque to the CPS method mismatch detector because ArtifactoryServer.upload is in a Plugin rather than directly in a shared library or the Pipeline itself, so what we did for sterrana's case doesn't apply here.

          CC jglick, any ideas on a fix or workaround? The only thing I can think of would be to ignore mismatch warnings in workflow-cps whenever the receiver is a plugin using something like this:

          diff --git a/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java b/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java
          index 7034810e..e96757cb 100644
          --- a/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java
          +++ b/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java
          @@ -12,4 +12,5 @@ import java.util.logging.Level;
           import java.util.logging.Logger;
           import javax.annotation.CheckForNull;
          +import jenkins.model.Jenkins;
           import jenkins.util.InterceptingExecutorService;
           
          @@ -113,4 +114,9 @@ class CpsVmExecutorService extends InterceptingExecutorService {
           
               private void handleMismatch(Object expectedReceiver, String expectedMethodName, Object actualReceiver, String actualMethodName) {
          +        Class receiverClass = expectedReceiver.getClass();
          +        if (Jenkins.get().getPluginManager().whichPlugin(receiverClass) != null) {
          +            // Plugin code is opaque to the mismatch detector.
          +            return;
          +        }
                   String mismatchMessage = mismatchMessage(className(expectedReceiver), expectedMethodName, className(actualReceiver), actualMethodName);
                   if (FAIL_ON_MISMATCH) {
          

          I haven't tested that to see if it even works, and I think that might also mask real problems in DSLs like pipeline-model-definition and docker-workflow, so it doesn't seem like a great solution.

          Devin Nusbaum added a comment - eyalbe Hmm, looks like your code is rather unique, you directly call CpsScript.invokeMethod inside of ArtifactoryServer.upload . This kind of metaprogramming is opaque to the CPS method mismatch detector because ArtifactoryServer.upload is in a Plugin rather than directly in a shared library or the Pipeline itself, so what we did for sterrana 's case doesn't apply here. CC jglick , any ideas on a fix or workaround? The only thing I can think of would be to ignore mismatch warnings in workflow-cps whenever the receiver is a plugin using something like this: diff --git a/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java b/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java index 7034810e..e96757cb 100644 --- a/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java +++ b/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsVmExecutorService.java @@ -12,4 +12,5 @@ import java.util.logging.Level; import java.util.logging.Logger; import javax.annotation.CheckForNull; +import jenkins.model.Jenkins; import jenkins.util.InterceptingExecutorService; @@ -113,4 +114,9 @@ class CpsVmExecutorService extends InterceptingExecutorService { private void handleMismatch(Object expectedReceiver, String expectedMethodName, Object actualReceiver, String actualMethodName) { + Class receiverClass = expectedReceiver.getClass(); + if (Jenkins.get().getPluginManager().whichPlugin(receiverClass) != null) { + // Plugin code is opaque to the mismatch detector. + return; + } String mismatchMessage = mismatchMessage(className(expectedReceiver), expectedMethodName, className(actualReceiver), actualMethodName); if (FAIL_ON_MISMATCH) { I haven't tested that to see if it even works, and I think that might also mask real problems in DSLs like pipeline-model-definition and docker-workflow, so it doesn't seem like a great solution.

          Jesse Glick added a comment -

          It does not seem like a great workaround either.

          We already explicitly discourage domain-specific plugins from using GlobalVariable and implementing DSLs, but what ArtifactoryServer is doing is an additional complication: the returned object, or one of its resultant objects, is implemented in Java rather than CPS-transformed Groovy. I would consider this to be an unsupported and unsupportable trick.

          Jesse Glick added a comment - It does not seem like a great workaround either. We already explicitly discourage domain-specific plugins from using GlobalVariable and implementing DSLs, but what ArtifactoryServer is doing is an additional complication: the returned object, or one of its resultant objects, is implemented in Java rather than CPS-transformed Groovy. I would consider this to be an unsupported and unsupportable trick.

          Jesse Glick added a comment -

          Although I should say that since docker-workflow and pipeline-model-definition use the less problematic approach of exposing CPS-transformed Groovy, the receiverClass in those cases should not be loaded by a Jenkins plugin class loader as far as I know, so I do not believe such a patch would mask mistakes in those plugins, though it is worth confirming this.

          (To confuse matters, pipeline-model-definition-plugin/pipeline-model-definition/src/main/groovy/ exists, but as far as Jenkins is concerned these are just plain old classes in the plugin. The CPS-transformed code that would be involved here is in pipeline-model-definition-plugin/pipeline-model-definition/src/main/groovy/**/*.groovy.

          Jesse Glick added a comment - Although I should say that since docker-workflow and pipeline-model-definition use the less problematic approach of exposing CPS-transformed Groovy, the receiverClass in those cases should not be loaded by a Jenkins plugin class loader as far as I know, so I do not believe such a patch would mask mistakes in those plugins, though it is worth confirming this. (To confuse matters, pipeline-model-definition-plugin/pipeline-model-definition/src/main/groovy/ exists, but as far as Jenkins is concerned these are just plain old classes in the plugin. The CPS-transformed code that would be involved here is in pipeline-model-definition-plugin/pipeline-model-definition/src/main/groovy/**/*.groovy .

          jglick, so what should we do to resolve this issue?

          Can we perhaps allow specific steps to be excluded and avoid this message?

          Eyal Ben Moshe added a comment - jglick , so what should we do to resolve this issue? Can we perhaps allow specific steps to be excluded and avoid this message?

          Jesse Glick added a comment -

          Best would be to reconsider what the artifactory plugin is doing, and simplify its implementation to be more normal. To the extent that is impossible for backward compatibility reasons, TBD—dnusbaum can check effectiveness & safety of the above patch.

          Jesse Glick added a comment - Best would be to reconsider what the artifactory plugin is doing, and simplify its implementation to be more normal. To the extent that is impossible for backward compatibility reasons, TBD— dnusbaum can check effectiveness & safety of the above patch.

          Can you please be more specific? What does simplifying the plugin and making it more normal mean? How can we maintain the existing APIs and avoid this warning message? 

          Eyal Ben Moshe added a comment - Can you please be more specific? What does simplifying the plugin and making it more normal mean? How can we maintain the existing APIs and avoid this warning message? 

          dnusbaum and jglick,

          Can we perhaps add support for an optional variable, which can be set by pipeline steps, to indicate the step name? This should resolve the "false mismatch warning" issue.

          Many pipelines out there are built and reply on the Artifactory pipeline APIs, so the solution we come up with cannot break the APIs. We'd be glad to create a PR, but we first need to know you're okay with this approach.

          Looking forward to your feedback. Thanks!

          Eyal Ben Moshe added a comment - dnusbaum and jglick , Can we perhaps add support for an optional variable, which can be set by pipeline steps, to indicate the step name? This should resolve the "false mismatch warning" issue. Many pipelines out there are built and reply on the Artifactory pipeline APIs, so the solution we come up with cannot break the APIs. We'd be glad to create a PR, but we first need to know you're okay with this approach. Looking forward to your feedback. Thanks!

          Jesse Glick added a comment -

          The patch above seems like the safest and more general fix, if it indeed works. Needs testing.

          Jesse Glick added a comment - The patch above seems like the safest and more general fix, if it indeed works. Needs testing.

          jglick,

          To which patch are you referring to? Is it my above suggestion? If so, we'll start working on a pull request.

          Eyal Ben Moshe added a comment - jglick , To which patch are you referring to? Is it my above suggestion? If so, we'll start working on a pull request.

          Jesse Glick added a comment -

          No, to the patch in the comment by dnusbaum in 2019-08-01.

          Jesse Glick added a comment - No, to the patch in the comment by dnusbaum in 2019-08-01.

          Robi Nino added a comment -

          Hi jglick and dnusbaum,
          I opened #314 with Devin's suggestion.
          The patch solves our issue, workflow-cps-plugin tests are passing.
          Thanks!

          Robi Nino added a comment - Hi jglick and dnusbaum , I opened #314 with Devin's suggestion. The patch solves our issue, workflow-cps-plugin tests are passing. Thanks!

          Robert Wedd added a comment -

          Do we know when to expect the fix to come in i see #314 is failing 

          Robert Wedd added a comment - Do we know when to expect the fix to come in i see  #314  is failing 

          Jake Mroz added a comment -

          We are seeing this issue in Jenkins 2.176.3, Artifactory Plugin 3.4.0, Groovy Plugin 2.74:

           

          expected to call org.jfrog.hudson.pipeline.common.types.ArtifactoryServer.promote but wound up catching artifactoryPromoteBuild; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/

          Jake Mroz added a comment - We are seeing this issue in Jenkins 2.176.3, Artifactory Plugin 3.4.0, Groovy Plugin 2.74:   expected to call org.jfrog.hudson.pipeline.common.types.ArtifactoryServer.promote but wound up catching artifactoryPromoteBuild; see: https: //jenkins.io/redirect/pipeline-cps-method-mismatches/

          Ward Hopeman added a comment -

          We are also seeing this on Jenkins v2.190.1; Artifactory plugin v3.40; Groovy plugin v2.2
          artifactoryUploadexpected to call org.jfrog.hudson.pipeline.common.types.ArtifactoryServer.upload but wound up catching artifactoryUpload; see:
           

          Ward Hopeman added a comment - We are also seeing this on Jenkins v2.190.1; Artifactory plugin v3.40; Groovy plugin v2.2 artifactoryUploadexpected to call org.jfrog.hudson.pipeline.common.types.ArtifactoryServer.upload but wound up catching artifactoryUpload; see:  

          Devin Nusbaum added a comment -

          A fix for this issue was just released in Pipeline: Groovy Plugin version 2.75.

          Devin Nusbaum added a comment - A fix for this issue was just released in Pipeline: Groovy Plugin version 2.75.

            Unassigned Unassigned
            robinino Robi Nino
            Votes:
            3 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated:
              Resolved: