Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-63384

load step fails to bind "complex" @Field defined variables

    • Icon: Task Task
    • Resolution: Unresolved
    • Icon: Critical Critical
    • workflow-cps-plugin
    • None

      for more complex variables that are annotated with @Field, the load step fails to bind the variable into the Script class.

      For example:

      // a.groovy
      import groovy.transform.Field
      
      @Field
      def LOADED_BUILD_NUMBER = ${env.BUILD_NUMBER}
      
      return this
      
      // Jenkinsfile
      node() {
        def a = load('a.groovy')
        echo(${env.BUILD_NUMBER})
        echo(${a.LOADED_BUILD_NUMBER})
      }
      

      This example will fail. However, if you replace ${env.BUILD_NUMBER} with a simple type such as `3`, the load step will succeed.

       

      This seems to be related to the security update in workflow-cps v2.64 and the subsequent regression fix for @Field in v2.71.

          [JENKINS-63384] load step fails to bind "complex" @Field defined variables

          Denys Digtiar created issue -
          Denys Digtiar made changes -
          Description Original: for more complex variables that are annotated with @Field, the load step fails to bind the variable into the Script class.

          For example:
          {code:java}
          // a.groovy
          import groovy.transform.Field

          @Field
          def LOADED_BUILD_NUMBER = ${env.BUILD_NUMBER}

          return this

          // Jenkinsfile
          node() {
            def a = load('a.groovy')
            echo(${env.BUILD_NUMBER})
            echo(${a.LOADED_BUILD_NUMBER})
          }
          {code}
          {{}}

          This example will fail. However, if you replace ${env.BUILD_NUMBER} with a simple type such as `3`, the load step will succeed.

           

          This seems to be related to the security update in workflow-cps v2.64 and the subsequent regression fix for @Field in v2.71.
          New: for more complex variables that are annotated with @Field, the load step fails to bind the variable into the Script class.

          For example:
          {code:java}
          // a.groovy
          import groovy.transform.Field

          @Field
          def LOADED_BUILD_NUMBER = ${env.BUILD_NUMBER}

          return this

          // Jenkinsfile
          node() {
            def a = load('a.groovy')
            echo(${env.BUILD_NUMBER})
            echo(${a.LOADED_BUILD_NUMBER})
          }
          {code}
          This example will fail. However, if you replace ${env.BUILD_NUMBER} with a simple type such as `3`, the load step will succeed.

           

          This seems to be related to the security update in workflow-cps v2.64 and the subsequent regression fix for @Field in v2.71.

          Dee Kryvenko added a comment -

          These "security" fixes start driving me crazy.

          There seems to be no way to preserve the state across Jenkins restarts anymore in my library.

          None of the classical Java singleton patterns (that relies on static fields) work because CPS does not serialize static fields.

          `@Singleton` annotation doesn't work because AST transformation will define a `getInstance()` method without `@NonCPS` annotation making this singleton unusable from my `@NonCPS` annotated code.

          Now this - I can't preserve state as a global var `@Field` anymore.

          Really starting to feel like the whole objective here is to make user's life harder...

          Dee Kryvenko added a comment - These "security" fixes start driving me crazy. There seems to be no way to preserve the state across Jenkins restarts anymore in my library. None of the classical Java singleton patterns (that relies on static fields) work because CPS does not serialize static fields. `@Singleton` annotation doesn't work because AST transformation will define a `getInstance()` method without `@NonCPS` annotation making this singleton unusable from my `@NonCPS` annotated code. Now this - I can't preserve state as a global var `@Field` anymore. Really starting to feel like the whole objective here is to make user's life harder...

          Dee Kryvenko added a comment -

          I just had a brilliant idea to implement InvisibleAction in my library and use it to store state... and what do you think? Yeah that's right - they took care of that too!

          https://www.jenkins.io/blog/2018/03/15/jep-200-lts/

          Well done Jenkins! Well done.

          Dee Kryvenko added a comment - I just had a brilliant idea to implement InvisibleAction in my library and use it to store state... and what do you think? Yeah that's right - they took care of that too! https://www.jenkins.io/blog/2018/03/15/jep-200-lts/ Well done Jenkins! Well done.

          Dee Kryvenko added a comment - - edited

          I'm feeling like Alice falling down the rabbit hole. I've tried to (for the sake of testing) executing this:

          def f = ExtensionList.lookup(jenkins.security.CustomClassFilter.Static.class)[0]
          f.overrides.put('my.JenkinsfileContext', true)
          

          Which is an equivalent of `-Dhudson.remoting.ClassFilter=my.JenkinsfileContext` so I don't have to restart my Jenkins for testing.
          And then I had this in my library:

          @NonCPS
          def diag() {
              echo "isBlacklisted(String) = ${hudson.remoting.ClassFilter.DEFAULT.isBlacklisted('my.JenkinsfileContext').toString()}"
              echo "isBlacklisted(Class) = ${hudson.remoting.ClassFilter.DEFAULT.isBlacklisted(my.JenkinsfileContext.class).toString()}"
          }
          

          It reports

          [Pipeline] echo
          isBlacklisted(String) = false
          [Pipeline] echo
          isBlacklisted(Class) = false
          

          But, I still get

          java.lang.UnsupportedOperationException: Refusing to marshal my.JenkinsfileContext for security reasons; see https://jenkins.io/redirect/class-filter/
          

          I am looking at https://github.com/jenkinsci/jenkins/blob/jenkins-2.289.2/core/src/main/java/hudson/util/XStream2.java#L561 and I can clearly see

          return ClassFilter.DEFAULT.isBlacklisted(name) || ClassFilter.DEFAULT.isBlacklisted(type);
          

          How in the world my `my.JenkinsfileContext ` is still end up in this `BlacklistedTypesConverter`??? How many more circles of Hell I'' have to go through for such a simple basic thing as preserving my groovy lib state between the Jenkins restarts?

          I am raising priority of this ticket. This does NOT conform to the "Minor". This is fundamental basic functionality that is broken.

          Dee Kryvenko added a comment - - edited I'm feeling like Alice falling down the rabbit hole. I've tried to (for the sake of testing) executing this: def f = ExtensionList.lookup(jenkins.security.CustomClassFilter.Static.class)[0] f.overrides.put( 'my.JenkinsfileContext' , true ) Which is an equivalent of `-Dhudson.remoting.ClassFilter=my.JenkinsfileContext` so I don't have to restart my Jenkins for testing. And then I had this in my library: @NonCPS def diag() { echo "isBlacklisted( String ) = ${hudson.remoting.ClassFilter.DEFAULT.isBlacklisted( 'my.JenkinsfileContext' ).toString()}" echo "isBlacklisted( Class ) = ${hudson.remoting.ClassFilter.DEFAULT.isBlacklisted(my.JenkinsfileContext.class).toString()}" } It reports [Pipeline] echo isBlacklisted( String ) = false [Pipeline] echo isBlacklisted( Class ) = false But, I still get java.lang.UnsupportedOperationException: Refusing to marshal my.JenkinsfileContext for security reasons; see https: //jenkins.io/redirect/ class- filter/ I am looking at https://github.com/jenkinsci/jenkins/blob/jenkins-2.289.2/core/src/main/java/hudson/util/XStream2.java#L561 and I can clearly see return ClassFilter.DEFAULT.isBlacklisted(name) || ClassFilter.DEFAULT.isBlacklisted(type); How in the world my `my.JenkinsfileContext ` is still end up in this `BlacklistedTypesConverter`??? How many more circles of Hell I'' have to go through for such a simple basic thing as preserving my groovy lib state between the Jenkins restarts? I am raising priority of this ticket. This does NOT conform to the "Minor". This is fundamental basic functionality that is broken.
          Dee Kryvenko made changes -
          Priority Original: Minor [ 4 ] New: Critical [ 2 ]

          Dee Kryvenko added a comment -

          jglick, oleg_nenashev, abayer, dnusbaum - tagging as participants on both SECURITY-1336 and JEP-200.

          We need a way to preserve groovy lib state across Jenkins restarts within the same pipeline run. One way or another - this is basic requirement. The whole CPS survivability makes no sense if my groovy lib is inoperable after a restart. From my perspective fixing the `@Field` annotation sounds like the right thing to do. But if the proper way to preserve state is through InvisibleAction - we need to be able to implement action in the the lib, which I'm not sure even feasible - it won't be on the class path outside of the `WorkflowRun` (probably that's why `BlacklistedTypesConverter` still caught me above). Meanwhile any other ideas/workarounds are highly appreciated.

          Dee Kryvenko added a comment - jglick , oleg_nenashev , abayer , dnusbaum - tagging as participants on both SECURITY-1336 and JEP-200. We need a way to preserve groovy lib state across Jenkins restarts within the same pipeline run. One way or another - this is basic requirement. The whole CPS survivability makes no sense if my groovy lib is inoperable after a restart. From my perspective fixing the `@Field` annotation sounds like the right thing to do. But if the proper way to preserve state is through InvisibleAction - we need to be able to implement action in the the lib, which I'm not sure even feasible - it won't be on the class path outside of the `WorkflowRun` (probably that's why `BlacklistedTypesConverter` still caught me above). Meanwhile any other ideas/workarounds are highly appreciated.

          Jesse Glick added a comment -

          llibicpep I think you are overthinking this. Do not write a plugin. If @Field is broken in this context, just do not use it unless and until the bug is fixed. (Obviously it had never occurred to anyone to write a functional test with load + @Field.)

          a way to preserve groovy lib state across Jenkins restarts within the same pipeline run

          Just have the Jenkinsfile pass a Map or other stateful object to the library or load’ed script? Or just use a local variable?

          Jesse Glick added a comment - llibicpep I think you are overthinking this. Do not write a plugin. If @Field is broken in this context, just do not use it unless and until the bug is fixed. (Obviously it had never occurred to anyone to write a functional test with load + @Field .) a way to preserve groovy lib state across Jenkins restarts within the same pipeline run Just have the Jenkinsfile pass a Map or other stateful object to the library or load ’ed script? Or just use a local variable?

          Dee Kryvenko added a comment -

          jglick - that's what I am already doing. The problem is with saving it or a pointer to it. Library doesn't have access to local variables defined in the Jenkinsfile context.

          Consider I have something like this

          ```groovy
          myLib(this, 'pipeline.yaml')
          ```

          As my Jenkinsfile. Yes, I was thinking about extending it to:

          ```groovy
          def state = [:]
          myLib(this, 'pipeline.yaml', state)
          ```

          But this library generates a pipeline (as String) and `evaluate`s it. I tried to evaluate it with a custom GroovyShell and passing a Binding inside - CPS is not happy about that. Otherwise - my dynamic Jenkinsfile has no access to that state object. When other steps being called from that dynamic Jenkinsfile - they don't have access to that either. I was trying to solve it with Singleton and it works - up until Controller restarts. `@Field` doesn't work at all.

          Even if I didn't had the dynamic Jenkinsfile generation - I would then had to recursively pass `state` object to every step and method in my library, which is an inconvenience at the very least.

          Dee Kryvenko added a comment - jglick - that's what I am already doing. The problem is with saving it or a pointer to it. Library doesn't have access to local variables defined in the Jenkinsfile context. Consider I have something like this ```groovy myLib(this, 'pipeline.yaml') ``` As my Jenkinsfile. Yes, I was thinking about extending it to: ```groovy def state = [:] myLib(this, 'pipeline.yaml', state) ``` But this library generates a pipeline (as String) and `evaluate`s it. I tried to evaluate it with a custom GroovyShell and passing a Binding inside - CPS is not happy about that. Otherwise - my dynamic Jenkinsfile has no access to that state object. When other steps being called from that dynamic Jenkinsfile - they don't have access to that either. I was trying to solve it with Singleton and it works - up until Controller restarts. `@Field` doesn't work at all. Even if I didn't had the dynamic Jenkinsfile generation - I would then had to recursively pass `state` object to every step and method in my library, which is an inconvenience at the very least.

          Jesse Glick added a comment -

          recursively pass state object to every step and method in my library

          Yes do this.

          Jesse Glick added a comment - recursively pass state object to every step and method in my library Yes do this.

            Unassigned Unassigned
            duemir Denys Digtiar
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: