Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44841

Load step within loaded script returns null

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • workflow-cps-plugin
    • None

      Hi,

      First of all - I will describe what I'm trying to do:

      I have multiple "library" pipeline scripts. I'm using the pipeline Load step to load them.

      Some scripts need to use methods in other scripts - think of the following scenario:
      pipeline Job loads script1.groovy and script2.groovy , script1.groovy loads script2.groovy as well. This results in script2.groovy being loaded twice for no good reason.

      I want to achieve lazy loading, meaning if the same script has already been loaded - I want to reuse it and not load it again, meaning script1.groovy should use script2.groovy which was loaded by the job itself.

      I tried to do it with the following code - in this example I do want the 2nd script to load the file on its own:

      job code:

      timestamps {

    node('some node') {
 
        //some code to get the files to the workspace - not relevant to the situation
        def test = load("./src/jenkins/pipeline/utils/test.groovy")

        test.test()

    }
}

       

      test.groovy

      /* ***** lazy loading - BEGIN ***** */
this._scripts = [:]

def getScript(def scriptName) {
    // lazy loading
    if (this._scripts."${scriptName}" == null) {
        this._scripts."${scriptName}" = load("${WORKSPACE}/src/jenkins/pipeline/utils/${scriptName}.groovy")
        this._scripts."${scriptName}".init(this._scripts)
    }
    return this._scripts."${scriptName}"
}

def init(def scripts) {
    if (scripts)
        this._scripts = scripts
}

/* ***** lazy loading - END ***** */

def test() {
    getScript("command_runner").runHostCommand("echo 'hello world'")
}

return this

      command_runner.groovy

      /* ***** lazy loading - BEGIN ***** */
this._scripts = [:]

def getScript(def scriptName) {
    // lazy loading
    if (this._scripts."${scriptName}" == null) {
        this._scripts."${scriptName}" = load("${WORKSPACE}/src/jenkins/pipeline/utils/${scriptName}.groovy")
        this._scripts."${scriptName}".init(this._scripts)
    }
    return this._scripts."${scriptName}"
}

def init(def scripts) {
    if (scripts)
        this._scripts = scripts
}

/* ***** lazy loading - END ***** */


/**
 * Runs a command and returns the exit code. assumes windows [for now]
 * @param command
 * @return
 */
def runHostCommand(def command) {
    int exitCode
    exitCode = bat script: command, returnStatus: true

    return exitCode
}

def runHostCommandWithOutput(def command) {
    currentDir = pwd()
    println "${currentDir}>${command}"
    def output = bat (script: command, returnStdout: true).trim()
    //output contains the call the command itself so we should remove it when printing
    output = output.substring(output.indexOf('\n')+1);
    println output
    return output
}

/* *** mandatory to be able to call from another bootstrap script *** */
return this

      When running it - I get the following exception

      ...
[Pipeline] load
[Pipeline] { (./src/jenkins/pipeline/utils/test.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] load
[Pipeline] { (F:\jenkins-remote\workspace\Mor\pipeline_tests/src/jenkins/pipeline/utils/command_runner.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] End of Pipeline
java.lang.NullPointerException: Cannot invoke method init() on null object
	at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:35)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:18)
	at Script1.getScript(Script1.groovy:13)
	at Script1.test(Script1.groovy:30)
	at WorkflowScript.run(WorkflowScript:26)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
	at sun.reflect.GeneratedMethodAccessor270.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.get(PropertyishBlock.java:76)
	at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.fixName(PropertyishBlock.java:66)
	at sun.reflect.GeneratedMethodAccessor355.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:173)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:162)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:162)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:330)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:82)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:242)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:230)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Finished: FAILURE

      According to the log, the Load command within test.groovy succeeded, but for some reason returned null.

       

      so I'm looking for either a solution to the problem, or a better way to achieve lazy loading. And yes - I am familiar with Shared Pipeline Libraries plugin which is not suitable for my case at this stage.

       

      Thanks in advance,

      Mor

          [JENKINS-44841] Load step within loaded script returns null

          Mor L added a comment -

          jglick would more than appreciate your feedback on this or lazy loading in general.
          Thanks.

          Mor L added a comment - jglick would more than appreciate your feedback on this or lazy loading in general. Thanks.

          Jesse Glick added a comment -

          No idea offhand if this is a product bug or a user error. Best to file issues containing minimal, self-contained, reproducible test cases. Otherwise this goes to the bottom of a long pile of unevaluated issues, I am afraid.

          Jesse Glick added a comment - No idea offhand if this is a product bug or a user error. Best to file issues containing minimal , self-contained, reproducible test cases. Otherwise this goes to the bottom of a long pile of unevaluated issues, I am afraid.

          Mor L added a comment -

          It looks as though if I replace the groovy property notation

           

          this._scripts."${scriptName}"

          with standard map put/get calls - it works (at least it doesn't fail with the same error for the exact same scenario - I haven't tested all scenarios yet)

           

          def getScript(def scriptName) {
    // lazy loading
    if (!this._scripts.get(scriptName)) {
        this._scripts.put(scriptName, load("${WORKSPACE}/src/jenkins/pipeline/utils/${scriptName}.groovy"))
        this._scripts.get(scriptName).init(this._scripts)
    }
    return this._scripts.get(scriptName)
}

          So this must be a pipeline bug...

          If it works for all scenarios - then I have my lazy loading working which is great

           

          Mor L added a comment - It looks as though if I replace the groovy property notation   this ._scripts. "${scriptName}" with standard map put/get calls - it works (at least it doesn't fail with the same error for the exact same scenario - I haven't tested all scenarios yet)   def getScript(def scriptName) { // lazy loading if (! this ._scripts.get(scriptName)) { this ._scripts.put(scriptName, load( "${WORKSPACE}/src/jenkins/pipeline/utils/${scriptName}.groovy" )) this ._scripts.get(scriptName).init( this ._scripts) } return this ._scripts.get(scriptName) } So this must be a pipeline bug... If it works for all scenarios - then I have my lazy loading working which is great  

          Jesse Glick added a comment -

          Is

          this._scripts."${scriptName}"

          even valid Groovy syntax? I have never seen that before; looks like it should be a compiler error to me. If you have a dynamic map key, the sugar is

          this._scripts[scriptName]

          Jesse Glick added a comment - Is this ._scripts. "${scriptName}" even valid Groovy syntax? I have never seen that before; looks like it should be a compiler error to me. If you have a dynamic map key, the sugar is this ._scripts[scriptName]

          Mor L added a comment - - edited

          using CodingGround 's groovy console for instance - the following works (they use groovy 2.4.4):

           

          def map = [:]
map."hello" = "world"

def key = "hello"

assert map[key] == "world"
assert map."${key}" == "world"

           But I do agree the [] notation is prettier. I will switch to that and continue testing

          Mor L added a comment - - edited using CodingGround 's groovy console for instance - the following works (they use groovy 2.4.4):   def map = [:] map. "hello" = "world" def key = "hello" assert map[key] == "world" assert map. "${key}" == "world"  But I do agree the [] notation is prettier. I will switch to that and continue testing

          Jesse Glick added a comment -

          It is possible the dot notation calls something different internally which is not yet supported by either groovy-sandbox or groovy-cps. I am guessing it has never been tested.

          Jesse Glick added a comment - It is possible the dot notation calls something different internally which is not yet supported by either groovy-sandbox or groovy-cps . I am guessing it has never been tested.

          Mor L added a comment -

          Do you agree its a bug then?
          The title needs to be changed though to reflect the actual issue.
          But, I do have a workaround so the priority can be lowered

          Mor L added a comment - Do you agree its a bug then? The title needs to be changed though to reflect the actual issue. But, I do have a workaround so the priority can be lowered

          Jesse Glick added a comment -

          Sounds like a bug. Do you know of some simpler way to reproduce that would clarify the issue?

          Jesse Glick added a comment - Sounds like a bug. Do you know of some simpler way to reproduce that would clarify the issue?

          Mor L added a comment -

          I will try to reproduce with minimal code tomorrow, hopefully without loading other scripts.

          Mor L added a comment - I will try to reproduce with minimal code tomorrow, hopefully without loading other scripts.

          Mor L added a comment -

          Unfortunately the same error reproduces even when using the [] notation.

          This happens when a script loads a script which loads another script - I get the same 
          java.lang.NullPointerException: Cannot invoke method init() on null object
          So I'm not entirely sure the problem is just with the dot notation.

          My current workaround is to use pre-loading (load all needed scripts in the first script and then init all other scripts).

           

          Mor L added a comment - Unfortunately the same error reproduces even when using the [] notation. This happens when a script loads a script which loads another script - I get the same  java.lang.NullPointerException: Cannot invoke method init() on null object So I'm not entirely sure the problem is just with the dot notation. My current workaround is to use pre-loading (load all needed scripts in the first script and then init all other scripts).  

          Jesse Glick added a comment -

          Will leave this open, but am not likely to spend time on it unless there is a minimal test case.

          Jesse Glick added a comment - Will leave this open, but am not likely to spend time on it unless there is a minimal test case.

          Mor L added a comment -

          I understand, I have more urgent issues to attend to but will do my best to have a minimal case sometime soon. Regardless, thanks for taking the time to reply

          Mor L added a comment - I understand, I have more urgent issues to attend to but will do my best to have a minimal case sometime soon. Regardless, thanks for taking the time to reply

            Unassigned Unassigned
            pyrocks Mor L
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: