-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
Fresh instance of Jenkins LTS version 2.375.2 (https://get.jenkins.io/war-stable/2.375.2/jenkins.war) which is the latest LTS at this time, with 'install suggested plugins' plus installing:
* pipeline-utility-steps:2.15.0 'Pipeline Utility Steps'
Which is the latest version of this plugin at this time.
Running Jenkins with Java:
openjdk version "11.0.17" 2022-10-18
OpenJDK Runtime Environment Temurin-11.0.17+8 (build 11.0.17+8)
OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (build 11.0.17+8, mixed mode)Fresh instance of Jenkins LTS version 2.375.2 ( https://get.jenkins.io/war-stable/2.375.2/jenkins.war ) which is the latest LTS at this time, with 'install suggested plugins' plus installing: * pipeline-utility-steps:2.15.0 'Pipeline Utility Steps' Which is the latest version of this plugin at this time. Running Jenkins with Java: openjdk version "11.0.17" 2022-10-18 OpenJDK Runtime Environment Temurin-11.0.17+8 (build 11.0.17+8) OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (build 11.0.17+8, mixed mode)
Expected behaviour
If I start up the controller, and use the 'readYaml' step with a yaml containing 51 aliases/anchors:
pipeline { agent any stages { stage('Hello') { steps { readYaml file: 'moreThan50anchors.yml' } } } }
I get the expected error (due to SnakeYAML's default configuration of 50):
org.yaml.snakeyaml.error.YAMLException: Number of aliases for non-scalar nodes exceeds the specified max=50
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:193)
...
at org.yaml.snakeyaml.Yaml$1.next(Yaml.java:499)
at org.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep$Execution.doRun(ReadYamlStep.java:203)
...
Thanks to the fix from https://issues.jenkins.io/browse/JENKINS-68830 I can then go to `Manage Jenkins` -> `Script console` and run:
org.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep.DEFAULT_MAX_ALIASES_FOR_COLLECTIONS=51 org.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep.MAX_MAX_ALIASES_FOR_COLLECTIONS=51
And run the same pipeline again, it is successful.
Ideally, I should not be parsing such a large yaml file using the readYaml step, This command loads the data into the controller's memory, and uses resources on the controller, which at scale (if the yaml is very large, or it runs very often from multiple pipelines, for example using a Pipeline shared library) can cause performance issues.
I should change my code to use `sh 'yq ,,,'` to use a shell step on an agent to run https://github.com/mikefarah/yq (as also recommended in the plugin documentation https://github.com/jenkinsci/pipeline-utility-steps-plugin/blob/5953af571306aed1ce61df12c6a86cd2f7a1d329/src/main/resources/org/jenkinsci/plugins/pipeline/utility/steps/conf/ReadYamlStep/help.groovy#L68-L77 ).
Unexpected behaviour
Instead of using the script console, if I add these as startup arguments:
java -Dorg.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep.DEFAULT_MAX_ALIASES_FOR_COLLECTIONS=51 -Dorg.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep.MAX_MAX_ALIASES_FOR_COLLECTIONS=51 -jar jenkins.war
Then I run my same pipeline (without running anything in the script console first), I get the error in the Pipeline log:
java.lang.IllegalArgumentException: 51 > 0. Reduce the required DEFAULT_MAX_ALIASES_FOR_COLLECTIONS or convince your administrator to increase the max allowed value with the system property "null"
at org.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep.setDefaultMaxAliasesForCollections(ReadYamlStep.java:105)
at org.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep.<clinit>(ReadYamlStep.java:68)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:330)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:272)
at org.jenkinsci.plugins.workflow.steps.StepDescriptor.newInstance(StepDescriptor.java:204)
at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepDescriptorImpl.newInstance(AbstractFileOrTextStepDescriptorImpl.java:24)
at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepDescriptorImpl.newInstance(AbstractFileOrTextStepDescriptorImpl.java:12)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:302)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:180)
at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:163)
at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:178)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:182)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:152)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
Caused: java.lang.ExceptionInInitializerError
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:330)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:272)
at org.jenkinsci.plugins.workflow.steps.StepDescriptor.newInstance(StepDescriptor.java:204)
at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepDescriptorImpl.newInstance(AbstractFileOrTextStepDescriptorImpl.java:24)
at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepDescriptorImpl.newInstance(AbstractFileOrTextStepDescriptorImpl.java:12)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:302)
...
This exception is coming from:
https://github.com/jenkinsci/pipeline-utility-steps-plugin/blob/0326b3cb497dee95348471fca1c7c62dbb6975fb/src/main/java/org/jenkinsci/plugins/pipeline/utility/steps/conf/ReadYamlStep.java#L105
Every subsequent pipeline run after that one encounters the error:
java.lang.NoClassDefFoundError: Could not initialize class org.jenkinsci.plugins.pipeline.utility.steps.conf.ReadYamlStep
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:330)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:272)
at org.jenkinsci.plugins.workflow.steps.StepDescriptor.newInstance(StepDescriptor.java:204)
at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepDescriptorImpl.newInstance(AbstractFileOrTextStepDescriptorImpl.java:24)
at org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepDescriptorImpl.newInstance(AbstractFileOrTextStepDescriptorImpl.java:12)