Major We are getting the below exception when we try to initialize and configure Jenkins plugins through post build Groovy Script under init.groovy.d. We understood this is the issue with classloader not able to find ivy in its path. However, this is working with 2.176.2 and started breaking from 2.176.3.
@Grab('org.yaml:snakeyaml:1.17')
import java.util.logging.Logger
import org.codehaus.groovy.control.CompilerConfiguration
import org.yaml.snakeyaml.Yaml
....
java.lang.ClassNotFoundException: org.apache.ivy.core.settings.IvySettings at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:565) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) Caused: java.lang.NoClassDefFoundError: org/apache/ivy/core/settings/IvySettings
We performed various work around to resolve this and none of them were successful
Job-DSL or Event Listener plugin cannot used since this is a config script which execute before Jenkins plugin installation/configuration. Similarly none of the pipeline utilities steps can be used at this point.
Cannot copy Ivy.jar to $Jenkins_Home/war/WEB_INF/lib due to the fact we run Jenkins as a docker container and war path will be only exposed after docker run which eventually invokes init.groovy.d script.
Tried adding the jar through classloader.
[JENKINS-61261] Ivy ClassNotFoundException when using Grab
Unlikely to be a core change given https://github.com/jenkinsci/jenkins/compare/jenkins-2.176.2...jenkins-2.176.3
My best guess is init.groovy scripts see all plugins' classes and their dependencies, and you updated a plugin at the same time that used to have Ivy on the classpath, and now don't.
See https://github.com/jenkinsci/jenkins/pull/2374 for a PR that never got merged, and recent comments indicating plugins that include Ivy.
Gowtham Subramanian
added a comment - We stood up 2.176.2 Jenkins with similar plugins (latest version) without any problem. Not sure, if this is a plugin issue.
As per your comment, init.groovy should find all plugin classes and their dependencies. So we also tried installing it with workflow-cps-global-lib as mentioned in the pull request mentioned as a work around. However, when the docker container is getting initialized we can see the below logs where it says started all plugins and fails to find ivy eventually
Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Mar 03, 2020 1:50:38 PM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @673ms to org.eclipse.jetty.util.log.JavaUtilLog
Mar 03, 2020 1:50:38 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Mar 03, 2020 1:50:39 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath
WARNING: Empty contextPath
Mar 03, 2020 1:50:39 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.z-SNAPSHOT; built: 2019-02-15T16:53:49.381Z; git: eb70b240169fcf1abbd86af36482d1c49826fa0b; jvm 1.8.0_222-b10
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.webapp.StandardDescriptorProcessor visitServlet
INFO: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
INFO: DefaultSessionIdManager workerName=node0
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
INFO: No SessionScavenger set, using defaults
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.session.HouseKeeper startScavenging
INFO: node0 Scavenging every 660000ms
Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.handler.ContextHandler doStart
INFO: Started w.@66971f6b{Jenkins v2.176.3,/,file:///var/jenkins_home/war/,AVAILABLE}
{/var/jenkins_home/war}Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.AbstractConnector doStart
INFO: Started ServerConnector@53499d85{HTTP/1.1,[http/1.1]}
{0.0.0.0:8080}Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.Server doStart
INFO: Started @3387ms
Mar 03, 2020 1:50:40 PM winstone.Logger logInternal
INFO: Winstone Servlet Engine v4.0 running: controlPort=disabled
Mar 03, 2020 1:50:43 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
Mar 03, 2020 1:50:43 PM jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
Mar 03, 2020 1:50:45 PM jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
Mar 03, 2020 1:50:45 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
Mar 03, 2020 1:50:45 PM jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
Mar 03, 2020 1:50:46 PM jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
Mar 03, 2020 1:50:46 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Started Download metadata
Mar 03, 2020 1:50:46 PM hudson.util.Retrier start
INFO: Attempt #1 to do the action check updates server
Mar 03, 2020 1:50:47 PM jenkins.util.groovy.GroovyHookScript execute
INFO: Executing /var/jenkins_home/init.groovy.d/run.groovy
Mar 03, 2020 1:50:47 PM jenkins.model.Jenkins$5 runTask
WARNING: GroovyInitScript.init failed perhaps due to plugin dependency issues
java.lang.Error: java.lang.reflect.InvocationTargetException
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1095)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
... 8 more
Gowtham Subramanian
added a comment - We stood up 2.176.2 Jenkins with similar plugins (latest version) without any problem. Not sure, if this is a plugin issue.
As per your comment, init.groovy should find all plugin classes and their dependencies. So we also tried installing it with workflow-cps-global-lib as mentioned in the pull request mentioned as a work around. However, when the docker container is getting initialized we can see the below logs where it says started all plugins and fails to find ivy eventually
Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Mar 03, 2020 1:50:38 PM org.eclipse.jetty.util.log.Log initialized
INFO: Logging initialized @673ms to org.eclipse.jetty.util.log.JavaUtilLog
Mar 03, 2020 1:50:38 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Mar 03, 2020 1:50:39 PM org.eclipse.jetty.server.handler.ContextHandler setContextPath
WARNING: Empty contextPath
Mar 03, 2020 1:50:39 PM org.eclipse.jetty.server.Server doStart
INFO: jetty-9.4.z-SNAPSHOT; built: 2019-02-15T16:53:49.381Z; git: eb70b240169fcf1abbd86af36482d1c49826fa0b; jvm 1.8.0_222-b10
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.webapp.StandardDescriptorProcessor visitServlet
INFO: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
INFO: DefaultSessionIdManager workerName=node0
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
INFO: No SessionScavenger set, using defaults
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.session.HouseKeeper startScavenging
INFO: node0 Scavenging every 660000ms
Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.handler.ContextHandler doStart
INFO: Started w.@66971f6b{Jenkins v2.176.3,/, file:///var/jenkins_home/war/,AVAILABLE }
{/var/jenkins_home/war}
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.AbstractConnector doStart
INFO: Started ServerConnector@53499d85{HTTP/1.1, [http/1.1] }
{0.0.0.0:8080}
Mar 03, 2020 1:50:40 PM org.eclipse.jetty.server.Server doStart
INFO: Started @3387ms
Mar 03, 2020 1:50:40 PM winstone.Logger logInternal
INFO: Winstone Servlet Engine v4.0 running: controlPort=disabled
Mar 03, 2020 1:50:43 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
Mar 03, 2020 1:50:43 PM jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
Mar 03, 2020 1:50:45 PM jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
Mar 03, 2020 1:50:45 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
Mar 03, 2020 1:50:45 PM jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
Mar 03, 2020 1:50:46 PM jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
Mar 03, 2020 1:50:46 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Started Download metadata
Mar 03, 2020 1:50:46 PM hudson.util.Retrier start
INFO: Attempt #1 to do the action check updates server
Mar 03, 2020 1:50:47 PM jenkins.util.groovy.GroovyHookScript execute
INFO: Executing /var/jenkins_home/init.groovy.d/run.groovy
Mar 03, 2020 1:50:47 PM jenkins.model.Jenkins$5 runTask
WARNING: GroovyInitScript.init failed perhaps due to plugin dependency issues
java.lang.Error: java.lang.reflect.InvocationTargetException
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1095)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
... 8 more Gowtham Subramanian
added a comment - Found the issue is related to downloaded plugins not present at the plugins path.
Gowtham Subramanian
added a comment - Found the issue is related to downloaded plugins not present at the plugins path. 
I don't think that this has anything to do with the postbuildscript plugin.