Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10238

Jenkins slave throws "Classloading from system classloader disabled" exception

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • maven-plugin
    • None
    • Master on AIX 5.3; slave on Windows 2008(R2)

      When running a Maven2 job on a slave instance, I get the following message (complete stack trace attached): "java.lang.ClassNotFoundException: Classloading from system classloader disabled". This happens with the first job after the slave instance is started (it's running as a Windows service). After that, I get a "java.lang.NoClassDefFoundError: Could not initialize class hudson.maven.MavenModuleSetBuild" error.
      If I create a freestyle job, and include the maven command as a batch command to run on the slave, it works fine. The Maven2 job appears to locate Maven on the slave machine, but it apparently can't find the maven-plugin (probably due to the classloading error).

          [JENKINS-10238] Jenkins slave throws "Classloading from system classloader disabled" exception

          Glenn Mayer created issue -

          Marco Rothe added a comment -

          Encountered the same issue after upgrading to 1.427 (from 1.421).

          The first job an each node fails with the same stacktrace already attached to this issue.
          All later starts of a job end with

          hudson.util.IOException2: remote file operation failed: /.../jenkins/workspace/$ourProject at hudson.remoting.Channel@40d39e96:NodeOne
          at hudson.FilePath.act(FilePath.java:754)
          at hudson.FilePath.act(FilePath.java:740)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1193)
          at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443)
          at hudson.model.Run.run(Run.java:1376)
          at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:479)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:230)
          Caused by: java.io.IOException: Remote call on NodeOne failed
          at hudson.remoting.Channel.call(Channel.java:677)
          at hudson.FilePath.act(FilePath.java:747)
          ... 10 more
          Caused by: java.lang.NoClassDefFoundError
          at hudson.scm.SubversionWorkspaceSelector.syncWorkspaceFormatFromMaster(SubversionWorkspaceSelector.java:85)
          at hudson.scm.SubversionSCM.createSvnClientManager(SubversionSCM.java:808)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:751)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:738)
          at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1994)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:287)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
          at java.util.concurrent.FutureTask.run(FutureTask.java:123)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)
          at hudson.remoting.Engine$1$1.run(Engine.java:60)
          at java.lang.Thread.run(Thread.java:595)

          After downgrade back to 1.421 all works fine again :-/

          Marco Rothe added a comment - Encountered the same issue after upgrading to 1.427 (from 1.421). The first job an each node fails with the same stacktrace already attached to this issue. All later starts of a job end with hudson.util.IOException2: remote file operation failed: /.../jenkins/workspace/$ourProject at hudson.remoting.Channel@40d39e96:NodeOne at hudson.FilePath.act(FilePath.java:754) at hudson.FilePath.act(FilePath.java:740) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676) at hudson.model.AbstractProject.checkout(AbstractProject.java:1193) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443) at hudson.model.Run.run(Run.java:1376) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:479) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:230) Caused by: java.io.IOException: Remote call on NodeOne failed at hudson.remoting.Channel.call(Channel.java:677) at hudson.FilePath.act(FilePath.java:747) ... 10 more Caused by: java.lang.NoClassDefFoundError at hudson.scm.SubversionWorkspaceSelector.syncWorkspaceFormatFromMaster(SubversionWorkspaceSelector.java:85) at hudson.scm.SubversionSCM.createSvnClientManager(SubversionSCM.java:808) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:751) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:738) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1994) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:287) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269) at java.util.concurrent.FutureTask.run(FutureTask.java:123) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676) at hudson.remoting.Engine$1$1.run(Engine.java:60) at java.lang.Thread.run(Thread.java:595) After downgrade back to 1.421 all works fine again :-/
          Marco Rothe made changes -
          Affects Version/s New: current [ 10162 ]

          Marco Rothe added a comment -

          I found a solution for me today: http://stackoverflow.com/questions/6653147/jenkins-slave-throws-classloading-from-system-classloader-disabled-exception/7364079#7364079

          The answer at stackoverflow is right. The problem is caused by the xercesImpl-2.9.1.jar which is shipped with the jenkins.war since ??? (a version after 1.421).

          After removing this lib from WEB-INF/lib and restarting the master the maven2 jobs start and build without any problem.

          Can somebody check the requierement for the xerxes dependecy? Thx.

          Marco Rothe added a comment - I found a solution for me today: http://stackoverflow.com/questions/6653147/jenkins-slave-throws-classloading-from-system-classloader-disabled-exception/7364079#7364079 The answer at stackoverflow is right. The problem is caused by the xercesImpl-2.9.1.jar which is shipped with the jenkins.war since ??? (a version after 1.421). After removing this lib from WEB-INF/lib and restarting the master the maven2 jobs start and build without any problem. Can somebody check the requierement for the xerxes dependecy? Thx.

          This doesn't seem to be Maven-specific.

          I got the same stack trace as the original attachment, but in my case it was caused by the Xvnc plugin shutting down.

          Here's the top of the stack trace.

          This happened to my when upgrading from the stable version 1.409.1 to 1.409.2.

          My slaves automatically download the latest slave.jar file from Jenkins at startup, so it appears to be (as mentioned in other comments) an issue with the slave JAR file.

          I have since downgraded back to 1.409.1 and everything works again.

          BUILD SUCCESSFUL
          Total time: 4 minutes 26 seconds
          [android] Stopping Android emulator
          [android] Archiving emulator log
          Terminating xvnc.
          FATAL: Remote call on SLAVE_NAME failed
          java.io.IOException: Remote call on SLAVE_NAME failed
          	at hudson.remoting.Channel.call(Channel.java:669)
          	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
          	at $Proxy52.kill(Unknown Source)
          	at hudson.Launcher$RemoteLauncher$ProcImpl.kill(Launcher.java:845)
          	at hudson.plugins.xvnc.Xvnc$3.tearDown(Xvnc.java:130)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:149)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423)
          	at hudson.model.Run.run(Run.java:1362)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:145)
          Caused by: java.lang.ExceptionInInitializerError
          	at hudson.slaves.SlaveComputer.getChannelToMaster(SlaveComputer.java:607)
          	at hudson.util.ProcessTree.getKillers(ProcessTree.java:158)
          	at hudson.util.ProcessTree$OSProcess.killByKiller(ProcessTree.java:219)
          	at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:552)
          	at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:559)
          	at hudson.util.ProcessTree.killAll(ProcessTree.java:147)
          	at hudson.Proc$LocalProc.destroy(Proc.java:379)
          	at hudson.Proc$LocalProc.kill(Proc.java:371)
          	at hudson.Launcher$RemoteLaunchCallable$1.kill(Launcher.java:940)
          

          Christopher Orr added a comment - This doesn't seem to be Maven-specific. I got the same stack trace as the original attachment, but in my case it was caused by the Xvnc plugin shutting down. Here's the top of the stack trace. This happened to my when upgrading from the stable version 1.409.1 to 1.409.2. My slaves automatically download the latest slave.jar file from Jenkins at startup, so it appears to be (as mentioned in other comments) an issue with the slave JAR file. I have since downgraded back to 1.409.1 and everything works again. BUILD SUCCESSFUL Total time: 4 minutes 26 seconds [android] Stopping Android emulator [android] Archiving emulator log Terminating xvnc. FATAL: Remote call on SLAVE_NAME failed java.io.IOException: Remote call on SLAVE_NAME failed at hudson.remoting.Channel.call(Channel.java:669) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158) at $Proxy52.kill(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.kill(Launcher.java:845) at hudson.plugins.xvnc.Xvnc$3.tearDown(Xvnc.java:130) at hudson.model.Build$RunnerImpl.doRun(Build.java:149) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423) at hudson.model.Run.run(Run.java:1362) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:145) Caused by: java.lang.ExceptionInInitializerError at hudson.slaves.SlaveComputer.getChannelToMaster(SlaveComputer.java:607) at hudson.util.ProcessTree.getKillers(ProcessTree.java:158) at hudson.util.ProcessTree$OSProcess.killByKiller(ProcessTree.java:219) at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:552) at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:559) at hudson.util.ProcessTree.killAll(ProcessTree.java:147) at hudson.Proc$LocalProc.destroy(Proc.java:379) at hudson.Proc$LocalProc.kill(Proc.java:371) at hudson.Launcher$RemoteLaunchCallable$1.kill(Launcher.java:940)
          Christopher Orr made changes -
          Assignee New: Kohsuke Kawaguchi [ kohsuke ]

          Yup, the WAR file for 1.409.1 does not include "WEB-INF/lib/xercesImpl-2.9.1.jar", but 1.409.2 does.

          I don't know why this is a problem, but as far as I can tell, this was caused by including "htmlunit" for the JDK download fix without excluding its "xerces" dependency:

          https://jenkins-ci.org/commit/jenkins/18ebe34

          Christopher Orr added a comment - Yup, the WAR file for 1.409.1 does not include "WEB-INF/lib/xercesImpl-2.9.1.jar", but 1.409.2 does. I don't know why this is a problem, but as far as I can tell, this was caused by including "htmlunit" for the JDK download fix without excluding its "xerces" dependency: https://jenkins-ci.org/commit/jenkins/18ebe34
          Kohsuke Kawaguchi made changes -
          Link New: This issue is duplicated by JENKINS-11088 [ JENKINS-11088 ]

          Romain Seguy added a comment - - edited

          I'm using 1.409.1 and the issue happens whereas the only one xercesImpl file is the one in the Maven plugin. Slave is Windows, master is Linux.

          hudson.util.IOException2: remote file operation failed: blabla at udson.remoting.Channel@73d373d3:SLAVE-NAME
          	at hudson.FilePath.act(FilePath.java:752)
          	at hudson.FilePath.act(FilePath.java:738)
          	at hudson.maven.MavenModuleSetBuild$RunnerImpl.parsePoms(MavenModuleSetBuild.java:817)
          	at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:617)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423)
          	at hudson.model.Run.run(Run.java:1362)
          	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:467)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:145)
          ..
          Caused by: javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found
          	at javax.xml.datatype.DatatypeFactory.newInstance(Unknown Source)
          	at com.thoughtworks.xstream.converters.extended.DurationConverter.<init>(DurationConverter.java:33)
          	... 24 more
          Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled
          	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:409)
          	at sun.reflect.GeneratedMethodAccessor3612.invoke(Unknown Source)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
          	at java.lang.reflect.Method.invoke(Method.java:599)
          	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274)
          	at hudson.remoting.Request$2.run(Request.java:270)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
          	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          	at java.lang.Thread.run(Thread.java:735)
          

          Romain Seguy added a comment - - edited I'm using 1.409.1 and the issue happens whereas the only one xercesImpl file is the one in the Maven plugin. Slave is Windows, master is Linux. hudson.util.IOException2: remote file operation failed: blabla at udson.remoting.Channel@73d373d3:SLAVE-NAME at hudson.FilePath.act(FilePath.java:752) at hudson.FilePath.act(FilePath.java:738) at hudson.maven.MavenModuleSetBuild$RunnerImpl.parsePoms(MavenModuleSetBuild.java:817) at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:617) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423) at hudson.model.Run.run(Run.java:1362) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:467) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:145) .. Caused by: javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found at javax.xml.datatype.DatatypeFactory.newInstance(Unknown Source) at com.thoughtworks.xstream.converters.extended.DurationConverter.<init>(DurationConverter.java:33) ... 24 more Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:409) at sun.reflect.GeneratedMethodAccessor3612.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:599) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274) at hudson.remoting.Request$2.run(Request.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:735)

          Kohsuke Kawaguchi added a comment - - edited

          So far I've been unable to reproduce this problem.

          The problem requires that you have Xerces (not the renamed JAXP in Sun JVMs but the real org.apache.xerces... classes) in the bootstrap classloader. This in turn requires that you have a copy of xercesImpl.jar in $JAVA_HOME/lib/jre/ on the master. But I kind of doubt if people are doing it.

          Please report back the exact JVM version used to run the master and the slave in question. Please pay extra attention to what's in the lib folder (or just do "ls -lR" for us.) Please also report the servlet container you use.

          Finally, please run the following probe script in the groovy script console and report back the results.

          println Thread.currentThread().contextClassLoader;
          dtf=javax.xml.datatype.DatatypeFactory.newInstance();
          println dtf;
          println dtf.class.classLoader;
          

          Kohsuke Kawaguchi added a comment - - edited So far I've been unable to reproduce this problem. The problem requires that you have Xerces (not the renamed JAXP in Sun JVMs but the real org.apache.xerces... classes) in the bootstrap classloader. This in turn requires that you have a copy of xercesImpl.jar in $JAVA_HOME/lib/jre/ on the master. But I kind of doubt if people are doing it. Please report back the exact JVM version used to run the master and the slave in question. Please pay extra attention to what's in the lib folder (or just do "ls -lR" for us.) Please also report the servlet container you use. Finally, please run the following probe script in the groovy script console and report back the results. println Thread.currentThread().contextClassLoader; dtf=javax.xml.datatype.DatatypeFactory.newInstance(); println dtf; println dtf.class.classLoader;

            kohsuke Kohsuke Kawaguchi
            glenn_m Glenn Mayer
            Votes:
            4 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: