Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10238

Jenkins slave throws "Classloading from system classloader disabled" exception

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • maven-plugin
    • None
    • Master on AIX 5.3; slave on Windows 2008(R2)

      When running a Maven2 job on a slave instance, I get the following message (complete stack trace attached): "java.lang.ClassNotFoundException: Classloading from system classloader disabled". This happens with the first job after the slave instance is started (it's running as a Windows service). After that, I get a "java.lang.NoClassDefFoundError: Could not initialize class hudson.maven.MavenModuleSetBuild" error.
      If I create a freestyle job, and include the maven command as a batch command to run on the slave, it works fine. The Maven2 job appears to locate Maven on the slave machine, but it apparently can't find the maven-plugin (probably due to the classloading error).

          [JENKINS-10238] Jenkins slave throws "Classloading from system classloader disabled" exception

          Marco Rothe added a comment -

          Encountered the same issue after upgrading to 1.427 (from 1.421).

          The first job an each node fails with the same stacktrace already attached to this issue.
          All later starts of a job end with

          hudson.util.IOException2: remote file operation failed: /.../jenkins/workspace/$ourProject at hudson.remoting.Channel@40d39e96:NodeOne
          at hudson.FilePath.act(FilePath.java:754)
          at hudson.FilePath.act(FilePath.java:740)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1193)
          at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555)
          at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443)
          at hudson.model.Run.run(Run.java:1376)
          at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:479)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:230)
          Caused by: java.io.IOException: Remote call on NodeOne failed
          at hudson.remoting.Channel.call(Channel.java:677)
          at hudson.FilePath.act(FilePath.java:747)
          ... 10 more
          Caused by: java.lang.NoClassDefFoundError
          at hudson.scm.SubversionWorkspaceSelector.syncWorkspaceFormatFromMaster(SubversionWorkspaceSelector.java:85)
          at hudson.scm.SubversionSCM.createSvnClientManager(SubversionSCM.java:808)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:751)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:738)
          at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1994)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:287)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
          at java.util.concurrent.FutureTask.run(FutureTask.java:123)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)
          at hudson.remoting.Engine$1$1.run(Engine.java:60)
          at java.lang.Thread.run(Thread.java:595)

          After downgrade back to 1.421 all works fine again :-/

          Marco Rothe added a comment - Encountered the same issue after upgrading to 1.427 (from 1.421). The first job an each node fails with the same stacktrace already attached to this issue. All later starts of a job end with hudson.util.IOException2: remote file operation failed: /.../jenkins/workspace/$ourProject at hudson.remoting.Channel@40d39e96:NodeOne at hudson.FilePath.act(FilePath.java:754) at hudson.FilePath.act(FilePath.java:740) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676) at hudson.model.AbstractProject.checkout(AbstractProject.java:1193) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443) at hudson.model.Run.run(Run.java:1376) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:479) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:230) Caused by: java.io.IOException: Remote call on NodeOne failed at hudson.remoting.Channel.call(Channel.java:677) at hudson.FilePath.act(FilePath.java:747) ... 10 more Caused by: java.lang.NoClassDefFoundError at hudson.scm.SubversionWorkspaceSelector.syncWorkspaceFormatFromMaster(SubversionWorkspaceSelector.java:85) at hudson.scm.SubversionSCM.createSvnClientManager(SubversionSCM.java:808) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:751) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:738) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1994) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:287) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269) at java.util.concurrent.FutureTask.run(FutureTask.java:123) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676) at hudson.remoting.Engine$1$1.run(Engine.java:60) at java.lang.Thread.run(Thread.java:595) After downgrade back to 1.421 all works fine again :-/

          Marco Rothe added a comment -

          I found a solution for me today: http://stackoverflow.com/questions/6653147/jenkins-slave-throws-classloading-from-system-classloader-disabled-exception/7364079#7364079

          The answer at stackoverflow is right. The problem is caused by the xercesImpl-2.9.1.jar which is shipped with the jenkins.war since ??? (a version after 1.421).

          After removing this lib from WEB-INF/lib and restarting the master the maven2 jobs start and build without any problem.

          Can somebody check the requierement for the xerxes dependecy? Thx.

          Marco Rothe added a comment - I found a solution for me today: http://stackoverflow.com/questions/6653147/jenkins-slave-throws-classloading-from-system-classloader-disabled-exception/7364079#7364079 The answer at stackoverflow is right. The problem is caused by the xercesImpl-2.9.1.jar which is shipped with the jenkins.war since ??? (a version after 1.421). After removing this lib from WEB-INF/lib and restarting the master the maven2 jobs start and build without any problem. Can somebody check the requierement for the xerxes dependecy? Thx.

          This doesn't seem to be Maven-specific.

          I got the same stack trace as the original attachment, but in my case it was caused by the Xvnc plugin shutting down.

          Here's the top of the stack trace.

          This happened to my when upgrading from the stable version 1.409.1 to 1.409.2.

          My slaves automatically download the latest slave.jar file from Jenkins at startup, so it appears to be (as mentioned in other comments) an issue with the slave JAR file.

          I have since downgraded back to 1.409.1 and everything works again.

          BUILD SUCCESSFUL
          Total time: 4 minutes 26 seconds
          [android] Stopping Android emulator
          [android] Archiving emulator log
          Terminating xvnc.
          FATAL: Remote call on SLAVE_NAME failed
          java.io.IOException: Remote call on SLAVE_NAME failed
          	at hudson.remoting.Channel.call(Channel.java:669)
          	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
          	at $Proxy52.kill(Unknown Source)
          	at hudson.Launcher$RemoteLauncher$ProcImpl.kill(Launcher.java:845)
          	at hudson.plugins.xvnc.Xvnc$3.tearDown(Xvnc.java:130)
          	at hudson.model.Build$RunnerImpl.doRun(Build.java:149)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423)
          	at hudson.model.Run.run(Run.java:1362)
          	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:145)
          Caused by: java.lang.ExceptionInInitializerError
          	at hudson.slaves.SlaveComputer.getChannelToMaster(SlaveComputer.java:607)
          	at hudson.util.ProcessTree.getKillers(ProcessTree.java:158)
          	at hudson.util.ProcessTree$OSProcess.killByKiller(ProcessTree.java:219)
          	at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:552)
          	at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:559)
          	at hudson.util.ProcessTree.killAll(ProcessTree.java:147)
          	at hudson.Proc$LocalProc.destroy(Proc.java:379)
          	at hudson.Proc$LocalProc.kill(Proc.java:371)
          	at hudson.Launcher$RemoteLaunchCallable$1.kill(Launcher.java:940)
          

          Christopher Orr added a comment - This doesn't seem to be Maven-specific. I got the same stack trace as the original attachment, but in my case it was caused by the Xvnc plugin shutting down. Here's the top of the stack trace. This happened to my when upgrading from the stable version 1.409.1 to 1.409.2. My slaves automatically download the latest slave.jar file from Jenkins at startup, so it appears to be (as mentioned in other comments) an issue with the slave JAR file. I have since downgraded back to 1.409.1 and everything works again. BUILD SUCCESSFUL Total time: 4 minutes 26 seconds [android] Stopping Android emulator [android] Archiving emulator log Terminating xvnc. FATAL: Remote call on SLAVE_NAME failed java.io.IOException: Remote call on SLAVE_NAME failed at hudson.remoting.Channel.call(Channel.java:669) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158) at $Proxy52.kill(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.kill(Launcher.java:845) at hudson.plugins.xvnc.Xvnc$3.tearDown(Xvnc.java:130) at hudson.model.Build$RunnerImpl.doRun(Build.java:149) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423) at hudson.model.Run.run(Run.java:1362) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:145) Caused by: java.lang.ExceptionInInitializerError at hudson.slaves.SlaveComputer.getChannelToMaster(SlaveComputer.java:607) at hudson.util.ProcessTree.getKillers(ProcessTree.java:158) at hudson.util.ProcessTree$OSProcess.killByKiller(ProcessTree.java:219) at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:552) at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:559) at hudson.util.ProcessTree.killAll(ProcessTree.java:147) at hudson.Proc$LocalProc.destroy(Proc.java:379) at hudson.Proc$LocalProc.kill(Proc.java:371) at hudson.Launcher$RemoteLaunchCallable$1.kill(Launcher.java:940)

          Yup, the WAR file for 1.409.1 does not include "WEB-INF/lib/xercesImpl-2.9.1.jar", but 1.409.2 does.

          I don't know why this is a problem, but as far as I can tell, this was caused by including "htmlunit" for the JDK download fix without excluding its "xerces" dependency:

          https://jenkins-ci.org/commit/jenkins/18ebe34

          Christopher Orr added a comment - Yup, the WAR file for 1.409.1 does not include "WEB-INF/lib/xercesImpl-2.9.1.jar", but 1.409.2 does. I don't know why this is a problem, but as far as I can tell, this was caused by including "htmlunit" for the JDK download fix without excluding its "xerces" dependency: https://jenkins-ci.org/commit/jenkins/18ebe34

          Romain Seguy added a comment - - edited

          I'm using 1.409.1 and the issue happens whereas the only one xercesImpl file is the one in the Maven plugin. Slave is Windows, master is Linux.

          hudson.util.IOException2: remote file operation failed: blabla at udson.remoting.Channel@73d373d3:SLAVE-NAME
          	at hudson.FilePath.act(FilePath.java:752)
          	at hudson.FilePath.act(FilePath.java:738)
          	at hudson.maven.MavenModuleSetBuild$RunnerImpl.parsePoms(MavenModuleSetBuild.java:817)
          	at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:617)
          	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423)
          	at hudson.model.Run.run(Run.java:1362)
          	at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:467)
          	at hudson.model.ResourceController.execute(ResourceController.java:88)
          	at hudson.model.Executor.run(Executor.java:145)
          ..
          Caused by: javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found
          	at javax.xml.datatype.DatatypeFactory.newInstance(Unknown Source)
          	at com.thoughtworks.xstream.converters.extended.DurationConverter.<init>(DurationConverter.java:33)
          	... 24 more
          Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled
          	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:409)
          	at sun.reflect.GeneratedMethodAccessor3612.invoke(Unknown Source)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
          	at java.lang.reflect.Method.invoke(Method.java:599)
          	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274)
          	at hudson.remoting.Request$2.run(Request.java:270)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
          	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          	at java.lang.Thread.run(Thread.java:735)
          

          Romain Seguy added a comment - - edited I'm using 1.409.1 and the issue happens whereas the only one xercesImpl file is the one in the Maven plugin. Slave is Windows, master is Linux. hudson.util.IOException2: remote file operation failed: blabla at udson.remoting.Channel@73d373d3:SLAVE-NAME at hudson.FilePath.act(FilePath.java:752) at hudson.FilePath.act(FilePath.java:738) at hudson.maven.MavenModuleSetBuild$RunnerImpl.parsePoms(MavenModuleSetBuild.java:817) at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:617) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:423) at hudson.model.Run.run(Run.java:1362) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:467) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:145) .. Caused by: javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found at javax.xml.datatype.DatatypeFactory.newInstance(Unknown Source) at com.thoughtworks.xstream.converters.extended.DurationConverter.<init>(DurationConverter.java:33) ... 24 more Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:409) at sun.reflect.GeneratedMethodAccessor3612.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:599) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274) at hudson.remoting.Request$2.run(Request.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:735)

          Kohsuke Kawaguchi added a comment - - edited

          So far I've been unable to reproduce this problem.

          The problem requires that you have Xerces (not the renamed JAXP in Sun JVMs but the real org.apache.xerces... classes) in the bootstrap classloader. This in turn requires that you have a copy of xercesImpl.jar in $JAVA_HOME/lib/jre/ on the master. But I kind of doubt if people are doing it.

          Please report back the exact JVM version used to run the master and the slave in question. Please pay extra attention to what's in the lib folder (or just do "ls -lR" for us.) Please also report the servlet container you use.

          Finally, please run the following probe script in the groovy script console and report back the results.

          println Thread.currentThread().contextClassLoader;
          dtf=javax.xml.datatype.DatatypeFactory.newInstance();
          println dtf;
          println dtf.class.classLoader;
          

          Kohsuke Kawaguchi added a comment - - edited So far I've been unable to reproduce this problem. The problem requires that you have Xerces (not the renamed JAXP in Sun JVMs but the real org.apache.xerces... classes) in the bootstrap classloader. This in turn requires that you have a copy of xercesImpl.jar in $JAVA_HOME/lib/jre/ on the master. But I kind of doubt if people are doing it. Please report back the exact JVM version used to run the master and the slave in question. Please pay extra attention to what's in the lib folder (or just do "ls -lR" for us.) Please also report the servlet container you use. Finally, please run the following probe script in the groovy script console and report back the results. println Thread.currentThread().contextClassLoader; dtf=javax.xml.datatype.DatatypeFactory.newInstance(); println dtf; println dtf.class.classLoader;

          Marco Rothe added a comment - - edited

          There are no explicit xerces related jar's in the bootstrap loader of the slave. But I found on the master

          sun.boot.class.path:.../jdk1.6.0_23-x64\lib\alt-string.jar;
          .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jaxb-api.jar;
          .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-jaxrpc.jar;
          .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-jaxws-ext.jar;
          .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-jaxws.jar;
          .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-saaj.jar;
          .../jboss-4.2.3.GA-buildsystem/lib/endorsed\serializer.
          jar;.../jboss-4.2.3.GA-buildsystem/lib/endorsed\xalan.jar;

          .../jboss-4.2.3.GA-buildsystem/lib/endorsed\xercesImpl.jar;

          .../jdk1.6.0_23-x64\lib\resources.jar;
          .../jdk1.6.0_23-x64\lib\rt.jar;
          .../jdk1.6.0_23-x64\lib\sunrsasign.jar;
          .../jdk1.6.0_23-x64\lib\jsse.jar;
          .../jdk1.6.0_23-x64\lib\jce.jar;
          .../jdk1.6.0_23-x64\lib\charsets.jar;
          .../jdk1.6.0_23-x64\lib\modules\jdk.boot.jar;
          .../jdk1.6.0_23-x64\classes,

          So the JBoss, which hosts the jenkins master, is shipped with its own xercesImpl.jar (it seems to be version 2.7.1).

          The master system itself is a 64bit Win2K8 Server which runs an JBoss 4.2.3 on JDK 1.6.23-x64:

          os.name:Windows Server 2008 R2, 
          os.arch:amd64
          java.runtime.version:1.6.0_23-b05 
          sun.management.compiler:HotSpot 64-Bit Server Compiler
          

          So maybe the xerxesImpl on JBoss made the difference?

          Otherwise XSTR-537 may also related to this problem because it stated:
          "...Either you are using not an XStream version >= 1.3.1 or you run with a JDK >= 1.5. Otherwise the DurationConverter is not registered automatically...".

          In our enviroment we run the master with JDK 1.6, but the slaves are still on JDK 1.5...
          The slave systems are Sun sparcs with SunOS 5.1

          os.name:SunOS
          os.version:5.10
          os.arch:sparc
          java.runtime.version:1.5.0_22-b03 
          sun.management.compiler:HotSpot Server Compiler
          

          The output ofthe probe script on one of the slaves is:

          hudson.remoting.RemoteClassLoader@1150b52
          com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl@b2cdf0
          null
          

          Marco Rothe added a comment - - edited There are no explicit xerces related jar's in the bootstrap loader of the slave. But I found on the master sun.boot.class.path:.../jdk1.6.0_23-x64\lib\alt-string.jar; .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jaxb-api.jar; .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-jaxrpc.jar; .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-jaxws-ext.jar; .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-jaxws.jar; .../jboss-4.2.3.GA-buildsystem/lib/endorsed\jboss-saaj.jar; .../jboss-4.2.3.GA-buildsystem/lib/endorsed\serializer. jar;.../jboss-4.2.3.GA-buildsystem/lib/endorsed\xalan.jar; .../jboss-4.2.3.GA-buildsystem/lib/endorsed\xercesImpl.jar; .../jdk1.6.0_23-x64\lib\resources.jar; .../jdk1.6.0_23-x64\lib\rt.jar; .../jdk1.6.0_23-x64\lib\sunrsasign.jar; .../jdk1.6.0_23-x64\lib\jsse.jar; .../jdk1.6.0_23-x64\lib\jce.jar; .../jdk1.6.0_23-x64\lib\charsets.jar; .../jdk1.6.0_23-x64\lib\modules\jdk.boot.jar; .../jdk1.6.0_23-x64\classes, So the JBoss, which hosts the jenkins master, is shipped with its own xercesImpl.jar (it seems to be version 2.7.1). The master system itself is a 64bit Win2K8 Server which runs an JBoss 4.2.3 on JDK 1.6.23-x64: os.name:Windows Server 2008 R2, os.arch:amd64 java.runtime.version:1.6.0_23-b05 sun.management.compiler:HotSpot 64-Bit Server Compiler So maybe the xerxesImpl on JBoss made the difference? Otherwise XSTR-537 may also related to this problem because it stated: "...Either you are using not an XStream version >= 1.3.1 or you run with a JDK >= 1.5. Otherwise the DurationConverter is not registered automatically...". In our enviroment we run the master with JDK 1.6, but the slaves are still on JDK 1.5... The slave systems are Sun sparcs with SunOS 5.1 os.name:SunOS os.version:5.10 os.arch:sparc java.runtime.version:1.5.0_22-b03 sun.management.compiler:HotSpot Server Compiler The output ofthe probe script on one of the slaves is: hudson.remoting.RemoteClassLoader@1150b52 com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl@b2cdf0 null

          Hi,

          Same issue with a more recent version of Jenkins.

          Configuration :

          • Jenkins 1.473 (running with Winstone)
          • Master : AIX 5.3 - java version "1.6.0" Java(TM) SE Runtime Environment (build pap3260sr9-20101125_01(SR9)) (IBM)
          • 2 slaves : RHEL 5 - java "1.6.0_25" Java(TM) SE Runtime Environment (build 1.6.0_25-b06) Java HotSpot(TM) 64-Bit Server VM (build 20.0-b11, mixed mode)

          Error when running the job (type = Maven) :

          Started by user ME
          Construction à distance sur SLAVE in workspace /path/to/jenkins/workspace/currentjob
          Checkout:currentjob / /path/to/jenkins/workspace/currentjob - hudson.remoting.Channel@583a583a:SLAVE
          Using strategy: Default
          Last Built Revision: Revision 885ed28b6e4e67fdfa52f5aed1a80e996cc13f5d (origin/HEAD, origin/master)
          Cloning the remote Git repository
          Cloning repository origin
          Fetching upstream changes from path://to/git/repo/currentjob.git
          Seen branch in repository origin/HEAD
          Seen branch in repository origin/master
          Commencing build of Revision 885ed28b6e4e67fdfa52f5aed1a80e996cc13f5d (origin/HEAD, origin/master)
          Checking out Revision 885ed28b6e4e67fdfa52f5aed1a80e996cc13f5d (origin/HEAD, origin/master)
          Warning : There are multiple branch changesets here
          Parsing POMs
          ERROR: Echec à la lecture des POMs
          hudson.util.IOException2: remote file operation failed: /path/to/jenkins/workspace/currentjob at hudson.remoting.Channel@583a583a:SLAVE
          at hudson.FilePath.act(FilePath.java:838)
          at hudson.FilePath.act(FilePath.java:824)
          at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:910)
          at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:654)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
          at hudson.model.Run.execute(Run.java:1488)
          at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:236)
          Caused by: java.io.IOException: Remote call on SLAVE failed
          at hudson.remoting.Channel.call(Channel.java:672)
          at hudson.FilePath.act(FilePath.java:831)
          ... 8 more
          Caused by: java.lang.ExceptionInInitializerError
          at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1182)
          at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1045)
          at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2196)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:326)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
          at java.util.concurrent.FutureTask.run(FutureTask.java:138)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at hudson.remoting.Engine$1$1.run(Engine.java:60)
          at java.lang.Thread.run(Thread.java:662)
          Caused by: com.thoughtworks.xstream.XStream$InitializationException: Could not instantiate converter : com.thoughtworks.xstream.converters.extended.DurationConverter : null
          at com.thoughtworks.xstream.XStream.dynamicallyRegisterConverter(XStream.java:735)
          at com.thoughtworks.xstream.XStream.setupConverters(XStream.java:699)
          at com.thoughtworks.xstream.XStream.<init>(XStream.java:445)
          at com.thoughtworks.xstream.XStream.<init>(XStream.java:385)
          at com.thoughtworks.xstream.XStream.<init>(XStream.java:323)
          at hudson.util.XStream2.<init>(XStream2.java:75)
          at hudson.model.Run.<clinit>(Run.java:2006)
          ... 13 more
          Caused by: java.lang.reflect.InvocationTargetException
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
          at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
          at com.thoughtworks.xstream.XStream.dynamicallyRegisterConverter(XStream.java:728)
          ... 19 more
          Caused by: javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found
          at javax.xml.datatype.DatatypeFactory.newInstance(DatatypeFactory.java:137)
          at com.thoughtworks.xstream.converters.extended.DurationConverter.<init>(DurationConverter.java:33)
          ... 24 more
          Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled
          at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:413)
          at sun.reflect.GeneratedMethodAccessor1340.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
          at java.lang.reflect.Method.invoke(Method.java:611)
          at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274)
          at hudson.remoting.Request$2.run(Request.java:326)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
          at java.util.concurrent.FutureTask.run(FutureTask.java:149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
          at java.lang.Thread.run(Thread.java:736)
          Saut de l&#39analyse sonar suite à un mauvais status de construction FAILURE
          Notifying upstream projects

          @Kohsuke Kawaguchi : I've tried to execute these commands of the Groovy console :
          println Thread.currentThread().contextClassLoader;
          dtf=javax.xml.datatype.DatatypeFactory.newInstance();
          println dtf;
          println dtf.class.classLoader;

          Result :
          hudson.remoting.RemoteClassLoader@8939e14
          javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found
          at javax.xml.datatype.DatatypeFactory.newInstance(DatatypeFactory.java:137)
          at javax.xml.datatype.DatatypeFactory$newInstance.call(Unknown Source)
          at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
          at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
          at Script1.run(Script1.groovy:2)
          at groovy.lang.GroovyShell.evaluate(GroovyShell.java:580)
          at groovy.lang.GroovyShell.evaluate(GroovyShell.java:618)
          at groovy.lang.GroovyShell.evaluate(GroovyShell.java:589)
          at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:150)
          at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:122)
          at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          at hudson.remoting.Request$2.run(Request.java:326)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
          at java.util.concurrent.FutureTask.run(FutureTask.java:138)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at hudson.remoting.Engine$1$1.run(Engine.java:60)
          at java.lang.Thread.run(Thread.java:662)
          Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled
          at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:413)
          at sun.reflect.GeneratedMethodAccessor1178.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
          at java.lang.reflect.Method.invoke(Method.java:611)
          at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274)
          at hudson.remoting.Request$2.run(Request.java:326)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
          at java.util.concurrent.FutureTask.run(FutureTask.java:149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
          at java.lang.Thread.run(Thread.java:736)

          It seems that xerces lib is missing in the slave.
          To test, I added xercesImpl-2.9.1.jar directly in the classpath when launching the slave :
          > java -jar slave.jar -cp xercesImpl-2.9.1.jar -jnlpUrl http://ic-host:8080/computer/SLAVE/slave-agent.jnlp

          I've launch again these commands in the Groovy console :
          Result :
          hudson.remoting.RemoteClassLoader@7a759e4f
          org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl@24d5578d
          sun.misc.Launcher$AppClassLoader@77cde100

          The bug seems to have disappeared and I have not found other regressions for the moment.
          But I'm not sure that is the best solution to add manually a jar file in the classpath of the slave launcher

          Michael Pailloncy added a comment - Hi, Same issue with a more recent version of Jenkins. Configuration : Jenkins 1.473 (running with Winstone) Master : AIX 5.3 - java version "1.6.0" Java(TM) SE Runtime Environment (build pap3260sr9-20101125_01(SR9)) (IBM) 2 slaves : RHEL 5 - java "1.6.0_25" Java(TM) SE Runtime Environment (build 1.6.0_25-b06) Java HotSpot(TM) 64-Bit Server VM (build 20.0-b11, mixed mode) Error when running the job (type = Maven) : Started by user ME Construction à distance sur SLAVE in workspace /path/to/jenkins/workspace/currentjob Checkout:currentjob / /path/to/jenkins/workspace/currentjob - hudson.remoting.Channel@583a583a:SLAVE Using strategy: Default Last Built Revision: Revision 885ed28b6e4e67fdfa52f5aed1a80e996cc13f5d (origin/HEAD, origin/master) Cloning the remote Git repository Cloning repository origin Fetching upstream changes from path://to/git/repo/currentjob.git Seen branch in repository origin/HEAD Seen branch in repository origin/master Commencing build of Revision 885ed28b6e4e67fdfa52f5aed1a80e996cc13f5d (origin/HEAD, origin/master) Checking out Revision 885ed28b6e4e67fdfa52f5aed1a80e996cc13f5d (origin/HEAD, origin/master) Warning : There are multiple branch changesets here Parsing POMs ERROR: Echec à la lecture des POMs hudson.util.IOException2: remote file operation failed: /path/to/jenkins/workspace/currentjob at hudson.remoting.Channel@583a583a:SLAVE at hudson.FilePath.act(FilePath.java:838) at hudson.FilePath.act(FilePath.java:824) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:910) at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:654) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1488) at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236) Caused by: java.io.IOException: Remote call on SLAVE failed at hudson.remoting.Channel.call(Channel.java:672) at hudson.FilePath.act(FilePath.java:831) ... 8 more Caused by: java.lang.ExceptionInInitializerError at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1182) at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1045) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2196) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at hudson.remoting.Engine$1$1.run(Engine.java:60) at java.lang.Thread.run(Thread.java:662) Caused by: com.thoughtworks.xstream.XStream$InitializationException: Could not instantiate converter : com.thoughtworks.xstream.converters.extended.DurationConverter : null at com.thoughtworks.xstream.XStream.dynamicallyRegisterConverter(XStream.java:735) at com.thoughtworks.xstream.XStream.setupConverters(XStream.java:699) at com.thoughtworks.xstream.XStream.<init>(XStream.java:445) at com.thoughtworks.xstream.XStream.<init>(XStream.java:385) at com.thoughtworks.xstream.XStream.<init>(XStream.java:323) at hudson.util.XStream2.<init>(XStream2.java:75) at hudson.model.Run.<clinit>(Run.java:2006) ... 13 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.thoughtworks.xstream.XStream.dynamicallyRegisterConverter(XStream.java:728) ... 19 more Caused by: javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found at javax.xml.datatype.DatatypeFactory.newInstance(DatatypeFactory.java:137) at com.thoughtworks.xstream.converters.extended.DurationConverter.<init>(DurationConverter.java:33) ... 24 more Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:413) at sun.reflect.GeneratedMethodAccessor1340.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:611) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:736) Saut de l&#39analyse sonar suite à un mauvais status de construction FAILURE Notifying upstream projects @Kohsuke Kawaguchi : I've tried to execute these commands of the Groovy console : println Thread.currentThread().contextClassLoader; dtf=javax.xml.datatype.DatatypeFactory.newInstance(); println dtf; println dtf.class.classLoader; Result : hudson.remoting.RemoteClassLoader@8939e14 javax.xml.datatype.DatatypeConfigurationException: Provider org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl not found at javax.xml.datatype.DatatypeFactory.newInstance(DatatypeFactory.java:137) at javax.xml.datatype.DatatypeFactory$newInstance.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112) at Script1.run(Script1.groovy:2) at groovy.lang.GroovyShell.evaluate(GroovyShell.java:580) at groovy.lang.GroovyShell.evaluate(GroovyShell.java:618) at groovy.lang.GroovyShell.evaluate(GroovyShell.java:589) at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:150) at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:122) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at hudson.remoting.Engine$1$1.run(Engine.java:60) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.ClassNotFoundException: Classloading from system classloader disabled at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch2(RemoteClassLoader.java:413) at sun.reflect.GeneratedMethodAccessor1178.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:611) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:274) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:736) It seems that xerces lib is missing in the slave. To test, I added xercesImpl-2.9.1.jar directly in the classpath when launching the slave : > java -jar slave.jar -cp xercesImpl-2.9.1.jar -jnlpUrl http://ic-host:8080/computer/SLAVE/slave-agent.jnlp I've launch again these commands in the Groovy console : Result : hudson.remoting.RemoteClassLoader@7a759e4f org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl@24d5578d sun.misc.Launcher$AppClassLoader@77cde100 The bug seems to have disappeared and I have not found other regressions for the moment. But I'm not sure that is the best solution to add manually a jar file in the classpath of the slave launcher

          Jesse Glick added a comment -

          remoting PR 309 may improve diagnostics.

          Jesse Glick added a comment - remoting PR 309 may improve diagnostics.

            kohsuke Kohsuke Kawaguchi
            glenn_m Glenn Mayer
            Votes:
            4 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: