Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48200

java.io.IOException: Remote call on JNLP4-connect connection failed

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • git-client-plugin
    • None
    • Jenkins version 2.73.2
      swarm client version 3.6
      Using Java 8
      OS: mixed Ubuntu 12.04 and CentOS7
      Jenkins client are physical machines

      We are using Jenkins version 2.73.2 and connected all the Jenkins slaves using Jenkins swarm client version 3.6. We are getting the below error while running a pipeline.

       

       

      java.lang.NoClassDefFoundError: Could not initialize class hudson.util.Secret
          at sun.misc.Unsafe.ensureClassInitialized(Native Method)
          at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
          at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:142)
          at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
          at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
          at java.lang.reflect.Field.getLong(Field.java:611)
          at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1707)
          at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72)
          at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:484)
          at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
          at java.security.AccessController.doPrivileged(Native Method)
          at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:472)
          at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
          at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:598)
          at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
          at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
          at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
          at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
          at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
          at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
          at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
          at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
          at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
          at hudson.remoting.UserRequest.deserialize(UserRequest.java:275)
          at hudson.remoting.UserRequest.perform(UserRequest.java:186)
          at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          at hudson.remoting.Request$2.run(Request.java:360)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at hudson.remoting.Engine$1$1.run(Engine.java:98)
          at java.lang.Thread.run(Thread.java:745)
          at ......remote call to JNLP4-connect connection from slave1/192.168.1.25:44581(Native Method)
          at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1654)
          at hudson.remoting.UserResponse.retrieve(UserRequest.java:311)
          at hudson.remoting.Channel.call(Channel.java:905)
      Caused: java.io.IOException: Remote call on JNLP4-connect connection from slave1/192.168.1.25:44581 failed
          at hudson.remoting.Channel.call(Channel.java:913)
          at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:257)
      Caused: hudson.remoting.RemotingSystemException
          at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:273)
          at com.sun.proxy.$Proxy115.addCredentials(Unknown Source)
          at org.jenkinsci.plugins.gitclient.RemoteGitImpl.addCredentials(RemoteGitImpl.java:200)
          at hudson.plugins.git.GitSCM.createClient(GitSCM.java:771)
          at hudson.plugins.git.GitSCM.createClient(GitSCM.java:747)
          at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1109)
          at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113)
          at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:85)
          at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:75)
          at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
          at hudson.security.ACL.impersonate(ACL.java:260)
          at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:748)
      Finished: FAILURE

          [JENKINS-48200] java.io.IOException: Remote call on JNLP4-connect connection failed

          Mark Waite added a comment -

          oleg_nenashev I don't know how the git client plugin can invoke command line git on the agent using secrets provided from the Jenkins master unless the secrets are serialized and sent to the agent for use. Is there another technique that I missed?

          Mark Waite added a comment - oleg_nenashev I don't know how the git client plugin can invoke command line git on the agent using secrets provided from the Jenkins master unless the secrets are serialized and sent to the agent for use. Is there another technique that I missed?

          Oleg Nenashev added a comment -

          markewaite See https://github.com/jenkinsci/git-client-plugin/blob/4dfc9368a7ce65bbb91294c22a0cd439fa7f77ea/src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java#L200. This call explicitly calls addCredentials() on an exported object of GitClient. Git Client uses a custom CommandInvocationHandler implementation for it, but it still causes an RPC call and makes the entire Credentials object to be serialized over the channel if I understand the code.

          Oleg Nenashev added a comment - markewaite See https://github.com/jenkinsci/git-client-plugin/blob/4dfc9368a7ce65bbb91294c22a0cd439fa7f77ea/src/main/java/org/jenkinsci/plugins/gitclient/RemoteGitImpl.java#L200 . This call explicitly calls addCredentials() on an exported object of GitClient. Git Client uses a custom CommandInvocationHandler implementation for it, but it still causes an RPC call and makes the entire Credentials object to be serialized over the channel if I understand the code.

          Oleg Nenashev added a comment -

          From what I see Remoting and Swarm client are fine. From what I see Git Plugin just should not send Credentials types with Secrets over Remoting.

          Oleg Nenashev added a comment - From what I see Remoting and Swarm client are fine. From what I see Git Plugin just should not send Credentials types with Secrets over Remoting.

          Mark Waite added a comment -

          oleg_nenashev I don't understand how the git client plugin can perform authenticated operations on the agent without sending credentials over remoting. Is it better to somehow extract the user name and password (or private key) from the credential on the master, then send the user name and password (or private key) as strings?

          Can you point me to a good example of the preferred technique for credential handling in a plugin which needs to use the credentials on an agent?

          Mark Waite added a comment - oleg_nenashev I don't understand how the git client plugin can perform authenticated operations on the agent without sending credentials over remoting. Is it better to somehow extract the user name and password (or private key) from the credential on the master, then send the user name and password (or private key) as strings? Can you point me to a good example of the preferred technique for credential handling in a plugin which needs to use the credentials on an agent?

          Oleg Nenashev added a comment -

          markewaite I would say that https://github.com/jenkinsci/credentials-binding-plugin from jglick is the best example. Maybe it would be possible to reuse it's codebase somehow

          Oleg Nenashev added a comment - markewaite I would say that https://github.com/jenkinsci/credentials-binding-plugin from jglick is the best example. Maybe it would be possible to reuse it's codebase somehow

          Mark Waite added a comment -

          Session with Stephen Connolly and Oleg Nenashev showed that the likely best path forward is to create a git client authentication token class which uses AuthenticationTokens.convert() to map specific credentials (username/password or ssh private key) to the git client, then that git client authentication token would be sent over the wire from the master to the agent, rather than sending the Secret over the wire.

          Stephen noted that the gitea plugin is an example of a plugin using authentication tokens in that way. He recommended using the strategy pattern so that the git client plugin authentication could be extended by other plugins when they want to support other credentials, like client certificates or personal access tokens.

          Mark Waite added a comment - Session with Stephen Connolly and Oleg Nenashev showed that the likely best path forward is to create a git client authentication token class which uses AuthenticationTokens.convert() to map specific credentials (username/password or ssh private key) to the git client, then that git client authentication token would be sent over the wire from the master to the agent, rather than sending the Secret over the wire. Stephen noted that the gitea plugin is an example of a plugin using authentication tokens in that way. He recommended using the strategy pattern so that the git client plugin authentication could be extended by other plugins when they want to support other credentials, like client certificates or personal access tokens.

          Any news on this issue?

          I see this issue on my setup and it is getting a big pain to deal with it. At the moment the only way I found it can be worked around is to restart the service on the slave this issue happens on. Of course this isn't something which can be easily automated across multiple jobs. I really would like to see a proper solution for this problem.

          Szabolcs Tolnai added a comment - Any news on this issue? I see this issue on my setup and it is getting a big pain to deal with it. At the moment the only way I found it can be worked around is to restart the service on the slave this issue happens on. Of course this isn't something which can be easily automated across multiple jobs. I really would like to see a proper solution for this problem.

          Mark Waite added a comment -

          szabi_tolnai no progress on the issue and no progress is expected for a long time. You're welcome to prepare a pull request that proposes a git client authentication token class to map specific credentials to the git client.

          Mark Waite added a comment - szabi_tolnai no progress on the issue and no progress is expected for a long time. You're welcome to prepare a pull request that proposes a git client authentication token class to map specific credentials to the git client.

          markewaite, I don't really want to get into the details of the plug-in development, just hoped that a solution is on its way.

          By the way, I didn't really understood the proposed solution. How will that help? Based on the error message, there is some issue with communication between the slave and the master to get the credentials to the other side. The same will have to happen with a token as well. Will not have the same issue with the communication channel? Or I completely misunderstood the whole issue, or maybe the reported error message isn't really the issue.

          Szabolcs Tolnai added a comment - markewaite , I don't really want to get into the details of the plug-in development, just hoped that a solution is on its way. By the way, I didn't really understood the proposed solution. How will that help? Based on the error message, there is some issue with communication between the slave and the master to get the credentials to the other side. The same will have to happen with a token as well. Will not have the same issue with the communication channel? Or I completely misunderstood the whole issue, or maybe the reported error message isn't really the issue.

          Mark Waite added a comment -

          If I've understood Oleg's comment correctly, the problem is that the git client plugin mistakenly serializes a complete Credentials object which contains a Secret. I believe the problem is that is "more object than is necessary".

          Unfortunately, I'm not available to work on this area of the plugin. Plugin maintenance and development is a hobby for my nights and weekends. Other topics in the plugin are higher priority currently than this case that I'm unable to duplicate and requires fundamental changes to the git client plugin.

          Mark Waite added a comment - If I've understood Oleg's comment correctly, the problem is that the git client plugin mistakenly serializes a complete Credentials object which contains a Secret. I believe the problem is that is "more object than is necessary". Unfortunately, I'm not available to work on this area of the plugin. Plugin maintenance and development is a hobby for my nights and weekends. Other topics in the plugin are higher priority currently than this case that I'm unable to duplicate and requires fundamental changes to the git client plugin.

            Unassigned Unassigned
            rengan Rajasekar Ramasamy
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: