Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26866

jenkins java.io.ioexception remote file operation failed

    XMLWordPrintable

Details

    Description

      Jenkins jobs assigned to java web-start slaves.

      The jobs in question were all working fine.
      Suddenly I get:

      jenkins java.io.ioexception remote file operation failed

      The call stack is essentially useless, and points to an exception handler in some Java class without indicating what exactly was attempted.

      Unfortunately this occurred just as I was preparing for a holiday. I do not have exact call stack now (I'm on a train).

      I fixed the problem by making the script explicitly perform a svn checkout/update rather than the normal method where one provides the URL and credentials in the fields for the "configure" page.

      That should be a good indication of where the issue lies.

      I really would like some kind of feedback on this issue, even though I will be away for a week. I would like to have the "normal" functionality and am willing to help investigate.

      Environment is:

      Fedora 21 x86_64 - all updates current as of 09.Feb.2015
      Master and slaves are identical.

      I appreciate any help.

      Attachments

        Activity

          thetaiter Ian Tait added a comment -

          I am also having this issue on various machines.

          thetaiter Ian Tait added a comment - I am also having this issue on various machines.

          dmir1988 I could not reproduce the bug. Could you provide more information? /cc smouch

          recena Manuel Recena Soto added a comment - dmir1988 I could not reproduce the bug. Could you provide more information? /cc smouch
          petervarga Peter Varga added a comment -

          Hi,

          we are facing a similar problem. We are using Jenkins version 1.591, and have two slaves configured. In case the job uses our slave-01, we are getting the NoClassDefFoundError shown below. If I configure the job, to use our slave-02, it works fine. We were not able to find any configuration differences between our slaves.

          [EnvInject] - Loading node environment variables.
          Building remotely on slave-01 (wfm-trunk linux tasktrack) in workspace /srv/jenkins-home/workspace/deployment-int-shutdown-instance-abc
          java.io.IOException: remote file operation failed: /srv/jenkins-home/workspace/deployment-int-shutdown-instance-abc at hudson.remoting.Channel@205b38e:slave-01: java.io.IOException: Remote call on slave-01 failed
          at hudson.FilePath.act(FilePath.java:977)
          at hudson.FilePath.act(FilePath.java:959)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:910)
          at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:845)
          at hudson.model.AbstractProject.checkout(AbstractProject.java:1259)
          at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:622)
          at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:528)
          at hudson.model.Run.execute(Run.java:1759)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
          at hudson.model.ResourceController.execute(ResourceController.java:89)
          at hudson.model.Executor.run(Executor.java:240)
          Caused by: java.io.IOException: Remote call on slave-01 failed
          at hudson.remoting.Channel.call(Channel.java:760)
          at hudson.FilePath.act(FilePath.java:970)
          ... 11 more
          Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins
          at hudson.scm.SubversionSCM.descriptor(SubversionSCM.java:2509)
          at hudson.scm.SubversionSCM.createDefaultSVNOptions(SubversionSCM.java:1059)
          at hudson.scm.SubversionSCM.createClientManager(SubversionSCM.java:1049)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:981)
          at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:961)
          at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2665)
          at hudson.remoting.UserRequest.perform(UserRequest.java:121)
          at hudson.remoting.UserRequest.perform(UserRequest.java:49)
          at hudson.remoting.Request$2.run(Request.java:324)
          at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
          at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:745)
          at ......remote call to slave-01(Native Method)
          at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356)
          at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
          at hudson.remoting.Channel.call(Channel.java:752)
          ... 12 more

          petervarga Peter Varga added a comment - Hi, we are facing a similar problem. We are using Jenkins version 1.591, and have two slaves configured. In case the job uses our slave-01, we are getting the NoClassDefFoundError shown below. If I configure the job, to use our slave-02, it works fine. We were not able to find any configuration differences between our slaves. [EnvInject] - Loading node environment variables. Building remotely on slave-01 (wfm-trunk linux tasktrack) in workspace /srv/jenkins-home/workspace/deployment-int-shutdown-instance-abc java.io.IOException: remote file operation failed: /srv/jenkins-home/workspace/deployment-int-shutdown-instance-abc at hudson.remoting.Channel@205b38e:slave-01: java.io.IOException: Remote call on slave-01 failed at hudson.FilePath.act(FilePath.java:977) at hudson.FilePath.act(FilePath.java:959) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:910) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:845) at hudson.model.AbstractProject.checkout(AbstractProject.java:1259) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:622) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:528) at hudson.model.Run.execute(Run.java:1759) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:89) at hudson.model.Executor.run(Executor.java:240) Caused by: java.io.IOException: Remote call on slave-01 failed at hudson.remoting.Channel.call(Channel.java:760) at hudson.FilePath.act(FilePath.java:970) ... 11 more Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins at hudson.scm.SubversionSCM.descriptor(SubversionSCM.java:2509) at hudson.scm.SubversionSCM.createDefaultSVNOptions(SubversionSCM.java:1059) at hudson.scm.SubversionSCM.createClientManager(SubversionSCM.java:1049) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:981) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:961) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2665) at hudson.remoting.UserRequest.perform(UserRequest.java:121) at hudson.remoting.UserRequest.perform(UserRequest.java:49) at hudson.remoting.Request$2.run(Request.java:324) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) at ......remote call to slave-01(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356) at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) at hudson.remoting.Channel.call(Channel.java:752) ... 12 more
          oleg_nenashev Oleg Nenashev added a comment -

          From what I see Jenkins class is not being propagated to slaves. It may happen if you enable Slave2Master security AFAIK.

          Anyway,SubversionSCM:: createDefaultSVNOptions() is misdesigned. It should not access descriptior() on its own. Once it is being called on slave, the method should retrieve the data from master via another remoting call. If the class is serializable, such double propagatiion will work correctly.

          Some bits of the issue have been introduced in https://github.com/jenkinsci/subversion-plugin/commit/208f10ba16f8317f64060262fa82477db18be299 , but I think the real cause is even older.

          oleg_nenashev Oleg Nenashev added a comment - From what I see Jenkins class is not being propagated to slaves. It may happen if you enable Slave2Master security AFAIK. Anyway,SubversionSCM:: createDefaultSVNOptions() is misdesigned. It should not access descriptior() on its own. Once it is being called on slave, the method should retrieve the data from master via another remoting call. If the class is serializable, such double propagatiion will work correctly. Some bits of the issue have been introduced in https://github.com/jenkinsci/subversion-plugin/commit/208f10ba16f8317f64060262fa82477db18be299 , but I think the real cause is even older.
          petervarga Peter Varga added a comment -

          oleg_nenashev thanks for your answer.

          The option "Enable Slave → Master Access Control" is not selected in our Jenkins configuration. When it had been selected it would affect all slaves, if I understand it correctly. As we have this problem only with one of our slaves, I don't think that this setting is relevant in our case.

          petervarga Peter Varga added a comment - oleg_nenashev thanks for your answer. The option "Enable Slave → Master Access Control" is not selected in our Jenkins configuration. When it had been selected it would affect all slaves, if I understand it correctly. As we have this problem only with one of our slaves, I don't think that this setting is relevant in our case.

          People

            Unassigned Unassigned
            smouch Smouch Smouch
            Votes:
            3 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated: