Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-5899

Hudson leaks file handles when archiving errors out

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • core
    • None
    • centos 5.4 64bit

    Description

      We have a job on a slave where the slave brings up a kvm/qemu vm, then executes the job on the vm via ssh. After the job is done, the vm is brought down, and then the image file is mounted so that archiving can occur.

      The problem is that we occassionally see errors in archiving (see below for stack trace). When this happens the slave.jar process leaks a file handle. This then leads to subsequent failures because new jobs on the slave need to unmount the image, start up a new vm with it, run a new job, then remount the image. This fails because we cannot unmount the image when there is an file open. This file is one of those that we were trying to archive, which presumably stayed open.

      I don't know what caused the original failure, but we really need protection from this file handle leakage. I'm assuming that the fix is to insert try-finally blocks somewhere to close the streams, but where?

      -P

      ERROR: Failed to archive artifacts: vm/ws/build/qrslogs/*,vm/ws/build/results/,vm/ws/build/logs/,vm/ws/build/ita/installs//logs/jtr.log,vm/ws/build/dbmake-.out,vm/ws/qres-server.out
      hudson.util.IOException2: java.io.IOException: Input/output error
      at hudson.FilePath.copyRecursiveTo(FilePath.java:1447)
      at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:117)
      at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:582)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildStep(AbstractBuild.java:563)
      at hudson.model.AbstractBuild$AbstractRunner.performAllBuildStep(AbstractBuild.java:550)
      at hudson.model.Build$RunnerImpl.post2(Build.java:152)
      at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:528)
      at hudson.model.Run.run(Run.java:1233)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:122)
      Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Input/output error
      at hudson.remoting.Channel$1.adapt(Channel.java:584)
      at hudson.remoting.Channel$1.adapt(Channel.java:579)
      at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
      at hudson.FilePath.copyRecursiveTo(FilePath.java:1445)
      ... 11 more
      Caused by: java.io.IOException: Input/output error
      at java.io.FileInputStream.readBytes(Native Method)
      at java.io.FileInputStream.read(FileInputStream.java:177)
      at hudson.FilePath$TarWriter.visit(FilePath.java:1523)
      at hudson.util.DirScanner$Glob.scan(DirScanner.java:91)
      at hudson.FilePath.writeToTar(FilePath.java:1606)
      at hudson.FilePath.access$900(FilePath.java:160)
      at hudson.FilePath$33.invoke(FilePath.java:1424)
      at hudson.FilePath$33.invoke(FilePath.java:1421)
      at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1962)
      at hudson.remoting.UserRequest.perform(UserRequest.java:114)
      at hudson.remoting.UserRequest.perform(UserRequest.java:48)
      at hudson.remoting.Request$2.run(Request.java:270)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)

      Attachments

        Activity

          abayer Andrew Bayer added a comment -

          This is actually in core, not the copyartifact plugin.

          abayer Andrew Bayer added a comment - This is actually in core, not the copyartifact plugin.

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/core/src/main/java/hudson/FilePath.java
          trunk/www/changelog.html
          http://jenkins-ci.org/commit/28500
          Log:
          [FIXED JENKINS-5899] in 1.351.

          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/FilePath.java trunk/www/changelog.html http://jenkins-ci.org/commit/28500 Log: [FIXED JENKINS-5899] in 1.351.

          People

            Unassigned Unassigned
            pgweiss pgweiss
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: