• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • jenkins 1.5.18; debian squeeze; jenkins package

      When fingerprinting large projects, jenkins fails with the following error message:

      Recording fingerprints
      ERROR: Failed to record fingerprints
      hudson.util.IOException2: Failed to compute digest for /var/lib/jenkins/workspace/zetbox-develop-Linux-Build-3-0/packages/NHibernate-dasz.3.3.3-DASZ2/src/Cfg/MappingSchema/HbmProperty.cs
      	at hudson.tasks.Fingerprinter$2.invoke(Fingerprinter.java:243)
      	at hudson.tasks.Fingerprinter$2.invoke(Fingerprinter.java:225)
      	at hudson.FilePath.act(FilePath.java:905)
      	at hudson.FilePath.act(FilePath.java:878)
      	at hudson.tasks.Fingerprinter.record(Fingerprinter.java:225)
      	at hudson.tasks.Fingerprinter.perform(Fingerprinter.java:121)
      	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:776)
      	at hudson.model.Build$BuildExecution.post2(Build.java:183)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:726)
      	at hudson.model.Run.execute(Run.java:1601)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:241)
      Caused by: java.io.FileNotFoundException: /var/lib/jenkins/workspace/zetbox-develop-Linux-Build-3-0/packages/NHibernate-dasz.3.3.3-DASZ2/src/Cfg/MappingSchema/HbmProperty.cs (Too many open files)
      	at java.io.FileInputStream.open(Native Method)
      	at java.io.FileInputStream.<init>(FileInputStream.java:137)
      	at hudson.FilePath$30.invoke(FilePath.java:1667)
      	at hudson.FilePath$30.invoke(FilePath.java:1664)
      	at hudson.FilePath.act(FilePath.java:905)
      	at hudson.FilePath.act(FilePath.java:878)
      	at hudson.FilePath.digest(FilePath.java:1664)
      	at hudson.tasks.Fingerprinter$2.invoke(Fingerprinter.java:241)
      	... 14 more
      Build step 'Record fingerprints of files to track usage' changed build result to FAILURE
      Finished: FAILURE
      

          [JENKINS-18351] Fingerprinting seems to leak Files

          kutzi added a comment -

          Yeah, looks like you're right. FilePath#digest opens a new FileInputstream but never closes it explicitly
          https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/FilePath.java#L1663

          kutzi added a comment - Yeah, looks like you're right. FilePath#digest opens a new FileInputstream but never closes it explicitly https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/FilePath.java#L1663

          Code changed in jenkins
          User: Christoph Kutzinski
          Path:
          changelog.html
          core/src/main/java/hudson/FilePath.java
          http://jenkins-ci.org/commit/jenkins/93c31be1b75940dfd2e8a3c5954a4d57f4c8e6d5
          Log:
          [FIXED JENKINS-18351] file descriptor leak while fingerprinting

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christoph Kutzinski Path: changelog.html core/src/main/java/hudson/FilePath.java http://jenkins-ci.org/commit/jenkins/93c31be1b75940dfd2e8a3c5954a4d57f4c8e6d5 Log: [FIXED JENKINS-18351] file descriptor leak while fingerprinting

          Jesse Glick added a comment -

          Presumably caused by https://github.com/jenkinsci/jenkins/commit/9d72311ba0222b9d0489605b8ec2cec711690993 though I am not sure how; Util.getDigestOf seems to close its parameter, and BufferedInputStream seems to close its delegate when it itself is closed.

          Jesse Glick added a comment - Presumably caused by https://github.com/jenkinsci/jenkins/commit/9d72311ba0222b9d0489605b8ec2cec711690993 though I am not sure how; Util.getDigestOf seems to close its parameter, and BufferedInputStream seems to close its delegate when it itself is closed.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2610
          [FIXED JENKINS-18351] file descriptor leak while fingerprinting (Revision 93c31be1b75940dfd2e8a3c5954a4d57f4c8e6d5)

          Result = SUCCESS
          Christoph Kutzinski : 93c31be1b75940dfd2e8a3c5954a4d57f4c8e6d5
          Files :

          • changelog.html
          • core/src/main/java/hudson/FilePath.java

          dogfood added a comment - Integrated in jenkins_main_trunk #2610 [FIXED JENKINS-18351] file descriptor leak while fingerprinting (Revision 93c31be1b75940dfd2e8a3c5954a4d57f4c8e6d5) Result = SUCCESS Christoph Kutzinski : 93c31be1b75940dfd2e8a3c5954a4d57f4c8e6d5 Files : changelog.html core/src/main/java/hudson/FilePath.java

          kutzi added a comment -

          Jesse, where did you see that Util.getDigestOf closes the passed InputStream? I cannot find such a place.
          So this issue has probably already been there before https://github.com/jenkinsci/jenkins/commit/9d72311ba0222b9d0489605b8ec2cec711690993

          It probably just manifests when fingerprinting a lot of files, since in that case the file descriptor run out, before the GC can discard and close the dangling FileOutputStreams

          kutzi added a comment - Jesse, where did you see that Util.getDigestOf closes the passed InputStream? I cannot find such a place. So this issue has probably already been there before https://github.com/jenkinsci/jenkins/commit/9d72311ba0222b9d0489605b8ec2cec711690993 It probably just manifests when fingerprinting a lot of files, since in that case the file descriptor run out, before the GC can discard and close the dangling FileOutputStreams

          Jesse Glick added a comment -

          My mistake, regressed from another commit: https://github.com/jenkinsci/jenkins/commit/51fbd2d

          Jesse Glick added a comment - My mistake, regressed from another commit: https://github.com/jenkinsci/jenkins/commit/51fbd2d

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/FilePath.java
          core/src/main/java/hudson/Util.java
          http://jenkins-ci.org/commit/jenkins/7f966f6f312f18bfc8a11166b1b2ebf474965ddb
          Log:
          [FIXED JENKINS-18351] Better fix for file handle leak.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/Util.java http://jenkins-ci.org/commit/jenkins/7f966f6f312f18bfc8a11166b1b2ebf474965ddb Log: [FIXED JENKINS-18351] Better fix for file handle leak.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2612
          [FIXED JENKINS-18351] Better fix for file handle leak. (Revision 7f966f6f312f18bfc8a11166b1b2ebf474965ddb)

          Result = SUCCESS
          Jesse Glick : 7f966f6f312f18bfc8a11166b1b2ebf474965ddb
          Files :

          • core/src/main/java/hudson/Util.java
          • core/src/main/java/hudson/FilePath.java

          dogfood added a comment - Integrated in jenkins_main_trunk #2612 [FIXED JENKINS-18351] Better fix for file handle leak. (Revision 7f966f6f312f18bfc8a11166b1b2ebf474965ddb) Result = SUCCESS Jesse Glick : 7f966f6f312f18bfc8a11166b1b2ebf474965ddb Files : core/src/main/java/hudson/Util.java core/src/main/java/hudson/FilePath.java

          Brian Brooks added a comment -

          I think this issue may be the root cause of problem I'm having with Jenkins 1.517. When using the native-maven-plugin in a Jenkins job, hudson.maven.reporters.MavenFingerprinter is leaving a file locked which System.gc() cleans up but in the interim causes "The process cannot access the file because it is being used by another process". Just putting a comment here in case someone else is affected by the issue. I'm going to try and upgrade to at least Jenkins 1.523 and see if the issue is resolved.

          1.0-alpha-7 manifest goal mt.exe : general error c101008d: Failed to write the updated manifest to the resource of file The process cannot access the file because it is being used by another process
          http://jira.codehaus.org/browse/MOJO-1941.

          Brian Brooks added a comment - I think this issue may be the root cause of problem I'm having with Jenkins 1.517. When using the native-maven-plugin in a Jenkins job, hudson.maven.reporters.MavenFingerprinter is leaving a file locked which System.gc() cleans up but in the interim causes "The process cannot access the file because it is being used by another process". Just putting a comment here in case someone else is affected by the issue. I'm going to try and upgrade to at least Jenkins 1.523 and see if the issue is resolved. 1.0-alpha-7 manifest goal mt.exe : general error c101008d: Failed to write the updated manifest to the resource of file The process cannot access the file because it is being used by another process http://jira.codehaus.org/browse/MOJO-1941 .

          Jesse Glick added a comment -

          @brian3791 the purported fix is in 1.520.

          Jesse Glick added a comment - @brian3791 the purported fix is in 1.520.

          Jim McCaskey added a comment -

          @brian3791 would probably need to confirm, but I'm still have the MOJO-1941 problem on 1.524.

          Jim McCaskey added a comment - @brian3791 would probably need to confirm, but I'm still have the MOJO-1941 problem on 1.524.

          Brian Brooks added a comment -

          @Jim that's odd. After upgrading to 1.523 my MOJO-1941 issue was resolved. No more maven artifact file locks caused by hudson.maven.reporters.MavenFingerprinter.

          Brian Brooks added a comment - @Jim that's odd. After upgrading to 1.523 my MOJO-1941 issue was resolved. No more maven artifact file locks caused by hudson.maven.reporters.MavenFingerprinter.

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/src/main/java/hudson/model/DirectoryBrowserSupport.java
          http://jenkins-ci.org/commit/jenkins/2ea0276e7b4ea9614961a530382320c49ab626a2
          Log:
          [CID-1205053] - Resource leak on fingerprint calculation in DirectoryBrowserSupport

          The issue is related to JENKINS-18351.

          Signed-off-by: Oleg Nenashev <o.v.nenashev@gmail.com>

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/model/DirectoryBrowserSupport.java http://jenkins-ci.org/commit/jenkins/2ea0276e7b4ea9614961a530382320c49ab626a2 Log: [CID-1205053] - Resource leak on fingerprint calculation in DirectoryBrowserSupport The issue is related to JENKINS-18351 . Signed-off-by: Oleg Nenashev <o.v.nenashev@gmail.com>

          Jesse Glick added a comment -

          https://github.com/jenkinsci/jenkins/pull/1434 may fix another issue here, not sure.

          Jesse Glick added a comment - https://github.com/jenkinsci/jenkins/pull/1434 may fix another issue here, not sure.

          We're seeing an issue with the exact same output. Unfortunately, we are still on Jenkins 1.498 and upgrading right now isn't an option. But looking at the link of where this regressed, it looks like the earliest tag I see is 1.517 - am I reading correctly that this bug wasn't introduced until then so we should be looking elsewhere as to the source of our issue?

          If it is this issue, is there anything we can do on the system running Jenkins to try to mitigate it?

          Kelsey Prantis added a comment - We're seeing an issue with the exact same output. Unfortunately, we are still on Jenkins 1.498 and upgrading right now isn't an option. But looking at the link of where this regressed, it looks like the earliest tag I see is 1.517 - am I reading correctly that this bug wasn't introduced until then so we should be looking elsewhere as to the source of our issue? If it is this issue, is there anything we can do on the system running Jenkins to try to mitigate it?

            jglick Jesse Glick
            davidschmitt David Schmitt
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: