Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30651

File descriptor leak when invoking for-loop

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Duplicate
    • Component/s: pipeline
    • Labels:
      None
    • Similar Issues:

      Description

      Steps to reproduce:

      Create a workflow job with the following:

      stage 'all'
      for (int i = 0; i < 9999; i++) {
        echo "running #${i}"
      }
      

      Expected output: "running #0" to "running #9998"

      Actual output:
      After 7940 the build will freeze, and the System.out will show:

      After upgrading to 1.7 I see the exception earlier on:

      Aug 14, 2015 9:03:37 PM org.jenkinsci.plugins.workflow.graph.FlowNode$1 persist
      WARNING: failed to save actions for FlowNode id=7945
      java.io.IOException: Failed to create a temporary file in /space/tmp/160911/jobs/folder/jobs/wf/builds/4/workflow
      	at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:68)
      	at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:55)
      	at hudson.XmlFile.write(XmlFile.java:175)
      	at org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage.saveActions(SimpleXStreamFlowNodeStorage.java:110)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.saveActions(CpsFlowExecution.java:734)
      	at org.jenkinsci.plugins.workflow.graph.FlowNode.save(FlowNode.java:254)
      	at org.jenkinsci.plugins.workflow.graph.FlowNode$1.persist(FlowNode.java:241)
      	at org.jenkinsci.plugins.workflow.graph.FlowNode$1.add(FlowNode.java:217)
      	at org.jenkinsci.plugins.workflow.graph.FlowNode$1.add(FlowNode.java:208)
      	at java.util.AbstractList.add(AbstractList.java:108)
      	at hudson.model.Actionable.addAction(Actionable.java:129)
      	at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:85)
      	at org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:124)
      	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:132)
      	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:98)
      	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:45)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
      	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:15)
      	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:69)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
      	at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:100)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
      	at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:71)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
      	at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
      	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
      	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:145)
      	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:271)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:71)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:180)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:178)
      	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.io.IOException: Too many open files in system
      	at java.io.UnixFileSystem.createFileExclusively(Native Method)
      	at java.io.File.createTempFile(File.java:2024)
      	at hudson.util.AtomicFileWriter.<init>(AtomicFileWriter.java:66)
      	... 55 more
      

      It happens after the 500 mark

      Attachments:

      fd.txt - After 6000 I ran "lsof -p $PID" to display the number of open file descriptors.

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            Marking as a duplicate of the older issue, though this one seems to add the critical information that a file handle leak (in DefaultStepContext) is responsible.

            Show
            jglick Jesse Glick added a comment - Marking as a duplicate of the older issue, though this one seems to add the critical information that a file handle leak (in DefaultStepContext ) is responsible.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/ScalabilityTest.java
            support/src/main/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamFlowNodeStorage.java
            http://jenkins-ci.org/commit/workflow-plugin/d3dc35a0f511610bd5ca56ea44317c4200c4953a
            Log:
            Reproduced the originally reported exception from JENKINS-30055, perhaps independent of JENKINS-30651.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: aggregator/src/test/java/org/jenkinsci/plugins/workflow/ScalabilityTest.java support/src/main/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamFlowNodeStorage.java http://jenkins-ci.org/commit/workflow-plugin/d3dc35a0f511610bd5ca56ea44317c4200c4953a Log: Reproduced the originally reported exception from JENKINS-30055 , perhaps independent of JENKINS-30651 .
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/ScalabilityTest.java
            support/src/main/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamFlowNodeStorage.java
            http://jenkins-ci.org/commit/workflow-support-plugin/ad18aafe913d801eeb88594d89b176df024495b4
            Log:
            Reproduced the originally reported exception from JENKINS-30055, perhaps independent of JENKINS-30651.

            Originally-Committed-As: d3dc35a0f511610bd5ca56ea44317c4200c4953a

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: aggregator/src/test/java/org/jenkinsci/plugins/workflow/ScalabilityTest.java support/src/main/java/org/jenkinsci/plugins/workflow/support/storage/SimpleXStreamFlowNodeStorage.java http://jenkins-ci.org/commit/workflow-support-plugin/ad18aafe913d801eeb88594d89b176df024495b4 Log: Reproduced the originally reported exception from JENKINS-30055 , perhaps independent of JENKINS-30651 . Originally-Committed-As: d3dc35a0f511610bd5ca56ea44317c4200c4953a

              People

              Assignee:
              jglick Jesse Glick
              Reporter:
              schristou Steven Christou
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: