Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44958

File handle leaks in *.log after hard kill

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Minor
    • Resolution: Won't Fix
    • workflow-cps-plugin
    • None
    • Jenkins ver. 2.46.3

    Description

      I use a Jenkins Pipeline to run my individual tests in parallel. I just noticed my test job that I run in parallel is failing with "Too many open files" errors.

      Next, I ssh into the master and did an lsof -p using the pid of Jenkins... i see over 4k file descriptors. The output has thousands of lines that look like this:

      java 7 root 4089w REG 202,96 28 2375888 /var/jenkins_home/jobs/test_pipeline/builds/15331/222.log
java 7 root 4090w REG 202,96 328 787837 /var/jenkins_home/jobs/test_pipeline/builds/15343/166.log
java 7 root 4091w REG 202,96 328 787851 /var/jenkins_home/jobs/test_pipeline/builds/15343/180.log
java 7 root 4094w REG 202,96 671 787839 /var/jenkins_home/jobs/test_pipeline/builds/15343/167.log
java 7 root 4095w REG 202,96 29 2361097 /var/jenkins_home/jobs/test_pipeline/builds/15331/157.log

      Each file has content that roughly looks like this:

      Starting building: ESC[8mha:////4CQAnoPVmHzYGq9xQUbp+LIf+xF38/zSfY1nZ0iZCPexAAAArB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3Rdl
LBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyZEgZF/az8JP2U/Pyi4pLUAt2ULKBh+fElqcUl+sYWliYGhvoAbOU0btYAAAA=ESC[0mmy_test #389401

       

      Attachments

        Issue Links

          relates to

          Task - A task that needs to be done. JENKINS-38381 [JEP-210] Optimize log handling in Pipeline and Durable Task

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            Ascending order - Click to sort in descending order
            piratejohnny Jon B created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Component/s pipeline-build-step-plugin [ 21707 ]
            Component/s workflow-support-plugin [ 21719 ]
            Component/s pipeline [ 21692 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-38381 [ JENKINS-38381 ]
            piratejohnny Jon B made changes -
            Attachment image-2017-06-27-12-41-39-630.png [ 38670 ]
            jglick Jesse Glick made changes -
            Component/s workflow-cps-plugin [ 21713 ]
            Component/s pipeline-build-step-plugin [ 21707 ]
            Component/s workflow-support-plugin [ 21719 ]
            Summary Pipeline plugin leaking file descriptors, causing jobs to fail File handle leaks in *.log after hard kill
            piratejohnny Jon B made changes -
            Attachment image-2017-09-09-09-48-57-939.png [ 39645 ]
            piratejohnny Jon B made changes -
            Attachment image-2017-09-09-09-49-15-788.png [ 39646 ]
            jglick Jesse Glick made changes -
            Resolution Won't Fix [ 2 ]
            Status Open [ 1 ] Resolved [ 5 ]

            People

              Unassigned Unassigned
              piratejohnny Jon B
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: