Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44958

File handle leaks in *.log after hard kill

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Minor
    • Resolution: Won't Fix
    • workflow-cps-plugin
    • None
    • Jenkins ver. 2.46.3

    Description

      I use a Jenkins Pipeline to run my individual tests in parallel. I just noticed my test job that I run in parallel is failing with "Too many open files" errors.

      Next, I ssh into the master and did an lsof -p using the pid of Jenkins... i see over 4k file descriptors. The output has thousands of lines that look like this:

      java 7 root 4089w REG 202,96 28 2375888 /var/jenkins_home/jobs/test_pipeline/builds/15331/222.log
java 7 root 4090w REG 202,96 328 787837 /var/jenkins_home/jobs/test_pipeline/builds/15343/166.log
java 7 root 4091w REG 202,96 328 787851 /var/jenkins_home/jobs/test_pipeline/builds/15343/180.log
java 7 root 4094w REG 202,96 671 787839 /var/jenkins_home/jobs/test_pipeline/builds/15343/167.log
java 7 root 4095w REG 202,96 29 2361097 /var/jenkins_home/jobs/test_pipeline/builds/15331/157.log

      Each file has content that roughly looks like this:

      Starting building: ESC[8mha:////4CQAnoPVmHzYGq9xQUbp+LIf+xF38/zSfY1nZ0iZCPexAAAArB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3Rdl
LBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyZEgZF/az8JP2U/Pyi4pLUAt2ULKBh+fElqcUl+sYWliYGhvoAbOU0btYAAAA=ESC[0mmy_test #389401

       

      Attachments

        Issue Links

          relates to

          Task - A task that needs to be done. JENKINS-38381 [JEP-210] Optimize log handling in Pipeline and Durable Task

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            Descending order - Click to sort in ascending order
            jglick Jesse Glick added a comment -

            Well, will not fix in original code base. That code is now gone.

            jglick Jesse Glick added a comment - Well, will not fix in original code base. That code is now gone.
            svanoort Sam Van Oort added a comment -

            totoroliu Since Jesse forgot to communicate the key info: there's a huge change to logging that's just gone into beta release (available via the Experimental Update Center – see https://jenkins.io/doc/developer/publishing/releasing-experimental-updates/ ) that completely removes these extra log files.

            I'd argue that qualifies as Resolution: Fixed rather than "Won't Fix" but ::shrug::

            svanoort Sam Van Oort added a comment - totoroliu Since Jesse forgot to communicate the key info: there's a huge change to logging that's just gone into beta release (available via the Experimental Update Center – see https://jenkins.io/doc/developer/publishing/releasing-experimental-updates/ ) that completely removes these extra log files. I'd argue that qualifies as Resolution: Fixed rather than "Won't Fix" but ::shrug::
            jglick Jesse Glick added a comment -

            No, https://jenkins.io/jep/210 (yes I complained when this system was introduced by rtyler that there was going to be confusion with Java Enhancement Proposals).

            jglick Jesse Glick added a comment - No, https://jenkins.io/jep/210 (yes I complained when this system was introduced by rtyler that there was going to be confusion with Java Enhancement Proposals).
            totoroliu Rick Liu added a comment -

            What's JEP-210?
            Is it this JEP-210: http://openjdk.java.net/jeps/210

            totoroliu Rick Liu added a comment - What's JEP-210? Is it this JEP-210: http://openjdk.java.net/jeps/210
            jglick Jesse Glick added a comment -

            Should not be an issue after JEP-210.

            jglick Jesse Glick added a comment - Should not be an issue after JEP-210.
            View 16 older comments

            People

              Unassigned Unassigned
              piratejohnny Jon B
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: