Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22607

LogRotator inefficient when number of builds to kept is very large

      If you have configured LogRotator to delete all builds older than 30 days, yet you do around 1000 builds a day, then at the end of each build LogRotator.perform will scan backwards from the previous build all the way to the first build (or nearby), loading each build along the way, just to find the (on average) one build which is now older than 30 days so it can be deleted. This forces thousands of builds to be kept in memory, or if they cannot all be held in memory, forces each build to reload thousands of older build records that it does not really need.

      Would be better to do a binary search through the build history to find the newest build which is suitable for deletion.

          [JENKINS-22607] LogRotator inefficient when number of builds to kept is very large

          Jesse Glick added a comment -

          Fixing this when numToKeep is set looks impractical, since AbstractLazyLoadRunMap.idOnDisk is not exposed (not even its size), and so there is no way of knowing which build is the 30000th oldest—you cannot rely on this having a particular build number. Fixing it for daysToKeep looks easier since you can simply browse forward from the oldest build and keep on deleting them so long as they are sufficiently old.

          Jesse Glick added a comment - Fixing this when numToKeep is set looks impractical, since AbstractLazyLoadRunMap.idOnDisk is not exposed (not even its size), and so there is no way of knowing which build is the 30000th oldest—you cannot rely on this having a particular build number. Fixing it for daysToKeep looks easier since you can simply browse forward from the oldest build and keep on deleting them so long as they are sufficiently old.

          Jesse Glick added a comment -

          Not sure if the fix should be lts-candidate, given that it does not work for numToKeep.

          Jesse Glick added a comment - Not sure if the fix should be lts-candidate , given that it does not work for numToKeep .

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/tasks/LogRotator.java
          http://jenkins-ci.org/commit/jenkins/d9188ce2ad2070eab5ea27cb7f96257b351392a7
          Log:
          [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/tasks/LogRotator.java http://jenkins-ci.org/commit/jenkins/d9188ce2ad2070eab5ea27cb7f96257b351392a7 Log: [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3307
          [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested. (Revision d9188ce2ad2070eab5ea27cb7f96257b351392a7)

          Result = SUCCESS
          Jesse Glick : d9188ce2ad2070eab5ea27cb7f96257b351392a7
          Files :

          • core/src/main/java/hudson/tasks/LogRotator.java
          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #3307 [FIXED JENKINS-22607] Avoid loading recent builds when deletion of builds older than some date has been requested. (Revision d9188ce2ad2070eab5ea27cb7f96257b351392a7) Result = SUCCESS Jesse Glick : d9188ce2ad2070eab5ea27cb7f96257b351392a7 Files : core/src/main/java/hudson/tasks/LogRotator.java changelog.html

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: