Although FilePath.validateAntFileMask is typically bounded to check at most 10000 files before giving up and announcing that it cannot find a match, it turns out that DirectoryScanner collects the names of non-matches in filesNotIncluded and dirsNotIncluded in the meantime. In a big workspace, this can cause the JVM (usually a slave agent) to allocate gigabytes of heap, eventually causing the slave to become overwhelmed and crash.

          [JENKINS-25759] FilePath.validateAntFileMask sucks up heap

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3833
          [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 534328b264f9338e48418d2bcc0d28daaf48b3a0)

          Result = SUCCESS
          jesse glick : 534328b264f9338e48418d2bcc0d28daaf48b3a0
          Files :

          • core/src/main/java/hudson/util/FormFieldValidator.java
          • core/src/main/java/hudson/FilePath.java
          • core/src/main/java/hudson/tasks/ArtifactArchiver.java
          • core/src/test/java/hudson/FilePathTest.java
          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #3833 [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 534328b264f9338e48418d2bcc0d28daaf48b3a0) Result = SUCCESS jesse glick : 534328b264f9338e48418d2bcc0d28daaf48b3a0 Files : core/src/main/java/hudson/util/FormFieldValidator.java core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/test/java/hudson/FilePathTest.java changelog.html

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/FilePath.java
          core/src/main/java/hudson/tasks/ArtifactArchiver.java
          core/src/main/java/hudson/util/FormFieldValidator.java
          core/src/test/java/hudson/FilePathTest.java
          http://jenkins-ci.org/commit/jenkins/534328b264f9338e48418d2bcc0d28daaf48b3a0
          Log:
          [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask.
          Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded.
          Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/java/hudson/util/FormFieldValidator.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/534328b264f9338e48418d2bcc0d28daaf48b3a0 Log: [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded. Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/FilePath.java
          core/src/main/java/hudson/tasks/ArtifactArchiver.java
          core/src/main/java/hudson/util/FormFieldValidator.java
          core/src/test/java/hudson/FilePathTest.java
          http://jenkins-ci.org/commit/jenkins/23e0d6ad1964e0efdd1e0f1c90360321acc8a073
          Log:
          [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask.
          Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded.
          Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound.
          (cherry picked from commit 534328b264f9338e48418d2bcc0d28daaf48b3a0)

          Conflicts:
          changelog.html

          Compare: https://github.com/jenkinsci/jenkins/compare/03600e925c3d...23e0d6ad1964

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/main/java/hudson/util/FormFieldValidator.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/23e0d6ad1964e0efdd1e0f1c90360321acc8a073 Log: [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. Not fully solved, since the scannedDirs field can still grow to be large, but at least clearing files/dirsNotIncluded. Also imposing a 5s timeout on the scan regardless of file count, and defining a user-customizable bound. (cherry picked from commit 534328b264f9338e48418d2bcc0d28daaf48b3a0) Conflicts: changelog.html Compare: https://github.com/jenkinsci/jenkins/compare/03600e925c3d...23e0d6ad1964

          dogfood added a comment -

          Integrated in jenkins_main_trunk #4292
          [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 23e0d6ad1964e0efdd1e0f1c90360321acc8a073)

          Result = UNSTABLE
          ogondza : 23e0d6ad1964e0efdd1e0f1c90360321acc8a073
          Files :

          • core/src/main/java/hudson/util/FormFieldValidator.java
          • core/src/main/java/hudson/tasks/ArtifactArchiver.java
          • core/src/test/java/hudson/FilePathTest.java
          • core/src/main/java/hudson/FilePath.java

          dogfood added a comment - Integrated in jenkins_main_trunk #4292 [FIXED JENKINS-25759] Avoid consuming too much memory while running validateAntFileMask. (Revision 23e0d6ad1964e0efdd1e0f1c90360321acc8a073) Result = UNSTABLE ogondza : 23e0d6ad1964e0efdd1e0f1c90360321acc8a073 Files : core/src/main/java/hudson/util/FormFieldValidator.java core/src/main/java/hudson/tasks/ArtifactArchiver.java core/src/test/java/hudson/FilePathTest.java core/src/main/java/hudson/FilePath.java

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: