Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-31981

Memory exhausted on slave when archive artifacts modified in GUI

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We see that the memory consumption on Jenkins slaves sometimes goes to maximum and the builds hang at archive the artifacts. The artifacts that we want to save do not contain any **. We also see in the heap dump that the string values in the hudsonFilePath is correct. But in the directoryScanner.scan, it is prepended with **. Since we have softlinks in our workspace, where we point to an underlying directory - forming a loop inthe directory structure -, the directoryScanner seems to create an infinite amount of objects, when trying to find the artifact.

      The above scenario happens when changing things in GUI for archive the Artifacts. And the code that does this I assume is from FilePath.java:
      FileSet fs = Util.createFileSet(reading(dir),"**/"+fileMask);

      Suggestion1: Detect loops in WS and if so handle it.
      Suggestion 2: Remove the feature to find where in the workspace a file is when entering something in the GUI
      Suggestion 3: Make this feature optional, with good explanation.

      Using memory analyzer we see:
      java.lang.Thread @ 0x7f9285dd3b78 pool-1-thread-7 for channel
      Shallow Size: 168 B Retained Size: 25.1 GB
      Thread Stack

      pool-1-thread-7 for channel
      at java.io.UnixFileSystem.list(Ljava/io/File[Ljava/lang/String; (Native Method)
      at java.io.File.list()[Ljava/lang/String; (File.java:973)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1257)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z[Ljava/lang/String;Ljava/util/LinkedList;)V (DirectoryScanner.java:1287)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Lorg/apache/tools/ant/types/selectors/TokenizedPath;Z)V (DirectoryScanner.java:1211)
      at org.apache.tools.ant.DirectoryScanner.scandir(Ljava/io/File;Ljava/lang/String;Z)V (DirectoryScanner.java:1173)
      at org.apache.tools.ant.DirectoryScanner.checkIncludePatterns()V (DirectoryScanner.java:955)
      at org.apache.tools.ant.DirectoryScanner.scan()V (DirectoryScanner.java:909)
      at org.apache.tools.ant.types.AbstractFileSet.getDirectoryScanner(Lorg/apache/tools/ant/Project;)Lorg/apache/tools/ant/DirectoryScanner; (AbstractFileSet.java:499)
      at hudson.FilePath$46.invoke(Ljava/io/File;Lhudson/remoting/VirtualChannel;)Ljava/lang/String; (FilePath.java:2406)
      at hudson.FilePath$46.invoke(Ljava/io/File;Lhudson/remoting/VirtualChannel;)Ljava/lang/Object; (FilePath.java:2364)
      at hudson.FilePath$FileCallableWrapper.call()Ljava/lang/Object; (FilePath.java:2688)
      at hudson.remoting.UserRequest.perform(Lhudson/remoting/Channel;)Lhudson/remoting/UserResponse; (UserRequest.java:121)
      at hudson.remoting.UserRequest.perform(Lhudson/remoting/Channel;)Ljava/io/Serializable; (UserRequest.java:49)
      at hudson.remoting.Request$2.run()V (Request.java:325)
      at hudson.remoting.InterceptingExecutorService$1.call()Ljava/lang/Object; (InterceptingExecutorService.java:68)
      at java.util.concurrent.FutureTask$Sync.innerRun()V (FutureTask.java:303)
      at java.util.concurrent.FutureTask.run()V (FutureTask.java:138)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Ljava/lang/Runnable;)V (ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:908)
      at java.lang.Thread.run()V (Thread.java:619)

        Attachments

          Issue Links

            Activity

            Hide
            ikedam ikedam added a comment -

            "archive artifacts" is provided by core, not by copyartifact.
            Removed copyartifactfrom component/s and label/s.

            Show
            ikedam ikedam added a comment - "archive artifacts" is provided by core, not by copyartifact. Removed copyartifactfrom component/s and label/s.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              qsimhen Simon Hensing
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated: