Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61368

Archiving artifacts takes a long time when the pattern does not match

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Component/s: core
    • Labels:
    • Environment:
      Debian 9 & Debian 10 with various Jenkins versions up to and including v2.204.4.
    • Similar Issues:

      Description

      I have a matrix job that is configured with:

      Archive the artifacts: bsoe/publish/*.bsfw.sha1.txt

      Not all of the matrix combinations are expected to generate artifacts. The build log for one that does not contains:

      14:34:48 Archiving artifacts
      14:38:34 ‘bsoe/publish/*.bsfw.sha1.txt’ doesn’t match anything: ‘bsoe’ exists but not ‘bsoe/publish/*.bsfw.sha1.txt’
      14:38:34 No artifacts found that match the file pattern "bsoe/publish/*.bsfw.sha1.txt". Configuration error?
      

      The error message is expected because the bsoe/publish directory is empty. However, I don't expect it to take ~4 minutes to discover that it needs to report that. If I run strace on the Java thread that appears to be busy during this time I can see it attempting to open (what looks like) every single file in the workspace with O_DIRECTORY and failing because the file is not a directory. If it successfully opens a directory then it calls fstat and getdents a few times before closing it. It looks like the glob pattern matcher has gone wild.

      If I make the job create a dummy file that does match the pattern then the error goes away and archiving is quick.

      (To be clear, the filenames I see in the strace output are not underneath bsoe/publish, they are elsewhere in the workspace.)

      Expected behaviour: Archiving no artifacts should take negligible time.
      Actual behaviour: Archiving no artifacts takes several minutes with a large workspace.
       

        Attachments

          Activity

          mcrowe Mike Crowe created issue -
          mcrowe Mike Crowe made changes -
          Field Original Value New Value
          Component/s core [ 15593 ]
          Component/s core [ 21134 ]
          Key INFRA-2511 JENKINS-61368
          Workflow classic default workflow [ 242421 ] JNJira + In-Review [ 242422 ]
          Project Infrastructure [ 10301 ] Jenkins [ 10172 ]
          mcrowe Mike Crowe made changes -
          Attachment plugins.xml [ 50600 ]
          mcrowe Mike Crowe made changes -
          Description I have a matrix job that is configured with:

          Archive the artifacts: {{bsoe/publish/*.bsfw.sha1.txt}}

          Not all of the matrix combinations are expected to generate artifacts. The build log for one that does not contains:
          {noformat}
          14:34:48 Archiving artifacts
          14:38:34 ‘bsoe/publish/*.bsfw.sha1.txt’ doesn’t match anything: ‘bsoe’ exists but not ‘bsoe/publish/*.bsfw.sha1.txt’
          14:38:34 No artifacts found that match the file pattern "bsoe/publish/*.bsfw.sha1.txt". Configuration error?
          {noformat}
          The error message is expected because the {{bsoe/publish}} directory is empty. However, I don't expect it to take ~4 minutes to discover that it needs to report that. If I run {{strace}} on the Java thread that appears to be busy during this time I can see it attempting to open (what looks like) every single file in the workspace with {{O_DIRECTORY}} and failing because the file is not a directory. If it successfully opens a directory then it calls {{fstat}} and {{getdents}} a few times before closing it. It looks like the glob pattern matcher has gone wild.

          If I make the job create a dummy file that does match the pattern then the error goes away and archiving is quick.

          Expected behaviour: Archiving no artifacts should take negligible time.
          Actual behaviour: Archiving no artifacts takes several minutes with a large workspace.
           
          I have a matrix job that is configured with:

          Archive the artifacts: {{bsoe/publish/*.bsfw.sha1.txt}}

          Not all of the matrix combinations are expected to generate artifacts. The build log for one that does not contains:
          {noformat}
          14:34:48 Archiving artifacts
          14:38:34 ‘bsoe/publish/*.bsfw.sha1.txt’ doesn’t match anything: ‘bsoe’ exists but not ‘bsoe/publish/*.bsfw.sha1.txt’
          14:38:34 No artifacts found that match the file pattern "bsoe/publish/*.bsfw.sha1.txt". Configuration error?
          {noformat}
          The error message is expected because the {{bsoe/publish}} directory is empty. However, I don't expect it to take ~4 minutes to discover that it needs to report that. If I run {{strace}} on the Java thread that appears to be busy during this time I can see it attempting to open (what looks like) every single file in the workspace with {{O_DIRECTORY}} and failing because the file is not a directory. If it successfully opens a directory then it calls {{fstat}} and {{getdents}} a few times before closing it. It looks like the glob pattern matcher has gone wild.

          If I make the job create a dummy file that does match the pattern then the error goes away and archiving is quick.

          (To be clear, the filenames I see in the {{strace}} output are not underneath {{bsoe/publish}}, they are elsewhere in the workspace.)

          Expected behaviour: Archiving no artifacts should take negligible time.
          Actual behaviour: Archiving no artifacts takes several minutes with a large workspace.
           

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            mcrowe Mike Crowe
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: