Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38005

Using archiveArtifacts with a non-matching pattern silently fails the build

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • Jenkins 2.7.3, Pipeline 2.3

      If you happen to specify a pattern that doesn't match any files in the workspace, archiveArtifacts will silently fail the build.

      Jenkinsfile

      node {
          sh 'echo "lol"'
          
          archiveArtifacts artifacts: 'lol*.zip'
      }
      

      Console Output

      Started by user admin
      [Pipeline] node
      Still waiting to schedule task
      Waiting for next available executor
      Running on Azure0907122157 in /home/azureuser/workspace/Experiments/bad-artifacts
      [Pipeline] {
      [Pipeline] sh
      [bad-artifacts] Running shell script
      + echo lol
      lol
      [Pipeline] step
      Archiving artifacts
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      Finished: FAILURE
      

      archive step

      In contrast, the use of the archive step (e.g. archive 'lol*.zip' in the above Jenkinsfile) will result in a silently successful build.

          [JENKINS-38005] Using archiveArtifacts with a non-matching pattern silently fails the build

          Max Wahler added a comment -

          I raised the priority to 'Critical' because this makes very difficult to see where a large build actually failed.

          Also, I saw a similar behavior:

          archiveArtifacts 'target/test.data'

          fails only if the directory 'target' does not exist. If it exists but not the file 'test.data', then the build fails but the build step was successful.

          Max Wahler added a comment - I raised the priority to 'Critical' because this makes very difficult to see where a large build actually failed. Also, I saw a similar behavior: archiveArtifacts 'target/test.data' fails only if the directory 'target' does not exist. If it exists but not the file 'test.data', then the build fails but the build step was successful.

          Max Wahler added a comment -

          Is there any progress on this issue?

          It would be really really great if there was a way to tell if archiveArtifacts worked or not! I have a job that builds a lot of stuff and has ~750 steps. It's totally impossible to find out where the pipeline failed in case archiveArtifacts failed, because every step is green and there's no output at all that points me to the correct place.

          My approach is now more a Poor Man's Debugger than a solution: I put masses of printlns in it which give me the current build result. But this doesn't necessarily mean that the error happened at that specific place if you build stuff in parallel.

          IMHO, archiveArtifacts should at least print out the amount of files it included. Also a return value containing the files or the number of files would be great. But instead of failing silently, I'd prefer an Exception.

          Max Wahler added a comment - Is there any progress on this issue? It would be really really great if there was a way to tell if archiveArtifacts worked or not! I have a job that builds a lot of stuff and has ~750 steps. It's totally impossible to find out where the pipeline failed in case archiveArtifacts failed, because every step is green and there's no output at all that points me to the correct place. My approach is now more a Poor Man's Debugger than a solution: I put masses of printlns in it which give me the current build result. But this doesn't necessarily mean that the error happened at that specific place if you build stuff in parallel. IMHO, archiveArtifacts should at least print out the amount of files it included. Also a return value containing the files or the number of files would be great. But instead of failing silently, I'd prefer an Exception.

          Jesse Glick added a comment -

          Just use allowEmptyArchive: true. See inline help in Pipeline Syntax.

          As to the lack of a message, I suspect the following would suffice:

          diff --git a/core/src/main/java/hudson/tasks/ArtifactArchiver.java b/core/src/main/java/hudson/tasks/ArtifactArchiver.java
          index ca1f154..9175e11 100644
          --- a/core/src/main/java/hudson/tasks/ArtifactArchiver.java
          +++ b/core/src/main/java/hudson/tasks/ArtifactArchiver.java
          @@ -242,7 +242,7 @@ public class ArtifactArchiver extends Recorder implements SimpleBuildStep {
                           }
                       } else {
                           Result result = build.getResult();
          -                if (result != null && result.isBetterOrEqualTo(Result.UNSTABLE)) {
          +                if (result == null || result.isBetterOrEqualTo(Result.UNSTABLE)) {
                               // If the build failed, don't complain that there was no matching artifact.
                               // The build probably didn't even get to the point where it produces artifacts. 
                               listenerWarnOrError(listener, Messages.ArtifactArchiver_NoMatchFound(artifacts));
          

          Jesse Glick added a comment - Just use allowEmptyArchive: true . See inline help in Pipeline Syntax . As to the lack of a message, I suspect the following would suffice: diff --git a/core/src/main/java/hudson/tasks/ArtifactArchiver.java b/core/src/main/java/hudson/tasks/ArtifactArchiver.java index ca1f154..9175e11 100644 --- a/core/src/main/java/hudson/tasks/ArtifactArchiver.java +++ b/core/src/main/java/hudson/tasks/ArtifactArchiver.java @@ -242,7 +242,7 @@ public class ArtifactArchiver extends Recorder implements SimpleBuildStep { } } else { Result result = build.getResult(); - if (result != null && result.isBetterOrEqualTo(Result.UNSTABLE)) { + if (result == null || result.isBetterOrEqualTo(Result.UNSTABLE)) { // If the build failed, don't complain that there was no matching artifact. // The build probably didn't even get to the point where it produces artifacts. listenerWarnOrError(listener, Messages.ArtifactArchiver_NoMatchFound(artifacts));

          Jesse Glick added a comment -

          From a script, it would probably also work to run

          currentBuild.result = 'STABLE'
          

          prior to running the archiver. (The result can only get worse, never better, but it can become nonnull.)

          Jesse Glick added a comment - From a script, it would probably also work to run currentBuild.result = 'STABLE' prior to running the archiver. (The result can only get worse, never better, but it can become nonnull.)

          Jesse, you're missing the point. Yes, once someone recognizes the problem, it can be worked around. Until that point, they are left with a failed build and no useful information as to its cause. Hence the request for additional output from archiveArtifacts.

          Mike Kingsbury added a comment - Jesse, you're missing the point. Yes, once someone recognizes the problem, it can be worked around. Until that point, they are left with a failed build and no useful information as to its cause. Hence the request for additional output from archiveArtifacts.

          Jesse Glick added a comment -

          you're missing the point

          I do not think I am missing anything. Hence my suggested patch above.

          (The ArtifactArchiver.NoMatchFound text should probably also suggest enabling the allowEmptyArchive mode—a later addition—if the lack of a match is not a bug.)

          Jesse Glick added a comment - you're missing the point I do not think I am missing anything. Hence my suggested patch above. (The ArtifactArchiver.NoMatchFound text should probably also suggest enabling the allowEmptyArchive mode—a later addition—if the lack of a match is not a bug.)

          My apologies, I misunderstood the intent of the patch.

          Mike Kingsbury added a comment - My apologies, I misunderstood the intent of the patch.

          Max Wahler added a comment -

          jglick: What would it look like with the modifications you proposed when there's nothing to archive?

          Max Wahler added a comment - jglick : What would it look like with the modifications you proposed when there's nothing to archive?

          Jesse Glick added a comment -

          It would print a warning before failing, as already happens for freestyle builds.

          Jesse Glick added a comment - It would print a warning before failing, as already happens for freestyle builds.

          Harry Mallon added a comment -

          jglick: Would this patch mean that the archive stage in pipeline steps view would show as failed?

          Harry Mallon added a comment - jglick : Would this patch mean that the archive stage in pipeline steps view would show as failed?

          Ian Leitch added a comment -

          Would be great to have this resolved. This is was a huge time sink for me to track down.

          Ian Leitch added a comment - Would be great to have this resolved. This is was a huge time sink for me to track down.

          Same for me. Almost half a day to figure out that this silently fails the build without any warning!

          Roman Bäriswyl added a comment - Same for me. Almost half a day to figure out that this silently fails the build without any warning!

          Daniel Beck added a comment -

          Notably (IIRC) freestyle jobs log when a step makes the build result worse. Looks like Pipeline does not?

          Daniel Beck added a comment - Notably (IIRC) freestyle jobs log when a step makes the build result worse. Looks like Pipeline does not?

          I would like to add that it looks to me that archiveArtifacts also (silently) fails when using absolute paths:

          def myfile1 = "foobar"
          sh "echo foobar > ${myfile1}"
          archiveArtifacts artifacts: "${myfile1}", fingerprint: true
          

          works correctly,

          def myfile1 = "${WORKSPACE}/foobar"
          sh "echo foobar > ${myfile1}"
          archiveArtifacts artifacts: "${myfile1}", fingerprint: true
          

          on the other hand does not.

          Christian Beushausen added a comment - I would like to add that it looks to me that archiveArtifacts also (silently) fails when using absolute paths: def myfile1 = "foobar" sh "echo foobar > ${myfile1}" archiveArtifacts artifacts: "${myfile1}" , fingerprint: true works correctly, def myfile1 = "${WORKSPACE}/foobar" sh "echo foobar > ${myfile1}" archiveArtifacts artifacts: "${myfile1}" , fingerprint: true on the other hand does not.

          dan turner added a comment -

          bump, just spent 1/2 a day looking for this!

          dan turner added a comment - bump, just spent 1/2 a day looking for this!

          Andrew Bayer added a comment -

          PR up at https://github.com/jenkinsci/jenkins/pull/2823 with Jesse's suggested patch and a change to the error message.

          Andrew Bayer added a comment - PR up at https://github.com/jenkinsci/jenkins/pull/2823  with Jesse's suggested patch and a change to the error message.

          Michael Neale added a comment -

          gosh that is terrible. The step result isn't failure, and there is no log message. 

          Michael Neale added a comment - gosh that is terrible. The step result isn't failure, and there is no log message. 

          Andrew Bayer added a comment -

          Not sure about the step result aspect - I can never remember how that actually gets set. But at a minimum, this will definitely log an error. If it turns out to still not cause the step in Blue Ocean to know it failed, we can revisit.

          Andrew Bayer added a comment - Not sure about the step result aspect - I can never remember how that actually gets set. But at a minimum, this will definitely log an error. If it turns out to still not cause the step in Blue Ocean to know it failed, we can revisit.

          Emil Styrke added a comment -

          abayer, as far as I can see, the step still shows with a green dot in "pipeline steps" view (and with a green check mark in BlueOcean). See JENKINS-45200 which I reported the other day.

          I get somewhere around 25 Mb of logs from my build, with multiple archiving steps, so even with the error message, finding out why my build failed was quite hard.

          Emil Styrke added a comment - abayer , as far as I can see, the step still shows with a green dot in "pipeline steps" view (and with a green check mark in BlueOcean). See JENKINS-45200 which I reported the other day. I get somewhere around 25 Mb of logs from my build, with multiple archiving steps, so even with the error message, finding out why my build failed was quite hard.

          Jesse Glick added a comment -

          If you continue to have an issue using Jenkins 2.59 or later please file a separate issue (linked) with steps to reproduce from scratch.

          Jesse Glick added a comment - If you continue to have an issue using Jenkins 2.59 or later please file a separate issue (linked) with steps to reproduce from scratch.

          Craig Ringer added a comment -

          The comments here about "silently" failing the build make me think it'd be very good to capture the hudson.Abort exception and its stack. Then when the pipeline aborts, print

          "Pipeline aborted. (See stack)", with "(See stack)" as a ConsoleAnnotation that lets you view the stack that led to the abort.

          Even "Pipeline aborted in 'build' step at line '42'" would be a massive help though.

          Craig Ringer added a comment - The comments here about "silently" failing the build make me think it'd be very good to capture the hudson.Abort exception and its stack. Then when the pipeline aborts, print "Pipeline aborted. (See stack)", with "(See stack)" as a ConsoleAnnotation that lets you view the stack that led to the abort. Even "Pipeline aborted in 'build' step at line '42'" would be a massive help though.

          Jesse Glick added a comment -

          ringerc as far as I know this issue is fixed, and when a Step fails with an AbortException, the log will display that message. That said, having an optional link in the (“classic”) UI to display the full stack trace from an AbortException (which is normally concealed) would be a neat enhancement for workflow-job-plugin.

          Jesse Glick added a comment - ringerc as far as I know this issue is fixed, and when a Step fails with an AbortException , the log will display that message. That said, having an optional link in the (“classic”) UI to display the full stack trace from an AbortException (which is normally concealed) would be a neat enhancement for workflow-job-plugin .

            abayer Andrew Bayer
            rtyler R. Tyler Croy
            Votes:
            9 Vote for this issue
            Watchers:
            22 Start watching this issue

              Created:
              Updated:
              Resolved: