Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38005

Using archiveArtifacts with a non-matching pattern silently fails the build

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: core
    • Labels:
    • Environment:
      Jenkins 2.7.3, Pipeline 2.3
    • Similar Issues:

      Description

      If you happen to specify a pattern that doesn't match any files in the workspace, archiveArtifacts will silently fail the build.

      Jenkinsfile

      node {
          sh 'echo "lol"'
          
          archiveArtifacts artifacts: 'lol*.zip'
      }
      

      Console Output

      Started by user admin
      [Pipeline] node
      Still waiting to schedule task
      Waiting for next available executor
      Running on Azure0907122157 in /home/azureuser/workspace/Experiments/bad-artifacts
      [Pipeline] {
      [Pipeline] sh
      [bad-artifacts] Running shell script
      + echo lol
      lol
      [Pipeline] step
      Archiving artifacts
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] End of Pipeline
      Finished: FAILURE
      

      archive step

      In contrast, the use of the archive step (e.g. archive 'lol*.zip' in the above Jenkinsfile) will result in a silently successful build.

        Attachments

          Issue Links

            Activity

            Hide
            maxzilla Max Wahler added a comment -

            I raised the priority to 'Critical' because this makes very difficult to see where a large build actually failed.

            Also, I saw a similar behavior:

            archiveArtifacts 'target/test.data'

            fails only if the directory 'target' does not exist. If it exists but not the file 'test.data', then the build fails but the build step was successful.

            Show
            maxzilla Max Wahler added a comment - I raised the priority to 'Critical' because this makes very difficult to see where a large build actually failed. Also, I saw a similar behavior: archiveArtifacts 'target/test.data' fails only if the directory 'target' does not exist. If it exists but not the file 'test.data', then the build fails but the build step was successful.
            Hide
            maxzilla Max Wahler added a comment -

            Is there any progress on this issue?

            It would be really really great if there was a way to tell if archiveArtifacts worked or not! I have a job that builds a lot of stuff and has ~750 steps. It's totally impossible to find out where the pipeline failed in case archiveArtifacts failed, because every step is green and there's no output at all that points me to the correct place.

            My approach is now more a Poor Man's Debugger than a solution: I put masses of printlns in it which give me the current build result. But this doesn't necessarily mean that the error happened at that specific place if you build stuff in parallel.

            IMHO, archiveArtifacts should at least print out the amount of files it included. Also a return value containing the files or the number of files would be great. But instead of failing silently, I'd prefer an Exception.

            Show
            maxzilla Max Wahler added a comment - Is there any progress on this issue? It would be really really great if there was a way to tell if archiveArtifacts worked or not! I have a job that builds a lot of stuff and has ~750 steps. It's totally impossible to find out where the pipeline failed in case archiveArtifacts failed, because every step is green and there's no output at all that points me to the correct place. My approach is now more a Poor Man's Debugger than a solution: I put masses of printlns in it which give me the current build result. But this doesn't necessarily mean that the error happened at that specific place if you build stuff in parallel. IMHO, archiveArtifacts should at least print out the amount of files it included. Also a return value containing the files or the number of files would be great. But instead of failing silently, I'd prefer an Exception.
            Hide
            jglick Jesse Glick added a comment -

            Just use allowEmptyArchive: true. See inline help in Pipeline Syntax.

            As to the lack of a message, I suspect the following would suffice:

            diff --git a/core/src/main/java/hudson/tasks/ArtifactArchiver.java b/core/src/main/java/hudson/tasks/ArtifactArchiver.java
            index ca1f154..9175e11 100644
            --- a/core/src/main/java/hudson/tasks/ArtifactArchiver.java
            +++ b/core/src/main/java/hudson/tasks/ArtifactArchiver.java
            @@ -242,7 +242,7 @@ public class ArtifactArchiver extends Recorder implements SimpleBuildStep {
                             }
                         } else {
                             Result result = build.getResult();
            -                if (result != null && result.isBetterOrEqualTo(Result.UNSTABLE)) {
            +                if (result == null || result.isBetterOrEqualTo(Result.UNSTABLE)) {
                                 // If the build failed, don't complain that there was no matching artifact.
                                 // The build probably didn't even get to the point where it produces artifacts. 
                                 listenerWarnOrError(listener, Messages.ArtifactArchiver_NoMatchFound(artifacts));
            
            Show
            jglick Jesse Glick added a comment - Just use allowEmptyArchive: true . See inline help in Pipeline Syntax . As to the lack of a message, I suspect the following would suffice: diff --git a/core/src/main/java/hudson/tasks/ArtifactArchiver.java b/core/src/main/java/hudson/tasks/ArtifactArchiver.java index ca1f154..9175e11 100644 --- a/core/src/main/java/hudson/tasks/ArtifactArchiver.java +++ b/core/src/main/java/hudson/tasks/ArtifactArchiver.java @@ -242,7 +242,7 @@ public class ArtifactArchiver extends Recorder implements SimpleBuildStep { } } else { Result result = build.getResult(); - if (result != null && result.isBetterOrEqualTo(Result.UNSTABLE)) { + if (result == null || result.isBetterOrEqualTo(Result.UNSTABLE)) { // If the build failed, don't complain that there was no matching artifact. // The build probably didn't even get to the point where it produces artifacts. listenerWarnOrError(listener, Messages.ArtifactArchiver_NoMatchFound(artifacts));
            Hide
            jglick Jesse Glick added a comment -

            From a script, it would probably also work to run

            currentBuild.result = 'STABLE'
            

            prior to running the archiver. (The result can only get worse, never better, but it can become nonnull.)

            Show
            jglick Jesse Glick added a comment - From a script, it would probably also work to run currentBuild.result = 'STABLE' prior to running the archiver. (The result can only get worse, never better, but it can become nonnull.)
            Hide
            mike_kingsbury Mike Kingsbury added a comment -

            Jesse, you're missing the point. Yes, once someone recognizes the problem, it can be worked around. Until that point, they are left with a failed build and no useful information as to its cause. Hence the request for additional output from archiveArtifacts.

            Show
            mike_kingsbury Mike Kingsbury added a comment - Jesse, you're missing the point. Yes, once someone recognizes the problem, it can be worked around. Until that point, they are left with a failed build and no useful information as to its cause. Hence the request for additional output from archiveArtifacts.
            Hide
            jglick Jesse Glick added a comment -

            you're missing the point

            I do not think I am missing anything. Hence my suggested patch above.

            (The ArtifactArchiver.NoMatchFound text should probably also suggest enabling the allowEmptyArchive mode—a later addition—if the lack of a match is not a bug.)

            Show
            jglick Jesse Glick added a comment - you're missing the point I do not think I am missing anything. Hence my suggested patch above. (The ArtifactArchiver.NoMatchFound text should probably also suggest enabling the allowEmptyArchive mode—a later addition—if the lack of a match is not a bug.)
            Hide
            mike_kingsbury Mike Kingsbury added a comment -

            My apologies, I misunderstood the intent of the patch.

            Show
            mike_kingsbury Mike Kingsbury added a comment - My apologies, I misunderstood the intent of the patch.
            Hide
            maxzilla Max Wahler added a comment -

            Jesse Glick: What would it look like with the modifications you proposed when there's nothing to archive?

            Show
            maxzilla Max Wahler added a comment - Jesse Glick : What would it look like with the modifications you proposed when there's nothing to archive?
            Hide
            jglick Jesse Glick added a comment -

            It would print a warning before failing, as already happens for freestyle builds.

            Show
            jglick Jesse Glick added a comment - It would print a warning before failing, as already happens for freestyle builds.
            Hide
            harryjenkins Harry Mallon added a comment -

            Jesse Glick: Would this patch mean that the archive stage in pipeline steps view would show as failed?

            Show
            harryjenkins Harry Mallon added a comment - Jesse Glick : Would this patch mean that the archive stage in pipeline steps view would show as failed?
            Hide
            ileitch Ian Leitch added a comment -

            Would be great to have this resolved. This is was a huge time sink for me to track down.

            Show
            ileitch Ian Leitch added a comment - Would be great to have this resolved. This is was a huge time sink for me to track down.
            Hide
            roemer Roman Bäriswyl added a comment -

            Same for me. Almost half a day to figure out that this silently fails the build without any warning!

            Show
            roemer Roman Bäriswyl added a comment - Same for me. Almost half a day to figure out that this silently fails the build without any warning!
            Hide
            danielbeck Daniel Beck added a comment -

            Notably (IIRC) freestyle jobs log when a step makes the build result worse. Looks like Pipeline does not?

            Show
            danielbeck Daniel Beck added a comment - Notably (IIRC) freestyle jobs log when a step makes the build result worse. Looks like Pipeline does not?
            Hide
            crb Christian Beushausen added a comment -

            I would like to add that it looks to me that archiveArtifacts also (silently) fails when using absolute paths:

            def myfile1 = "foobar"
            sh "echo foobar > ${myfile1}"
            archiveArtifacts artifacts: "${myfile1}", fingerprint: true
            

            works correctly,

            def myfile1 = "${WORKSPACE}/foobar"
            sh "echo foobar > ${myfile1}"
            archiveArtifacts artifacts: "${myfile1}", fingerprint: true
            

            on the other hand does not.

            Show
            crb Christian Beushausen added a comment - I would like to add that it looks to me that archiveArtifacts also (silently) fails when using absolute paths: def myfile1 = "foobar" sh "echo foobar > ${myfile1}" archiveArtifacts artifacts: "${myfile1}" , fingerprint: true works correctly, def myfile1 = "${WORKSPACE}/foobar" sh "echo foobar > ${myfile1}" archiveArtifacts artifacts: "${myfile1}" , fingerprint: true on the other hand does not.
            Hide
            danturn dan turner added a comment -

            bump, just spent 1/2 a day looking for this!

            Show
            danturn dan turner added a comment - bump, just spent 1/2 a day looking for this!
            Hide
            abayer Andrew Bayer added a comment -

            PR up at https://github.com/jenkinsci/jenkins/pull/2823 with Jesse's suggested patch and a change to the error message.

            Show
            abayer Andrew Bayer added a comment - PR up at https://github.com/jenkinsci/jenkins/pull/2823  with Jesse's suggested patch and a change to the error message.
            Hide
            michaelneale Michael Neale added a comment -

            gosh that is terrible. The step result isn't failure, and there is no log message. 

            Show
            michaelneale Michael Neale added a comment - gosh that is terrible. The step result isn't failure, and there is no log message. 
            Hide
            abayer Andrew Bayer added a comment -

            Not sure about the step result aspect - I can never remember how that actually gets set. But at a minimum, this will definitely log an error. If it turns out to still not cause the step in Blue Ocean to know it failed, we can revisit.

            Show
            abayer Andrew Bayer added a comment - Not sure about the step result aspect - I can never remember how that actually gets set. But at a minimum, this will definitely log an error. If it turns out to still not cause the step in Blue Ocean to know it failed, we can revisit.
            Hide
            estyrke Emil Styrke added a comment -

            Andrew Bayer, as far as I can see, the step still shows with a green dot in "pipeline steps" view (and with a green check mark in BlueOcean). See JENKINS-45200 which I reported the other day.

            I get somewhere around 25 Mb of logs from my build, with multiple archiving steps, so even with the error message, finding out why my build failed was quite hard.

            Show
            estyrke Emil Styrke added a comment - Andrew Bayer , as far as I can see, the step still shows with a green dot in "pipeline steps" view (and with a green check mark in BlueOcean). See JENKINS-45200 which I reported the other day. I get somewhere around 25 Mb of logs from my build, with multiple archiving steps, so even with the error message, finding out why my build failed was quite hard.
            Hide
            jglick Jesse Glick added a comment -

            If you continue to have an issue using Jenkins 2.59 or later please file a separate issue (linked) with steps to reproduce from scratch.

            Show
            jglick Jesse Glick added a comment - If you continue to have an issue using Jenkins 2.59 or later please file a separate issue (linked) with steps to reproduce from scratch.
            Hide
            ringerc Craig Ringer added a comment -

            The comments here about "silently" failing the build make me think it'd be very good to capture the hudson.Abort exception and its stack. Then when the pipeline aborts, print

            "Pipeline aborted. (See stack)", with "(See stack)" as a ConsoleAnnotation that lets you view the stack that led to the abort.

            Even "Pipeline aborted in 'build' step at line '42'" would be a massive help though.

            Show
            ringerc Craig Ringer added a comment - The comments here about "silently" failing the build make me think it'd be very good to capture the hudson.Abort exception and its stack. Then when the pipeline aborts, print "Pipeline aborted. (See stack)", with "(See stack)" as a ConsoleAnnotation that lets you view the stack that led to the abort. Even "Pipeline aborted in 'build' step at line '42'" would be a massive help though.
            Hide
            jglick Jesse Glick added a comment -

            Craig Ringer as far as I know this issue is fixed, and when a Step fails with an AbortException, the log will display that message. That said, having an optional link in the (“classic”) UI to display the full stack trace from an AbortException (which is normally concealed) would be a neat enhancement for workflow-job-plugin.

            Show
            jglick Jesse Glick added a comment - Craig Ringer as far as I know this issue is fixed, and when a Step fails with an AbortException , the log will display that message. That said, having an optional link in the (“classic”) UI to display the full stack trace from an AbortException (which is normally concealed) would be a neat enhancement for workflow-job-plugin .

              People

              Assignee:
              abayer Andrew Bayer
              Reporter:
              rtyler R. Tyler Croy
              Votes:
              9 Vote for this issue
              Watchers:
              22 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: