Ulli Hafner added a comment - 2020-02-18 18:04 The button changes only the selection of the reference in the following way: as reference any previous build will be chosen that is not failing (regardless of the quality gate). So I am assuming your builds #3-#6 are failing? Or let me ask the other way around: what do you want to achieve?

R. Fitzner added a comment - 2020-02-19 08:35 - edited With build #2 the repository suppressed JavaDoc warnings/errors via Gradle options. With #3 I disabled suppressing it and off course now the "new" issues increased dramatically. See How can I make build #3 as a new baseline? PS. Yes build #3-#7 are failing because the quality gates insists that the "overall result is FAILED".

R. Fitzner added a comment - 2020-02-21 16:39 - edited The official Pipeline Syntax documentation is: 1. Ignore Quality Gate (Default for ignoreQualityGate: false): If checked, then the last finished build of the reference job will be used as baseline no matter if the quality gate of that build has been passed or not. " ... Jenkins will mark all builds as unstable until the issues have been resolved." 2. Ignore Failed Builds (Default for ignoreFailedBuilds: true): If checked, then only successful or unstable builds will be considered as baseline.   I have noticed that: 1. It seems like that against the documentation "unstable builds" are treated as failed! Even if QualityGate is configured with unstable: true neither the "Reset quality gate" button is displayed nor after re-running the job the previous UNSTABLE build is concidered as new baseline. 00:02:39.415 [Static Analysis] Attaching ResultAction with ID 'analysis' to run 'XYZ/aaaaaa/develop #12'. 00:02:39.436 [Static Analysis] Using reference build 'XYZ/aaaaaa/develop #10' to compute new, fixed, and outstanding issues 00:02:39.436 [Static Analysis] Issues delta (vs. reference build): outstanding: 61, new: 52, fixed: 1 00:02:39.436 [Static Analysis] Evaluating quality gates 00:02:39.436 [Static Analysis] -> PASSED - New (errors only): 0 - Quality QualityGate: 1 00:02:39.436 [Static Analysis] -> PASSED - New (severity high only): 0 - Quality QualityGate: 1 00:02:39.436 [Static Analysis] -> WARNING - New (severity normal only): 51 - Quality QualityGate: 4 00:02:39.436 [Static Analysis] -> PASSED - New (severity low only): 1 - Quality QualityGate: 10 00:02:39.436 [Static Analysis] -> Some quality gates have been missed: overall result is WARNING 00:02:39.436 [Static Analysis] Enabling health report (Healthy=1, Unhealthy=12, Minimum Severity=LOW) 00:02:39.483 [Static Analysis] Created analysis result for 113 issues (found 52 new issues, fixed 1 issues) This explains why adding "ignoreQualityGate: true" to my pipeline did not solve my problem because the parameter ignoreFailedBuilds was still true (default). 2. Since "unstable" and "failed" builds are treated equally the plugin now would force me to set "ignoreFailedBuilds: false". If I would do this then defining a QualityGate does not realy make sense anymore because the baseline is always the previous build of the last one. Even if I would define a QualityGate and set it to unstable I cannot differenciate between UNSTABLE (publish SNAPSHOTS) and FAILED (abort and do not publish RELEASE) anymore because the plugin checks the build status and not its quality gate status. Or am I missing something?

Ulli Hafner added a comment - 2020-02-22 10:17 This is exactly what I mean: the reference build selection for quality gates that fail a build is not really working, see JENKINS-61140 . The options currently available make only sense if you are using a quality gate that sets a build to unstable.