Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68419

Help text of quality gate threshold is wrong: values less or equal zero are NOT ignored

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Minor
    • Resolution: Fixed
    • warnings-ng-plugin
    • None
    • Jenkins 2.319.1
      Warnings Next Generation 9.11.1

    Description

      In the documentation of the Warnings Next Generation plugin, under the publishIssues and recordIssues, the description of thresholds on qualityGates states "The threshold defines the minimum number of warnings that will fail a build. Values less or equal zero are ignored. So if you want to fail a build that has one warning, set this field to 1."

      Setting the threshold values less or equal to zero does NOT cause the quality gate to be ignored. All quality gates fail since even a value of zero succeeds the greater than or equal test.

      For now I have worked around this by setting the thresholds to 10,000 instead of 0. But it would be better if a the quality gates were ignored instead of comparing to a 10,000 threshold.

      Attachments

        Issue Links

          Activity

            mikedeco Michael DeCourcey created issue -
            sravya_2402 Chandrika Sravya made changes -
            Field Original Value New Value
            Status Open [ 1 ] In Progress [ 3 ]
            drulli Ulli Hafner made changes -
            Status In Progress [ 3 ] Open [ 1 ]
            drulli Ulli Hafner added a comment -

            Thanks for spotting! Seems that I did not change the documentation after the quality gates have been migrated from thresholds to quality gate objects. Are you not using the UI and the field validation to create the step parameters? In the UI I get an error message when using values <= 0: "Threshold must be an integer value greater 0."

            Why do you want to use a quality gate < 0? It should be sufficient to remove the quality gate in such a case.

            drulli Ulli Hafner added a comment - Thanks for spotting! Seems that I did not change the documentation after the quality gates have been migrated from thresholds to quality gate objects. Are you not using the UI and the field validation to create the step parameters? In the UI I get an error message when using values <= 0: "Threshold must be an integer value greater 0." Why do you want to use a quality gate < 0? It should be sufficient to remove the quality gate in such a case.
            drulli Ulli Hafner made changes -
            Summary Quality gate thresholds with values less or equal zero are NOT ignored Help text of quality gate thresholds wrong: values less or equal zero are NOT ignored
            drulli Ulli Hafner made changes -
            Summary Help text of quality gate thresholds wrong: values less or equal zero are NOT ignored Help text of quality gate threshold is wrong: values less or equal zero are NOT ignored

             

            Sorry for the untimely response.

             

            No, we are not using the UI to se the threshold. We are using a pipeline coded in a Jenkinsfile that we supply config.json file to. We have a default set of configs that are used by all repositories and then where needed we override particular the configs for a particular repository.

            The default config looked like this prior to updating to warnings-ng-plugin:

             

            {
              ...,
              "checkstyle": {
                "profile": "checkstyle",
                "failed": {
                  "all": "", "high": "", "low": "", "normal": ""
                },
                "unstable": {
                  "all": "", "high": "", "low": "", "normal": ""
                }
              },
              "findbugs": {
                "failed": {
                  "all": "", "high": "0", "low": "", "normal": ""
                },
                "unstable": {
                  "all": "", "high": "", "low": "", "normal": "0"
                }
              },
              ...
            } 

            And the override config for a specific repo (when required) might look like this:

             

            {
              "checkstyle": {
                "failed": {
                  "high": "0"
                },
              },
              "findbugs": {
                "failed": {
                  "all": "0"
                },
                "unstable": {
                  "normal": ""
                }
              },
              ...
            } 

             

             

            After upgrading to warnings-ng-plugin I expected it to change to:

             

             

            {
              ...,
              "checkstyle": {
                "profile": "checkstyle",
                "failed": {
                  "all": 0, "high": 0, "low": 0, "normal": 0
                },
                "unstable": {
                  "all": 0, "high": 0, "low": 0, "normal": 0
                }
              },
              "findbugs": {
                "failed": {
                  "all": 0, "high": 1, "low": 0, "normal": 0
                },
                "unstable": {
                  "all": 0, "high": 0, "low": 0, "normal": 1
                }
              },
              ...
            } 

            And

             

             

            {
              "checkstyle": {
                "failed": {
                  "high": 1
                },
              },
              "findbugs": {
                "failed": {
                  "all": 1
                },
                "unstable": {
                  "normal": 0
                }
              },
              ...
            }
             

            The code that references these config changes after updating to warnings-ng-plugin looks like this:

                def findbugsFailedAll = config['findbugs']['failed']['all']
                def findbugsFailedHigh = config['findbugs']['failed']['high']
                def findbugsFailedLow = config['findbugs']['failed']['low']
                def findbugsFailedNormal = config['findbugs']['failed']['normal']
                def findbugsUnstableAll = config['findbugs']['unstable']['all']
                def findbugsUnstableHigh = config['findbugs']['unstable']['high']
                def findbugsUnstableLow = config['findbugs']['unstable']['low']
                def findbugsUnstableNormal = config['findbugs']['unstable']['normal']
            
                recordIssues tool: findBugs(pattern: findbugsResults, useRankAsPriority: true), enabledForFailure: true,
                    qualityGates: [[threshold: findbugsFailedAll, type: 'TOTAL', unstable: false],
                      [threshold: findbugsFailedHigh, type: 'TOTAL_HIGH', unstable: false],
                      [threshold: findbugsFailedLow, type: 'TOTAL_LOW', unstable: false],
                      [threshold: findbugsFailedNormal, type: 'TOTAL_NORMAL', unstable: false],
                      [threshold: findbugsUnstableAll, type: 'TOTAL', unstable: true],
                      [threshold: findbugsUnstableHigh, type: 'TOTAL_HIGH', unstable: true],
                      [threshold: findbugsUnstableLow, type: 'TOTAL_LOW', unstable: true],
                      [threshold: findbugsUnstableNormal, type: 'TOTAL_NORMAL', unstable: true]]

            As stated in the issue description, my workaround was to use 10,000 instead of 0 as my value where I want the gate to be ignored. If I was better at coding in Groovy I would probably change the code to dynamically build the qualityGates array depending on whether the value was 0 or greater than zero. But since the documentation indicated that passing zero was the same as saying that the gate should be ignored I decided to go with a more temporary workaround.

             

            mikedeco Michael DeCourcey added a comment -   Sorry for the untimely response.   No, we are not using the UI to se the threshold. We are using a pipeline coded in a Jenkinsfile that we supply config.json file to. We have a default set of configs that are used by all repositories and then where needed we override particular the configs for a particular repository. The default config looked like this prior to updating to warnings-ng-plugin:   { ...,   "checkstyle" : {     "profile" : "checkstyle" ,     "failed" : {       "all" : "", " high ": " ", " low ": " ", " normal ": " "     },     "unstable" : {       "all" : "", " high ": " ", " low ": " ", " normal ": " "     }   },   "findbugs" : {     "failed" : {       "all" : "", " high ": " 0 ", " low ": " ", " normal ": " "     },     "unstable" : {       "all" : "", " high ": " ", " low ": " ", " normal ": " 0"     }   },   ... } And the override config for a specific repo (when required) might look like this:   {   "checkstyle" : {     "failed" : {       "high" : "0"     },   },   "findbugs" : { "failed" : { "all" : "0" },     "unstable" : {       "normal" : ""     }   },   ... }     After upgrading to warnings-ng-plugin I expected it to change to:     {   ...,   "checkstyle" : {     "profile" : "checkstyle" ,     "failed" : {       "all" : 0, "high" : 0, "low" : 0, "normal" : 0     },     "unstable" : {       "all" : 0, "high" : 0, "low" : 0, "normal" : 0     }   },   "findbugs" : {     "failed" : {       "all" : 0, "high" : 1, "low" : 0, "normal" : 0     },     "unstable" : {       "all" : 0, "high" : 0, "low" : 0, "normal" : 1     }   },   ... } And     {   "checkstyle" : {     "failed" : {       "high" : 1     },   },   "findbugs" : {   "failed" : {       "all" : 1     },     "unstable" : {       "normal" : 0     }   }, ... } The code that references these config changes after updating to warnings-ng-plugin looks like this:     def findbugsFailedAll = config[ 'findbugs' ][ 'failed' ][ 'all' ]     def findbugsFailedHigh = config[ 'findbugs' ][ 'failed' ][ 'high' ]     def findbugsFailedLow = config[ 'findbugs' ][ 'failed' ][ 'low' ]     def findbugsFailedNormal = config[ 'findbugs' ][ 'failed' ][ 'normal' ]     def findbugsUnstableAll = config[ 'findbugs' ][ 'unstable' ][ 'all' ]     def findbugsUnstableHigh = config[ 'findbugs' ][ 'unstable' ][ 'high' ]     def findbugsUnstableLow = config[ 'findbugs' ][ 'unstable' ][ 'low' ]     def findbugsUnstableNormal = config[ 'findbugs' ][ 'unstable' ][ 'normal' ]   recordIssues tool: findBugs(pattern: findbugsResults, useRankAsPriority: true ), enabledForFailure: true ,         qualityGates: [[threshold: findbugsFailedAll, type: 'TOTAL' , unstable: false ],           [threshold: findbugsFailedHigh, type: 'TOTAL_HIGH' , unstable: false ],           [threshold: findbugsFailedLow, type: 'TOTAL_LOW' , unstable: false ],           [threshold: findbugsFailedNormal, type: 'TOTAL_NORMAL' , unstable: false ],           [threshold: findbugsUnstableAll, type: 'TOTAL' , unstable: true ],           [threshold: findbugsUnstableHigh, type: 'TOTAL_HIGH' , unstable: true ],           [threshold: findbugsUnstableLow, type: 'TOTAL_LOW' , unstable: true ],           [threshold: findbugsUnstableNormal, type: 'TOTAL_NORMAL' , unstable: true ]] As stated in the issue description, my workaround was to use 10,000 instead of 0 as my value where I want the gate to be ignored. If I was better at coding in Groovy I would probably change the code to dynamically build the qualityGates array depending on whether the value was 0 or greater than zero. But since the documentation indicated that passing zero was the same as saying that the gate should be ignored I decided to go with a more temporary workaround.  

            Assuming that you want the documentation to remain the same and allow setting threshold to zero to still mean ignoring the quality gate, I took a stab a pull request against my own fork of warnings-ng-plugin.

            https://github.com/miked-rr/warnings-ng-plugin/pull/1/commits/e323e3ea29e2e08e999953d509d3a4c9193a2506

            mikedeco Michael DeCourcey added a comment - Assuming that you want the documentation to remain the same and allow setting threshold to zero to still mean ignoring the quality gate, I took a stab a pull request against my own fork of warnings-ng-plugin. https://github.com/miked-rr/warnings-ng-plugin/pull/1/commits/e323e3ea29e2e08e999953d509d3a4c9193a2506
            drulli Ulli Hafner added a comment -

            I think your implementation is more smart than the current one. Even though we have the validation, users still can press ok and then something not expected is happening.

            Can you please file your changes as PR for my plugin as well?

            drulli Ulli Hafner added a comment - I think your implementation is more smart than the current one. Even though we have the validation, users still can press ok and then something not expected is happening. Can you please file your changes as PR for my plugin as well?
            drulli Ulli Hafner made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            drulli Ulli Hafner made changes -
            Assignee Ulli Hafner [ drulli ] Michael DeCourcey [ mikedeco ]
            drulli Ulli Hafner made changes -
            Remote Link This issue links to "PR #1367 (Web Link)" [ 28249 ]
            drulli Ulli Hafner made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            drulli Ulli Hafner made changes -
            Released As https://github.com/jenkinsci/warnings-ng-plugin/releases/tag/v9.20.1
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]

            People

              mikedeco Michael DeCourcey
              mikedeco Michael DeCourcey
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: