Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70711

Need to remove critical tests from the dashboard and build is marked as pass even though failures in all tests exists

XMLWordPrintable

        1. jenkins plugin result.PNG
          jenkins plugin result.PNG
          13 kB

          [JENKINS-70711] Need to remove critical tests from the dashboard and build is marked as pass even though failures in all tests exists

          RaZeR created issue -
          RaZeR made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          RaZeR made changes -
          Status Original: In Progress [ 3 ] New: Open [ 1 ]

          Aleksi Simell added a comment -

          Hi,

          what makes this a critical issue? You have the `onlyCritical` option to bypass test criticality checks, and you also have the possibility to add pass thresholds to tests to allow build to pass even though you have failures in tests.

          For example `passThreshold: 70, unstableThreshold: 50` will mark the build as

          • passed if at least 70% of your tests pass
          • unstable if 50-69% of your tests pass
          • failed if under 50% of your tests pass

          Aleksi Simell added a comment - Hi, what makes this a critical issue? You have the `onlyCritical` option to bypass test criticality checks, and you also have the possibility to add pass thresholds to tests to allow build to pass even though you have failures in tests. For example `passThreshold: 70, unstableThreshold: 50` will mark the build as passed if at least 70% of your tests pass unstable if 50-69% of your tests pass failed if under 50% of your tests pass

          Aleksi Simell added a comment -

          I created JENKINS-71249 to be treated as the general "remove support from Robot Framework 3.x feature". Please refer to that issue for further information Closing this as duplicate.

          Aleksi Simell added a comment - I created JENKINS-71249 to be treated as the general "remove support from Robot Framework 3.x feature". Please refer to that issue for further information Closing this as duplicate.
          Aleksi Simell made changes -
          Link New: This issue duplicates JENKINS-71249 [ JENKINS-71249 ]
          Aleksi Simell made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Closed [ 6 ]

            aleksisimell Aleksi Simell
            testing12345 RaZeR
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: