• Type: Task
• Resolution: Duplicate
• Priority: Critical
• Component/s: robot-plugin
• Labels:

  • robot-plugin

1. 

   jenkins plugin result.PNG

   13 kB

   2023-03-01 16:43

[JENKINS-70711] Need to remove critical tests from the dashboard and build is marked as pass even though failures in all tests exists

Collapse comment: Aleksi Simell added a comment - 2023-03-14 10:48

Aleksi Simell added a comment - 2023-03-14 10:48

Hi,

what makes this a critical issue? You have the `onlyCritical` option to bypass test criticality checks, and you also have the possibility to add pass thresholds to tests to allow build to pass even though you have failures in tests.

For example `passThreshold: 70, unstableThreshold: 50` will mark the build as

• passed if at least 70% of your tests pass
• unstable if 50-69% of your tests pass
• failed if under 50% of your tests pass

Expand comment: Aleksi Simell added a comment - 2023-03-14 10:48

Aleksi Simell added a comment - 2023-03-14 10:48 Hi, what makes this a critical issue? You have the `onlyCritical` option to bypass test criticality checks, and you also have the possibility to add pass thresholds to tests to allow build to pass even though you have failures in tests. For example `passThreshold: 70, unstableThreshold: 50` will mark the build as passed if at least 70% of your tests pass unstable if 50-69% of your tests pass failed if under 50% of your tests pass

Collapse comment: Aleksi Simell added a comment - 2023-05-11 10:42

Aleksi Simell added a comment - 2023-05-11 10:42

I created JENKINS-71249 to be treated as the general "remove support from Robot Framework 3.x feature". Please refer to that issue for further information Closing this as duplicate.

Expand comment: Aleksi Simell added a comment - 2023-05-11 10:42

Aleksi Simell added a comment - 2023-05-11 10:42 I created JENKINS-71249 to be treated as the general "remove support from Robot Framework 3.x feature". Please refer to that issue for further information Closing this as duplicate.