-
Bug
-
Resolution: Unresolved
-
Major
-
Jenkins version: 2.89.2
junit plugin version: 1.23
os: Description: CentOS release 6.8 (Final)
Hi,
we are using Jenkins doing CI with our product and running smoke test on it, we adopt Junit plugin to collect information and report smoke test results, but there is some problem when we using it. In the history of one of the smoke tests, Jenkins is showing what seems like false information as you can see in the pic:
Build 47 is labeled as Passed, but Build 48 is Fixed. The same thing happens with 49 and 50. This results in build 51 reporting: Failed 2 times in the last 30 runs. Flakiness: 13%, Stability: 93% . That seems incorrect.
Is that a bug in the Jenkins plugin? Do we have an understanding explanation for that?
[JENKINS-48839] Junit plugin reporting Fixed when apparently the previous run did not fail
We observed similar issue, See below snapshot. Buid #718 got "Regression" and said "Failed 1 times in the last 30 runs. Flakiness: 3%, Stability: 96%". The following builds (since build #719) passed. But they said "(No known failures. Flakiness 0%, Stability 100%)". This should be incorrect.
I'm hitting the exact same issue where a test that did not fail in the previous run is marked as FIXED. In this case the same test has been marked with status FIXED for three consecutive builds.
Build 44:
{ "_class": "hudson.tasks.junit.ClassResult", "testActions": [ ], "child": [{ "testActions": [ ], "age": 0, "className": "Test /ep/GetEp.ExistingEp", "duration": 0.439, "failedSince": 0, "name": "should get status 200", "skipped": false, "status": "FIXED" }], "failCount": 0, "name": "ExistingEp", "passCount": 1, "skipCount": 0 }Build 43:
{ "_class": "hudson.tasks.junit.ClassResult", "testActions": [ ], "child": [{ "testActions": [ ], "age": 0, "className": "Test /ep/GetEp.ExistingEp", "duration": 0.506, "failedSince": 0, "name": "should get status 200", "skipped": false, "status": "FIXED" }], "failCount": 0, "name": "ExistingEp", "passCount": 1, "skipCount": 0 }Build 42:
{ "_class": "hudson.tasks.junit.ClassResult", "testActions": [ ], "child": [{ "testActions": [ ], "age": 0, "className": "Test /ep/GetEp.ExistingEp", "duration": 0.472, "failedSince": 0, "name": "should get status 200", "skipped": false, "status": "FIXED" }], "failCount": 0, "name": "ExistingEp", "passCount": 1, "skipCount": 0 }