• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Component/s: core
• Labels:

  • tests

[JENKINS-69650] Fix test reporting of Security2779Test

Alexander Brandes created issue - 2022-09-20 13:42

Mark Waite made changes - 2022-09-20 22:22

Remote Link

New: This issue links to "PR 7134 for easier diagnosis of the failure (Web Link)" [ 28233 ]

Jesse Glick made changes - 2022-09-21 15:49

Assignee

New: Mark Waite [ markewaite ]

Jesse Glick made changes - 2022-09-21 15:49

Status

Original: Open [ 1 ]

New: In Progress [ 3 ]

Jesse Glick made changes - 2022-09-21 15:49

Status

Original: In Progress [ 3 ]

New: In Review [ 10005 ]

Mark Waite made changes - 2022-09-21 17:56

Description

Original: Security2779Test#noXssInHelp() is flaky, according to the build logs: [build log on ci.jenkins.io|https://ci.jenkins.io/job/Core/job/jenkins/job/master/lastSuccessfulBuild/consoleText] The first test passes, but the following five fail: {code} [2022-09-20T02:56:55.267Z] [WARNING] Flakes: [2022-09-20T02:56:55.267Z] [WARNING] Security2779Test.noXssInHelp [2022-09-20T02:56:55.267Z] [INFO] Run 1: PASS [2022-09-20T02:56:55.267Z] [ERROR] Run 2: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 3: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 4: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 5: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 6: expected:<0> but was:<1> {code}

New: Security2779Test#noXssInHelp() is flaky, according to the build logs: [build log on ci.jenkins.io|https://ci.jenkins.io/job/Core/job/jenkins/job/master/lastSuccessfulBuild/consoleText] The first test passes, but the following five fail: {code} [2022-09-20T02:56:55.267Z] [WARNING] Flakes: [2022-09-20T02:56:55.267Z] [WARNING] Security2779Test.noXssInHelp [2022-09-20T02:56:55.267Z] [INFO] Run 1: PASS [2022-09-20T02:56:55.267Z] [ERROR] Run 2: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 3: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 4: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 5: expected:<0> but was:<1> [2022-09-20T02:56:55.267Z] [ERROR] Run 6: expected:<0> but was:<1> {code} Improve the test so that it accurately reports the state of the code rather than incorrectly reporting a real failure as a "flaky test". The test is not unreliable. It was correctly identifying a failing condition. The Maven surefire plugin seems to incorrectly classify the test as unreliable.

Mark Waite made changes - 2022-09-21 17:57

Summary

Original: Flaky test in Security2779Test#noXssInHelp()

New: Fix test reporting of Security2779Test

Alexander Brandes made changes - 2022-09-27 12:23

Released As		New: 2.371
Resolution		New: Fixed [ 1 ]
Status	Original: In Review [ 10005 ]	New: Fixed but Unreleased [ 10203 ]

Mark Waite made changes - 2022-09-28 19:31

Status

Original: Fixed but Unreleased [ 10203 ]

New: Closed [ 6 ]