Hello, I've been struggling with a problem related to testng-plugin, that consist of making the entire pipeline to be marked as UNSTABLE even when a retry option is used.

Example:

pipeline {
  ...
  stages {    
    stage('Execute the tests') {
      options {
        retry(10)
      }
      stages {
        stage('Cleanup last test results') {
          ...
        }
        stage('Run ant tests') {
          steps {
            container('jdk11') {
              withAnt(installation: 'ant') {
                sh """
                  export ANT_OPTS="-Xmx2048m"
                  ant ...
                """
              }
            }
          }
          post {
            always {
              step(
                [ $class: 'Publisher',
                  reportFilenamePattern: 'test-output/testng-results.xml'
                ]
              )
              ...
            }
            unstable {
              timeout(time: 30, unit: 'SECONDS') {
                echo 'Pipeline finished with unstable result. Retrying...'
                sleep 35
              }
            }
          }
        }
      }
    }
  }
}

Expected result:

• 1st execution: 1 test successful and 1 test failed -> Marked as UNSTABLE -> retry due to post: unstable
• 2nd execution: 1 test skipped (previously succeeded) and 1 successful -> Marked as SUCCESS

Current result:

• 1st execution: 1 test successful and 1 test failed -> Marked as UNSTABLE -> retry due to post: unstable
• 2nd execution: 1 test skipped (previously succeeded) and 1 successful -> Marked as UNSTABLE -> retry due to post: unstable
• 3rd execution: 2/2 tests skipped (previously succeeded) -> Marked as UNSTABLE -> retry due to post: unstable
• ... 10 times as set per retry limit

I guess the problem here is that the plugin is using inside the Publisher.java class the method .setResult(...); to set the build result without using the new API WarningAction from Pipeline: Basic Steps Plugin.

• New API in Pipeline: API Plugin 2.34

I would be very glad if it could be fixed. Thank you!