Search

I like the idea, make a pull request when you think you are finished.

moved to separate repository for a standalone plugin: https://github.com/samsta/quarantine

is someone still working on that or is there a different solution for muting specific failing tests?

donatello, the quarantine plugin at https://github.com/samsta/quarantine is functional enough for me and we're using it extensively at work. i failed getting the answers i needed from the developers' list that would allow me to bring it to a stage where it's good enough to be hosted on the jenkins plugin repository. since it's good enough for me i didn't bother to keep nagging them. feel free to suggest improvements or even make them yourself!

here's what i posted on the jenkins developers list a while ago. if someone could give me a few hints i might be able to move this forward:

hello everyone

i have created a plugin that adds the ability to quarantine a test case that fails intermittently:

https://github.com/samsta/quarantine

we are using it on our jenkins instance, but it has one major area of concern. because the build status can't be made better, but only worse, i can't just run another step after the test result archiver, so i had to piggy-back functionality on a sub-class of JUnitResultArchiver:

https://github.com/samsta/quarantine/blob/master/src/main/java/org/jenkinsci/plugins/quarantine/QuarantinableJUnitResultArchiver.java

personally, i do not like this approach because of the following issues:

• it is brittle
• it is confusing and error prone in the job configuration
• it is too tightly coupled to the junit result format

i would really like to resolve this before releasing this plugin into the wider world as i am not really willing to provide support, knowing of these architectural deficiencies.

can anyone suggest a way forward, please? all ideas i have so far require changes in the core:

• add an extension point that allows to assess test results after the archiver has run
  • this would mean that the archiver is no longer making the decision about the build status
  • not really backwards compatible
• drop the requirement that builds can only be made worse
  • yuck! potentially opens a can of worms
• allow to modify a test result from within a TestAction or TestDataPublisher, either using the skipped state or by adding a new ignored/quarantined state
  • test result has been immutable so far
  • a fair bit of work in TestResult and friends, but these could do with some work because the relation between the hudson.tasks.test.TestResult and hudson.tasks.junit.TestResult is a bit confusing

cheers,
sam

This issue/feature should deserve some more attention. This is a great plugin which needs some support.

Any update on this? It looks like the plugin has fallen out of date but it'd be a really helpful feature to have

No update. We're using the plugin as is and it works fine for what we need it. Just build it from source: https://github.com/samsta/quarantine

samsta - do you know of any way to make the Quarantine Plugin work in pipelines? Currently it freezes. Easy way to reproduce:

node { 
  def mvnHome 
  stage('Preparation') { // for display purposes 
     // Get some code from a GitHub repository 
     git 'https://github.com/jglick/simple-maven-project-with-tests.git' 
     // Get the Maven tool. 
     // ** NOTE: This 'M3' Maven tool must be configured 
     // **       in the global configuration.            
     mvnHome = tool 'M3' 
  } 
  stage('Build') { 
     // Run the maven build 
     if (isUnix()) { 
        sh "'${mvnHome}/bin/mvn' -Dmaven.test.failure.ignore clean package" 
     } else { 
        bat(/"${mvnHome}\bin\mvn" -Dmaven.test.failure.ignore clean package/) 
     } 
  } 
  stage('Results') { 
     step([$class: 'QuarantinableJUnitResultArchiver', testDataPublishers: [[$class: 'QuarantineTestDataPublisher']], testResults: '**/target/surefire-reports/TEST-*.xml']
) 
    // junit '**/target/surefire-reports/TEST-*.xml' 

  } 
}

mireksz, sorry, no idea. I have never used pipelines before