-
Bug
-
Resolution: Fixed
-
Minor
-
None
Jacoco delta threshold got failure if some coverage metrics decreases less than delta threshold but some increases over delta threshold.
Problematic code is
JacocoPublisher.java
public Result checkBuildOverBuildResult(Run<?,?> run, PrintStream logger){ JacocoDeltaCoverageResultSummary deltaCoverageResultSummary = JacocoDeltaCoverageResultSummary.build(run); logger.println("[JaCoCo plugin] Delta coverage: class: " + deltaCoverageResultSummary.getClassCoverage() + ", method: " + deltaCoverageResultSummary.getMethodCoverage() + ", line: " + deltaCoverageResultSummary.getLineCoverage() + ", branch: " + deltaCoverageResultSummary.getBranchCoverage() + ", instruction: " + deltaCoverageResultSummary.getInstructionCoverage() + ", complexity: " + deltaCoverageResultSummary.getComplexityCoverage()); if(Math.abs(deltaCoverageResultSummary.getInstructionCoverage()) <= deltaHealthReport.getDeltaInstruction() && Math.abs(deltaCoverageResultSummary.getBranchCoverage()) <= deltaHealthReport.getDeltaBranch() && Math.abs(deltaCoverageResultSummary.getComplexityCoverage()) <= deltaHealthReport.getDeltaComplexity() && Math.abs(deltaCoverageResultSummary.getLineCoverage()) <= deltaHealthReport.getDeltaLine() && Math.abs(deltaCoverageResultSummary.getMethodCoverage()) <= deltaHealthReport.getDeltaMethod() && Math.abs(deltaCoverageResultSummary.getClassCoverage()) <= deltaHealthReport.getDeltaClass()) return Result.SUCCESS; else if(deltaCoverageResultSummary.isCoverageBetterThanPrevious()) return Result.SUCCESS; else return Result.FAILURE; }
The above code uses abs. This means that check not make any difference of increase or decrease. If not all metrics are increased then isCoverageBetterThanPrevious gives false. so Despite all decreases are below delta threshold build fails if some of increase more than delta. This is not logical because coverage increase should not be measured against delta at all.
[JENKINS-58184] Jacoco delta threshold failures if coverage *increases* over threshold
centic
added a comment - this was contributed, so I am not sure what the original intent was on these checks, however I also would not expect a failure when coverage actually increases. Can you provide a PR which simplifies/improves this handling?
centic
added a comment - this was contributed, so I am not sure what the original intent was on these checks, however I also would not expect a failure when coverage actually increases. Can you provide a PR which simplifies/improves this handling? centic I am also facing the same issue where a positive increase in coverage is failing the build. I will raise a PR for the same.
centic since this PR has been merged should I mark this issue to resolved? I just want to make sure that I am following the correct process.
centic
added a comment - Yes, I usually resolve them with the next "released as" version that is planned, currently 3.1 and a comment that it will be included in the next release.
centic
added a comment - Yes, I usually resolve them with the next "released as" version that is planned, currently 3.1 and a comment that it will be included in the next release. 
Why not check before abs that coverage delta is negative? It should be ok to ignore positive delta from that check.