Search

This is the use case:

In our company, we are using Gerrit and Jenkins to have each change set be tested by Jenkins before it being allowed to be merged into master.

We decided to start using a coverage tool (that interfaces with Cobertura) to also encourage test-writing. We would like to have the Cobertura Plugin fail the build if a metric falls below some threshold.

Now, the current Cobertura Plugin does have this, but it only allows to test against the "current" job running (ie, Job A runs, if the metrics are above Job A's set metrics, then it passes. Else, fails).

Now this doesn't work with our Gerrit-Jenkins set up due to the following use case:

Lets say that the Job's initial metrics is 30% and set to fail if it goes below it.

Developer A submits some amount of code and some amount of tests, resulting in an increase of coverage to 40%. 40% > 30% so the build passes and the threshold is increased to 40% for the Job. However, for some reason, Developer A does not submit to master (so the code is still gerrit, not in master. A common reason could be that he found a bug, or he is waiting for code review, or the feature is not ready, and they are using gerrit to share code).

At the same time, Developer B was writing code for some other feature. Her code is smaller and her tests are too, so her coverage is 35%. Since Developer A's code is not in master, her build SHOULD pass (master's coverage is 30%, not 40%), however, since it is the same Job running the coverage tool and the Cobertura Plugin, her build will fail (because 35% < 40%).

I would like to have some static coverage of master (maybe have a build only run when stuff gets into master) and then have every gerrit change set build compare their metrics against THAT build, not their own.

Does anyone know of a way of doing this in the current system? If not, could we get that feature in there?

Thanks.