Add option to fail commits on misses instead of percentage threshold.
Description
^ Looking there, it seems like the commit status depends on threshold. We generally don’t care about the percentage change, just that PR’s doesn’t introduce less coverage (ie. coverage misses).
Examples
The simplest example would be when removing some lines of code, then coverage% decreases. In our workflow, that’s completely fine as it doesn’t introduce misses, just less code. We don’t use coverage% that much.
Hi @bjornba, I’m not completely sure what you mean here. If you are looking for a status that will prevent coverage from dropping at all the link provided will do so. Setting target to auto will enforce that a new commit must have equal or higher coverage than the base commit. Threshold allows for some leniency around that number (0% means no leniency). Can you provide me an example of what you mean if this isn’t accurate?
Sure. The simplest example would be when removing some lines of code, then coverage% decreases. In our workflow, that’s completely fine as it doesn’t introduce misses, just less code. We don’t use coverage% that much.
Absolutely, I understand where you’re coming from here. Unfortunately based on the way we calculate coverage, your example falls into what we call coverage offset.
I’ve looped in the product team about this, but thanks for the request.