Better detection of high level coverage changes

[cinderblock]

Summary

Adding code to coverage reports makes it look like we’re going backwards instead of showing that we’re covering more.

Description

I have a project with existing coverage data for JavaScript sources. I’m adding coverage tests for the C++ sources as well. Combining the new coverage info shows the absolute coverage has decreased dramatically (because the C++ code is less covered on average than the JS). However I have only added lines of code to coverage. So coverage has not truly decreased.

Now I’m looking to merge my high level project changes and the codecov CI check is blocking merging. I realize an admin should be able to bypass this check, but it would be nice if this check was a little more “soft”.

As it stands, this also slightly addresses the issue of simply adding a lot of code to the codebase. I appreciate the goal of keeping the percent coverage on an upward trend but that can be a detriment to adding useful features.

My intent here is just to spark this discussion if it hasn’t already been addressed. Maybe an option doesn’t make sense but adjusting the visuals/graphs to more accurately reflect the changes might be nice. For instance, the sunburst/icicle graphs could more clearly reflect the blocks of code that were added to coverage.

Examples

Generate C++ coverage reports too!