Relative coverage increase

Support
1

Hi,
I want to use codecov in the ceph project, but I got struggled with the following message that is shown as the codecov job passes but it shouldn’t because it’s decreasing the coverage.

Absolute coverage decreased by -0.8266% but relative coverage increased by +2.3331% compared to 6a9d458

This is my current testing PR.

The current report looks like shown in the picture below:

codecov_2019-07-12_14-43-30
codecov_2019-07-12_14-43-30699×809 59.9 KB

My current config looks like this:

coverage:
  precision: 4
  range: "30..90"
  status:
    changes: no
    project:
      default: off
      dashboard:
        target: auto
        threshold: 0%
        base: auto
        flags: dashboard
        path: "src/pybind/mgr/dashboard/"
    patch:
      default: off
      dashboard:
        target: 50%
        threshold: 0%
        base: auto

comment:
  require_changes: yes

How can I use the absolute coverage value instead of the relative one?

Thanks in advance :slight_smile:

2

@Devp00l Did you ever figure out how to get this to work? I’m in the exact same situation.

It seems the default (and only?) comparison method is to use the relative delta rather than the absolute delta. This is especially problematic if trying to use the auto target to use the base branch as the threshold to compare against. It pulls in the project’s absolute coverage as a base, then compares the relative diff coverage to that. Depending on which parts of the project were updated this value may be significantly higher or lower than the entire project and provides zero value whatsoever.

In other words, what I want to do is ensure that the coverage across the project never goes down. I want to ensure that the delta change value is always nonnegative.

Wondering if this has something to do with patch vs project. The yaml file defines the configuration as a project, but the status getting posted back to github is “codecov/patch.”

3

Hey @Devp00l and @noahw3, sorry for lengthy delay on this. To be clear, Codecov supports either relative changes in coverage OR absolute changes.

@noahw3, you are correct. Project coverage indeed is the coverage of the whole repo, setting the threshold to 0% would ensure that the affect of coverage on repo on the PR is always >=0%.

The patch coverage, on the other hand, is only analyzing how covered the changes to the code you are making e.g., “leave the world better than you found it, but don’t worry about overall repo coverage.”

Let me know if this helps!