Help with codecov.yml

Support
,
1

Description

I maintain an open source Python library, spaghetti, and we have recently migrated our testing+coverage from TravisCI/Appveyor+Coveralls to GitHub Actions+Codecov. I am very much happy with our choice to adopt Codecov, but I am having some trouble getting our codecov.yml to function properly (though this is most definitely my misunderstanding the parameters). Basically, what I would like to have is a delayed report to PRs until at least 24 jobs with the GitHub Actions matrix are complete (see matrix below). This is because the tests with only core dependencies skip ~5% of the coverage. Also I would like to increase the threshold from the default (0%) to 2%. I thought I had set these parameters correctly in the codecov.yml, but it doesn’t seem to be kicking in. Any review of the .yml and dive on how to fix it would be greatly appreciated.

Our testing matrix is:

  • MacOS, Ubuntu, Windows
  • Python 3.6, 3.7, 3.8
  • core and optional dependencies

Relevant files:

Repository

2

Hi @jGaboardi,

  1 codecov:
  2   notify:
  3    after_n_builds: 24
  4 coverage:
  5   range: 50..95
  6   round: nearest
  7   precision: 1
  8   status:
  9     project:
 10       default:
 11         threshold: 2%
 12     patch:
 13       default:
 14         threshold: 2%
 15         target: 80%
 16   ignore:
 17     - "tests/*"
 18 comment:
 19   layout: "reach, diff, files"
 20   branches:
 21     - *
 22   behavior: once
 23   after_n_builds: 24
 24   require_changes: true

Do you mind trying that and seeing how it works for you?

3

@tom Thanks for this! However, I tried it (see pysal/spaghetti#479) and:

  • codecov is still immediately posting after the first build is complete
  • the precision is not 1

Any other thoughts as to what I am doing incorrectly? Maybe some setting I have missed in either of the spaghetti GitHub or Codecov repos?

4

Hi @jGaboardi, this is my mistake, the yaml I gave you wasn’t fully valid

codecov:
  notify:
   after_n_builds: 24
coverage:
  range: 50..95
  round: nearest
  precision: 1
  status:
    project:
      default:
        threshold: 2%
    patch:
      default:
        threshold: 2%
        target: 80%
  ignore:
    - "tests/*"
comment:
  layout: "reach, diff, files"
  behavior: once
  after_n_builds: 24
  require_changes: true

should be right (branches does not need to be specified if it’s all of them)

5

Nice! I’ll give this a try!

6

@tom After implementing this codecov.yml, CodeCov is still commenting on the PR directly after the first build is complete, not waiting until 24 builds are complete. Maybe I am not understanding how the after_n_builds should behave. My interpretation is that after a PR is made CodeCov should wait until 24 builds (in my case) out a matrix of builds are completed before reporting the coverage/changes to the GitHub PR. Is this incorrect?

7

Hi @jGaboardi,

Looks like you might not have indented properly here: spaghetti/codecov.yml at main · pysal/spaghetti · GitHub

8

OMG! That was such a careless mistake. Thanks so much, this fixed the issue perfectly!

1 Like
9

Happens to the best of us :joy:, glad it worked for you!

1 Like