I maintain an open source Python library,
spaghetti, and we have recently migrated our testing+coverage from TravisCI/Appveyor+Coveralls to GitHub Actions+Codecov. I am very much happy with our choice to adopt Codecov, but I am having some trouble getting our
codecov.yml to function properly (though this is most definitely my misunderstanding the parameters). Basically, what I would like to have is a delayed report to PRs until at least 24 jobs with the GitHub Actions matrix are complete (see matrix below). This is because the tests with only core dependencies skip ~5% of the coverage. Also I would like to increase the threshold from the default (0%) to 2%. I thought I had set these parameters correctly in the
codecov.yml, but it doesn’t seem to be kicking in. Any review of the
.yml and dive on how to fix it would be greatly appreciated.
Our testing matrix is:
- MacOS, Ubuntu, Windows
- Python 3.6, 3.7, 3.8
- core and optional dependencies
3 after_n_builds: 24
5 range: 50..95
6 round: nearest
7 precision: 1
11 threshold: 2%
14 threshold: 2%
15 target: 80%
17 - "tests/*"
19 layout: "reach, diff, files"
21 - *
22 behavior: once
23 after_n_builds: 24
24 require_changes: true
Do you mind trying that and seeing how it works for you?
@tom Thanks for this! However, I tried it (see
codecov is still immediately posting after the first build is complete
- the precision is not
Any other thoughts as to what I am doing incorrectly? Maybe some setting I have missed in either of the
spaghetti GitHub or Codecov repos?
Hi @jGaboardi, this is my mistake, the yaml I gave you wasn’t fully valid
layout: "reach, diff, files"
should be right (branches does not need to be specified if it’s all of them)
Nice! I’ll give this a try!
@tom After implementing this
codecov.yml, CodeCov is still commenting on the PR directly after the first build is complete, not waiting until 24 builds are complete. Maybe I am not understanding how the
after_n_builds should behave. My interpretation is that after a PR is made CodeCov should wait until 24 builds (in my case) out a matrix of builds are completed before reporting the coverage/changes to the GitHub PR. Is this incorrect?
OMG! That was such a careless mistake. Thanks so much, this fixed the issue perfectly!
Happens to the best of us , glad it worked for you!