Description
Codecov_io is very much useless for our repo, no two runs give the same coverage, even with a repeated force-push with no content changes, variance is about ±2%.
- Clicking a PR link [1] either says the report could not be found in the archives, or it gives a 500 error.
- Checking the commits [2] on codecov_io often shows some of the commits in an endless “Reports are queued for processing…” state (Even many months later).
- Github comment [3] only very very very rarely (1% or so?) shows a correct looking report for each flag. Usual behavior is it only shows a coverage percentage for 5-6 of the flags (~30 flags total), or it shows correct-looking percentage for 5-6 flags and 100% for all the rest (and we know that is impossible). Sometimes 5-6 correct and a couple 100% with the rest empty.
1: Codecov
2: https://codecov.io/gh/zlib-ng/zlib-ng/commit/af399ea6c309dd8fbb2ce107115fa7020c388e6d/graphs
3: Codecov Improvements by Dead2 · Pull Request #523 · zlib-ng/zlib-ng · GitHub
We have made sure we don’t have any codecov.yaml file, and not anything filled out for that in the project config either.
We have tried deleting all project data on codecov_io (that is why there is still very little data there now).
We have often had upload errors, to the point we forked codecov-python, but even with the rare occasion of all uploads succeeding the things above don’t work.
All permissions in github etc have been checked repeatedly.
We currently use Github Actions for CI builds, but used to use Travis CI with the exact same problems (In fact we switched because we thought it might fix the upload problems).
Our CI builds run from Linux, MacOS and Windows. gcc and clang. x86, x86_64, arm, aarch64, ppc, ppc64 and s390x.
We first tried to solve these problems ~a year ago, but figured it was probably a codecov_io bug that might be fixed in their next release, but it seems it is rare enough that it hasn’t been fixed yet. That was when we changed over from Travis to Github Actions too actually.
Less important, but likely to be related; The sourcegraph integration does not seem to work properly for our repo. It works when viewing other repos using codecov on github, but in ours it won’t show coverage data in the header or hit counts. Browsing our code through sourcegraph_com shows hit counts per line perhaps once in a blue moon (I have seen it work once, but then it broke when I switched to another file, going back to the first file was then also broken).
We have followed the docs repeatedly and checked all permissions etc.
I am not allowed to post another link.
Repository
Versions
Not relevant, but Win10+Chrome, Win10+Waterfox, Fedora 30+Chrome, Fedora 30+Waterfox
