I suspect that there is some issue with parsing that is not caught by the validation check. However, I have not be able to access any traceback from parsing.
My question is then kind of double, first it would be great if someone could help and point me what I am missing in order to fix this. Secondly, it would be eve better to get some guidance (or be pointed to any resource) on how to troubleshoot this kind of issues by myself. I am open to help on this once I know how to do it.
Here is a PR where CI has run ONLY ONCE, and eve though codecov.yml says to wait until 5 builds were finished, the bot commented and edited as new builds were finishing.
Extra checks I have done:
make sure codecov.yml is in repo main directory
make sure yaml file is named codecov.yml and not .codecov.yml
The validate endpoint is the best bet, but it’s not currently updated (should be very soon) and as such can not be fully trusted.
In the case of your b1316f41e3cfd34fe66196a98c1ca11d4a2745e6 commit, the YAML has the following error: after_n_builds: 5 is not a valid key under comment, it only works under notify (and affects all types of notify, so no need to get more detailed)
If things are still wrong after this fix, please share a commit SHA and I’ll check the logs again.
commit 4c8e16e91c405091efe0b306f8c752387175657d may still have after_n_builds in codecov (due to forgotten rebase) but both comment and check are waiting! 3 builds have finished (according to codecov, 5 have actually finished), but no message whatsoever
How can builds finished hours ago still not appear in codecov? codecov page for commit 4c8e16e... still says " Notifications are pending CI completion. Waiting for GitHub’s status webhook to queue notifications."
I have opened a PR only for troubleshooting this so it won’t get merged because of the other changes.
I found an error message in one of the builds where not all reports were uploaded! It actually is in the base commit of the PR linked above, which is why we get a nearly 2% coverage increase by adding a name to the uploaded codecov builds. Would using --required flag solve the issue by failing CI build when codecov upload fails? Or is there a better way to tackle this problem?
I am copying the error message, for the full traceback and context see link:
Error: HTTPSConnectionPool(host='codecov.io', port=443): Max retries exceeded with url: /codecov/v4/raw/2020-03-06/2C016F20EC330FC563151DA316E11499/00c6d5c057944966e765399d1508181b89c1ce3d/8abc87f8-533f-42ca-b73d-fa182d6ef3f0.txt?AWSAccessKeyId=AKIAIHLZSCQCS4WIHD4A&Expires=1583500724&Signature=%2FGe0uHJMOuRL06J0nbZCI9uPdLs%3D (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fea47cab6d0>: Failed to establish a new connection: [Errno 110] Connection timed out'))
I there any way to lower the number of times this error happens? I am not sure if this is an issue with azure sending the reports or with codecov receiving them. And if possible we’d like to avoid rerunning the whole job to reupload coverage to codecov.
We are using python uploader, would using the bash make any difference?
Note: Just to be extra clear with what I said above, we don’t really know at all the reason why the uploads are failing nor have any idea about how to investigate the reason, so any help on this direction is greatly appreciated too.
Changed to using the bash uploader, there is no comparison between them! The bash uploader retries automatically to reupload results when there is a connexion error, the verbose flag does print useful info, thanks for the pointer!
Is there somewhere in the docs discouraging the use of the Python uploader? Maybe there should?