I have the codecov/project and codecov/patch status
set as required for protected branches in GitHub.
I have tried every combination of the codecov.yml
settings to achieve the following:
If for a given commit coverage is uploaded
all codecov checks must pass. If no coverage
is uploaded the status checks should be set
to “success” automatically. (IE. updating docs, README, etc).
In the codecov.yml reference
it says about the “if_not_found” setting:
" success : the status will pass if there is no report for the head. Use this on commits / PRs where you won’t be uploading coverage but still want codecov status checks to pass. "
So what I’m expecting is that codecov status checks
are set to success if no coverage is uploaded after
all actions are run. However, its always stays stuck
at "Expected — Waiting for status to be reported "
I’m thinking that maybe it has something to
do with “skipped” actions not being recognized as
hi @calvinalkan I’m not sure this is quite possible. I see that the if_not_found reference is misleading, as it is more specific to flags.
The reason that it’s still waiting on a status, is that Codecov is not made aware of the commit (since no uploads are sent up). So Codecov doesn’t send up notifications. A way around this would be to send up a blank coverage report.
There’re both if_not_found (Status Checks) and if_no_uploads (seen in some public gists), and both are not making status check green. If both references are just wrong, and the only way is to upload a blank report, I think there’s definitely something to fix inside codecov so we can finally update stuff not influencing code coverage.
Seems there’s one more thing until we get this working. After uploading the fake report json to codecov, my codecov will be stuck like below. Pushing another commit will get this to pass, but that new one will be stuck again. Forcing pushing notification will also work but it doesnt seem right.
In this case, what CI completion exactly is codecov waiting for? The fake report upload workflow is the only CI running here and it’s already complete. Note: in normal PRs where real report will be uploaded, this works totally fine so our GHE integration with codecov app is functioning.
Medium term: I agree this is a problem and a good feature request. It had been in the backlog from a couple months back but is starting to get more attention now as a feature to better integrate with Github.
Short term: As far as remediation before we can build around this, I’m posting a response from our team:
A) You need to be using carryforward flags. That way, we won’t have to worry about any large swings in code coverage. You need the entire set of their coverage reports uploaded under one flag
B) We still need a coverage report for our notification system to send out the PR comment/status check. So “blank report” is actually a misnomer, because it has to contain data. What I would suggest is to find a file that will always exist where the first line will always be covered, and upload that file like this named coverage.xml