Github status check reported too early

Description

I’ve finally got a 4-job GitHub Actions CI workflow reporting all CodeCov reports correctly, as reported here: Codecov (it seems a little lower than the 80% I knew before but… closer than earlier attempts).

The PR is: add wait_for_ajax for barnstar system test by jywarren · Pull Request #9909 · publiclab/plots2 · GitHub

However the reported coverage in GitHub’s check is 59.59% (-22.56%).

If you look at the commits, that’s one behind the most recent assessment, so it skips the final merge commit listed at Codecov which would bring it up to 74%.

Commit SHAs

Repository

CI/CD or Build URL

Uploader

It’s uploading via the codecov gem. The results were the same using the CodeCov GitHub Action, which I tested earlier.

Codecov Output

In mine it’s done in 4 different jobs; but here is the final job:


  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _\`|/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                               Ruby-0.5.2
==> GitHub Actions detected
==> Gzipping contents
==> Uploading reports
    url:   https://codecov.io
    query: token=secret&flags&package=ruby-0.5.2&service=github-actions&branch=barnstar-wait&pr=9909&slug=publiclab%2Fplots2&build=1038545312&commit=8b6a73e9d7d7c44a8f5b4ea627f70192c64ecd15
->  Pinging Codecov
https://codecov.io/upload/v4?token=secret&flags&package=ruby-0.5.2&service=github-actions&branch=barnstar-wait&pr=9909&slug=publiclab%2Fplots2&build=1038545312&commit=8b6a73e9d7d7c44a8f5b4ea627f70192c64ecd15
->  Uploading to
https://storage.googleapis.com/codecov/v4/raw/2021-07-16/86C0015BA4E3819F6F955369719E30A5/450780ff05cb1cf5e64f27510dff36cadb200031/8a0820d2-e380-4d98-ae7f-e1f36a42b079.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EKKHVKCKHW7KBCGM7IHR55T63V2PAVJWLVFNITJHDU5G6R5IRN3LMWJA%2F20210716%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20210716T193342Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=614b99264492d96cd8f8f622e3a672814df130f25f44db1bba4d96fd883241f4
    View reports at https://codecov.io/github/publiclab/plots2/commit/450780ff05cb1cf5e64f27510dff36cadb200031

Expected Results

I would expect it to either wait until the final job is complete before reporting a status check, OR to update once it got the last report.

Actual Results

It seems to have reported a failure due to lower than expected coverage because it didn’t wait until the final job report uploaded, or it didn’t re-report after that.

Additional Information

For commenting, i set after_n_builds to 4, which is a really nice feature:

However i don’t see a way to tell CodeCov to wait on the status check until 4 builds. That would solve this problem.

Thank you, i love CodeCov!!!

1 Like

Hi @jywarren I think you’re looking for notify#after_n_builds

1 Like

That looks just right, thank you so much!!!

1 Like