PR Merge "Not Updating" Base Coverage - Misunderstanding

Support
1

Please search for similar issues before creating a ticket.
All fields below are required.

Description

Mainly just looking for some clarification on expected behavior of Codecov. I searched through the Codecov docs and this support forum and, to the best of my abilities, was unable to find a concrete answer.

My expectation of how Codecov works would be something like the following:

  • I connect a new repository to Codecov and by some means, whether it be manual or automated, upload a base report for the entire repository to Codecov
  • I make some changes on a branch and create a PR to main for those changes
  • Github Actions runs the effected tests and uploads the coverage report to Codecov. For this scenario, let’s say the change that was made increases the coverage by 5%
  • On merge of the PR to main, Codecov recognizes that the branch was merged and now uses the coverage report from the PR as the base coverage report.

However, it doesn’t appear like that last piece is happening. If I pull main and get the new code that increases the coverage by 5%, and then branch, make changes, push, and PR, the report that is uploaded also shows an increase of coverage by the same 5%. This would indicate to me that the base report was never updated to reflect the new coverage on main.

Is it expected that after merging to main, we once again run the tests, and upload the coverage report? Even though I just ran the same tests on the branch?

CI/CD or Build URL

Github

Uploader

Github Actions

2

@brandon-lile that is currently the expected behavior. On merges to main, we still expect coverage reports to be uploaded if that commit SHA is different as coverage numbers are tied to commits.

A way around that is to upload the coverage reports to some artifact and pull them down on merges to main (maybe this?) and use those (as opposed to re-running tests).

3

Thank you Tom! That is the confirmation I was looking for.

1 Like