Before submitting a topic, please confirm the following
I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my
codecov.yaml configuration file.
I have filled out the below sections to the best of my ability.
Say your main branch is on commit A and then two things happen. First, commit B lands in your main branch in a way that affects its coverage data. Coverage data for both commits is uploaded to codecov. Second, a PR is opened against main, but the first common commit between the PR and main is the commit A. In other words, the PR isn’t up-to-date with its target branch.
In this case, for a PR build, most CI systems would fetch
pull/<PR num>/merge from GitHub which is the result of merging that PR into the current state of its target branch. In this case, it would merge the PR into main at commit B and pick up the coverage changes from that commit.
I set up this scenario in a dummy repo at change something by bmw · Pull Request #3 · bmw/codecov-test · GitHub.
In the codecov report for the pull request, I’d expect only coverage changes that would take place as a result of that pull request to be reported.
When codecov computes coverage changes, it selects commit A as the base commit since it’s the first common commit between main and the PR branch. As a result, codecov will report that the PR will cause the change in coverage caused by commit B despite the PR having nothing to do with commit B.
It seems to me like codecov’s uploader should report that main was on commit B when CI ran and then compare the coverage change between the PR and commit B.
Assuming that’s not going to happen anytime soon, what’s the best way to solve this? Is it to manually set the
--parent option on the codecov uploader? Is it to force all PRs to be up-to-date with the target branch?