Codecov applying wrong commit order (coverage showing as decreased when it has actually increased)

Support
1

Description

I have a number of commits on different branches which were created at different times. When these are added to my “main” branch, Codecov is ordering these by the commit date, rather than the order that these were applied to the main branch (via a “Rebase & Merge”). A PR that showed an overall increase to the project’s coverage now shows a decrease on the site.

CI/CD URL

This is the relevant PR on Github: Add better handling of widgets in unit tests by elParaguayo · Pull Request #7 · elParaguayo/qtile-extras · GitHub

Expected Results

I would expect the commit order on the dashboard to match the github log order.

github log --oneline gives me

596c365 (HEAD -> main, origin/main) Add tox requirements
b3ccd6a StravaWidget: handle FileNotFoundError
a3d9ba8 Change to codecov github acton with no token
...

Actual Results

Dashboard shows different order: Codecov

Additional Information

It seems clear that this is because of the commit date.

The “build time” is correct (Code coverage done right.) and would seem to be a better indicator of when this should be applied.

2

@elParaguayo I’m a little confused here, I think because I’m not seeing the 596c365 and b3ccd6a commits on Codecov. Could you share with me what the order you’re seeing on Codecov look like alongside what you think it should be?

3

@tom - Thanks for the reply.

I’m perfectly willing to accept that this problem is one of my own making and could come down to the fact that I’ve just used Codecov wrong!

Looking at the “main” branch on Codecov (Codecov), the “Recent Commits” shows:

Change to codecov github acton with no token

elParaguayo 4 days agomain a3d9ba8

Try codecov

elParaguayo 5 days agomain a861b58

Add tox requirements

elParaguayo 6 days agomain 596c365

In answer to your question about those commits, 596c365 is shown at the bottom of that list. b3ccd6a was a second commit that was part of the PR that included 596c365. They were both merged into the main branch.

In addition, the two commits that are shown above 596c365 (a3d9ba8 and a861b58) were both merged into main before 596c365 but are shown as having been merged afterwards).

Have I done something wrong? The commits were originally pushed to a different branch, a PR was submitted to merge them into main and they were then merged via a “rebase and merge”. Given these were all my own commits, I would have preferred to merge locally and just push to main but it seems that this didn’t get picked up by Codecov, hence the PR.

4

Following on from this, I pushed an update last night (Code coverage done right.) which should not have had any impact on coverage (just changing some documentation).

The graph shows the correct coverage level because this commit was correctly shown as being based on 596c365 but, because of the issue above, the graph shows that this commit increased coverage when it didn’t.

5

Hi @elParaguayo sorry for the delay here. I think what you are seeing is a difference in author date vs commit date. Currently there is no way to use one over the other, but I added a request to our product team to look into it. Is this blocking you?