Incorrect repots merging


In my project, I use Cypress and Jest to run E2E and unit tests suites.
I have a Github workflows to run those tests and the two coverage are uploaded to codecov by the action.

When I look at Codecov report on the app, some lines are reported as uncovered (example: VPhoneInput.vue line 497.

When I download the uploaded coverage from my CI execution and check the unit test report for the same file and line, there are hits for the statement.

Expected Results

The line should be marked as covered by codecov too.

Actual Results

It is not covered in the codecov report.

Additional Information

Before using codecov, I was having similar problems with NYC report merging tools. This is probably due to the Codecov report merging tools.

@paul-thebaud, this is actually coming from your e2e tests.

I forked the repo and uploaded just the unit tests, and you can see coverage for VPhoneInput.vue line 497 is not touched.

However, when I only upload e2e tests, you see that line 497 is showing as not covered.

If you dig into the coverage report, you’ll see that file path=coverage-e2e/coverage-final.json, shows two bits of information that is useful.

  1. The coverage report notes the lines starting at 497 as a branch (as it is in this ternary)
  2. That branch (0) is never run

Hi @tom, thanks for the quick reply.

I don’t understand. How could a map callback could be touched without touching the map call itself?
And since some lines are covered by one part of tests (eg. E2E), why is this not getting merged with other to get a full cover?

Thanks for your time.

I’m not sure because I’m not familiar enough with how cypress collects coverage. What happens if you run e2e tests only and spit out the HTML file as described here? I suspect you will see line 497 being uncovered.

I’m not sure I understand the question here. Line 497 does not have coverage information from unit tests, only E2E. So on the merge, we would take the result from E2E.

Hi @tom,

Ok, I think I understand the problem: it looks like Jest won’t manage map, filter, etc. as branch where Cypress does. This will result in incorrect coverage because my unit tests covers only the filter part where E2E covers only the other part (see the attached screen captures).

This does no seems to be related to Codecov, so I’m closing the issue and will try to improve the coverage with this difference in mind!

Thanks again for your help.

Codecov capture

E2E with Cypress capture

Unit with Jest capture

1 Like