Upload of Coverage of several Python versions?

Support
1

Not really a support request, more like a general FAQ-like question:

So a bit of background: I am one of the maintainers of the aiosmtpd library. We are currently transitioning from Travis CI to GitHub Actions (GA). One of the steps in the GA Workflow we’ve made is the uploading of the coverage testing result to codecov.io.

Now since this is a FOSS library recommended for use by Python documentation, naturally we have to test it against several Python versions & implementations (currently 3.6 to 3.9, inclusive, and PyPy3). We implement the testing using tox which sets things up and perform the actual unit testing & coverage testing (using pytest & pytest-cov).

This resulted in several coverage*.xml files, one per each Python version/implementation.

So my questions:

  1. Should we upload all of the XML files (in parallel through several GA jobs) or just one?
  2. If we upload all of them, what will happen with the coverage report? Will Codecov merge them additively, or provide separate reports based on the version/implementation, or both? Or?
  3. Should I somehow tag/flag/mark the uploads separately? And if so, how?

Thank you in advance.

2

Hi @pepoluan, glad to help here.

  1. Yes, I would recommend uploading all of them, but on different jobs
    2+3. The coverage report will merge them all additively. However, you can use flags to designate different versions and implementations. You can configure the GitHub comment and status checks to show up by flag.

Let me know if this helps!