Description
Coverage in one of the PRs in our repo is not being correctly reported. The test suite logs show 100% coverage of the problematic file (and we obtain 100% coverage running locally), but codecov is reporting lines in the file as untested.
Commit SHAs
3d9c2b5285cf77cc289fd0a96e152fd90e1564a6 (most recent)
Link to the branch
Link to the open PR
Repository
CI/CD or Build URL
Link to the folder on the codecov page
Uploader
Running via Github Actions:
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
file: ./coverage.xml
The full workflow file is available here.
Codecov Output
(This is from the core-tests (3.9) job; results are comparable for other jobs).
Run codecov/codecov-action@v2
with:
file: ./coverage.xml
env:
TF_VERSION: 2.4
TORCH_VERSION: 1.7
COVERAGE_FLAGS: --cov=pennylane --cov-report=term-missing --cov-report=xml --no-flaky-report -p no:warnings --tb=native
pythonLocation: /opt/hostedtoolcache/Python/3.9.6/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.6/x64/lib
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (d577b7c828ebd0a51ad4b398547d188e839440ac045c686b303c8fe16d2858a6 codecov)
/home/runner/work/_actions/codecov/codecov-action/v2/dist/codecov -n -Q github-action-2.0.2 -f ./coverage.xml -C 3d9c2b5285cf77cc289fd0a96e152fd90e1564a6
[2021-08-04T12:44:26.031Z] ['info']
_____ _
/ ____| | |
| | ___ __| | ___ ___ _____ __
| | / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| | __/ (_| (_) \ V /
\_____\___/ \__,_|\___|\___\___/ \_/
Codecov report uploader 0.1.0
[2021-08-04T12:44:26.037Z] ['info'] => Project root located at: /home/runner/work/pennylane/pennylane
[2021-08-04T12:44:26.040Z] ['info'] -> No token specified or token is empty
[2021-08-04T12:44:26.072Z] ['info'] Processing ./coverage.xml...
[2021-08-04T12:44:26.085Z] ['info'] Detected GitHub Actions as the CI provider.
[2021-08-04T12:44:26.086Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-2.0.2-uploader-0.1.0&token=*******&branch=ch7122-add-a-top-level-qml-compile-function-with&build=1097705386&build_url=https%3A%2F%2Fgithub.com%2FPennyLaneAI%2Fpennylane%2Factions%2Fruns%2F1097705386&commit=3d9c2b5285cf77cc289fd0a96e152fd90e1564a6&job=Tests&pr=1475&service=github-actions&slug=PennyLaneAI/pennylane&name=&tag=&flags=&parent=
[2021-08-04T12:44:26.681Z] ['info'] Uploading...
[2021-08-04T12:44:26.872Z] ['info'] {"status":"success","resultURL":"https://codecov.io/github/PennyLaneAI/pennylane/commit/3d9c2b5285cf77cc289fd0a96e152fd90e1564a6"}
Expected Results
The file pennylane/transforms/compile.py is fully tested in tests/transforms/test_compile.py, and we expect to see 100% coverage based on local results and results of each job in the workflow.
Actual Results
Coverage is shown as missing on most lines in the file. See details in patch and project.
Additional Information
This seems to be the only PR we are having issues with. We have tried a number of things already to fix the issue: switching from v1 to v2 of codecov in the yml file; placement of tests both within and outside of a class; changing the names of the tests to include the function name; making a new PR with the full set of changes to rule out an issue in commit history; changing the way the module is imported, etc.