Code showing as not covered despite coverage in workflows recorded as 100%


Coverage in one of the PRs in our repo is not being correctly reported. The test suite logs show 100% coverage of the problematic file (and we obtain 100% coverage running locally), but codecov is reporting lines in the file as untested.

Commit SHAs

3d9c2b5285cf77cc289fd0a96e152fd90e1564a6 (most recent)

Link to the branch
Link to the open PR


CI/CD or Build URL

Link to the folder on the codecov page

Text output of the build.


Running via Github Actions:

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v2
          file: ./coverage.xml

The full workflow file is available here.

Codecov Output

(This is from the core-tests (3.9) job; results are comparable for other jobs).

Run codecov/codecov-action@v2
    file: ./coverage.xml
    TF_VERSION: 2.4
    COVERAGE_FLAGS: --cov=pennylane --cov-report=term-missing --cov-report=xml --no-flaky-report -p no:warnings --tb=native
    pythonLocation: /opt/hostedtoolcache/Python/3.9.6/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.6/x64/lib
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (d577b7c828ebd0a51ad4b398547d188e839440ac045c686b303c8fe16d2858a6  codecov)
/home/runner/work/_actions/codecov/codecov-action/v2/dist/codecov -n  -Q github-action-2.0.2 -f ./coverage.xml -C 3d9c2b5285cf77cc289fd0a96e152fd90e1564a6
[2021-08-04T12:44:26.031Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/

  Codecov report uploader 0.1.0
[2021-08-04T12:44:26.037Z] ['info'] => Project root located at: /home/runner/work/pennylane/pennylane
[2021-08-04T12:44:26.040Z] ['info'] -> No token specified or token is empty
[2021-08-04T12:44:26.072Z] ['info'] Processing ./coverage.xml...
[2021-08-04T12:44:26.085Z] ['info'] Detected GitHub Actions as the CI provider.
[2021-08-04T12:44:26.086Z] ['info'] Pinging Codecov:*******&branch=ch7122-add-a-top-level-qml-compile-function-with&build=1097705386&
[2021-08-04T12:44:26.681Z] ['info'] Uploading...
[2021-08-04T12:44:26.872Z] ['info'] {"status":"success","resultURL":""}

Expected Results

The file pennylane/transforms/ is fully tested in tests/transforms/, and we expect to see 100% coverage based on local results and results of each job in the workflow.

Actual Results

Coverage is shown as missing on most lines in the file. See details in patch and project.

Additional Information

This seems to be the only PR we are having issues with. We have tried a number of things already to fix the issue: switching from v1 to v2 of codecov in the yml file; placement of tests both within and outside of a class; changing the names of the tests to include the function name; making a new PR with the full set of changes to rule out an issue in commit history; changing the way the module is imported, etc.

1 Like

@glassnotes this seems to be accurate given the coverage reports we are being sent.

You can see the uploads. Doing a search on every report, you will see either

<class name="" filename="pennylane/transforms/" complexity="0" line-rate="1" branch-rate="0">
    <line number="14" hits="1"/>
    <line number="16" hits="1"/>
    <line number="18" hits="1"/>
    <line number="19" hits="1"/>
    <line number="20" hits="1"/>
    <line number="22" hits="1"/>
    <line number="23" hits="1"/>
    <line number="26" hits="1"/>
    <line number="29" hits="1"/>


<class branch-rate="0" complexity="0" filename="/usr/share/miniconda/lib/python3.7/site-packages/pennylane/transforms/" line-rate="0.3333" name="">
    <line hits="1" number="14"/>
    <line hits="1" number="16"/>
    <line hits="1" number="18"/>
    <line hits="1" number="19"/>
    <line hits="1" number="20"/>
    <line hits="1" number="22"/>
    <line hits="1" number="23"/>
    <line hits="1" number="26"/>
    <line hits="1" number="29"/>
    <line hits="1" number="30"/>
    <line hits="0" number="128"/>
    <line hits="0" number="129"/>
    <line hits="0" number="131"/>
    <line hits="0" number="132"/>
    <line hits="0" number="133"/>
    <line hits="0" number="134"/>
    <line hits="0" number="136"/>
    <line hits="0" number="137"/>
    <line hits="0" number="143"/>
    <line hits="0" number="145"/>
    <line hits="0" number="146"/>
    <line hits="0" number="147"/>
    <line hits="0" number="152"/>
    <line hits="0" number="155"/>
    <line hits="0" number="156"/>
    <line hits="0" number="157"/>
    <line hits="0" number="158"/>
    <line hits="0" number="160"/>
    <line hits="0" number="163"/>
    <line hits="0" number="164"/>

This would account for the misses after line 30