Coverage result not correct for yarn2 monorepo

Support
1

Description

When I change my project to use yarn PnP for the monorepo, one of the files code coverage goes from 100% to 33%.

The actual coverage is 100% in the CI:

just-func:coverage: ----------------|---------|----------|---------|---------|-------------------
just-func:coverage: File            | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
just-func:coverage: ----------------|---------|----------|---------|---------|-------------------
just-func:coverage: All files       |     100 |      100 |     100 |     100 |                   
just-func:coverage:  Just.ts        |       0 |        0 |       0 |       0 |                   
just-func:coverage:  index.ts       |       0 |        0 |       0 |       0 |                   
just-func:coverage:  testFn.ts      |     100 |      100 |     100 |     100 |                   
just-func:coverage:  toJustValue.ts |     100 |      100 |     100 |     100 |                   
just-func:coverage: ----------------|---------|----------|---------|---------|-------------------

and in the clover.xml:

<file name="toJustValue.ts">
  <metrics statements="3" coveredstatements="3" conditionals="4" coveredconditionals="4" methods="1" coveredmethods="1"/>
</file>

That is the file being sent to codecov:

[2022-05-07T23:04:17.741Z] ['info'] => Found 1 possible coverage files:
  packages/just-func/coverage/clover.xml

CI/CD URL

Codecov Output

==> linux OS detected
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (b2cb60098c868462baebdf4efe586686062997eb9d407dc5ab6085c009f932b7  codecov)
==> Running version latest
==> Running version v0.2.2
/home/runner/work/_actions/codecov/codecov-action/v3/dist/codecov -n  -Q github-action-3.1.0 -f **/coverage/clover.xml -C 3aa24d2a7366f4b333c3fd1309794143400e9097
[2022-05-07T23:04:17.628Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/

  Codecov report uploader 0.2.2
[2022-05-07T23:04:17.637Z] ['info'] => Project root located at: /home/runner/work/just-func-typescript/just-func-typescript
[2022-05-07T23:04:17.639Z] ['info'] -> No token specified or token is empty
[2022-05-07T23:04:17.647Z] ['info'] Searching for coverage files...
[2022-05-07T23:04:17.740Z] ['info'] Warning: Some files passed via the -f flag would normally be excluded from search.
[2022-05-07T23:04:17.740Z] ['info'] If Codecov encounters issues processing your reports, please review https://docs.codecov.com/docs/supported-report-formats
[2022-05-07T23:04:17.741Z] ['info'] => Found 1 possible coverage files:
  packages/just-func/coverage/clover.xml
[2022-05-07T23:04:17.741Z] ['info'] Processing /home/runner/work/just-func-typescript/just-func-typescript/packages/just-func/coverage/clover.xml...
[2022-05-07T23:04:17.745Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-05-07T23:04:17.747Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-3.1.0-uploader-0.2.2&token=*******&branch=pr&build=2287827418&build_url=https%3A%2F%2Fgithub.com%2Fjustland%2Fjust-func-typescript%2Factions%2Fruns%2F2287827418&commit=3aa24d2a7366f4b333c3fd1309794143400e9097&job=pull-request&pr=11&service=github-actions&slug=justland%2Fjust-func-typescript&name=&tag=&flags=&parent=
[2022-05-07T23:04:19.001Z] ['info'] https://codecov.io/github/justland/just-func-typescript/commit/3aa24d2a7366f4b333c3fd1309794143400e9097
https://storage.googleapis.com/codecov/v4/raw/2022-05-07/E4E644B4FB1A443E99A123D3CDA10494/3aa24d2a7366f4b333c3fd1309794143400e9097/dbdb7797-ae70-47fc-8433-6319b70a3342.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EKKHVKCKHW7KBCGM7IHR55T63V2PAVJWLVFNITJHDU5G6R5IRN3LMWJA%2F20220507%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20220507T230418Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=faf04c75558235c5ac1613e288e06ee7a80bb242cd43d3f340ab5f3f3256410d
[2022-05-07T23:04:19.001Z] ['info'] Uploading...
[2022-05-07T23:04:19.202Z] ['info'] {"status":"success","resultURL":"https://codecov.io/github/justland/just-func-typescript/commit/3aa24d2a7366f4b333c3fd1309794143400e9097"}

Expected Results

The coverage should match what’s in the CI

Actual Results

Incorrect coverage

Additional Information

One thing I changed is sending coverage to codecov for all 3 NodeJS runs (14, 16, 18).
In the past did some check and only send coverage to codecov if it is running with 16.
But I’m not sure if that would make a difference.

2

@unional, I believe this is actually a difference in how jest and codecov calculates branch coverage.

I made a PR on a fork of this repository and got CI results. Note the Cat coverage task

You should find lines

    <file name="toJustValue.ts" path="/home/runner/work/just-func-typescript/just-func-typescript/packages/types/ts/toJustValue.ts">
      <metrics statements="3" coveredstatements="3" conditionals="4" coveredconditionals="4" methods="1" coveredmethods="1"/>
      <line num="8" count="12" type="cond" truecount="2" falsecount="0"/>
      <line num="9" count="10" type="cond" truecount="2" falsecount="0"/>

While jest will count having a truecount as being covered, Codecov considers it only partially covered (as there are no falsecounts)

If you look at the Codecov UI, you’ll see we mark it as partially covered,

image
image857×384 80.1 KB

Codecov again does NOT consider partially covered lines as covered.

3

I think it is related to yarn2 or something. These repo was reporting as 100% before I update the repo. The code haven’t changed and the coverage haven’t changed.

Tom via Codecov <notifications@codecov.discoursemail.com>於 2022年5月16日 週一，下午10:20寫道：

4

Here is another example:

The code between these two PR have not changed. But the coverage drops from 100% to 83.33%.

Does codecov only need clover.xml or does it need other files?

There is a change I made to GitHub Action to make it support monorepo:

      - name: codecov
        if: matrix.os == 'ubuntu-latest' && matrix.node-version == 16
        uses: codecov/codecov-action@v3
+        with:
+          files: "**/coverage/clover.xml"

thx

5

The code between these two PR have not changed. But the coverage drops from 100% to 83.33%.

The first PR has no change.
The second PR has a decrease in coverage as noted here. My guess is that normalizePath(actual) === normalizePath(expected) is never getting run and the full line is partially covered. I would guess that actual === expected always happens to be true, and the normalize is not getting used.

Does codecov only need clover.xml or does it need other files?

clover.xml should be fine, but I don’t think you need to specify. The uploader should be able to pick it up.

6

I would guess that actual === expected always happens to be true, and the normalize is not getting used.

Not really, there is a test to cover the false | false case:

test('same style different value fails', () => {
  t.strictEqual(pathEqual('a', 'b'), false)
})

You can see that the lcov-report itself also says 100%:

image
image2171×434 23.8 KB

And the detailed report shows it hits 25 times (can’t attach a second screenshot).

thx.

7

I pulled up an old branch and the coverage goes back to 100%
https://codecov.io/api/gh/unional/path-equal/download/build?path=v4/raw/2022-05-26/BAD291DE9B135E7405413B61B4C93878/c329492bf6a15ffe216ae0c4831344fe3821bffb/215e0c18-cdfe-4840-b8ef-9fb2f73a1a32.txt

It seems like just the clover.xml is not sufficient.

The code are still the same.

8

@unional I’m sorry I lost context on this for a bit. Do you have a more recent PR showing this issue?