Can't get Codecov to work with monorepo

Description

I’m trying to make Codecov for monorepo, which has this setup:

./
./api
./frontend

The API is hosted and CI-managed by Heroku. Due to Heroku’s specifications, the CI flow looks like this:

  1. Remove ./frontend
  2. Move ./app/* to ./ (the root dir)
  3. Run tests and code coverage there.

For Codecov to work, we applied some path fixing:

fixes:
  - "::api/"

This works for the API, but the frontend coverage, which is handled by GitHub Actions, doesn’t appear on Codecov, even though the Action runs OK and the coverage is reported to be uploaded. Instead, we’re receiving this error:

image

I’m guessing here that the fix "::api/" applies to the frontend as well and breaks the paths, but I’m not sure. I’ve tried these to no avail:

  • Rewriting the fixes with RegEx to not rewrite paths that start with “frontend”

  • Use a granular “/home/runner/work/eve/eve/frontend::frontend/” fix that matches GitHub’s directory structure

  • Use a different codecov.yml for api and frontend

  • Mixing flags and stuff, e.g.,

    flags:
     frontend:
       paths:
         - ./frontend
     api:
       paths:
         - ./api
    
    fixes:
     - "::api/"
    

    for api: bash <(curl -s https://codecov.io/bash) -F api
    for frontend (GH Actions):

     uses: codecov/codecov-action@v1
     with:
       token: ${{ secrets.CODECOV_TOKEN }}
       flags: frontend
    

Commit SHAs

a2263fdbc07de51171dab7febc2ceaf887d0ab34 (private)

Repository

https://github.com/Eveio/eve (private)

CI/CD or Build URL

https://github.com/Eveio/eve/runs/1680232543 (private)

Uploader

Bash uploader.

Codecov Output

/bin/bash codecov.sh -n  -F frontend -Q github-action

  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-20201231-23d4836


==> git version 2.29.2 found
==> curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1g zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL 
==> GitHub Actions detected.
->  Issue detecting commit SHA. Please run actions/checkout with fetch-depth > 1 or set to 0
    project root: .
--> token set from env
    Yaml found at: codecov.yml
==> Running gcov in . (disable via -X gcov)
==> Python coveragepy not found
==> Searching for coverage reports in:
    + .
    -> Found 3 reports
==> Detecting git/mercurial file structure
==> Reading reports
    + ./frontend/coverage/clover.xml bytes=328322
    + ./frontend/coverage/coverage-final.json bytes=817145
    + ./frontend/coverage/lcov.info bytes=159384
==> Appending adjustments
    docs.codecov.io/docs/fixing-reports
    + Found adjustments
==> Gzipping contents
        148K	/tmp/codecov.eEL7ns.gz
==> Uploading reports
    url: codecov.io
    query: branch=dev%2FDEV-0-fe-cov&commit=29d8dfe294623cc727527397ec09d2da617edca2&build=477182479&build_url=http%3A%2F%2Fgithub.com%2FEveio%2Feve%2Factions%2Fruns%2F477182479&name=&tag=&slug=Eveio%2Feve&service=github-actions&flags=frontend&pr=623&job=FrontendUnitTests&cmd_args=n,F,Q
->  Pinging Codecov
codecov.io/upload/v4?package=github-action-20201231-23d4836&token=secret&branch=dev%2FDEV-0-fe-cov&commit=29d8dfe294623cc727527397ec09d2da617edca2&build=477182479&build_url=http%3A%2F%2Fgithub.com%2FEveio%2Feve%2Factions%2Fruns%2F477182479&name=&tag=&slug=Eveio%2Feve&service=github-actions&flags=frontend&pr=623&job=FrontendUnitTests&cmd_args=n,F,Q
->  Uploading to
storage.googleapis.com/codecov/v4/raw/2021-01-11/F99736FF3FD5A25101C81FD741998BC0/29d8dfe294623cc727527397ec09d2da617edca2/3ed2cdfc-0005-4989-8618-667dc132e6a4.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20210111%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20210111T093337Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=93e8c1f55986c1ed08d9e65ac37cfb82c0656a822fc0675d9fadfd7ad6cb6110
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  145k    0     0  100  145k      0   648k --:--:-- --:--:-- --:--:--  648k
    -> View reports at codecov.io/github/Eveio/eve/commit/29d8dfe294623cc727527397ec09d2da617edca2

Any help would be much appreciated.

@phanan, what happens if you remove the paths from the flag section of the yaml?

1 Like

@tom Thanks for answering. With that, I’m assuming that I should remove the -F flag from codecov command as well?

@tom So I have removed everything flag from the config and build commands. My codecov.yml is now a minimal

fixes:
  - "::api/"

The results are as follow:

  • The API coverage (processed and uploaded by Heroku) still works fine

  • The frontend coverage (processed and uploaded by GitHub) shows up in Codecov with correct metrics, but all paths are prefixed by api/ (I guess due to the path fixing) and are broken:

  • As you can see, there are two separated reports. My ultimate goal (if possible) is to merge them into one (and have it reported by Codecov bot in GitHub as one).

Basically this is the “reversed” version of where we initially started (having no codecov.yml)—both reports appeared separately, frontend worked fine, api showed up with broken paths.

What do you reckon should be the next thing to try?

Update: I’ve modified the path fix to be a regex that negates frontend, like this:

fixes:
  - "^(?!frontend)::api/"

and the coverages now correctly show up in Codecov, but still as two separate reports. What can be done to merge them into one, preferably properly flagged?

Edit: I’ve also tried uploading each coverage with the corresponding flag (without specifying them in codecod.yml) e.g. bash <(curl -s https://codecov.io/bash) -F frontend and bash <(curl -s https://codecov.io/bash) -F api. This still doesn’t work as expected. Instead, it goes like this:

  1. Codecov bot initially displays both api and frontend statistics
  2. Upon frontend finishes coverage reporting, the statistics are replaced with frontend specifics
  3. Upon api finishes coverage reporting (as it takes longer than frontend), the (frontend) statistics are replaced by api specifics.

What I’m looking for is essentially a report for both of them.

Hi @phanan, I’m a little confused about your question.

the coverages now correctly show up in Codecov, but still as two separate reports

Can you share a link or screenshot of this happening and explain what you would want it to look like instead?

1 Like

Sure @tom.

You’ll notice that there’s only one flag (api) and its statistics are shown here, even though there are actually two coverages generated in this commit—one by Heroku, and one by GitHub Actions. The reason why it’s api is that, as I explained above, Heroku’s CI finished later and replaced the previous frontend statistics.

I’d like both coverages api and frontend merged and shown here instead.

To add more to my previous answer:

After GitHub Actions finishes its frontend run and uploads the coverage, I can follow the uploaded link and see the coverage on Codecov:

A little while later, Heroku finishes its api run, and I can see its coverage as well (I’m sorry I can’ upload another image due to the forum’s limitation for new users, but replace frontend/src from the image above with api and you’ll have it.)

These reports are, however, separate from each other, even though they’re initialized from the same commit.

@phanan, hmmm… ok that’s strange. Can you share the Codecov outputs from Heroku and GHA?

Sure @tom (and I really appreciate your patience, thank you!)

GitHub:

Run bash <(curl -s codecov.io/bash) -t *** -F frontend

  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-20201231-23d4836


==> git version 2.29.2 found
==> curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1g zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL 
==> GitHub Actions detected.
->  Issue detecting commit SHA. Please run actions/checkout with fetch-depth > 1 or set to 0
    project root: /home/runner/work/eve/eve
    Yaml found at: codecov.yml
==> Running gcov in /home/runner/work/eve/eve (disable via -X gcov)
==> Python coveragepy not found
==> Searching for coverage reports in:
    + /home/runner/work/eve/eve
    -> Found 3 reports
==> Detecting git/mercurial file structure
==> Reading reports
    + /home/runner/work/eve/eve/frontend/coverage/clover.xml bytes=329607
    + /home/runner/work/eve/eve/frontend/coverage/coverage-final.json bytes=820061
    + /home/runner/work/eve/eve/frontend/coverage/lcov.info bytes=160061
==> Appending adjustments
    docs.codecov.io/docs/fixing-reports
/dev/fd/63: line 1632: echo: write error: Broken pipe
    + Found adjustments
==> Gzipping contents
        148K	/tmp/codecov.dQwzYd.gz
==> Uploading reports
    url: codecov.io
    query: branch=dev%2FDEV-0-fe-cov2&commit=1d5802fb1f7973cf27fcd4bf61741d1bf27a753d&build=480449241&build_url=http%3A%2F%2Fgithub.com%2FEveio%2Feve%2Factions%2Fruns%2F480449241&name=&tag=&slug=Eveio%2Feve&service=github-actions&flags=frontend&pr=640&job=FrontendUnitTests&cmd_args=t,F
->  Pinging Codecov
codecov.io/upload/v4?package=bash-20201231-23d4836&token=secret&branch=dev%2FDEV-0-fe-cov2&commit=1d5802fb1f7973cf27fcd4bf61741d1bf27a753d&build=480449241&build_url=http%3A%2F%2Fgithub.com%2FEveio%2Feve%2Factions%2Fruns%2F480449241&name=&tag=&slug=Eveio%2Feve&service=github-actions&flags=frontend&pr=640&job=FrontendUnitTests&cmd_args=t,F
->  Uploading to
storage.googleapis.com/codecov/v4/raw/2021-01-12/F99736FF3FD5A25101C81FD741998BC0/1d5802fb1f7973cf27fcd4bf61741d1bf27a753d/43789d49-8f73-4cce-b59b-a206c7b2b439.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20210112%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20210112T144008Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=0250285bab01375476fe17ce74bd1fc56cef8a079d1da205344711bcee45996e
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
 43  145k    0     0   43 65535      0  3764k --:--:-- --:--:-- --:--:-- 3555k
100  145k    0     0  100  145k      0   641k --:--:-- --:--:-- --:--:--  638k
    -> View reports at codecov.io/github/Eveio/eve/commit/1d5802fb1f7973cf27fcd4bf61741d1bf27a753d

Heroku:

Generating code coverage report in Clover XML format ... done [00:01.705]
  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-20201231-23d4836
==> git version 2.17.1 found
==> curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL 
==> Heroku CI detected.
    project root: .
--> token set from env
not found!
    Yaml found at: ./codecov.yml
==> Running gcov in . (disable via -X gcov)
==> Searching for coverage reports in:
    + .
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Reading reports
    + ./coverage.xml bytes=514026
==> Appending adjustments
    https://docs.codecov.io/docs/fixing-reports
    + Found adjustments
==> Gzipping contents
        320K	/tmp/codecov.iPaXAu.gz
==> Uploading reports
    url: https://codecov.io
    query: branch=dev%2FDEV-0-fe-cov2&commit=adfcfd86148ac68a08d87cd52b67edfbd7273a05&build=7dbaed02-d816-4c77-8583-78ce7b8c9e93&build_url=&name=&tag=&slug=&service=heroku&flags=api&pr=&job=&cmd_args=F
->  Pinging Codecov
https://codecov.io/upload/v4?package=bash-20201231-23d4836&token=secret&branch=dev%2FDEV-0-fe-cov2&commit=adfcfd86148ac68a08d87cd52b67edfbd7273a05&build=7dbaed02-d816-4c77-8583-78ce7b8c9e93&build_url=&name=&tag=&slug=&service=heroku&flags=api&pr=&job=&cmd_args=F
->  Uploading to
https://storage.googleapis.com/codecov/v4/raw/2021-01-12/F99736FF3FD5A25101C81FD741998BC0/adfcfd86148ac68a08d87cd52b67edfbd7273a05/9b9be579-6d9d-4743-b586-2b0c9fad79a7.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20210112%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20210112T144221Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=3d02174a59a61f651265e360db723e52c3586f6437407951acc4f91e8e6b1edc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
    -> View reports at https://codecov.io/github/Eveio/eve/commit/adfcfd86148ac68a08d87cd52b67edfbd7273a05
-----> test command `composer cs && composer deptrac && composer test:ci && bash <(curl -s https://codecov.io/bash) -F api` completed successfully

@phanan, if you take a look, you’ll see that the GHA and Heroku builds have different commit SHAs. For GitHub Actions, add fetch-depth: 2 (or similar as above).

1 Like

That’s a great catch, thanks @tom! Indeed, after setting fetch-depth to 2, the commits are fixed and both flags show up in GitHub. However, the “Continue to review full report at Codecov” link now yields a server error:

Is it because I’m doing something funny with the config or…?

@phanan, I think that PR might be in a strange state on our side due to the previous uploads, would you be able to open a new PR with applicable changes?

Strange, it happened to some others around that time, but since then everything is back to normal.
Thanks a lot @tom, my issue is now completely resolved. You guys rock!

1 Like

FWIW, in following up on my thread on tags, I also had a few minutes where the PR view would come back with a 500 page. And then it would be just fine.