Codecov doesn't fail the CI - github actions

Support
1

Please search for similar issues before creating a ticket.
All fields below are required.

Description

I’ve setup the CI with codecov. It shows the report. It doesn’t fail the CI though and reports success.

CI/CD URL

https://codecov.io/gh/Vonage/vivid-3/commit/7e82b31ee4bda75ab29399c52d4579c64536042a/https://codecov.io/gh/Vonage/vivid-3/commit/7e82b31ee4bda75ab29399c52d4579c64536042a/

Codecov Output

==> linux OS detected
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (be64e2915c5f1ca777044cf1994d474dc9b4d6fe74339111163b00dbb4c1b9d3  codecov)
==> Running version latest
==> Running version v0.1.18
/home/runner/work/_actions/codecov/codecov-action/v2/dist/codecov -n codecov-umbrella -Q github-action-2.1.0 -Z -f ./coverage/lcov.info -F unittests -C 7e82b31ee4bda75ab29399c52d4579c64536042a -s ./coverage/ -v
[2022-03-15T11:59:18.094Z] ['verbose'] Start of uploader: 1647345558094...
[2022-03-15T11:59:18.097Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/

  Codecov report uploader 0.1.18
[2022-03-15T11:59:18.103Z] ['info'] => Project root located at: /home/runner/work/vivid-3/vivid-3
[2022-03-15T11:59:18.108Z] ['info'] -> No token specified or token is empty
[2022-03-15T11:59:18.108Z] ['verbose'] Start of network processing...
[2022-03-15T11:59:18.108Z] ['verbose'] Searching for files in /home/runner/work/vivid-3/vivid-3
[2022-03-15T11:59:18.113Z] ['info'] Searching for coverage files...
[2022-03-15T11:59:18.155Z] ['verbose'] Preparing to clean the following coverage paths: ./coverage/lcov.info
[2022-03-15T11:59:18.155Z] ['info'] => Found 1 possible coverage files:
  ./coverage/lcov.info
[2022-03-15T11:59:18.155Z] ['verbose'] End of network processing
[2022-03-15T11:59:18.156Z] ['info'] Processing ./coverage/lcov.info...
[2022-03-15T11:59:18.161Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-03-15T11:59:18.161Z] ['verbose'] -> Using the following env variables:
[2022-03-15T11:59:18.161Z] ['verbose']      GITHUB_ACTION: __codecov_codecov-action
[2022-03-15T11:59:18.162Z] ['verbose']      GITHUB_HEAD_REF: block-PR-if-coverage-drops
[2022-03-15T11:59:18.162Z] ['verbose']      GITHUB_REF: refs/pull/254/merge
[2022-03-15T11:59:18.162Z] ['verbose']      GITHUB_REPOSITORY: Vonage/vivid-3
[2022-03-15T11:59:18.162Z] ['verbose']      GITHUB_RUN_ID: 1986485916
[2022-03-15T11:59:18.162Z] ['verbose']      GITHUB_SERVER_URL: https://github.com
[2022-03-15T11:59:18.162Z] ['verbose']      GITHUB_SHA: 1dd8cbeefdc461f0a24534f2f0fea0197b07e2c4
[2022-03-15T11:59:18.162Z] ['verbose']      GITHUB_WORKFLOW: Pull Request
[2022-03-15T11:59:18.162Z] ['verbose'] Using the following upload parameters:
[2022-03-15T11:59:18.162Z] ['verbose'] branch
[2022-03-15T11:59:18.162Z] ['verbose'] build
[2022-03-15T11:59:18.163Z] ['verbose'] buildURL
[2022-03-15T11:59:18.163Z] ['verbose'] commit
[2022-03-15T11:59:18.163Z] ['verbose'] job
[2022-03-15T11:59:18.163Z] ['verbose'] pr
[2022-03-15T11:59:18.163Z] ['verbose'] service
[2022-03-15T11:59:18.163Z] ['verbose'] slug
[2022-03-15T11:59:18.163Z] ['verbose'] name
[2022-03-15T11:59:18.163Z] ['verbose'] tag
[2022-03-15T11:59:18.163Z] ['verbose'] flags
[2022-03-15T11:59:18.163Z] ['verbose'] parent
[2022-03-15T11:59:18.164Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-2.1.0-uploader-0.1.18&token=*******&branch=block-PR-if-coverage-drops&build=1986485916&build_url=https%3A%2F%2Fgithub.com%2FVonage%2Fvivid-3%2Factions%2Fruns%2F1986485916&commit=7e82b31ee4bda75ab29399c52d4579c64536042a&job=Pull+Request&pr=254&service=github-actions&slug=Vonage%2Fvivid-3&name=codecov-umbrella&tag=&flags=unittests&parent=
[2022-03-15T11:59:18.164Z] ['verbose'] Passed token was 0 characters long
[2022-03-15T11:59:18.164Z] ['verbose'] https://codecov.io/upload/v4?package=github-action-2.1.0-uploader-0.1.18&branch=block-PR-if-coverage-drops&build=1986485916&build_url=https%3A%2F%2Fgithub.com%2FVonage%2Fvivid-3%2Factions%2Fruns%2F1986485916&commit=7e82b31ee4bda75ab29399c52d4579c64536042a&job=Pull+Request&pr=254&service=github-actions&slug=Vonage%2Fvivid-3&name=codecov-umbrella&tag=&flags=unittests&parent=
        Content-Type: 'text/plain'
        Content-Encoding: 'gzip'
        X-Reduced-Redundancy: 'false'
[2022-03-15T11:59:18.695Z] ['info'] https://codecov.io/github/Vonage/vivid-3/commit/7e82b31ee4bda75ab29399c52d4579c64536042a
https://storage.googleapis.com/codecov/v4/raw/2022-03-15/4D545C4B2C0886EFA8D1EA42CBB2F096/7e82b31ee4bda75ab29399c52d4579c64536042a/3551c12d-9a87-4f99-bc9f-d06c15128be0.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EKKHVKCKHW7KBCGM7IHR55T63V2PAVJWLVFNITJHDU5G6R5IRN3LMWJA%2F20220315%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20220315T115918Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=f1f2129e98a1a8e347aa5037e5cde61d3462010a5ce15fd27ef96dbe241a1c4c
[2022-03-15T11:59:18.696Z] ['verbose'] Returned upload url: https://storage.googleapis.com/codecov/v4/raw/2022-03-15/4D545C4B2C0886EFA8D1EA42CBB2F096/7e82b31ee4bda75ab29399c52d4579c64536042a/3551c12d-9a87-4f99-bc9f-d06c15128be0.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EKKHVKCKHW7KBCGM7IHR55T63V2PAVJWLVFNITJHDU5G6R5IRN3LMWJA%2F20220315%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20220315T115918Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=f1f2129e98a1a8e347aa5037e5cde61d3462010a5ce15fd27ef96dbe241a1c4c
[2022-03-15T11:59:18.696Z] ['info'] Uploading...
[2022-03-15T11:59:18.863Z] ['info'] {"status":"success","resultURL":"https://codecov.io/github/Vonage/vivid-3/commit/7e82b31ee4bda75ab29399c52d4579c64536042a"}
[2022-03-15T11:59:18.863Z] ['verbose'] End of uploader: 769 milliseconds

Expected Results

I’d expect the CI to fail as I intentionally dropped coverage from 95% to 85%.

Actual Results

CI passed

Additional Information

The whole repo is open source: GitHub - Vonage/vivid-3: Vonage's web UI 🎨 toolbelt

2

Can you try adding the Codecov GitHub app?

3

Thanks for your reply.
Is using the app a must? Can’t I just use the github action?

4

There’s a discrepancy between the codecov UI and the response returned from the github action:
image
In the codecov UI it shows the CI fails, as shown in the image above. Also, when I check the codecov.yml file I see my config as I’ve set it in the branch.
The CI itself passed as I get a success response from codecov

5

The app is a must, it allows us to interface with GitHub for API calls.

Do you mind sharing a link for the CI discrepancy?

6

Hi,
We’ve activated and configured the app.
This has solved the issue.
Thanks!

1 Like