Hi,
I’m trying to setup CI/CD pipeline (on this repo: GitHub - ptrkalm/hello-shuttle) with Github Actions and I created following configuration for github jobs:
name: CI/CD
on:
push:
branches:
- "main"
jobs:
check:
name: Check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- uses: actions-rs/cargo@v1
with:
command: check
test:
name: Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- uses: actions-rs/cargo@v1
with:
command: test
coverage:
name: Coverage
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --all-features --lcov --ignore-filename-regex main.rs --output-path lcov.info
- uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
fail_ci_if_error: true
deploy:
needs: [check, test, coverage]
name: Deploy
runs-on: ubuntu-latest
steps:
- uses: shuttle-hq/deploy-action@main
with:
deploy-key: ${{ secrets.SHUTTLE_DEPLOY_KEY }}
As you can see in step “coverage” I generate coverage info with llvm-cov for Rust and then I’m trying to upload it ro codecov.io, which seems to work just fine since I can see on dashboards at codecov.io all of my commits with correctly calculated coverage.
However, I on purpose commited changes without tests to lower the coverage (which worked), but I expected that since I configured in codecov.yaml that I want 100% coverage, that the status will change to Failure, but it stays as passed all the time.
My configuration of codecov.yml that is in root directory:
coverage:
status:
project:
default:
target: 100%
threshold: 0%
Response from codecov.io in Github Actions console:
[2023-07-21T00:56:00.114Z] ['info']
_____ _
/ ____| | |
| | ___ __| | ___ ___ _____ __
| | / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| | __/ (_| (_) \ V /
\_____\___/ \__,_|\___|\___\___/ \_/
Codecov report uploader 0.6.1
[2023-07-21T00:56:00.123Z] ['info'] => Project root located at: /home/runner/work/hello-shuttle/hello-shuttle
[2023-07-21T00:56:00.126Z] ['info'] -> Token found by environment variables
[2023-07-21T00:56:00.133Z] ['info'] Searching for coverage files...
[2023-07-21T00:56:00.275Z] ['info'] Warning: Some files located via search were excluded from upload.
[2023-07-21T00:56:00.275Z] ['info'] If Codecov did not locate your files, please review https://docs.codecov.com/docs/supported-report-formats
[2023-07-21T00:56:00.276Z] ['info'] => Found 1 possible coverage files:
lcov.info
[2023-07-21T00:56:00.276Z] ['info'] Processing /home/runner/work/hello-shuttle/hello-shuttle/lcov.info...
[2023-07-21T00:56:00.280Z] ['info'] Detected GitHub Actions as the CI provider.
[2023-07-21T00:56:00.667Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-3.1.4-uploader-0.6.1&token=*******&branch=main&build=5617376383&build_url=https%3A%2F%2Fgithub.com%2Fptrkalm%2Fhello-shuttle%2Factions%2Fruns%2F5617376383&commit=5367e7f3297718ad1df15de098255e1c336af4f7&job=CI%2FCD&pr=&service=github-actions&slug=ptrkalm%2Fhello-shuttle&name=&tag=&flags=&parent=
[2023-07-21T00:56:01.049Z] ['info'] https://app.codecov.io/github/ptrkalm/hello-shuttle/commit/5367e7f3297718ad1df15de098255e1c336af4f7
https://storage.googleapis.com/codecov/v4/raw/2023-07-21/4860789E6ED39C81BD52AEB9AFA147CA/5367e7f3297718ad1df15de098255e1c336af4f7/7a4817bb-5f94-41db-b1c0-dfa59f479e88.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJOGFN2JQ4OCTGA2MU5AEIT7OT5Z7HTFOAN2SPG4NWSN2UJYOY5U6LZQ%2F20230721%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20230721T005601Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=20fac72b1bc47de6a4647e5d8e5a989b9d10f9211ddf8f3883f8169121b92c32
[2023-07-21T00:56:01.050Z] ['info'] Uploading...
[2023-07-21T00:56:01.257Z] ['info'] {"status":"success","resultURL":"https://app.codecov.io/github/ptrkalm/hello-shuttle/commit/5367e7f3297718ad1df15de098255e1c336af4f7"}
Current behaviour:
Codecov.io fetches report generated and uploaded with github actions, but passes all commits despite lower coverage than expected.
Expected:
Status is not success if coverage after commit (on whole project, not just new code) and since I added this to job:
with:
fail_ci_if_error: true
it will stop my pipeline and not deploy it.
I must do some stupid mistake, but don’t know what to do anymore xd
Thanks in advance!
Piotrek
