Please search for similar issues before creating a ticket.
All fields below are required.
Description
The codecov PR bot shows a decrease in coverage when there is no decrease in coverage.
CI/CD URL
https://github.com/spinalcordtoolbox/spinalcordtoolbox/runs/5039672929?check_suite_focus=true
Codecov Output
Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.
Run codecov/codecov-action@v2
==> linux OS detected
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (330066ecce62f2cdcfd2fe1845095475f51bbad9649dce81ac7b663dc884bbef codecov)
==> Running version latest
==> Running version v0.1.15
/home/runner/work/_actions/codecov/codecov-action/v2/dist/codecov -n -Q github-action-2.1.0 -Z -f ./cov-api.xml -F api-tests -C 7605b9e21599a848fc78d8fa26732a0a9258fc61
[2022-02-02T16:37:17.705Z] ['info']
_____ _
/ ____| | |
| | ___ __| | ___ ___ _____ __
| | / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| | __/ (_| (_) \ V /
\_____\___/ \__,_|\___|\___\___/ \_/
Codecov report uploader 0.1.15
[2022-02-02T16:37:17.712Z] ['info'] => Project root located at: /home/runner/work/spinalcordtoolbox/spinalcordtoolbox
[2022-02-02T16:37:17.714Z] ['info'] -> No token specified or token is empty
[2022-02-02T16:37:17.723Z] ['info'] Searching for coverage files...
[2022-02-02T16:37:18.313Z] ['info'] => Found 1 possible coverage files:
./cov-api.xml
[2022-02-02T16:37:18.313Z] ['info'] Processing ./cov-api.xml...
[2022-02-02T16:37:18.330Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-02-02T16:37:18.331Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-2.1.0-uploader-0.1.15&token=*******&branch=jn%2F3669-test-release-path-in-CI&build=1784986182&build_url=https%3A%2F%2Fgithub.com%2Fspinalcordtoolbox%2Fspinalcordtoolbox%2Factions%2Fruns%2F1784986182&commit=7605b9e21599a848fc78d8fa26732a0a9258fc61&job=Tests&pr=3678&service=github-actions&slug=spinalcordtoolbox%2Fspinalcordtoolbox&name=&tag=&flags=api-tests&parent=
[2022-02-02T16:37:18.776Z] ['info'] Uploading...
[2022-02-02T16:37:18.969Z] ['info'] {"status":"success","resultURL":"https://codecov.io/github/spinalcordtoolbox/spinalcordtoolbox/commit/7605b9e21599a848fc78d8fa26732a0a9258fc61"}
Expected Results
The “flag” table should show 0.00% change.
Actual Results
See https://github.com/spinalcordtoolbox/spinalcordtoolbox/pull/3678#issuecomment-1028125606:
Codecov Report
Merging #3678 (7605b9e) into master (5d422d9) will not change coverage.
The diff coverage isn/a.
Flag Coverage Δ api-tests 24.20% <ø> (-0.25%)cli-tests 56.84% <ø> (ø)Flags with carried forward coverage won’t be shown. Click here to find out more.
Even more strangely, the web UI shows a coverage increase? Codecov
While the changes page shows no change at all: Codecov
Additional Information
My best guess is that maybe the cause of this is because the changes happened outside of the monitored folder:
- The file changed is:
./.github/workflows/create-release.yml - However we’re only monitoring
./spinalcordtoolbox(the package directory)
So, maybe the bot is including other files in how it calculates coverage %?
Even still, it’s odd that the changes are only showing for the api-tests flag and not the cli-tests flag. 
