Description
When we set up CodeCov, we noticed that often project coverage would be failing, often saying that “the coverage will decrease by < 0.01%”. We would still like to get project reports, but since it would fail our builds too often, we decided to keep it in informational mode:
informational
Use Codecov in informational mode. Default is
false
. Iftrue
is specified the resulting status will pass > no matter what the coverage is or what other settings are specified. Informational mode is great to use > if you want to expose codecov information to other developers in your pull request without necessarily gating PRs on that information.
This would suggest that the status should pass even if coverage dropped too low. However, we’re still getting failed “informational” status:
Is this desired behaviour?
Commit SHAs
d092e839d92a55908de8b84bf330882cbcc29f6d
Repository
https://github.com/eatclub/web/
CI/CD or Build URL
The CI is private.
Uploader
Codecov bash uploader, version “20200728-9fb7d93”.
Codecov Output
-
bash build/scripts/code_cov_uploader.sh
/ | | |
| | ___ __| | ___ ___ _____ __
| | / _ \ / ` |/ _ / / _ \ \ / /
| || () | (| | __/ (| () \ V /
______/ _,|_|____/ _/
Bash-20200728-9fb7d93
==> Jenkins CI detected.
project root: .
Fixing merge commit SHA
Yaml found at: codecov.yml
→ token set from yaml
==> Running gcov in . (disable via -X gcov)
==> Searching for coverage reports in:
+ .
→ Found 2 reports
==> Detecting git/mercurial file structure
==> Reading reports
+ ./nosetests.xml bytes=574109
+ ./coverage.xml bytes=2260886
==> Appending adjustments
Fixing Reports
+ Found adjustments
==> Gzipping contents
==> Uploading reports
url: https://codecov.io
query: branch=COM-686-bye-bye-base-template&commit=d092e839d92a55908de8b84bf330882cbcc29f6d&build=15751&build_url=https%3A%2F%2Fjenkins.eatclub.net%2Fjob%2FWeb_unittest%2F15751%2F&name=&tag=&slug=eatclub%2Fweb&service=jenkins&flags=&pr=6489&job=&cmd_args=
→ Pinging Codecov
https://codecov.io/upload/v4?package=bash-20200728-9fb7d93&token=secret&branch=COM-686-bye-bye-base-template&commit=d092e839d92a55908de8b84bf330882cbcc29f6d&build=15751&build_url=https%3A%2F%2Fjenkins.eatclub.net%2Fjob%2FWeb_unittest%2F15751%2F&name=&tag=&slug=eatclub%2Fweb&service=jenkins&flags=&pr=6489&job=&cmd_args=
→ Uploading to
https://storage.googleapis.com/codecov/v4/raw/2020-12-17/C0CBE9440179AE1D090801ABC07BD16B/d092e839d92a55908de8b84bf330882cbcc29f6d/e6a7fb0e-a8e1-4e12-8749-67c9cecb800a.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20201217%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20201217T172235Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=0e3efd001e4758a4caa8f7c7f5b01ac9706c013031a7beb72586b210384a6e92
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
100 213k 0 0 100 213k 0 521k --:–:-- --:–:-- --:–:-- 522k
→ View reports at Codecov
Expected Results
I’d have expected to have informational status that informs us about the drop in coverage, but still has a passing status check on Github.
Actual Results
The status check on Github is showing up as failed
Additional Information
Our codecov.yml:
coverage:
status:
project:
# Treat Project coverage info as informational only
# CodeCov too often marks the build as failed when it says that coverage dropped by < 0.01%
informational: true
patch:
default:
target: 80
if_not_found: error
I guess my question is:
Is this the expected behaviour of “informational” check and we’re misunderstanding how it’s meant to work? Or is the behaviour we’re seeing a result of some misconfigurion/error/bug?