I was working in a PR in a repository that disabled codecov a while ago (it seems that they were having problems with codecov failing a lot with small decrements). When I tried to re-activate the Github Action for codecov to see if it still works, my build failed because codecov is comparing it to a very old commit (I guess it was the last time coverage data was uploaded?).
This comparison is unfair. Is there a way of preventing codecov from comparing coverage changes with commits other than the actual PR base?
I had a look on previous discussions, specially Flag -N to set the base report does not work - #6 by jerrod, but I don’t think it is applicable.
Please provide a link to the CI/CD run.
Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.
/usr/bin/bash codecov.sh -n -F Linux, 3.10 -Q github-action-v1.5.2 -C 9ce6385df46d6aa897c9d7905d0f9fc660d390bc _____ _ / ____| | | | | ___ __| | ___ ___ _____ __ | | / _ \ / _` |/ _ \/ __/ _ \ \ / / | |___| (_) | (_| | __/ (_| (_) \ V / \_____\___/ \__,_|\___|\___\___/ \_/ Bash-1.0.3 ==> git version 2.35.1 found ==> curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3 Release-Date: 2020-01-08 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets ==> GitHub Actions detected. Env vars used: -> GITHUB_ACTIONS: true -> GITHUB_HEAD_REF: disable-coverage-pypy -> GITHUB_REF: refs/pull/3112/merge -> GITHUB_REPOSITORY: pypa/setuptools -> GITHUB_RUN_ID: 1855218487 -> GITHUB_SHA: 1e690451bef88aad5c323fb5eb77e565a06bfe92 -> GITHUB_WORKFLOW: tests project root: . Yaml found at: .codecov.yml ==> Running gcov in . (disable via -X gcov) ==> Searching for coverage reports in: + . -> Found 1 reports ==> Detecting git/mercurial file structure ==> Reading reports + ./coverage.xml bytes=1061907 ==> Appending adjustments https://docs.codecov.io/docs/fixing-reports -> No adjustments found ==> Gzipping contents 76K /tmp/codecov.yXkurD.gz ==> Uploading reports url: https://codecov.io query: branch=disable-coverage-pypy&commit=9ce6385df46d6aa897c9d7905d0f9fc660d390bc&build=1855218487&build_url=http%3A%2F%2Fgithub.com%2Fpypa%2Fsetuptools%2Factions%2Fruns%2F1855218487&name=&tag=&slug=pypa%2Fsetuptools&service=github-actions&flags=Linux,3.10&pr=3112&job=tests&cmd_args=n,F,Q,C -> Pinging Codecov https://codecov.io/upload/v4?package=github-action-v1.5.2-1.0.3&token=secret&branch=disable-coverage-pypy&commit=9ce6385df46d6aa897c9d7905d0f9fc660d390bc&build=1855218487&build_url=http%3A%2F%2Fgithub.com%2Fpypa%2Fsetuptools%2Factions%2Fruns%2F1855218487&name=&tag=&slug=pypa%2Fsetuptools&service=github-actions&flags=Linux,3.10&pr=3112&job=tests&cmd_args=n,F,Q,C -> Uploading to https://storage.googleapis.com/codecov/v4/raw/2022-02-16/19BC5522785AF0C590C8AE9698BF7D5D/9ce6385df46d6aa897c9d7905d0f9fc660d390bc/cde37ce3-eae7-4177-92d9-bf584debb847.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EKKHVKCKHW7KBCGM7IHR55T63V2PAVJWLVFNITJHDU5G6R5IRN3LMWJA%2F20220216%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20220216T211259Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=a813d5a85a3c381470e1a3f43869365c069257b56905ce1b3bdc99a86226b0f3 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 73784 0 0 100 73784 0 324k --:--:-- --:--:-- --:--:-- 324k -> Reports have been successfully queued for processing at https://codecov.io/github/pypa/setuptools/commit/9ce6385df46d6aa897c9d7905d0f9fc660d390bc
I would expect codecov to not try to compare coverage changes if it cannot find the base commit for the PR
Codecov failed (in a very unfair manner) when comparing coverage results more than 200 commits old.