How to prevent codecov from using a commit older than the base of the PR to do comparison?


I was working in a PR in a repository that disabled codecov a while ago (it seems that they were having problems with codecov failing a lot with small decrements). When I tried to re-activate the Github Action for codecov to see if it still works, my build failed because codecov is comparing it to a very old commit (I guess it was the last time coverage data was uploaded?).

This comparison is unfair. Is there a way of preventing codecov from comparing coverage changes with commits other than the actual PR base?

I had a look on previous discussions, specially Flag -N to set the base report does not work - #6 by jerrod, but I don’t think it is applicable.


Please provide a link to the CI/CD run.

Codecov Output

Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.

/usr/bin/bash -n  -F Linux, 3.10 -Q github-action-v1.5.2 -C 9ce6385df46d6aa897c9d7905d0f9fc660d390bc

  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/

==> git version 2.35.1 found
==> curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets
==> GitHub Actions detected.
    Env vars used:
      -> GITHUB_ACTIONS:    true
      -> GITHUB_HEAD_REF:   disable-coverage-pypy
      -> GITHUB_REF:        refs/pull/3112/merge
      -> GITHUB_REPOSITORY: pypa/setuptools
      -> GITHUB_RUN_ID:     1855218487
      -> GITHUB_SHA:        1e690451bef88aad5c323fb5eb77e565a06bfe92
      -> GITHUB_WORKFLOW:   tests
    project root: .
    Yaml found at: .codecov.yml
==> Running gcov in . (disable via -X gcov)
==> Searching for coverage reports in:
    + .
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Reading reports
    + ./coverage.xml bytes=1061907
==> Appending adjustments
    -> No adjustments found
==> Gzipping contents
        76K	/tmp/codecov.yXkurD.gz
==> Uploading reports
    query: branch=disable-coverage-pypy&commit=9ce6385df46d6aa897c9d7905d0f9fc660d390bc&build=1855218487&,3.10&pr=3112&job=tests&cmd_args=n,F,Q,C
->  Pinging Codecov,3.10&pr=3112&job=tests&cmd_args=n,F,Q,C
->  Uploading to
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 73784    0     0  100 73784      0   324k --:--:-- --:--:-- --:--:--  324k
    -> Reports have been successfully queued for processing at

Expected Results

I would expect codecov to not try to compare coverage changes if it cannot find the base commit for the PR

Actual Results

Codecov failed (in a very unfair manner) when comparing coverage results more than 200 commits old.

hi @abravalheri, really appreciate you bringing Codecov back up. the only way for Codecov to compare against is by the coverage reported from a commits’ parents. If those don’t exist, Codecov will keep going backwards until it hits a commit that has coverage.

I realize that in your case, it’s causing some heartache, but there’s no way to turn that off. What you could do is change the status checks to be informational until the repository stabilizes.

        informational: true
        informational: true

Also of note, the Action is on v1, but it is recommended to upgrade to v2

1 Like