Codecov GitHub app does not report back in GitHub PRs anymore

Before submitting a topic, please confirm the following

I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my codecov.yaml configuration file.
I have filled out the below sections to the best of my ability.
I understand that the community boards are a free, best-effort tool. While we hope to have someone on the Codecov team resolve your problem quickly, this is not always possible.

Description

We recently found out that the Codecov GitHub is not reporting any more reports in PRs in the scikit-learn repository.

We could find out the following:

  • Our Azure builds are correctly uploading the CI report to Codecov
  • Reports are available in the “Commits” tab: Codecov
  • Reports are not available in the “Pulls” tab: Codecov

We thought that the last item could be the reason for the app not reporting on the GitHub PRs.
Note that the report is available for the commit history of the main branch.

CI/CD URL

A potential PR is the following: MAINT check status of codecov by glemaitre · Pull Request #25875 · scikit-learn/scikit-learn · GitHub
All CIs run are available here: https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=53067&view=results
A CI build showing the coverage reporting and upload to Codecov is available here: https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=53067&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e&t=80793e4d-ec39-5212-d625-56647bcbed6e

Codecov Output

Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.

2023-03-17T11:48:03.6852529Z ---------------------------------------------------------------------------------------------------------------------------------------------
2023-03-17T11:48:03.6853076Z TOTAL                                                                                            101597   2176  21258    991    97%
2023-03-17T11:48:03.7024848Z ~/work/1/s
2023-03-17T11:48:23.1985529Z 
2023-03-17T11:48:23.1986740Z       _____          _
2023-03-17T11:48:23.1987247Z      / ____|        | |
2023-03-17T11:48:23.1987731Z     | |     ___   __| | ___  ___ _____   __
2023-03-17T11:48:23.1988228Z     | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
2023-03-17T11:48:23.1988725Z     | |___| (_) | (_| |  __/ (_| (_) \ V /
2023-03-17T11:48:23.1989221Z      \_____\___/ \____|\___|\___\___/ \_/
2023-03-17T11:48:23.1989708Z                                     v2.1.12
2023-03-17T11:48:23.1990027Z 
2023-03-17T11:48:23.1990968Z e[90m==>e[0m Detecting CI provider
2023-03-17T11:48:23.1991676Z   -> Got branch from git/hg
2023-03-17T11:48:23.1992325Z   -> Got sha from git/hg
2023-03-17T11:48:23.1992965Z e[90m==>e[0m Preparing upload
2023-03-17T11:48:23.1993681Z e[90m==>e[0m Processing gcov (disable by -X gcov)
2023-03-17T11:48:23.1994393Z e[90m==>e[0m Collecting reports
2023-03-17T11:48:23.1995266Z     e[32m+e[0m /home/vsts/work/1/s/build_tools/azure/pylatest_conda_forge_mkl_no_coverage_linux-64_conda.lock bytes=21504
2023-03-17T11:48:23.1995981Z     Generating coverage xml reports for Python
2023-03-17T11:48:23.1997002Z     e[32m+e[0m /home/vsts/work/1/s/coverage.xml bytes=4281281
2023-03-17T11:48:23.1997690Z e[90m==>e[0m Uploading
2023-03-17T11:48:23.1998296Z     .url https://codecov.io
2023-03-17T11:48:23.1998938Z     .query commit=91e6a0ae0604773520c9042f89eb877b24b43763&branch=HEAD&token=<secret>&yaml=.codecov.yml&package=py2.1.12
2023-03-17T11:48:23.1999774Z -------------------- Reports --------------------
... Check the output of the raw report
2023-03-17T11:48:27.2714156Z -------------------------------------------------
2023-03-17T11:48:27.2715166Z     Gzipping contents..
2023-03-17T11:48:27.2715506Z     Compressed contents to 284028 bytes
2023-03-17T11:48:27.2715861Z     Pinging Codecov...
2023-03-17T11:48:27.2716166Z     Uploading to S3...
2023-03-17T11:48:27.2716646Z     Uploading to S3 took 0:00:00.222914
2023-03-17T11:48:27.2717473Z e[92m    https://app.codecov.io/github/scikit-learn/scikit-learn/commit/91e6a0ae0604773520c9042f89eb877b24b43763e[0m
2023-03-17T11:48:27.2723631Z ##[section]Finishing: Upload To Codecov

Expected Results

We would expect the app to report coverage reports as in main but in PR.

Actual Results

For the PR above, we don’t get codecov report.

@glemaitre the python codecov uploader is deprecated, please switch to this uploader.

That said, it looks like there was an issue on our side in that time frame where you weren’t getting back a PR comment. I noticed that you are getting the Codecov statuses now, is that true?

Thanks, I will make the switch now.

On a recent PR (FIX Adds requires_y tag to TargetEncoder by thomasjpfan · Pull Request #25917 · scikit-learn/scikit-learn · GitHub) I cannot see any status reported.

Switching to the new uploader seems to have solved the issue. Thanks @tom

1 Like