Before submitting a topic, please confirm the following
I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my codecov.yaml configuration file.
(NB: we don’t have a codecov.yaml in the repository).
I have filled out the below sections to the best of my ability.
Description
Firstly, thanks for providing this tool. I’m hoping I can get it working as it looks great!
This question concerns the use of the Codecov GitHub app on a public open-source repository, located in an organisation which implements third-party app access restrictions. We cannot grant OAuth access to the organisation.
I am a member of the organisation, but not an admin. I asked our organisation admin to approve the access request I made for my individual repo within the org. They clicked through on the access request email and approved the GitHub app on the repo. As part of that approval flow, our org admin was then asked to allow OAuth access to (as I understand it) their github account and the entire organisation. We cannot grant access to all repositories within the organisation, as this runs counter to the principle of least privilege.
I initially put this down to weird UI flow design, but looking at this support post, I now think it might be a bigger limitation that we cannot allow OAuth access to the organisation.
I have been able to upload coverage results from our GitHub Actions workflow successfully, but in the Codecov dashboard get very similar results to this post. The first upload is showing fine, but each subsequent run is labelled commit message unavailable and the CI Status is Failed.
CI/CD URL
https://github.com/DurhamARC/ManyFEWS/actions/runs/3372319468
Codecov Output
Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.
Run codecov/codecov-action@v3
==> linux OS detected
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (20f9c9d78483fce977b6cc39e231a734a23bcd36f4d536bb7355222fb88d02bc codecov)
==> Running version latest
==> Running version v0.3.2
/home/runner/work/_actions/codecov/codecov-action/v3/dist/codecov -n -Q github-action-3.1.1
[2022-11-01T19:42:20.918Z] ['info']
_____ _
/ ____| | |
| | ___ __| | ___ ___ _____ __
| | / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| | __/ (_| (_) \ V /
\_____\___/ \__,_|\___|\___\___/ \_/
Codecov report uploader 0.3.2
[2022-11-01T19:42:20.926Z] ['info'] => Project root located at: /home/runner/work/ManyFEWS/ManyFEWS
[2022-11-01T19:42:20.926Z] ['info'] -> Token found by environment variables
[2022-11-01T19:42:20.935Z] ['info'] Searching for coverage files...
[2022-11-01T19:42:21.021Z] ['info'] Warning: Some files located via search were excluded from upload.
[2022-11-01T19:42:21.021Z] ['info'] If Codecov did not locate your files, please review https://docs.codecov.com/docs/supported-report-formats
[2022-11-01T19:42:21.021Z] ['info'] => Found 1 possible coverage files:
manyfews/coverage.xml
[2022-11-01T19:42:21.021Z] ['info'] Processing /home/runner/work/ManyFEWS/ManyFEWS/manyfews/coverage.xml...
[2022-11-01T19:42:21.043Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-11-01T19:42:21.045Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-3.1.1-uploader-0.3.2&token=*******&branch=main&build=3372319468&build_url=https%3A%2F%2Fgithub.com%2FDurhamARC%2FManyFEWS%2Factions%2Fruns%2F3372319468&commit=303c5d18ae220fa9271a8ede43764bc9e21ee706&job=Unit+Tests&pr=&service=github-actions&slug=DurhamARC%2FManyFEWS&name=&tag=&flags=&parent=
[2022-11-01T19:42:21.904Z] ['info'] https://app.codecov.io/github/DurhamARC/ManyFEWS/commit/303c5d18ae220fa9271a8ede43764bc9e21ee706
https://storage.googleapis.com/codecov/v4/raw/2022-11-01/2BFA8792505232B8964C241D1D7D34EC/303c5d18ae220fa9271a8ede43764bc9e21ee706/cd3acb36-e4d5-4328-8cd6-35e1200061ac.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJOGFN2JQ4OCTGA2MU5AEIT7OT5Z7HTFOAN2SPG4NWSN2UJYOY5U6LZQ%2F20221101%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20221101T194221Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=69689cd585c5870be4756f79c072b5d09bd89b42882c3a16294e3febdf1277d0
[2022-11-01T19:42:21.904Z] ['info'] Uploading...
[2022-11-01T19:42:22.108Z] ['info'] {"status":"success","resultURL":"https://app.codecov.io/github/DurhamARC/ManyFEWS/commit/303c5d18ae220fa9271a8ede43764bc9e21ee706"}
Expected Results
Visibility of commit messages and CI status in Codecov dashboard.
Actual Results
commit message unavailable and CI workflow status failed (but actually passed in GitHub).
Additional Information
To reproduce:
- A GitHub organisation with minimum two members, whose roles are admin and member respectively
- Third-party OAuth Access Restrictions turned on for the organisation
- A repository within the organisation, to request GitHub App Integration for the Codecov app on GitHub marketplace
- Request app authorisation on GitHub as the member with least permissions, for “Only select repositories”.
- As the admin, approve the GitHub app integration on the repository, but decline OAuth access to the organisation
- Set up a CI/CD workflow which pushes results to Codecov, using a CODECOV_TOKEN
- View results in Codecov dashboard. The first run will work, but subsequent runs show
commit message unavailable
Potential Solutions
Potential solutions seem to involve configuring an account to use a Codecov Team Bot to provide access to commit messages. But, this is a public github repo. The documentation itself says “You do not need to set a Team Bot because Codecov will use the [GitHub App] integration to post statuses and comments.” It implies this shouldn’t be necessary, but perhaps in this instance, it is?
I note that this reply to the previously linked post indicates there is little interest in removing the necessity to request OAuth app permissions. I can add my “+1” to that, but, perhaps there is a potential Codecov documentation improvement to be made around organisations with Third Party App Restriction turned on?
I’m liking the tool so far and it seems perfect for what I need in terms of improving our test coverage. My main goal was in implementing the README.md status badge for code coverage, and that works, but ideally I’d like to be able to make proper use of the full suite of metrics in the dashboard.
