Codecov GitHub App: single repo authed, but no OAuth access to the wider GitHub Organisation

Before submitting a topic, please confirm the following

I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my codecov.yaml configuration file.
(NB: we don’t have a codecov.yaml in the repository).
I have filled out the below sections to the best of my ability.


Firstly, thanks for providing this tool. I’m hoping I can get it working as it looks great!

This question concerns the use of the Codecov GitHub app on a public open-source repository, located in an organisation which implements third-party app access restrictions. We cannot grant OAuth access to the organisation.

I am a member of the organisation, but not an admin. I asked our organisation admin to approve the access request I made for my individual repo within the org. They clicked through on the access request email and approved the GitHub app on the repo. As part of that approval flow, our org admin was then asked to allow OAuth access to (as I understand it) their github account and the entire organisation. We cannot grant access to all repositories within the organisation, as this runs counter to the principle of least privilege.

I initially put this down to weird UI flow design, but looking at this support post, I now think it might be a bigger limitation that we cannot allow OAuth access to the organisation.

I have been able to upload coverage results from our GitHub Actions workflow successfully, but in the Codecov dashboard get very similar results to this post. The first upload is showing fine, but each subsequent run is labelled commit message unavailable and the CI Status is Failed.


Codecov Output

Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.

Run codecov/codecov-action@v3
==> linux OS detected
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (20f9c9d78483fce977b6cc39e231a734a23bcd36f4d536bb7355222fb88d02bc  codecov)
==> Running version latest
==> Running version v0.3.2
/home/runner/work/_actions/codecov/codecov-action/v3/dist/codecov -n  -Q github-action-3.1.1
[2022-11-01T19:42:20.918Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/

  Codecov report uploader 0.3.2
[2022-11-01T19:42:20.926Z] ['info'] => Project root located at: /home/runner/work/ManyFEWS/ManyFEWS
[2022-11-01T19:42:20.926Z] ['info'] ->  Token found by environment variables
[2022-11-01T19:42:20.935Z] ['info'] Searching for coverage files...
[2022-11-01T19:42:21.021Z] ['info'] Warning: Some files located via search were excluded from upload.
[2022-11-01T19:42:21.021Z] ['info'] If Codecov did not locate your files, please review
[2022-11-01T19:42:21.021Z] ['info'] => Found 1 possible coverage files:
[2022-11-01T19:42:21.021Z] ['info'] Processing /home/runner/work/ManyFEWS/ManyFEWS/manyfews/coverage.xml...
[2022-11-01T19:42:21.043Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-11-01T19:42:21.045Z] ['info'] Pinging Codecov:*******&branch=main&build=3372319468&
[2022-11-01T19:42:21.904Z] ['info']
[2022-11-01T19:42:21.904Z] ['info'] Uploading...
[2022-11-01T19:42:22.108Z] ['info'] {"status":"success","resultURL":""}

Expected Results

Visibility of commit messages and CI status in Codecov dashboard.

Actual Results

commit message unavailable and CI workflow status failed (but actually passed in GitHub).

Additional Information

To reproduce:

  • A GitHub organisation with minimum two members, whose roles are admin and member respectively
  • Third-party OAuth Access Restrictions turned on for the organisation
  • A repository within the organisation, to request GitHub App Integration for the Codecov app on GitHub marketplace
  • Request app authorisation on GitHub as the member with least permissions, for “Only select repositories”.
  • As the admin, approve the GitHub app integration on the repository, but decline OAuth access to the organisation
  • Set up a CI/CD workflow which pushes results to Codecov, using a CODECOV_TOKEN
  • View results in Codecov dashboard. The first run will work, but subsequent runs show commit message unavailable

Potential Solutions

Potential solutions seem to involve configuring an account to use a Codecov Team Bot to provide access to commit messages. But, this is a public github repo. The documentation itself says “You do not need to set a Team Bot because Codecov will use the [GitHub App] integration to post statuses and comments.” It implies this shouldn’t be necessary, but perhaps in this instance, it is?

I note that this reply to the previously linked post indicates there is little interest in removing the necessity to request OAuth app permissions. I can add my “+1” to that, but, perhaps there is a potential Codecov documentation improvement to be made around organisations with Third Party App Restriction turned on?

I’m liking the tool so far and it seems perfect for what I need in terms of improving our test coverage. My main goal was in implementing the status badge for code coverage, and that works, but ideally I’d like to be able to make proper use of the full suite of metrics in the dashboard.

Hi @sjmf, thanks for your thoughtful and in-depth write-up. I can’t answer everything you have mentioned, but I’d like to inform you of what’s happening on our side.

  1. Codecov Team Bot, implementing this will probably not resolve the issue. It’s still a third-party app access restriction
  2. Migrating to GitHub App permissions, this is actively on our product roadmap. I don’t have a time estimate of when it’ll be complete, but we are willing to invest some resources here.
  3. Third-party OAuth Access permissions, I realize that this does not seem like a viable option for you at the time. I’m sorry, I don’t have any better answers here for you right now. Thanks for sticking with us while we move to a new set of GitHub permissions.
1 Like

Hi Tom– thanks for your reply and giving your time to address those points.

Really, it’s just good to know that this is on the roadmap for the codecov team. I can get by for now with what’s available, but will be looking forward to release for the GitHub App permission migration. :slight_smile: