Codecov actions and comments suddenly appeared in PRs/CI

Support
,
1

Before submitting a topic, please confirm the following

I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my codecov.yaml configuration file.
I have filled out the below sections to the best of my ability.
I understand that the community boards are a free, best-effort tool. While we hope to have someone on the Codecov team resolve your problem quickly, this is not always possible.

Description

The codecov/patch and codecov/project actions appeared on pushes and PRs and codecov-commenter started commenting on PRs a few days ago (it seem it started on March 13, 2023). We have been using the codecov/codecov-action@v3 since September 2022. The usage of codecov has been introduced in March 2020 and we have never seen the aforementioned feedback.
I checked the release notes of the latest version of the action (v3.1.1) and there’s no mention of any changes or new versions since September 2022.

CI/CD URL

Expected Results

The actions/comments should have been opt-in via a configuration setting or a new branch of codecov/codecov-action

Actual Results

The actions/comments appeared without any changes.

Additional Information

I omitted some checkmarks since we neither use the App or do we have a configuration files. Both have not been necessary so far.

2

@firewave this is actually the expected behavior. There is a lot to unwrap here, so please let me know if this doesn’t make sense.

  1. Codecov’s default behavior is to send the codecov/patch and codecov/project status checks. The comment is also the default behavior.
  2. On some of the previous commits (without the status checks or comments), we did not have the proper credentials to send those status checks or comments.
  3. As such, you did not see any of the status checks or comments.
  4. Something happened in the past 2 days that gave us the proper credentials. It is likely that danmar (repo owner) signed in and refreshed an old token we had.
  5. As a result, you are now getting the expected behavior.

So how do I get rid of the comment and status checks?

You will need to add a codecov.yml file with the following

comment: false
coverage:
  status:
    project: off
    patch: off

That said, I would recommend that if the comment or statuses are not valuable, we would love to hear your feedback.

3

Thanks a lot for the fast and detailed reply.

I was wondering about that. Interestingly we did see the commit message on some of them but it was missing for most of them. They are also still shown as “processing” which seems to indicate that they might finish at some point in the future. Maybe that needs to be re-worded or handled differently if the credentials are not correct.

That is quite possible. We were evaluating including badges and actions into our PRs.

Thanks. I will add that for the time being and monitor if the information improves. I just don’t have the time at the moment to be completely on top of it.

I already filed some feedback and support requests. There will be more coming up based on we are seeing so far. Hopefully we can enable this again in the future.

4

For

They are also still shown as “processing” which seems to indicate that they might finish at some point in the future. Maybe that needs to be re-worded or handled differently if the credentials are not correct.

I would recommend installing the Codecov GitHub app. It’s usually how we communicate with GitHub.

5

That’s definitely something we will be looking into when we are preparing enabling it again. Doing incremental steps.

But it seems there might still be something wrong.

Starting with Codecov we were getting commit messages but somehow there are commits where it says No report uploaded yet but if you check the action you see that it it did do the upload.

Drilling into some minor `QtCharts` usage cleanups (#4890) · danmar/cppcheck@0b8af4f · GitHub you see that the workflow was restarted because it failed because of the following error: (I guess that’s the rate-limit issue you run into because of no token):

[2023-03-13T15:00:56.447Z] ['error'] There was an error running the uploader: Error uploading to https://codecov.io: Error: There was an error fetching the storage URL during POST: 404 - {'detail': ErrorDetail(string='Unable to locate build via Github Actions API. Please upload with the Codecov repository upload token to resolve issue.', code='not_found')}

So it seems that re-running a failed build does overwrite the previously incomplete one leading to the missing coverage data.

Then there is Codecov which has commit message unavailable although the previous commits were fine. Looking at Fix ctuuninitvar false positive for struct that is partly initialized · danmar/cppcheck@a4d2178 · GitHub there’s no apparent issues at all with the upload so that’s a bit baffling.

6

@firewave yes, this is due to not having a token as well as the GitHub app that I was talking about.

As for the commit message unavailable, this is the GitHub app as well.

7

It is strange that it is not consistently happening though. It is hard to tell with only so few commits since data actually appeared. So we should wait until there is actually some traffic on the repo to make conclusions.

8

@firewave, yup, the GitHub app helps us get past the rate-limiting issues with GitHub. Without the app, we can’t guarantee that we can get back successful API calls to GitHub.