Issues using Codecov on Gitlab.com (public/OSS) - reports upload, partial info on codecov site, no bot messages

Support
1

I have three open source project repos on Gitlab. Each of them are configured to upload to Codecov as part of their CI. None of them seem to… completely… work with it though. But also, none of them are “broken” in quite the same way. I have detailed the issues below.

Note that I recently moved these from public projects under my personal account to public projects under a group. This was to qualify for Gitlab’s open source CI benefits. The issues I describe were the same before the move though.

I’ll break it down for each repo:

Repo 1 (Rust Test Binary)

Output from Codecov run:

$ ./codecov -R $CI_PROJECT_DIR
[2023-01-22T17:55:06.375Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/
  Codecov report uploader 0.3.2
[2023-01-22T17:55:06.378Z] ['info'] => Project root located at: /builds/rust-test-binary/test-binary
[2023-01-22T17:55:06.379Z] ['info'] ->  Token found by environment variables
[2023-01-22T17:55:06.409Z] ['info'] Searching for coverage files...
[2023-01-22T17:55:06.565Z] ['info'] Warning: Some files located via search were excluded from upload.
[2023-01-22T17:55:06.565Z] ['info'] If Codecov did not locate your files, please review https://docs.codecov.com/docs/supported-report-formats
[2023-01-22T17:55:06.566Z] ['info'] => Found 1 possible coverage files:
  cobertura.xml
[2023-01-22T17:55:06.566Z] ['info'] Processing /builds/rust-test-binary/test-binary/cobertura.xml...
[2023-01-22T17:55:06.594Z] ['info'] Detected GitLab CI as the CI provider.
[2023-01-22T17:55:06.598Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=uploader-0.3.2&token=*******&branch=show-compiler-errors-on-output&build=3641644845&build_url=&commit=08bb923eb99d30b89845f4d7a4562bfc0959c425&job=&pr=&service=gitlab&slug=rust-test-binary%2Ftest-binary&name=&tag=&flags=&parent=
[2023-01-22T17:55:06.853Z] ['info'] https://app.codecov.io/gitlab/rust-test-binary/test-binary/commit/08bb923eb99d30b89845f4d7a4562bfc0959c425
https://storage.googleapis.com/codecov/v4/raw/2023-01-22/C75E7D4D5CDCF8CF6F5EFE2210F07108/08bb923eb99d30b89845f4d7a4562bfc0959c425/244c0fc5-7fd9-4530-a238-0ac4ca2ceeb4.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJOGFN2JQ4OCTGA2MU5AEIT7OT5Z7HTFOAN2SPG4NWSN2UJYOY5U6LZQ%2F20230122%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20230122T175506Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=f9c71845c5788b32be6affcf77edc73847010a9ba3e95c176e597eadff6738a7
[2023-01-22T17:55:06.854Z] ['info'] Uploading...
[2023-01-22T17:55:07.048Z] ['info'] {"status":"success","resultURL":"https://app.codecov.io/gitlab/rust-test-binary/test-binary/commit/08bb923eb99d30b89845f4d7a4562bfc0959c425"}

Seems to be successful, so I assume this means I have the token correctly entered in my CI settings. I’ve confirmed it matches the one from the repo settings.

Issues:

  1. Despite the upload appearing to complete successfully from this MR, the MR does not appear in the list. (Cannot post screenshot due to forum policy.)

  2. I can browse to the branch on the commits page but all commits are listed as “commit message unavailable”. (Cannot post screenshot due to forum policy.)

  3. There are no coverage statistics messages on the MR from Codecov. As far as I can tell, this is what is documented as the “Team Bot” feature? But the instructions say I need to create a new user on Gitlab for it, which I can’t do, because I can’t create multiple Gitlab accounts with the same email.

Repo 2 (Calloop ZeroMQ)

Output from Codecov run:

$ ./codecov -R $CI_PROJECT_DIR
[2023-02-13T05:32:46.777Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/
  Codecov report uploader 0.3.2
[2023-02-13T05:32:46.779Z] ['info'] => Project root located at: /builds/calloop-zmq/calloop-zmq
[2023-02-13T05:32:46.780Z] ['info'] ->  Token found by environment variables
[2023-02-13T05:32:46.825Z] ['info'] Searching for coverage files...
[2023-02-13T05:32:47.047Z] ['info'] Warning: Some files located via search were excluded from upload.
[2023-02-13T05:32:47.051Z] ['info'] If Codecov did not locate your files, please review https://docs.codecov.com/docs/supported-report-formats
[2023-02-13T05:32:47.052Z] ['info'] => Found 1 possible coverage files:
  cobertura.xml
[2023-02-13T05:32:47.052Z] ['info'] Processing /builds/calloop-zmq/calloop-zmq/cobertura.xml...
[2023-02-13T05:32:47.062Z] ['info'] Detected GitLab CI as the CI provider.
[2023-02-13T05:32:47.067Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=uploader-0.3.2&token=*******&branch=add-tests&build=3758356074&build_url=&commit=a768193d557a0c8c4c7fd0e0b2482afe70337e2b&job=&pr=&service=gitlab&slug=calloop-zmq%2Fcalloop-zmq&name=&tag=&flags=&parent=
[2023-02-13T05:32:47.300Z] ['info'] https://app.codecov.io/gitlab/calloop-zmq/calloop-zmq/commit/a768193d557a0c8c4c7fd0e0b2482afe70337e2b
https://storage.googleapis.com/codecov/v4/raw/2023-02-13/B02131A76A1D7B519BFF3BB173E00306/a768193d557a0c8c4c7fd0e0b2482afe70337e2b/a3f1034a-2bfd-497f-914b-438b7b66f024.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJOGFN2JQ4OCTGA2MU5AEIT7OT5Z7HTFOAN2SPG4NWSN2UJYOY5U6LZQ%2F20230213%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20230213T053247Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=1175b6daf218d4bbb7eff2dbd05af7a9954f7512cacbeae5f41d1e947438f25e
[2023-02-13T05:32:47.301Z] ['info'] Uploading...
[2023-02-13T05:32:47.494Z] ['info'] {"status":"success","resultURL":"https://app.codecov.io/gitlab/calloop-zmq/calloop-zmq/commit/a768193d557a0c8c4c7fd0e0b2482afe70337e2b"}

The MR does appear in the list! But:

  1. The commit message is also unavailable on the commits page. (Cannot post screenshot due to forum policy.)

  2. There does appear to be a “Team Bot” message on this MR… from 6 months ago…? The MR was up 6 months ago, and I did see the message then, but my point is there’s nothing since, nor on any other project. I also didn’t have to do anything to get this to happen (eg. no adding a user to the projects on codecov.io). Clicking on the username takes me to this profile page. The account shown there seems to have performed no activity for months.

    Screenshot from 2023-02-13 14-00-52
    Screenshot from 2023-02-13 14-00-52681×787 57.1 KB

Repo 3 (Calloop Subprocess)

I won’t dump the details of this one like with the other two, but basically: this one has commits properly listed on codecov.io, but no Team Bot message on the MRs.

I realise this is a massive dump of symptoms pulled from different places, but they all relate to the same single thing — I cannot figure out how to get Codecov to work on Gitlab projects, the documentation is lacking some details, and there seems to be a non-operational(?) Codecov bot account on Gitlab that isn’t doing anything but… maybe should be?

What do I need to do to fully use Codecov on these projects? Do you need any more information about these issues?

2

Hi @detly if you don’t mind making separate threads for all 3 of the issues, it will help back and forth. Also, filling out the template will also make things easier for debugging.

For Repo 1, this smells like a team bot issue. You don’t need to make a GitLab account with the same email, it just needs to be a part of the organization and have the right permissions (like setting statuses and reading the repo). It can be your own username, as well if you have the correct permission set.

3

Sure thing, but do you mean I should make a separate thread for each repo, or a separate thread for each issue, which might mention all three repos that I see it in?

Also, I don’t understand this:

I can’t make a Gitlab account with the same email. You can’t make multiple Gitlab accounts off one email. But also

How can the Team Bot also be my own account? How would I log in to Codecov as a Team Bot when I can only log in with my Gitlab account, which already gives me my existing Codecov account.

(Also, why is there a Codecov account on Gitlab, and why did it comment on one of my repos, but only one?)

4

Sure thing, but do you mean I should make a separate thread for each repo, or a separate thread for each issue, which might mention all three repos that I see it in?

That would be a separate topic for each of the issues.

I can’t make a Gitlab account with the same email. You can’t make multiple Gitlab accounts off one email. But also

This isn’t relevant to your situation, but you could have used a different email to make a new GitLab account. That new GitLab account would need to be added to your GitLab org and given access to the repos in question.

Again, that’s not important if you are using your own account as the Team Bot.

How can the Team Bot also be my own account?

The team bot can be your own account (if your account has the necessary permissions). You need to set it by following these instructions.

How would I log in to Codecov as a Team Bot when I can only log in with my Gitlab account, which already gives me my existing Codecov account.

You would log in to Codecov as yourself using your GitLab account.

(Also, why is there a Codecov account on Gitlab, and why did it comment on one of my repos, but only one?)

Because in lieu of a team bot, we try to notify you of changes. This is not always possible, but you would need to give me more details here to tell you why explicitly.

5

Some issues with the instructions:

Step 3 is Set the bot in the Team Yaml. I have “repository yml” and “global yml”. Neither are named “Team Yaml”. The rest of this step says by navigating to /account/<PROVIDER>/<ORG>/yaml. So I can infer that PROVIDER is eg. Gitlab, and ORG is the group name, and sure enough, that takes me to the “global yml” section of the settings for the rust-test-binary group.

But these names don’t match up - with Codecov’s own terms, nor Gitlab’s. “Team yml” actually means “Global yml”. What Codecov calls an “org” (and also a “team” on the very same page), Gitlab calls a “group” (of which there may be many under an organisation’s account, which is only relevant if we’re dealing with an organisation, which we aren’t here). This is confusing when you’re already trying to figure out if you even need to create a Team Bot account or not.

Anyway, I gave this a go - I put this

codecov:
  bot: detly

into the global yaml. I then re-ran the job for code coverage on my MR’s pipeline. There is no statistics post on the MR yet though.

I’d argue that it’s more confusing to have intermittent messages from an official-looking account than to simply not have it. I spent ages trying to figure out why it worked once, and what I did to “break” it.

What details would you need? By the look of its public activity, it’s not just me that it’s stopped working for, right?

6

Also does that mean the Team Bot would be posting “as me” on Gitlab? That might be confusing.

7

Set the bot in the Team Yaml

Thanks, looks like we need to update the docs on this page :grimacing:.

Next steps

Looks like you’re running into a GitLab token issue. Can you log out of Codecov, remove all Codecov cookies, and log back in? That should refresh the token. Then, push up a new commit.

8

I believe that it will still comment as Codecov. Here is an example on an older MR.

9

I’ve logged out of codecov.io, cleared all site data in Firefox, logged back in (which prompted a re-auth with Gitlab).

Does it need to be a new commit or can I just run the Codecov job again? I pushed a new, empty commit just to be sure.

The MR currently shows no Team Bot message, I will check back in a while in case it takes some time.

10

Well, it’s been a couple of days, so I think I can safely say that wasn’t the issue.

11

@detly, yeah I think this is a different issue from our side. We made a small change, but I’m not totally positive it’s the right fix. Do you mind pushing a new commit/re-running the Codeocv job to that MR?

12

It works!

Oh, except Codecov is now adding failing jobs to my CI pipeline. Seems to have resolved itself eventually. It works!