Codecov GitHub action doesn't work well compared to bash uploader

Support Bug Fixes
1

Before submitting a topic, please confirm the following

[ x] I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
[ x] I have validated my codecov.yaml configuration file.
[ x] I have filled out the below sections to the best of my ability.
[x ] I understand that the community boards are a free, best-effort tool. While we hope to have someone on the Codecov team resolve your problem quickly, this is not always possible.

Description

CodeCov GitHub action coverage is low

CI/CD URL

Please provide a link to the CI/CD run.

Here’s the low coverage using the action:

Codecov Output

Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.

Output too long to post here, here’s the link:

Expected Results

Please provide what you expect to have happened (e.g. a file that has missing coverage on a particular line).

Here’s the results when I use the bash uploader:

Actual Results

Please provide what actually happened.

This were the results using the Codecov GitHub Action:

Additional Information

Please provide applicable commit SHAs or file names that are extraneous or missing. Any additional information, configuration, or data that might be necessary to reproduce the issue.

2

@acgetchell I’ll take a look at this. What I’m getting is that you have different coverage between using the bash uploader and the new uploader (which is what underlies the GitHub Action v3).

Can you provide a commit SHA using the bash uploader and one using the new one?

3

Bash uploader:

GitHub action:

4

@acgetchell the 2nd link has logs that are a bit too old (I can’t see what the Codecov output is). Do you have a more recent commit that uses the GitHub Action v2+?

5

Thanks @tom !

Here’s a recent commit which switches to using the CodeCov GitHub Action:

Here is the GitHub Workflow file:

You can see here that coverage drops by 62%:

Thanks again for looking into it!

6

@acgetchell, ok, so the first thing that I noticed is that the bash uploader runs gcov by default. We decided to not do that for the underlying node uploader for the Action@v3. Can you add the last line below

      - name: Submit to codecov.io
        uses: codecov/codecov-action@v3
        with:
          directory: build/gcov-reports
          fail_ci_if_error: false
          verbose: true
          gcov: true

and see if that fixes things

7

Okay, I did that.

Switching back from the bash uploader to the GitHub Action using gcov dropped my coverage from 84% to 22%:

8

@acgetchell sorry for the delay here. I had an idea and was working on a fix. Could you try running the action again, I just released a new version of it.

9

I tried the new action, coverage still dropped:

10

@acgetchell ok, I think I got this a little bit closer. I’m off on vacation tomorrow for a week, so I won’t be super responsive.

I made this PR off of your commit and it looks like removing this line

directory: build/gcov-reports

helps to find the coverage files. I also get coverage of 79%. Let me know if this looks good

11

Works with gcc-11, not gcc-12.

12

@acgetchell this looks like a problem with your conversion before running Codecov.

13

Yes, that is probably a mismatch between gcov and gcc versions. Let me see if that can be fixed.

14

Nope, even without gcov/gcc mismatch errors, it still fails on gcc-12:

But works on gcc-11:

15

@acgetchell it looks like there is an error when you are generating coverage files. See this line.

16

I think the gcc and gcov mismatch has been resolved in gcc 13.

Coverage is also now working well.