Something broke ca. 25 days ago. How can I find out what exactly?


A month ago, my codecov setup was working properly.
It includes both C++ tests (with CMake, Catch2, gcov) and Python tests.
In a commit 25 days ago, that did not include any changes to CI or CMake or gcov or codecov.yaml, the setup stopped working: only the Python tests are reported.

Now, I see that I might should be reporting this is the “But Fixes” rather than the Support category, but I can also imagine that my initial setup was only working because of a bug that was fixed; or that GitHub changed something about the directory structure that I did not notice.


Unfortunately, for now, it is a private GitHub repository ( The relevant codecov upload (GitHub action workflow file) looks like this:

name: Run Tests

      - master
      - main

    runs-on: ubuntu-latest

    name: Run Python + C++ Tests
      - uses: actions/checkout@v3
          submodules: "recursive"

      - uses: actions/cache@v2
          path: |
          key: ${{ runner.os }}-py-3.10.6-${{ hashFiles('**/requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-py-3.10.6-
      - uses: actions/setup-python@v4
          python-version: '3.10.6'
          cache: 'pip'

      - name: Install Dependencies
        run: pip install -r requirements.txt
      - name: Install dev dependencies
        run: sudo apt-get install bison flex lcov libeigen3-dev
      - name: Run Tests
        run: ./bin/
      # Generate Coverage Badge
      - name: Install Badge Generator
        run: pip install genbadge[coverage]

      - name: Coverage Badge
        run: genbadge coverage -i coverage.xml -o ".github/coverage.svg"
      - name: Upload to CodeCov
        uses: codecov/codecov-action@v3
          token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos
          fail_ci_if_error: false # optional (default = false)
          verbose: true # optional (default = false)
          # gcov: true

Codecov Output

The full output of running the uploader on your CI/CD.

==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (20f9c9d78483fce977b6cc39e231a734a23bcd36f4d536bb7355222fb88d02bc  codecov)
==> Running version latest
==> Running version v0.3.2
/home/runner/work/_actions/codecov/codecov-action/v3/dist/codecov -n  -Q github-action-3.1.1 -v
[2022-12-21T15:13:55.562Z] ['verbose'] Start of uploader: 1671635635562...
[2022-12-21T15:13:55.564Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/

  Codecov report uploader 0.3.2
[2022-12-21T15:13:55.570Z] ['info'] => Project root located at: /home/runner/work/pylimer-tools/pylimer-tools
[2022-12-21T15:13:55.574Z] ['info'] ->  Token found by environment variables
[2022-12-21T15:13:55.575Z] ['verbose'] Start of network processing...
[2022-12-21T15:13:55.575Z] ['verbose'] Searching for files in /home/runner/work/pylimer-tools/pylimer-tools
[2022-12-21T15:13:55.712Z] ['info'] Running coverage xml...
[2022-12-21T15:13:55.986Z] ['verbose'] Wrote XML report to coverage.xml
[2022-12-21T15:13:55.986Z] ['info'] Searching for coverage files...
[2022-12-21T15:13:56.337Z] ['info'] Warning: Some files located via search were excluded from upload.
[2022-12-21T15:13:56.337Z] ['info'] If Codecov did not locate your files, please review
[2022-12-21T15:13:56.337Z] ['verbose'] Preparing to clean the following coverage paths: codecov.yaml,coverage.xml,tests/build/#home#runner#work#pylimer-tools#pylimer-tools#src#pylimer_tools_cpp#MyClass.cpp.gcov 
// ...etc., very many .gcov files redacted
[2022-12-21T15:13:56.341Z] ['info'] => Found 489 possible coverage files:
// long list of gcov files again, redacted for brevity
[2022-12-21T15:13:56.341Z] ['verbose'] End of network processing
[2022-12-21T15:13:56.341Z] ['info'] Processing /home/runner/work/pylimer-tools/pylimer-tools/codecov.yaml...
[2022-12-21T15:13:56.343Z] ['info'] Processing /home/runner/work/pylimer-tools/pylimer-tools/coverage.xml...
[2022-12-21T15:13:56.343Z] ['info'] Processing /home/runner/work/pylimer-tools/pylimer-tools/tests/build/#home#runner#work#pylimer-tools#pylimer-tools#src#pylimer_tools_cpp#MyClass.cpp.gcov 
// etc., redacted for brevity
[2022-12-21T15:13:57.166Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-12-21T15:13:57.166Z] ['verbose'] -> Using the following env variables:
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_ACTION: __codecov_codecov-action
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_HEAD_REF: 
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_REF: refs/heads/main
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_REPOSITORY: GenieTim/pylimer-tools
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_RUN_ID: ...
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_SERVER_URL:
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_SHA: ...
[2022-12-21T15:13:57.166Z] ['verbose']      GITHUB_WORKFLOW: Run Tests
[2022-12-21T15:13:57.166Z] ['verbose'] Using the following upload parameters:
[2022-12-21T15:13:57.166Z] ['verbose'] branch
[2022-12-21T15:13:57.166Z] ['verbose'] build
[2022-12-21T15:13:57.166Z] ['verbose'] buildURL
[2022-12-21T15:13:57.166Z] ['verbose'] commit
[2022-12-21T15:13:57.166Z] ['verbose'] job
[2022-12-21T15:13:57.166Z] ['verbose'] pr
[2022-12-21T15:13:57.166Z] ['verbose'] service
[2022-12-21T15:13:57.166Z] ['verbose'] slug
[2022-12-21T15:13:57.166Z] ['verbose'] name
[2022-12-21T15:13:57.166Z] ['verbose'] tag
[2022-12-21T15:13:57.166Z] ['verbose'] flags
[2022-12-21T15:13:57.166Z] ['verbose'] parent
[2022-12-21T15:13:57.168Z] ['info'] Pinging Codecov:*******&branch=main&build=3750309808&
[2022-12-21T15:13:57.169Z] ['verbose'] Passed token was 36 characters long
[2022-12-21T15:13:57.169Z] ['verbose']
        Content-Type: 'text/plain'
        Content-Encoding: 'gzip'
        X-Reduced-Redundancy: 'false'
[2022-12-21T15:13:57.828Z] ['info']
[2022-12-21T15:13:57.829Z] ['verbose'] Returned upload url:
[2022-12-21T15:13:57.829Z] ['info'] Uploading...
[2022-12-21T15:13:58.141Z] ['info'] {"status":"success","resultURL":""}
[2022-12-21T15:13:58.141Z] ['verbose'] End of uploader: 2579 milliseconds

Expected Results

Coedecoverage being reported consistently.
As I understand the logs, all the required gcov files are uploaded.

Actual Results

C++ test’s coverage is no longer listed.

Additional Information

And this is the content of my codecov.yaml:

  # strict_yaml_branch: "yaml-config"
  # max_report_age: 24
  disable_default_path_fixes: no
  require_ci_to_pass: yes
  # notify:
  #   after_n_builds: 2
  #   wait_for_ci: yes
  - "/home/runner/work/pylimer-tools/pylimer-tools/::/"

@GenieTim, it seems like there is something going wrong with how you are collecting coverage for C++ files. If you take a look at the coverage files for 19224f3d compared to bef924e0, you’ll see that some files are showing covered lines previously, while those recently are showing no coverage. I’m being a little vague with details on this because it’s a private repo.

Have you changed versions of gcov or how you collect coverage reports? Codecov looks to be showing the coverage as reported.

Thank you for your response, tom. Sorry for answering late, but I needed some time to figure things out; I have not yet managed to understand, what it is, that changed, maybe it was a version of a program in the GitHub Action, or something, but at least I got back to good coverage.

Your point of looking into the files that arrive in Codecov was a good one. They look different from what is generated on my local computer.

By outputting the contents of an example gcov file after each step in the CI, I found that the point where my coverage was reset was when running cmake pylimer_tools-gcov (as suggested by the used cmake-gcov integration), and that the gcov file was already populated after running the tests before that.

By just not running the cmake pylimer_tools-gcov step I got back to a nice amount of coverage.

@GenieTim absolutely, thanks for replying with all the juicy details. I wonder why cmake pylimer_tools-gcov reset the coverage files. I’ll have to dig in a little at some point.

Just wanted to double-check that things are looking good for you now?

Yes, I am wondering as well, especially since it does not make sense to me that this would change that way. Maybe there are other factors at play as well that I have not yet understood.
As it works for now, I will not prioritize any further investigations on my part until I get problems again.

Yes, things are looking good for me now, I guess I could accept any of our answers as the solution.

1 Like