Files missing from report

Support
,
1

Before submitting a topic, please confirm the following

I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my codecov.yaml configuration file.
I have filled out the below sections to the best of my ability.

Description

The open source project distributed is having issues in their coverage report, see Codecov

Specifically, the report is missing a couple of top-level files, e.g. scheduler.py, worker.py, semaphore.py.
Interestingly, this affects all top-level files that have a similarly named file in a subfolder. For instance, there is
distributed/scheduler.py
and
distributed/dashboard/scheduler.py
distributed/dashboard/components/scheduler.py
I verified that the uploaded coverage report includes the top level scheduler.py. The same is true for the other top-level files I mentioned above that have a sibling in a lower directory folder.

CI/CD URL

Codecov Output

Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.

Run codecov/codecov-action@v3
==> linux OS detected
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (20f9c9d78483fce977b6cc39e231a734a23bcd36f4d536bb7355222fb88d02bc  codecov)
==> Running version latest
==> Running version v0.3.2
/home/runner/work/_actions/codecov/codecov-action/v3/dist/codecov -n  -Q github-action-3.1.1
[2022-10-07T06:23:25.631Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/

  Codecov report uploader 0.3.2
[2022-10-07T06:23:25.640Z] ['info'] => Project root located at: /home/runner/work/distributed/distributed
[2022-10-07T06:23:25.645Z] ['info'] -> No token specified or token is empty
[2022-10-07T06:23:25.653Z] ['info'] Searching for coverage files...
[2022-10-07T06:23:25.740Z] ['info'] Warning: Some files located via search were excluded from upload.
[2022-10-07T06:23:25.740Z] ['info'] If Codecov did not locate your files, please review https://docs.codecov.com/docs/supported-report-formats
[2022-10-07T06:23:25.741Z] ['info'] => Found 1 possible coverage files:
  coverage.xml
[2022-10-07T06:23:25.741Z] ['info'] Processing /home/runner/work/distributed/distributed/coverage.xml...
[2022-10-07T06:23:25.759Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-10-07T06:23:25.760Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-3.1.1-uploader-0.3.2&token=*******&branch=main&build=3202612408&build_url=https%3A%2F%2Fgithub.com%2Fdask%2Fdistributed%2Factions%2Fruns%2F3202612408&commit=3d1f53dc57421a44181acb718a5f86864ab9be25&job=Tests&pr=&service=github-actions&slug=dask%2Fdistributed&name=&tag=&flags=&parent=
[2022-10-07T06:23:26.501Z] ['info'] https://app.codecov.io/github/dask/distributed/commit/3d1f53dc57421a44181acb718a5f86864ab9be25
https://storage.googleapis.com/codecov/v4/raw/2022-10-07/2AA2EC756C47ABD17A802CF8D77E3F7E/3d1f53dc57421a44181acb718a5f86864ab9be25/30c47129-54e8-4e58-9f5a-f35ccc9bb2db.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJOGFN2JQ4OCTGA2MU5AEIT7OT5Z7HTFOAN2SPG4NWSN2UJYOY5U6LZQ%2F20221007%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20221007T062326Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=53a24be7ac32c1f6b1bbd9fed7109593abd1aa92e01b568b941d8ef5098e7390
[2022-10-07T06:23:26.501Z] ['info'] Uploading...
[2022-10-07T06:23:26.800Z] ['info'] {"status":"success","resultURL":"https://app.codecov.io/github/dask/distributed/commit/3d1f53dc57421a44181acb718a5f86864ab9be25"}

Expected Results

All files are in the report

Actual Results

Top-level files with duplicates in sub folders are missing

Additional Information

2

@fjetter I’m wondering if there is a conflict in our path fixing. Can you try running the action with

- name: Upload coverage to Codecov
  uses: codecov/codecov-action@v3
  with:
    functionalities: network

in particular the last two lines

3

Thank you @tom ! I think this fixed the problem. At least the main view shows the files again but I do get an error when trying to load the file / line view.

I get something like There was a problem getting the source code from your provider. Unable to show line by line coverage.. I assume this is unrelated?

4

Ok, pretty sure it is related. I can get either source code or all files but never both. I tried a couple of different manual fixes but couldn’t get it to work. When I set functionalities: network it appears the path fixing doesn’t have any effect. is this correct?

5

Hi @fjetter I haven’t forgotten about this one. I’m trying to find a solution that works, but the big thing is that the coverage report doesn’t have the root folder specified which is giving us problems.

6

Is there any way for me to provide this? I’m happy to use a customer upload script or something like that if it helps. I really would like to get this working again

7

@fjetter I’m going to try to code up something really hacky. I’ll make a PR to the repo today.

1 Like
8

Thank you for looking into this. Let me know if there is something I should test

9

@fjetter I made a first pass here. Let’s wait and see the test results.

10

@fjetter looks like this is resolved due to the PR merge. Let me know if there’s anything else!

For any future readers,

What is the underlying problem?
The files listed in the coverage report do not match the files that come from doing git -C . ls-files. This caused name collisions on our side which cause some files from showing up.

How do I know if I have this problem?
Check your coverage reports for filenames. You can do this in the Codecov UI by clicking on an upload. You’ll notice a section at the top which lists all of the files from git -C . ls-files. These filenames should match with the filenames in your coverage reports below.

If they do not match and you are missing files, you are likely having the problem in this topic.

What is the fix?
Since the issue is that the filenames are not matching, we need to change them. You can run a step before you upload a coverage report to transform all the filenames. In this topic, I did something like

      - name: Prepare coverage report
        run: sed -i'' -e 's/filename="/filename="distributed\//g' coverage.xml

to append distributed/ to the beginning of all filenames. You can view the PR for the full change.