# Codecov.yml Ignore not working

**URL:** <https://community.codecov.com/t/codecov-yml-ignore-not-working/794>\
**Category:** Bug Fixes\
**Created:** [December 12, 2019, 9:32pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794 "2019-12-12T21:32:10Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ornias1993](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/ornias1993/32/428_2.png) [@Ornias1993](https://community.codecov.com/u/Ornias1993)\
**Post date:** [December 12, 2019, 9:32pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/1 "2019-12-12T21:32:10Z")

</div>

### Description

As requested here:  
[https://twitter.com/thomasrockhu/status/1204075395978584064](https://twitter.com/thomasrockhu/status/1204075395978584064)

I hereby report the following bug:  
Adding “Ignore” to the codecov.yml file in following project, does not work:

> <https://github.com/openzfs/zfs/blob/master/.github/codecov.yml>

Other settings from the codecov.yml work fine, making changes to those also can be previewed fine in a Pull Request. As can be seen here:  
[CANT POST LINK 1]  
(ignore the wrong ignore syntax in that PR, this has been fixed later)

We tested multiple writingstyles, paths and moving codecov.yml to root in 2 PR’s, which did not work out:  
[CANT POST LINK 2]  
[CANT POST LINK 3]

Goal is to make sure the following path is fully excluded:  
/tests/\*\*/\*  
(/tests, recursively very file and folder under it)

### Repository

See above

### Steps to Reproduce

See above, no known way to reproduce and as it is very much connected to a whole test suite not feasable to repeat outside this repo.

**Expected behavior:**  
Those tests folders being completely ignored

**Actual behavior:**  
No ignore happening

**Flakiness?**  
All the time

### Versions

Irrelevant

### Additional Information

codecov validate results:

```
   {
      "comment": {
        "require_changes": true,
        "layout": "reach, diff, flags, footer",
        "behavior": "once"
      },
      "ignore": [
        "^tests/.*/.*"
      ],
      "coverage": {
        "status": {
          "project": {
            "default": {
              "threshold": 1.0
            }
          },
          "patch": {
            "default": {
              "threshold": 1.0
            }
          }
        },
        "range": [
          50.0,
          90.0
        ],
        "precision": 0,
        "round": "nearest"
      },
      "codecov": {
        "notify": {
          "after_n_builds": 2
        },
        "require_ci_to_pass": false
      }
    }

```

Taking above results into account this might be related to this:  
[CANT POST LINK 4]

---

<div class="post-metadata">

**Author:** ![Ornias1993](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/ornias1993/32/428_2.png) [@Ornias1993](https://community.codecov.com/u/Ornias1993)\
**Post date:** [December 12, 2019, 9:33pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/2 "2019-12-12T21:33:05Z")

</div>

[CANT POST LINK 1]

> <https://github.com/openzfs/zfs/pull/9650/files>
>
> Codecov has been hit-or-mis for a time now
> 
> It seems to have 2 flaws:
> \- Tests… are actually taken into account when comparing coverage
> \- There is a random variance in coverage leading in the 0.0-0.4 range.
> 
> 
> \### Motivation, Context and Description
> \- As far as I know there is no plan to write tests to cover tests, so this folder could just as well be excluded.
> \- Precision has been limited to whole percents only, but will round to nearest. This means 0.0-0.49 will round to zero (no change) and 0.51 will round to 1%. 
> 
> This might also lead to a more realistic and stable codecov report.
> 
> \### How Has This Been Tested?
> Automated tests for as far as thats even possible
> 
> \### Types of changes
> \- \[X\] Bug fix (non-breaking change which fixes an issue)
> \- \[\] New feature (non-breaking change which adds functionality)
> \- \[\] Performance enhancement (non-breaking change which improves efficiency)
> \- \[X\] Code cleanup (non-breaking change which makes code smaller or more readable)
> \- \[\] Breaking change (fix or feature that would cause existing functionality to change)
> \- \[\] Documentation (a change to man pages or other documentation)
> 
> \### Checklist:
> \- \[X\] My code follows the ZFS on Linux \[code style requirements\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#coding-conventions).
> \- \[X\] I have updated the documentation accordingly.
> \- \[X\] I have read the \[\*\*contributing\*\* document\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md).
> \- \[X\] I have added \[tests\](https://github.com/zfsonlinux/zfs/tree/master/tests) to cover my changes.
> \- \[X\] All new and existing tests passed.
> \- \[X\] All commit messages are properly formatted and contain \[\`Signed-off-by\`\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#signed-off-by).

[CANT POST LINK 2]

> <https://github.com/openzfs/zfs/pull/9695>
>
> Test using relative pathing for codecov yml.
> Personal experiment based on reque…st by coedcov support
> If moving works, maybe relative pathing too.
> 
> Both PR's added at once, because I would've the results of both by tomorrow morning (my timezone)
> 
> 
> \## TBD:
> 
> \### Motivation and Context
> 
> 
> 
> \### Description
> 
> 
> \### How Has This Been Tested?
> 
> 
> 
> 
> 
> \### Types of changes
> 
> \- \[\] Bug fix (non-breaking change which fixes an issue)
> \- \[\] New feature (non-breaking change which adds functionality)
> \- \[\] Performance enhancement (non-breaking change which improves efficiency)
> \- \[\] Code cleanup (non-breaking change which makes code smaller or more readable)
> \- \[\] Breaking change (fix or feature that would cause existing functionality to change)
> \- \[\] Documentation (a change to man pages or other documentation)
> 
> \### Checklist:
> 
> 
> \- \[\] My code follows the ZFS on Linux \[code style requirements\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#coding-conventions).
> \- \[\] I have updated the documentation accordingly.
> \- \[\] I have read the \[\*\*contributing\*\* document\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md).
> \- \[\] I have added \[tests\](https://github.com/zfsonlinux/zfs/tree/master/tests) to cover my changes.
> \- \[\] All new and existing tests passed.
> \- \[\] All commit messages are properly formatted and contain \[\`Signed-off-by\`\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#signed-off-by).

---

<div class="post-metadata">

**Author:** ![Ornias1993](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/ornias1993/32/428_2.png) [@Ornias1993](https://community.codecov.com/u/Ornias1993)\
**Post date:** [December 12, 2019, 9:33pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/3 "2019-12-12T21:33:39Z")

</div>

[CANT POST LINK 3]

> <https://github.com/openzfs/zfs/pull/9694>
>
> This moves codecov.yml to root
> As requested by codecov support
> see: https://tw…itter.com/drazisil/status/1203338067676934144
> 
> \## TBD
> 
> \### Motivation and Context
> 
> 
> 
> \### Description
> 
> 
> \### How Has This Been Tested?
> 
> 
> 
> 
> 
> \### Types of changes
> 
> \- \[\] Bug fix (non-breaking change which fixes an issue)
> \- \[\] New feature (non-breaking change which adds functionality)
> \- \[\] Performance enhancement (non-breaking change which improves efficiency)
> \- \[\] Code cleanup (non-breaking change which makes code smaller or more readable)
> \- \[\] Breaking change (fix or feature that would cause existing functionality to change)
> \- \[\] Documentation (a change to man pages or other documentation)
> 
> \### Checklist:
> 
> 
> \- \[\] My code follows the ZFS on Linux \[code style requirements\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#coding-conventions).
> \- \[\] I have updated the documentation accordingly.
> \- \[\] I have read the \[\*\*contributing\*\* document\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md).
> \- \[\] I have added \[tests\](https://github.com/zfsonlinux/zfs/tree/master/tests) to cover my changes.
> \- \[\] All new and existing tests passed.
> \- \[\] All commit messages are properly formatted and contain \[\`Signed-off-by\`\](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#signed-off-by).

Mod, please add these links to the opening posts for readability and remove these additional posts)  
(an serieusly, if it’s this easy to circumvent that link limit, please just remove the link limit \>.\< )

---

<div class="post-metadata">

**Author:** ![Ornias1993](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/ornias1993/32/428_2.png) [@Ornias1993](https://community.codecov.com/u/Ornias1993)\
**Post date:** [December 12, 2019, 9:54pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/4 "2019-12-12T21:54:08Z")

</div>

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

---

<div class="post-metadata">

**Author:** ![Ornias1993](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/ornias1993/32/428_2.png) [@Ornias1993](https://community.codecov.com/u/Ornias1993)\
**Post date:** [December 15, 2019, 10:45am UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/5 "2019-12-15T10:45:01Z")

</div>

Taking the following into account:

- There are multiple references to this issue/bug
- They are being mostly ignored
- Support is acting like they don’t know those reports exist
- When confronted support starts: Lying, Putting up smokescreens, Ignoring, putting blame on unrelated “issues” and so forth…

I hereby conclude Codecov is unwilling and/or unable to fix this bug.

I’m just going the way of ignoring before content even gets send to Codecov, because we surely cant rely on codecov.

Maybe it would help if their developers where putting more time in development and less time in enforcing wrong english pronoun use? (no you are still not multiple people, otherwise you might actually have manpower to… you know… fix these bugs you keep ignoring 😉 )

---

<div class="post-metadata">

**Author:** ![drazisil](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/drazisil/32/496_2.png) [@drazisil](https://community.codecov.com/u/drazisil)\
**Post date:** [December 16, 2019, 2:51pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/7 "2019-12-16T14:51:30Z")

</div>

I’m investigating this.

---

<div class="post-metadata">

**Author:** ![jerrod](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/jerrod/32/41_2.png) [@jerrod](https://community.codecov.com/u/jerrod)\
**Post date:** [December 20, 2019, 7:31pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/8 "2019-12-20T19:31:37Z")

</div>

Crossposting our response on this matter here: [Coverage report ignoring the configuration file on main branch - #12 by jerrod](https://community.codecov.com/t/coverage-report-ignoring-the-configuration-file-on-main-branch/618/12)

---

<div class="post-metadata">

**Author:** ![hootener](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/hootener/32/15_2.png) [@hootener](https://community.codecov.com/u/hootener)\
**Post date:** [December 20, 2019, 7:33pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/9 "2019-12-20T19:33:55Z")

</div>

(Edit: it looks like @jerrod has crossposted another response as well. Both posts lead to the same solution. I’m leaving this one here, though, since it specifically addresses your issue using your own project’s commit history, @Ornias1993 .)

Hi @Ornias1993,

My name is Eli, and I’m the CTO of Codecov. Your posts here were brought to my attention by the support team, and I felt that I needed to step in a bit and address your issue directly.

First, a dive into your issue, what’s causing it, and how to remedy it…

Your core issue stems from the fact that your codecov.yml file is located in a subdirectory of your project. While our docs state that this behavior is supported, a deep dive of our codebase makes me believe that this functionality was never actually working fully as expected. The fast remedy to your solution is one of the following:

1. move the codecov.yml to the project’s root directory
2. Uploading using the -y flag to specify the path to your codecov.yml in your project.

For the curious, a bit of a forensic investigation into how we came to this conclusion…

There was a commit in your repository, shown here:

[https://codecov.io/gh/zfsonlinux/zfs/tree/f12999148d2f3620c39bfebbb6ffd3c95009bc23](https://codecov.io/gh/zfsonlinux/zfs/tree/f12999148d2f3620c39bfebbb6ffd3c95009bc23)

that properly ignored the tests directory. Viewing that commit on github at:

> **[GitHub - openzfs/zfs at f12999148d2f3620c39bfebbb6ffd3c95009bc23](https://github.com/openzfs/zfs/tree/f12999148d2f3620c39bfebbb6ffd3c95009bc23)**
>
> f12999148d2f3620c39bfebbb6ffd3c95009bc23

we see your codecov.yml, with the `ignore` section included, located in the root of the repository. Our internal logs also revealed that ignore was being properly applied for this commit.

Additional commits, such as the current HEAD of master:

> **[zfs/.github at 3035f14d82d0ba6a68eec0242dd37bfeace424bc · openzfs/zfs](https://github.com/openzfs/zfs/tree/3035f14d82d0ba6a68eec0242dd37bfeace424bc/.github)**
>
> 3035f14d82d0ba6a68eec0242dd37bfeace424bc/.github

show that the codecov.yml was moved to a .github directory at some point. It was at this point I’m assuming the ignore functionality began to fail, as codecov’s recursive folder search for the codecov.yml appears to not be functioning as intended in this case. Therefore, I believe adopting one of the two solutions above, moving the codecov.yml to root or specifying its location at upload with the -y flag, should solve your problem.

Upon additional internal discussion with the team, we will very likely deprecate the recursive codecov.yml search functionality for a few reasons:

1. It appears to be flaky at best
2. A vast, vast majority of codecov users place their codecov.yml in the root
3. Codecov provides a straightforward workaround, the -y flag at upload time, if an alternative path is desired.

I believe this will solve your issue, but if it does not, please let us know.

---

<div class="post-metadata">

**Author:** ![jerrod](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/jerrod/32/41_2.png) [@jerrod](https://community.codecov.com/u/jerrod)\
**Post date:** [December 20, 2019, 7:34pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/10 "2019-12-20T19:34:41Z")

</div>

Crossposting our general response on this topic here: [Coverage report ignoring the configuration file on main branch - #12 by jerrod](https://community.codecov.com/t/coverage-report-ignoring-the-configuration-file-on-main-branch/618/12)

Separately, @Ornias1993 – some of the things you said here were pretty hurtful to us – we are indeed humans on the receiving end of your words 😢. I hope our discourse can be more productive and forward-looking as we continue to work together on this. Code coverage is our absolute passion and we support our open source community for free always and forever.

Jerrod

---

<div class="post-metadata">

**Author:** ![Ornias1993](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/ornias1993/32/428_2.png) [@Ornias1993](https://community.codecov.com/u/Ornias1993)\
**Post date:** [December 20, 2019, 7:56pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/11 "2019-12-20T19:56:48Z")

</div>

The reason the above commit worked is because i’ve stopped trying to get codecov ignore to work and just changed LCOV to removed files before sending things to codecov.

So no: Moving it to root never solved any issue, the 4 times I tried.

Like it says:  
“Try lcov ignore” and that worked out great, in stark contrast with codecov.

I’m not really interested in spending time on the issue, it is still broken in root or not. I fixed it for us, but not using codecov, because everything I tried (including moving it to root) didn’t work.

---

<div class="post-metadata">

**Author:** ![jerrod](https://yyz1.discourse-cdn.com/flex029/user_avatar/community.codecov.com/jerrod/32/41_2.png) [@jerrod](https://community.codecov.com/u/jerrod)\
**Post date:** [December 20, 2019, 8:23pm UTC](https://community.codecov.com/t/codecov-yml-ignore-not-working/794/12 "2019-12-20T20:23:40Z")

</div>

@Ornias1993 – ok, sorry to hear to this didn’t get to work for you. We will continue to investigate and hope to earn back your trust and usage in the future.

Jerrod