Wrong coverage percentage

Description

Codecov is wrongly informing the coverage percentage on my project, https://codecov.io/gh/alumna/unitflow

As can be seen on it, the original source file has 100% coverage, but there is a strange copy of the same file from Travis where Codecov informs only 83.67%, giving the project and average of 91.84%

The original file is src/unitflow.js and the copy is Users/travis/build/alumna/unitflow/src/unitflow.js

Repository

CI/CD

Travis CI

Uploader

I don’t understand how to implement the bash uploader on our public project on Travis CI

Commit SHAs

8e03ccfac25c17d6f08867da233cd8bf82b89b58

Codecov YAML

I have no codecov.yml on my project, just a .travis.yml with:

os:
  - linux
  - osx
  - windows
language: node_js
install:
  - npm ci
  - npm install -g codecov
node_js:
  - "node"
after_script:
  - codecov

Codecov Output

$ codecov

  _____          _  

 / ____|        | |  

| |     ___   __| | ___  ___ _____   __  

| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /  

| |___| (_) | (_| |  __/ (_| (_) \ V /  

 \_____\___/ \__,_|\___|\___\___/ \_/  

                                v3.7.1

==> Detecting CI Provider

    Travis CI Detected

==> Configuration: 

    Endpoint: https://codecov.io

{

  commit: '8e03ccfac25c17d6f08867da233cd8bf82b89b58',

  branch: 'v1.0.1',

  package: 'node-v3.7.1'

}

==> Building file structure

==> Generating gcov reports (skip via --disable=gcov)

    Failed to run gcov command.

==> Scanning for reports

    X Failed to read file at 

==> Uploading reports

    Success!

    View report at: https://codecov.io/github/alumna/unitflow/commit/8e03ccfac25c17d6f08867da233cd8bf82b89b58

Done. Your build exited with 0.

Steps to Reproduce

  1. Created a new PR on my project
  2. Travis CI identified it and runned new tests
  3. It executed the codecov command in the end

Expected behavior:
Since the original file has 100% coverage, as Codecov recognizes, I would not like to have a copy file with wrong coverage percentage changing the project overall coverage to a wrong value (below 100%)

Actual behavior:

Codecov is considering also this wrong file and its strange coverage percentage.

Flakiness?

This only started to happen recently

Hi @paulocoghi, we noticed an issue with 3.7.1 and have patched it to 3.7.2. Could you try that out and see if that fixes your issue?

1 Like

Hi Tom! I will do a new test! Thanks for your reply!

To workaround this issue, I had added a rule to codecov.yml to ignore all files from Users/travis, that seems to be from Windows test.

With the new Travis version 3.7.2 I will remove this rule and see how it goes.

Tested and confirmed that on 3.7.2 my workaround isn’t necessary anymore, since Codecov is not listing that additional copy anymore!

Everything is nice again! Thanks @tom !

1 Like

I also have an issue with wrong coverage percentage, I have a 100% coverage but Codecov give me a wrong percentage. But I use the bash script and Github actions.

@mikbry , it seems to me that HueSlider.jsx is not covered by your tests, and maybe this is the reason your total coverage is not 100%

If this is not the case, post all the details I posted to make it easy to the Codecov team to analyze your issue, including commit SHAs, etc

1 Like

@mikbry, you are seeing less than 100% coverage because we do not consider partial hits to be covered lines. If you look at the raw reports by going to the build tab here, you will see that these lines do not traverse the false path

      <file name="ColorButton.jsx" path="/home/runner/work/material-ui-color/material-ui-color/src/components/ColorButton.jsx">
        <metrics statements="27" coveredstatements="27" conditionals="8" coveredconditionals="8" methods="11" coveredmethods="11"/>
        <line num="19" count="4" type="stmt"/>
        <line num="20" count="164" type="stmt"/>
        <line num="22" count="164" type="stmt"/>
        <line num="23" count="164" type="stmt"/>
        <line num="24" count="164" type="stmt"/>
        <line num="25" count="164" type="stmt"/>
        <line num="27" count="164" type="cond" truecount="4" falsecount="0"/>
        <line num="30" count="164" type="stmt"/>
        <line num="31" count="164" type="stmt"/>
        <line num="32" count="164" type="stmt"/>
        <line num="34" count="164" type="stmt"/>
        <line num="44" count="4" type="stmt"/>
        <line num="52" count="4" type="stmt"/>
        <line num="53" count="164" type="stmt"/>
        <line num="54" count="164" type="stmt"/>
        <line num="55" count="164" type="stmt"/>
        <line num="56" count="164" type="stmt"/>
        <line num="57" count="164" type="stmt"/>
        <line num="58" count="164" type="cond" truecount="2" falsecount="0"/>
        <line num="59" count="164" type="stmt"/>
        <line num="60" count="164" type="stmt"/>
        <line num="62" count="164" type="stmt"/>
        <line num="76" count="164" type="cond" truecount="2" falsecount="0"/>
        <line num="77" count="144" type="stmt"/>
        <line num="83" count="20" type="stmt"/>
        <line num="86" count="4" type="stmt"/>
        <line num="117" count="4" type="stmt"/>
      </file>
2 Likes

There is no code logic in HueSlider.jsx.

@mikbry Sorry. Noob response on my part. I believe @tom 's response is the point.

1 Like

You are right @tom. It is confusing to see some differences between Codecov and jest coverage. And it is more tricky to avoid them during dev phase as partials are not displayed in html report… I will investigate.

1 Like

This is strange, because I also use Jest and partials are displayed for me, both on the results summarized on the command line (with line numbers partially covered being enumerated and printed with different colors, like yellow), as well as in html report.

The html reports are even more detailed, showing exactly where in the line the issue of partial coverage is, and I remember many cases where I was only able to solve after opening the html report, to understand precisely what logic scenarios were missing on my tests.

1 Like

Finally I found why such differences: I was sending cover.xml report which includes partials, but my html report is using lcov.info without partials support. I changed my script to send to codecov the lcov.info instead and now I have same 100%. But yes i didn’t fix partials…

2 Likes