Coverage merging does not work as expected

boesing · June 2, 2020, 7:09pm

Description

Hey there, I’ve a repository I need to run tests with multiple composer versions.
Thus, I skip some tests in composer v1 and some tests in composer v2.
At the end, two files should be tested.

I’ve added “flags” to the codecov bash file by appending -cF composer<version> to the upload bash script.

If I have a look into the final results, it still states that only 48% of my code is covered.

Did I missunderstood the merging feature or is there something I have to change?

I introduced codecov in this PR: https://github.com/laminas/laminas-dependency-plugin/pull/18

I would like to know how to get the merged coverage percentage. Should be at least more than 70% I think.

Repository

github.com/laminas/laminas-dependency-plugin

Slip stream in composer 2.0

laminas:develop ← boesing:feature/slip-stream-2.0

opened 09:30PM - 23 May 20 UTC

boesing

+2447 -404

| Q | A |-------------- | ------ | New Feature | yes ### De…scription While trying to migrate to composer 2.0, I've tried many ways on how to achieve the functionality of the current plugin. The current plugin "slip streams" into the composer process, which is probably the best solution to solve the following scenarios: - If 3rd-party library has a zend-dependency and updates to laminas, there wont be any changes to the project as the laminas package is already required. - If a 3rd-party library has a zend-dependency and requires a newer zend-dependency, the project will just upgrade to the laminas equivalent. No further action needed. - If a 3rd-party library totally drops a zend-dependency and does not need the laminas equivalent, the slip-streamed laminas dependency will automatically being removed from the project if its not needed anymore. I came up with a way in #15 which should have introduced some more user interactions. The main idea was: If a 3rd-party library has a zend-dependency, ask the user if he wants to replace that zend-dependency the laminas pendant, write the dependency to `composer.json` of the project and remember that we introduced that dependency due to a non-migrated library. ### TL;DR This is finally ready for review. ### Story on why I decided to go this way After many sleepless nights and a total rewrite of the dependency plugin, I realized that this approach would probably be a future proof solution but its way too complex. The complexity exceeded my brain capacity too often. Its unmaintainable due to all those possible combinations: - 3rd-party library no.1 uses zend-dependency and is required via root `composer.json` as a `require` package - 3rd party library no.2 uses same zend-dependency and is required via root `composer.json` as a `require-dev` package - 3rd party library no.1 is being uninstalled - root `composer.json` should change the `require` of the laminas equivalent to `require-dev` as there is only a dev-requirement left which requires the zend-package And thats just one of those complex scenarios... However, after I realized that #15 wont be a solution I am happy with, I started to work on this approach today. I don't know why I tried this earlier (as I was debugging what exactly changed in composer 2.0 that the slip-stream solution we are actually use does not work anymore)... That would probably saved my a ton of work.

CI/CD

Travis CI

Uploader

Bash Uploader with -cF to provide flags

Commit SHAs

8e1528a5b74945ecfb3cd7211191bf58e44518d3

Codecov YAML

none

Codecov Output

 bash <(curl -s https://codecov.io/bash) -cF composer${COMPOSER_VERSION}
  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-20200602-f809a24
==> Travis CI detected.
    project root: .
    Fixing merge commit SHA
    Yaml not found, that's ok! Learn more at http://docs.codecov.io/docs/codecov-yaml
==> Running gcov in . (disable via -X gcov)
==> Python coveragepy not found
==> Searching for coverage reports in:
    + .
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Appending build variables
    + TRAVIS_OS_NAME
    + 7.4
==> Reading reports
    + ./clover.xml bytes=25027
==> Appending adjustments
    https://docs.codecov.io/docs/fixing-reports
    + Found adjustments
==> Gzipping contents
==> Uploading reports
    url: https://codecov.io
    query: branch=develop&commit=4c1574a5bdde07079d5947f20a6bb9c6748201af&build=113.12&build_url=&name=&tag=&slug=laminas%2Flaminas-dependency-plugin&service=travis&flags=composer1&pr=18&job=342815503
    -> Pinging Codecov
https://codecov.io/upload/v4?package=bash-20200602-f809a24&token=secret&branch=develop&commit=4c1574a5bdde07079d5947f20a6bb9c6748201af&build=113.12&build_url=&name=&tag=&slug=laminas%2Flaminas-dependency-plugin&service=travis&flags=composer1&pr=18&job=342815503
    -> Uploading
    -> View reports at https://codecov.io/github/laminas/laminas-dependency-plugin/commit/4c1574a5bdde07079d5947f20a6bb9c6748201af

bash <(curl -s https://codecov.io/bash) -cF composer${COMPOSER_VERSION}
  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-20200602-f809a24
==> Travis CI detected.
    project root: .
    Fixing merge commit SHA
    Yaml not found, that's ok! Learn more at http://docs.codecov.io/docs/codecov-yaml
==> Running gcov in . (disable via -X gcov)
==> Python coveragepy not found
==> Searching for coverage reports in:
    + .
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Appending build variables
    + TRAVIS_OS_NAME
    + 7.4
==> Reading reports
    + ./clover.xml bytes=25027
==> Appending adjustments
    https://docs.codecov.io/docs/fixing-reports
    + Found adjustments
==> Gzipping contents
==> Uploading reports
    url: https://codecov.io
    query: branch=develop&commit=4c1574a5bdde07079d5947f20a6bb9c6748201af&build=113.11&build_url=&name=&tag=&slug=laminas%2Flaminas-dependency-plugin&service=travis&flags=composer2&pr=18&job=342815502
    -> Pinging Codecov
https://codecov.io/upload/v4?package=bash-20200602-f809a24&token=secret&branch=develop&commit=4c1574a5bdde07079d5947f20a6bb9c6748201af&build=113.11&build_url=&name=&tag=&slug=laminas%2Flaminas-dependency-plugin&service=travis&flags=composer2&pr=18&job=342815502
    -> Uploading
    -> View reports at https://codecov.io/github/laminas/laminas-dependency-plugin/commit/4c1574a5bdde07079d5947f20a6bb9c6748201af

Additional Information

None.

tom · June 3, 2020, 1:46pm

Hi @boesing, in order to use flags, you need to specify them in a codecov.yml file.

Borda · July 1, 2020, 6:56am

so is there a way how to aggregate two reports from two different CI, one using Github action and other just running with bash? I have tried the -cF and it faile as unrecognised opion
http://35.192.60.23/PyTorchLightning/pytorch-lightning/5279

tom · July 1, 2020, 9:39pm

Hi @Borda, I’m not sure what you are asking here. Coverage reports are automatically merged together, regardless of the CI. What uploader of Codecov are you using (bash, ruby, python, etc) and what version?

Borda · July 3, 2020, 8:46pm

it turned out that for some reason one reported uploaded repo with parent folder in its path…

Topic		Replies	Views
Codecov merge report issue Bug Fixes	12	313	August 27, 2021
Codecov not correctly merging reports Support	26	1735	October 6, 2020
Coverage dropped by > 60% after merging a commit Support	18	266	March 7, 2023
Codecov no longer merges multiple reports correctly Bug Fixes	11	757	September 24, 2020
Coverage seems to be off Support javascript , github	3	467	October 6, 2020