Codecov cannot analyze Cobertura reports from AppVeyor

Size of Cobertura report generated by AppVeyor is 200-250 KB. You can find it in “Console” view of particular AppVeyor build job (at the end of log) - like https://ci.appveyor.com/project/mabrarov/asio-samples/builds/30321782/job/h8c6lcmwl1r6mu01

==> Detecting CI provider
    AppVeyor Detected
==> Preparing upload
XX> Skip processing gcov
==> Collecting reports
XX> Searching for reports disabled
    Targeting specific files
    + C:/projects/asio-samples/build/coverage/coverage.xml bytes=213422
==> Uploading
    .url https://codecov.io
    .query service=appveyor&build=h8c6lcmwl1r6mu01&commit=3852d780ba46d7d50affd63615c4e109d0b457d9&token=<secret>&slug=mabrarov%2Fasio_samples&flags=windows_6_3_9600__x64__msvc_14_0__boost_1_72_0__qt_5_11_3&job=mabrarov%2Fasio-samples%2F1.0.1007&package=py2.0.15&yaml=codecov.yml&branch=feature%2Fcodecov
    Pinging Codecov...
    Uploading to S3...
    https://codecov.io/github/mabrarov/asio_samples/commit/3852d780ba46d7d50affd63615c4e109d0b457d9

Cobertura report (coverage.xml file) can be found among build artifacts (“Artifacts” view / tab of job) of particular AppVeyor build job, like at https://ci.appveyor.com/project/mabrarov/asio-samples/builds/30321782/job/h8c6lcmwl1r6mu01/artifacts

It looks like issues started ~2 months ago

Thanks for that, I’ll keep digging here.

I see you are using the python uploader, do you use the same uploader across CI providers? Would it be possible to try the bash uploader to rule that out? The python uploader isn’t as maintained, and I know there have been a couple Appvayor-related fixes to the bash uploader in that time that might help.

I’d prefer to use python uploader if it’s not the root cause (I’ll test bash uploader soon).

I use python uploader for both Travis CI (refer to https://travis-ci.org/mabrarov/asio_samples/jobs/641131620) and AppVeyor. According to logs:

      _____          _
     / ____|        | |
    | |     ___   __| | ___  ___ _____   __
    | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
    | |___| (_) | (_| |  __/ (_| (_) \ V /
     \_____\___/ \____|\___|\___\___/ \_/
                                    v2.0.15

version of python uploader used by both CI systems is the same

I tried Codecov bash uploader in another project facing the same issue - https://github.com/mabrarov/yatest.

Here is Travis CI build which uses python uploader and is visible in Codecov: https://travis-ci.org/mabrarov/yatest/jobs/641593992

Here is AppVeyor build which uses bash uploader, completed successfully but is not visible in Codecov: https://ci.appveyor.com/project/mabrarov/yatest/builds/30346304/job/hvx0pfw2fhthjgpq

Here is Codecov view for 19e71b10 commit: https://codecov.io/gh/mabrarov/yatest/commit/19e71b106937b1a90dce277650e9064ac924e47a

I think I am running into the same issue.

I have an AppVeyor job that creates a bunch of partial cobertura reports, merges them and then uploads them to codecov. According to the log, the upload succeeds, but I cannot see it on codecov.

Job log: https://ci.appveyor.com/project/CatchOrg/catch2/builds/30419141/job/nw6k4trce09cpt7o (sorry it is noisy, but I enabled all the prints to help me debug this… the relevant part is at the end)

Codecov for that commit: https://codecov.io/github/catchorg/Catch2/commit/f118a63a5ca11a137d771bd4efb0c0cf075a38b4

Hi @mabrarov and @horenmar

Besides you two, I have a couple attitudinal reports of this. I’m not sure what’s going on, but I’m investigating, wanted to let you know I wasn’t ignoring this.

@mabrarov, @horenmar, @yitam

It looks like the reports you are uploading have an empty timestamp, which is completly breaking our parsing library. Can you check if this is the case?

Here is an example of what I’m seeing

<coverage line-rate="0.77029569892473115" branch-rate="0" complexity="0" branches-covered="0" branches-valid="0" timestamp="0" lines-covered="5731" lines-valid="7440" version="0">

Hi @drazisil,

According to your answer timestamp is zero and not empty. You are right - the latest failed build (6b4dabd2 commit) has

<coverage line-rate="0.9755576649264357" branch-rate="0" complexity="0" branches-covered="0" branches-valid="0" timestamp="0" lines-covered="4111" lines-valid="4214" version="0">

but timestamp had the same value

<coverage line-rate="0.9755576649264357" branch-rate="0" complexity="0" branches-covered="0" branches-valid="0" timestamp="0" lines-covered="4111" lines-valid="4214" version="0">

previously - when Codecov worked fine. Refer to artifacts of https://ci.appveyor.com/project/mabrarov/asio-samples/builds/27250158/job/71id95s4ypdu59vb/artifacts build job (to coverage.xml artifact) for e8997afe commit and to related Codecov report - https://codecov.io/gh/mabrarov/asio_samples/commit/e8997afe0ed393b9c10300fd1ef2069e66e6a011/build - which contains reports uploaded from AppVeyor

@mabrarov

Well, drat! I though we had this solved. The code that’s throwing the error hasn’t been touched in years: https://github.com/codecov/timestring/blame/master/timestring/Date.py#L51

Will continue digging.

Can I ask one of you 3 to try adding this to your codecov.yml and see if it helps?

codecov:
  max_report_age: off

Hi @drazisil,

I added

max_report_age: off

in yatest project which has the same issue. Refer to bae0fd69ce5bdb976658ba9b0f3a245846083cd8 commit and to related AppVeyor build job which completed successfully (as well as related Travis CI build job).

Codecov (refer to https://codecov.io/gh/mabrarov/yatest/branch/feature%2Fcodecov/commits) looks being hanged for this commit - it shows “Processing…” on commits view of feature/codecov branch

and shows " Unable to find report content in the storage archive" when trying to look at the report for this commit:

It looks like I added new line into wrong section of codecov.yml… Let me fix and retest.

@drazisil,

It looks like your suggestion implemented in 01f1f87cccbf3efdc0b072e54b4ef39bbbfd334d commit fixed this issue for yatest project - refer to https://codecov.io/gh/mabrarov/yatest/commit/01f1f87cccbf3efdc0b072e54b4ef39bbbfd334d/build where both Linux (from Travis CI) and Windows (from AppVeyor) builds are visible.

I’m going to test the same solution for asio_samples project now.

Is it permanent solution I can keep or is it needed just for your investigation?

1 Like

Nope, you can totally keep it. I’ll mark as the solution for now, since tracking down what changed will take us a while.

The yaml change has also fixed the problem for me, with a small exception that for some reason codecov does not add its own “tickmark” to the commit the way it used to.

That shouldn’t have changed anything, it’s possible there’s a delay. Can you share a SHA?

I gave it some time, and for 12557773c8f9e6632fb240f1bce8e07a74c50208 the AppVeyor jobs report that they uploaded the coverage information, and if I go to the reported URL manually, it is indeed there. However, the CI tickmark still hasn’t been updated.

The next commit in the branch, 6d98573d919976e1f2b93143e87b1d46a135cf3e, has the tickmark updated.

Honestly it isn’t much of a problem, I just found it curious.

Sorry @drazisil we were busy with release activities until now. Our project does not use a codecov.yml because the default one suffices so far. But if this is required then we will add one. Please advise

No worries!

Yes, in this case it is needed.

Yes it works! Wonderful :smile: Just one more question, how long does it take for the code coverage badge to show a percentage rather than “unknown”?

1 Like