Description
Codecov stopped analyzing reports successfully uploaded from AppVeyor (according to build log) in Cobertura format.
Around ~4 months ago analysis worked correctly and builds uploaded from AppVeyor were visible under “Build” tab of particular commit as well as Lcov reports uploaded from Travis CI. Lines covered by tests executed during AppVeyor build only (i.e. lines which are covered by Cobertura reports uploaded from AppVeyor only) were shown as covered and were included into total coverage.
Now Codecov shows “Reports are queued for processing…” message infinitely (for more than a week) and builds successfully (!) uploaded from AppVeyor are not shown on “Build” tab of particular commit (while Travis CI builds are shown). Coverage reported by missing reports only is not taken into account - lines are shown as not covered and total coverage has reduced.
Repository
GitHub repository: GitHub - mabrarov/asio_samples: Examples (code samples) describing the construction of active objects on the top of Boost.Asio. A code-based guide for client/server creation with usage of active object pattern by means of Boost C++ Libraries.
Links to Travis CI and AppVeyor can be found in README (I cannot put them here as new user)
Versions
Language: C++
Format: Cobertura
Tool: OpenCppCoverage
Platform / OS: Windows Server x64 on AppVeyor
Last successful Codecov analysis: Code coverage done right.
Last failed Codecov analysis: https://codecov.io/gh/mabrarov/asio_samples/commit/c6cfb877e428026247acf4d4662191f6c61dc8b8/build and AppVeyor
Additional Information
The same issue happens with yatest repository (same mabrarov GitHub user) for which I even recreated Codecov project but this didn’t help.
Hi @mabrarov
I fixed your ability to post additional links and images.
Can you share a recent commit SHA for an Appveyor build that does not show? I’d like to check our logs and see what happened to it.
Hi @drazisil,
Here are the links for the latest (just completed) builds and analysis.
Commit: Eliminated upgrade of pip in Travis CI · mabrarov/asio_samples@3852d78 · GitHub
AppVeyor build: AppVeyor (consists of multiple jobs, some of which send report to Codecov, like AppVeyor)
Codecov analysis of commit: Codecov
It looks like we failed to process the Appveyor upload. It’s not clear from the logs if the upload timed out, or something else, still investigating.
Couple of questions:
- Do you know how large the Appveyor upload is?
- Do you know roughly when it stopped working?
Size of Cobertura report generated by AppVeyor is 200-250 KB. You can find it in “Console” view of particular AppVeyor build job (at the end of log) - like AppVeyor
==> Detecting CI provider
AppVeyor Detected
==> Preparing upload
XX> Skip processing gcov
==> Collecting reports
XX> Searching for reports disabled
Targeting specific files
+ C:/projects/asio-samples/build/coverage/coverage.xml bytes=213422
==> Uploading
.url https://codecov.io
.query service=appveyor&build=h8c6lcmwl1r6mu01&commit=3852d780ba46d7d50affd63615c4e109d0b457d9&token=<secret>&slug=mabrarov%2Fasio_samples&flags=windows_6_3_9600__x64__msvc_14_0__boost_1_72_0__qt_5_11_3&job=mabrarov%2Fasio-samples%2F1.0.1007&package=py2.0.15&yaml=codecov.yml&branch=feature%2Fcodecov
Pinging Codecov...
Uploading to S3...
https://codecov.io/github/mabrarov/asio_samples/commit/3852d780ba46d7d50affd63615c4e109d0b457d9
Cobertura report (coverage.xml file) can be found among build artifacts (“Artifacts” view / tab of job) of particular AppVeyor build job, like at AppVeyor
It looks like issues started ~2 months ago
Thanks for that, I’ll keep digging here.
I see you are using the python uploader, do you use the same uploader across CI providers? Would it be possible to try the bash uploader to rule that out? The python uploader isn’t as maintained, and I know there have been a couple Appvayor-related fixes to the bash uploader in that time that might help.
I’d prefer to use python uploader if it’s not the root cause (I’ll test bash uploader soon).
I use python uploader for both Travis CI (refer to Travis CI - Test and Deploy Your Code with Confidence) and AppVeyor. According to logs:
_____ _
/ ____| | |
| | ___ __| | ___ ___ _____ __
| | / _ \ / _ |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| | __/ (_| (_) \ V /
\_____\___/ \____|\___|\___\___/ \_/
v2.0.15
version of python uploader used by both CI systems is the same
I tried Codecov bash uploader in another project facing the same issue - GitHub - mabrarov/yatest: Test for C++.
Here is Travis CI build which uses python uploader and is visible in Codecov: Travis CI - Test and Deploy Your Code with Confidence
Here is AppVeyor build which uses bash uploader, completed successfully but is not visible in Codecov: AppVeyor
Here is Codecov view for 19e71b10 commit: Codecov
I think I am running into the same issue.
I have an AppVeyor job that creates a bunch of partial cobertura reports, merges them and then uploads them to codecov. According to the log, the upload succeeds, but I cannot see it on codecov.
Job log: AppVeyor (sorry it is noisy, but I enabled all the prints to help me debug this… the relevant part is at the end)
Codecov for that commit: https://codecov.io/github/catchorg/Catch2/commit/f118a63a5ca11a137d771bd4efb0c0cf075a38b4
Hi @mabrarov and @horenmar
Besides you two, I have a couple attitudinal reports of this. I’m not sure what’s going on, but I’m investigating, wanted to let you know I wasn’t ignoring this.
@mabrarov, @horenmar, @yitam
It looks like the reports you are uploading have an empty timestamp, which is completly breaking our parsing library. Can you check if this is the case?
Here is an example of what I’m seeing
<coverage line-rate="0.77029569892473115" branch-rate="0" complexity="0" branches-covered="0" branches-valid="0" timestamp="0" lines-covered="5731" lines-valid="7440" version="0">
Hi @drazisil,
According to your answer timestamp is zero and not empty. You are right - the latest failed build (6b4dabd2 commit) has
<coverage line-rate="0.9755576649264357" branch-rate="0" complexity="0" branches-covered="0" branches-valid="0" timestamp="0" lines-covered="4111" lines-valid="4214" version="0">
but timestamp had the same value
<coverage line-rate="0.9755576649264357" branch-rate="0" complexity="0" branches-covered="0" branches-valid="0" timestamp="0" lines-covered="4111" lines-valid="4214" version="0">
previously - when Codecov worked fine. Refer to artifacts of AppVeyor build job (to coverage.xml artifact) for e8997afe commit and to related Codecov report - Code coverage done right. - which contains reports uploaded from AppVeyor
@mabrarov
Well, drat! I though we had this solved. The code that’s throwing the error hasn’t been touched in years: timestring/timestring/Date.py at master · codecov/timestring · GitHub
Will continue digging.
Can I ask one of you 3 to try adding this to your codecov.yml and see if it helps?
codecov:
max_report_age: off
Hi @drazisil,
I added
max_report_age: off
in yatest project which has the same issue. Refer to bae0fd69ce5bdb976658ba9b0f3a245846083cd8 commit and to related AppVeyor build job which completed successfully (as well as related Travis CI build job).
Codecov (refer to https://codecov.io/gh/mabrarov/yatest/branch/feature%2Fcodecov/commits) looks being hanged for this commit - it shows “Processing…” on commits view of feature/codecov branch
and shows " Unable to find report content in the storage archive" when trying to look at the report for this commit:
It looks like I added new line into wrong section of codecov.yml… Let me fix and retest.
@drazisil,
It looks like your suggestion implemented in 01f1f87cccbf3efdc0b072e54b4ef39bbbfd334d commit fixed this issue for yatest project - refer to https://codecov.io/gh/mabrarov/yatest/commit/01f1f87cccbf3efdc0b072e54b4ef39bbbfd334d/build where both Linux (from Travis CI) and Windows (from AppVeyor) builds are visible.
I’m going to test the same solution for asio_samples project now.
Is it permanent solution I can keep or is it needed just for your investigation?
1 Like
Nope, you can totally keep it. I’ll mark as the solution for now, since tracking down what changed will take us a while.
The yaml change has also fixed the problem for me, with a small exception that for some reason codecov does not add its own “tickmark” to the commit the way it used to.
That shouldn’t have changed anything, it’s possible there’s a delay. Can you share a SHA?
