CLOSED ISSUE INVESTIGATION: Report stuck in processing + Unable to find report content in storage archive

UPDATE DECEMBER 2ND 2019:

We have found root cause of central issue regarding Unable to find report content in storage archive errors.

In summary: a refactored worker we delivered to Codecov.io application was in some cases picking the wrong file processor. That bug has now been squashed.

There are users that may still see this error: Unable to find report content in storage archive, but that would only be in cases where the error existed prior to the current bug in question. Reasons are likely:

  1. If the repo never had a report in first place to locate
  2. If reports are intermittently stuck in “Processing” state, potentially due to number of uploads or size of uploads (or both).

If we had added you to our workaround approach, we will begin removing users from that workaround and adding them back to our main processing worker.

Thank you!


Original Post: November 5th 2019

Hello Codecov Community,

Over the last two weeks, we’ve been noticing an issue substantially impacting about 1% of organizations.

Users experiencing this error will see general issues like Report stuck in processing and the specific issue Unable to find report content in storage archive.

We are doing in-depth investigation to fix the root cause issue, and simultaneously looking for workarounds for affected users.

If we find a workaround before a root cause fix, we will manually help users utilize the workaround and update here accordingly.

Thank you everyone for your patience, and apologies for the issue.

Best,
Jerrod and the Codecov Team

4 Likes

Hello everyone,

Thanks for your patience while we investigated this issue. We’ve been able to identify and push a manual workaround approach for affected users, simply message us:

Your relevant code host (Github/Gitlab/Bitbucket) organization name to hello@codecov.io and mention you are experiencing Unable to find report content in storage archive. We will provide change to your account.

Meanwhile, we are still investigating a longer term fix and will update thread again once found and deployed.

Jerrod

Hey @bofeizhu – very unlikely to be the same issue. I saw you filed a separate post on this, thanks!

Thanks for looking into this, which also affects our GitHub project.
The default branch is dev, and I wonder if this current issue is related to the project badge showing “unknown” or is it a different issue?

Hi, codecov report is blank and status badge unknown still. Wondering if this has been resolved what we should do to see the reports again?

Hi @yitam

This has been resolved. Can you share a recent commit SHA so I can take a look?

The recent commits have been streaming but no reports so far. It seems to assume our repo is private

You seem to be running into the this issue: Codecov cannot analyze Cobertura reports from AppVeyor - #12 by drazisil

Do you happen to know when this started happening?

I see. I think you’re right. The last time it actually worked was back in late Oct 2019. For details, please check this pull request

1 Like

Does Appveyor have a built-in reporter/Codecov uploader that may have changed, somehow? I don’t believe all cobertura reports from Appveyor are failing, I’d hope I would have gotten sooner/more reports if this was case.

As stated in the other thread, we are investigating.

Good point. Let me also do some investigation. Thank you

We’re having a similar issue:

We use tags, and upload the reports from different jobs in a single CircleCI workflow.

Two directories are missing from the results: vm and account [1]

The account job has finished, and vm job is not even reported here. [2]

This is the full commit SHA: 67da6a40628a26264b32714d83319f1539908683

[1] https://codecov.io/gh/evertonfraga/ethereumjs-vm/tree/67da6a40628a26264b32714d83319f1539908683/packages-

[2] https://codecov.io/gh/evertonfraga/ethereumjs-vm/commit/67da6a40628a26264b32714d83319f1539908683/build

The issue was on our end. NYC now reports relative paths and it wasn’t conformant with the repository file structure. Solved!

2 Likes

I am experiencing this issue suddenly again. It has come up before, and I post on here and someone fixes it on the backend, but every month or so, I get the same issue again. Here is the first commit where it started again recently:
https://codecov.io/gh/APB9785/phxBB/commit/699128c16f3f17d63c459ad0487bd94776c0e8a3/

Backlog is empty according to status.codecov.io so I don’t think it’s a load issue

@APB9785 please open a new issue for this