"CI Failed" on many commits which have CI passing

snafu109 · December 31, 2022, 3:03am

Before submitting a topic, please confirm the following

I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my codecov.yaml configuration file.
I have filled out the below sections to the best of my ability.

Description

Public repo uses GitHub action to upload JaCoCo coverage reports on each commit to main branch and for each PR.

More than half the time the Codecov report says “CI Failed” and doesn’t report any coverage data, even though the build is successful.

CI/CD URL

Codecov Output

Please provide the full output of running the uploader on your CI/CD. This will typically have the Codecov logo as ASCII.

Run codecov/codecov-action@d9f34f8cd5cb3b3eb79b3e4b5dae3a16df499a70
{
  execArgs: [
    '-n',
    '',
    '-Q',
    'github-action-3.1.1',
    '-f',
    './code-coverage-report/build/reports/jacoco/jacocoMergedReport/jacocoMergedReport.xml',
    '-v'
  ]
}
==> linux OS detected
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
Received SHA256SUM 20f9c9d78483fce977b6cc39e231a734a23bcd36f4d536bb7355222fb88d02bc  codecov
Received SHA256SUM signature -----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEJwNOf9uFDgu8LGL/gGuyiu13mGkFAmMsvyQACgkQgGuyiu13
mGmStA/+N4roe/N6hA78uO55SL8/8tfS75QYe+PbRF3nN+mpXsyik+gMZWfu9SLT
lQJK2T9PfU6dFx5VnceoENXgnH+SUvDLorKUOwTolXMIEos9bLCPRcgfe3AkTeZR
2sHZcDj75nmcfEpO9Tf/Jpq2PPrP24J3S822/v3ZT8lca9jJe8CJlBL15yXC1kgD
Vk72+r9KzAoAu6bCEvBOOIFHHNfVBf0635vamRFCHFgjN1rRJW5cuyieZr9rYa22
1KDpN6vijJwUlrwB7V8wTvaeOEinR5Fzo55TtjTtE4brfaOF2KCzM33Czzep3SfH
OPJENE1+A+/bleRcVok5qmRfvMThSITE98INNk4qCVsB1YqTOvtJs5rFtyuH/Xqf
674HAqeNjzXvFME5oX4Hm31X1Y4rfzH0738H25J//3GqZWahivq7e8ElyYP+CI0H
pXbeN6E9Zup/QR6YjYWBV35AvRscC7v4e6HnlN9ctTuAd0VZJXI8JLM1XTgiQnIQ
rql/7lIYrvzLerPmzVoAmYZuV2iAE7pxfo3Iy9zEtgV8cF42peE3TDyrqj5Ylbtc
1pKlfIIAhoB/ff2hIvGonXG4OgTCCQrUvbtI8KsNi6UFPX5jhCaltKMsLG/JPM+Q
vXPa43eZ+aWPHOi9dFdlg98GCFesSVKLECkU4DjoZxJsI493PrQ=
=IHR5
-----END PGP SIGNATURE-----

==> SHASUM file signed by key id 806bb28aed779869
==> Uploader SHASUM verified (20f9c9d78483fce977b6cc39e231a734a23bcd36f4d536bb7355222fb88d02bc  codecov)
==> Running version latest
==> Running version v0.3.2
/home/runner/work/_actions/codecov/codecov-action/d9f34f8cd5cb3b3eb79b3e4b5dae3a16df499a70/dist/codecov -n  -Q github-action-3.1.1 -f ./code-coverage-report/build/reports/jacoco/jacocoMergedReport/jacocoMergedReport.xml -v
[2022-12-30T22:06:54.775Z] ['verbose'] Start of uploader: 1672438014775...
[2022-12-30T22:06:54.777Z] ['info'] 
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/

  Codecov report uploader 0.3.2
[2022-12-30T22:06:54.783Z] ['info'] => Project root located at: /home/runner/work/detekt/detekt
[2022-12-30T22:06:54.787Z] ['info'] -> No token specified or token is empty
[2022-12-30T22:06:54.787Z] ['verbose'] Start of network processing...
[2022-12-30T22:06:54.788Z] ['verbose'] Searching for files in /home/runner/work/detekt/detekt
[2022-12-30T22:06:54.797Z] ['verbose'] coveragepy is not installed
[2022-12-30T22:06:54.797Z] ['info'] Searching for coverage files...
[2022-12-30T22:06:55.099Z] ['verbose'] Preparing to clean the following coverage paths: ./code-coverage-report/build/reports/jacoco/jacocoMergedReport/jacocoMergedReport.xml
[2022-12-30T22:06:55.099Z] ['info'] => Found 1 possible coverage files:
  ./code-coverage-report/build/reports/jacoco/jacocoMergedReport/jacocoMergedReport.xml
[2022-12-30T22:06:55.100Z] ['verbose'] End of network processing
[2022-12-30T22:06:55.100Z] ['info'] Processing ./code-coverage-report/build/reports/jacoco/jacocoMergedReport/jacocoMergedReport.xml...
[2022-12-30T22:06:55.145Z] ['info'] Detected GitHub Actions as the CI provider.
[2022-12-30T22:06:55.145Z] ['verbose'] -> Using the following env variables:
[2022-12-30T22:06:55.146Z] ['verbose']      GITHUB_ACTION: __codecov_codecov-action
[2022-12-30T22:06:55.146Z] ['verbose']      GITHUB_HEAD_REF: 
[2022-12-30T22:06:55.146Z] ['verbose']      GITHUB_REF: refs/heads/main
[2022-12-30T22:06:55.146Z] ['verbose']      GITHUB_REPOSITORY: detekt/detekt
[2022-12-30T22:06:55.146Z] ['verbose']      GITHUB_RUN_ID: 3809284713
[2022-12-30T22:06:55.146Z] ['verbose']      GITHUB_SERVER_URL: https://github.com
[2022-12-30T22:06:55.147Z] ['verbose']      GITHUB_SHA: a0cbe25cf9026514e6d758a1b0ed57c2fa29e69a
[2022-12-30T22:06:55.147Z] ['verbose']      GITHUB_WORKFLOW: Code Coverage
[2022-12-30T22:06:55.147Z] ['verbose'] Using the following upload parameters:
[2022-12-30T22:06:55.147Z] ['verbose'] branch
[2022-12-30T22:06:55.147Z] ['verbose'] build
[2022-12-30T22:06:55.147Z] ['verbose'] buildURL
[2022-12-30T22:06:55.147Z] ['verbose'] commit
[2022-12-30T22:06:55.147Z] ['verbose'] job
[2022-12-30T22:06:55.148Z] ['verbose'] pr
[2022-12-30T22:06:55.148Z] ['verbose'] service
[2022-12-30T22:06:55.148Z] ['verbose'] slug
[2022-12-30T22:06:55.148Z] ['verbose'] name
[2022-12-30T22:06:55.148Z] ['verbose'] tag
[2022-12-30T22:06:55.148Z] ['verbose'] flags
[2022-12-30T22:06:55.148Z] ['verbose'] parent
[2022-12-30T22:06:55.149Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=github-action-3.1.1-uploader-0.3.2&token=*******&branch=main&build=3809284713&build_url=https%3A%2F%2Fgithub.com%2Fdetekt%2Fdetekt%2Factions%2Fruns%2F3809284713&commit=a0cbe25cf9026514e6d758a1b0ed57c2fa29e69a&job=Code+Coverage&pr=&service=github-actions&slug=detekt%2Fdetekt&name=&tag=&flags=&parent=
[2022-12-30T22:06:55.149Z] ['verbose'] Passed token was 0 characters long
[2022-12-30T22:06:55.149Z] ['verbose'] https://codecov.io/upload/v4?package=github-action-3.1.1-uploader-0.3.2&branch=main&build=3809284713&build_url=https%3A%2F%2Fgithub.com%2Fdetekt%2Fdetekt%2Factions%2Fruns%2F3809284713&commit=a0cbe25cf9026514e6d758a1b0ed57c2fa29e69a&job=Code+Coverage&pr=&service=github-actions&slug=detekt%2Fdetekt&name=&tag=&flags=&parent=
        Content-Type: 'text/plain'
        Content-Encoding: 'gzip'
        X-Reduced-Redundancy: 'false'
[2022-12-30T22:06:55.764Z] ['info'] https://app.codecov.io/github/detekt/detekt/commit/a0cbe25cf9026514e6d758a1b0ed57c2fa29e69a
https://storage.googleapis.com/codecov/v4/raw/2022-12-30/2A3CE3E75A8557C4C960F700F50C5FE6/a0cbe25cf9026514e6d758a1b0ed57c2fa29e69a/85721b29-63b5-4fd3-b72c-c7c0643acbc8.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJOGFN2JQ4OCTGA2MU5AEIT7OT5Z7HTFOAN2SPG4NWSN2UJYOY5U6LZQ%2F20221230%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20221230T220655Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=ba3f4ca3f3a4b7fdddf25acd58728588aaf1b774db249d5bea3fb8d6715f623c
[2022-12-30T22:06:55.765Z] ['verbose'] Returned upload url: https://storage.googleapis.com/codecov/v4/raw/2022-12-30/2A3CE3E75A8557C4C960F700F50C5FE6/a0cbe25cf9026514e6d758a1b0ed57c2fa29e69a/85721b29-63b5-4fd3-b72c-c7c0643acbc8.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJOGFN2JQ4OCTGA2MU5AEIT7OT5Z7HTFOAN2SPG4NWSN2UJYOY5U6LZQ%2F20221230%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20221230T220655Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=ba3f4ca3f3a4b7fdddf25acd58728588aaf1b774db249d5bea3fb8d6715f623c
[2022-12-30T22:06:55.765Z] ['info'] Uploading...
[2022-12-30T22:06:56.076Z] ['info'] {"status":"success","resultURL":"https://app.codecov.io/github/detekt/detekt/commit/a0cbe25cf9026514e6d758a1b0ed57c2fa29e69a"}
[2022-12-30T22:06:56.076Z] ['verbose'] End of uploader: 1301 milliseconds

Expected Results

Coverage report generated for a successful build and successful upload.

Actual Results

Multiple errors, nothing really explaining what’s going on. Some uploads from the same branch with same config have worked, many don’t.

CI Failed
There is an error processing the coverage reports. Common issues are files paths, empty files or expired reports. See error reference page for additional troubleshooting to resolve error.
The following uploads failed to process:

Github-Actions

3809284713

We recommend checking the Codecov step of this commit’s CI Run to make sure it uploaded properly and, if needed, run your CI again.

Additional Information

Any additional information, configuration, or data that might be necessary to reproduce the issue.

tom · January 7, 2023, 10:13pm

Hi @snafu109, I started taking a look into this but it seems like the issue has resolved a few days ago. Is this true or are you running into the issue still?

snafu109 · January 9, 2023, 6:34am

Thanks Tom. It’s intermittent, and you’ll see the latest commit on main has shown this occurring again: Codecov

snafu109 · January 10, 2023, 11:44am

Now it looks like the account has disappeared? Message when viewing any commit’s coverage report, or accessing Codecov directly, shows message:

Not found

You may be able to locate the content by visiting [Codecov’s home page](https://app.codecov.io/) and browsing to it.

**Error 404**

snafu109 · January 10, 2023, 9:49pm

The link to the account now works again. Very strange.

tom · January 11, 2023, 10:47pm

@snafu109 ahhh, I think this was a result (both the commit and the link) of our outage uncovered yesterday. Will you ping me back if this is still intermittent?

snafu109 · January 13, 2023, 9:35pm

The last 4 successful CI runs on commits on our main branch have failed to generate reports, with the same original issue: Codecov

tom · January 16, 2023, 8:02pm

@snafu109 I’m seeing that a lot of your reports are showing as expired

Expired reports will show as CI Failed in the Codecov UI. I would recommend adding this tidbit to your codecov.yml file

codecov:
  max_report_age: off

snafu109 · January 17, 2023, 12:28pm

I think that will fix it actually, thanks! A couple of suggestions for improvement, which might have helped me figure this out myself:

In the Codecov UI, instead of just saying “upload expired”, say something like “timestamp in uploaded report is 2023-01-17T12:21:36+0000, which is more than 12 hours old” and link to that section of the docs. I probably would have put two and two together quite quickly.
In the Expired Reports docs, mention the fact that if a build caches reports, the cached report may have an old timestamp that exceeds the 12 hour threshold.
Saying “CI Failed” in Codecov for this scenario is incorrect. CI is passing. The actual issue is that the report timestamp is more than 12 hours old.
Not sure if it impacted my case, but the JaCoCo merged report that I uploaded has many different timestamps in it, as it might itself be compiling report results from different modules that were cached at different times. Is the 12 hour threshold on the expired report based on the oldest or latest timestamp?

utybo · February 8, 2023, 6:46pm

Managed to fix the issue in the same way, thank you for your thread! I think option 4 is what may have happened in my case as I did have fresh reports that were a few hours old, but it’s possible that parts of the merged report were much older due to the use of the results of a build cache.

tom · February 8, 2023, 7:26pm

Yeah, I suspect that there may be something on our end with timezone matching, but I’m not positive. I’ll let the product team know you’re suggestions @snafu109

Topic		Replies	Views
Codecov + private repo on gitlab.com getting CI Failed Support	7	918	December 17, 2022
Codecov.yaml is valid, but github comment function doesn't work as expected Support java , github	3	265	July 23, 2023
"There is an error processing the coverage reports " Support	2	364	September 17, 2022
Codecov GitHub action doesn't work well compared to bash uploader Bug Fixes github	15	416	October 6, 2023
There was an error processing coverage reports for reports that look fine Support python , github	1	413	April 26, 2021