400 "too many uploads to this commit"

Description

My CI is failing is a 400 error being reported by the uploader, “too many uploads to this commit”. I’ve never seen this before and nothing special is going on in the repository. This is on the daily scheduled CI run on the main branch. I’m using the new/binary uploader, latest (1.0.5 on this run).

Commit SHAs

61ba6ea7e69e11a1616287d5996a025da8482fd3

Repository

CI/CD or Build URL

Uploader

binary uploader 1.0.5

Codecov Output

Full output
       _____          _
      / ____|        | |
     | |     ___   __| | ___  ___ _____   __
     | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
     | |___| (_) | (_| |  __/ (_| (_) \ V /
      \_____\___/ \__,_|\___|\___\___/ \_/
  
    Codecov report uploader 0.1.5
  [2021-10-08T15:10:58.442Z] ['info'] => Project root located at: /home/runner/work/community.hashi_vault/community.hashi_vault
  [2021-10-08T15:10:58.444Z] ['info'] -> No token specified or token is empty
  [2021-10-08T15:10:58.448Z] ['info'] Searching for coverage files...
  [2021-10-08T15:10:58.482Z] ['info'] => Found 20 possible coverage files:
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.5.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.6.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.8.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-2.6.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-2.7.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.7.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.5.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.6.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.8.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-2.6.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-2.7.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.9.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.7.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.5.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.6.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.8.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-2.6.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-2.7.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.9.xml
    /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.7.xml
  [2021-10-08T15:10:58.482Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.5.xml...
  [2021-10-08T15:10:58.487Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.6.xml...
  [2021-10-08T15:10:58.488Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.8.xml...
  [2021-10-08T15:10:58.489Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-2.6.xml...
  [2021-10-08T15:10:58.490Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-2.7.xml...
  [2021-10-08T15:10:58.491Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.9=data/coverage=units=units=docker-default=python-3.7.xml...
  [2021-10-08T15:10:58.492Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.5.xml...
  [2021-10-08T15:10:58.493Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.6.xml...
  [2021-10-08T15:10:58.499Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.8.xml...
  [2021-10-08T15:10:58.502Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-2.6.xml...
  [2021-10-08T15:10:58.503Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-2.7.xml...
  [2021-10-08T15:10:58.503Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.9.xml...
  [2021-10-08T15:10:58.504Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.10=data/coverage=units=units=docker-default=python-3.7.xml...
  [2021-10-08T15:10:58.504Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.5.xml...
  [2021-10-08T15:10:58.505Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.6.xml...
  [2021-10-08T15:10:58.505Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.8.xml...
  [2021-10-08T15:10:58.506Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-2.6.xml...
  [2021-10-08T15:10:58.506Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-2.7.xml...
  [2021-10-08T15:10:58.507Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.9.xml...
  [2021-10-08T15:10:58.508Z] ['info'] Processing /home/runner/work/community.hashi_vault/community.hashi_vault/cov-units-pre2.12/coverage=units=ansible_stable-2.11=data/coverage=units=units=docker-default=python-3.7.xml...
  [2021-10-08T15:10:58.525Z] ['info'] Detected GitHub Actions as the CI provider.
  [2021-10-08T15:10:58.526Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=uploader-0.1.5&token=*******&branch=main&build=1320652759&build_url=https%3A%2F%2Fgithub.com%2Fansible-collections%2Fcommunity.hashi_vault%2Factions%2Fruns%2F1320652759&commit=61ba6ea7e69e11a1616287d5996a025da8482fd3&job=CI&pr=&service=github-actions&slug=ansible-collections%2Fcommunity.hashi_vault&name=&tag=&flags=env_docker-default%2Cunits&parent=
  [2021-10-08T15:10:58.837Z] ['error'] Error POSTing to https://codecov.io: 400 [ErrorDetail(string='Too many uploads to this commit.', code='invalid')]
  [2021-10-08T15:10:58.838Z] ['error'] There was an error running the uploader: Error uploading to https://codecov.io: Error: Bad Request

Expected Results

Success

Actual Results

400 error from codecov servers

Additional Information

Re-running CI did not help. There is no additional or unusual CI activity happening on this commit.

@tom are you able to help with this? My CI is still failing because of the issue and I have no insight into why.

Hi @briantist, the main issue is that you are, in fact, uploading too many builds for the same commit. This isn’t strictly documented as it rarely occurs.

Has your daily scheduled build run on the same commit for the past few days? Do you know about how many uploads you do per CI run?

There are a lot of uploads per run, because of the matrix of all the testing scenarios, plus using the flags feature, so that results in lots more reports. I don’t have a specific number but you could take a look at the codecov job in CI on a successful run.

It had definitely run on the same commit for several days; possibly longer. But that’s not new; when there are no new commits in the repo, the daily will keep running against the same commit. That’s never caused a problem before, even on commits that had more active development, and remained the HEAD of main for a much longer time. So this really came as a surprise.

As the repo gets more and more stable, I expect that will happen more over time…

  • What is the upper limit on number of uploads per commit?
  • Why have a strict maximum number of uploads vs. some kind of rate limiting (max uploads per Y time period)?
  • Please document this!

This is happening again: token auth - unhandled exception when `token_file` is a directory (#154) · ansible-collections/community.hashi_vault@a7c2a19 · GitHub

This commit is only 5 days old. Something has changed on the codecov side… how can I expect to work around this??

Reiterating these points:

  • What is the upper limit on number of uploads per commit?
  • Why have a strict maximum number of uploads vs. some kind of rate limiting (max uploads per Y time period)?
  • Please document this!