HTTP 400 while uploading to s3 with python codecov from Travis

Support Bug Fixes
1

A travis run that worked yesterday morning is now getting a HTTP 400 while uploading using python codecov 2.0.22.

note: links were broken up due to limit of 4 links in posts by new users

I have two fairly old public projects which I’ve tried hooking up to codecov in the past 48 hours. Both are using codecov 2.0.22 from the output of pytest-cov. The first originally succeeded, but when I moved onto the second it has never passed, and the first gets the same issue when rerunning the Travis job that previously passed.

First project which used to pass:

Second project that always failed:

Additional Information

This may coincide with the scheduled maintenance that occurred yesterday.

I installed the codecov app in GhtHub Marketplace between the two changes, so I also tried uninstalling that but it still fails.

travis-ci.org use to be for free/public projects, and travis-ci.com for paid/private projects, but a few years ago they started migrating everything towards the latter. It seems python-jsonstore wasn’t migrated to the latter infrastructure - I don’t know if that would have any impact. [src].

Codecov Output

$ codecov
      _____          _
     / ____|        | |
    | |     ___   __| | ___  ___ _____   __
    | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
    | |___| (_) | (_| |  __/ (_| (_) \ V /
     \_____\___/ \____|\___|\___\___/ \_/
                                    v2.0.22
==> Detecting CI provider
    Travis Detected
==> Preparing upload
==> Processing gcov (disable by -X gcov)
    Executing gcov (['find', '/home/travis/build/Code0x58/python-stripzip', "-not -path './bower_components/**' -not -path './node_modules/**' -not -path './vendor/**'", '-type', 'f', '-name', '*.gcno', '', '-exec', 'gcov', '-pb', '', '{}', '+'])
find: unknown predicate `-not -path './bower_components/**' -not -path './node_modules/**' -not -path './vendor/**''
    Error running `['find', '/home/travis/build/Code0x58/python-stripzip', "-not -path './bower_components/**' -not -path './node_modules/**' -not -path './vendor/**'", '-type', 'f', '-name', '*.gcno', '', '-exec', 'gcov', '-pb', '', '{}', '+']`: Command '['find', '/home/travis/build/Code0x58/python-stripzip', "-not -path './bower_components/**' -not -path './node_modules/**' -not -path './vendor/**'", '-type', 'f', '-name', '*.gcno', '', '-exec', 'gcov', '-pb', '', '{}', '+']' returned non-zero exit status 1.
==> Collecting reports
    Generating coverage xml reports for Python
    + /home/travis/build/Code0x58/python-stripzip/coverage.xml bytes=3249
==> Appending environment variables
    + TRAVIS_OS_NAME
    + TRAVIS_PYTHON_VERSION
==> Uploading
    .url https://codecov.io
    .query commit=38b91d3db01c8298941661b260b3d3b3faf1c756&branch=master&job=335714598&pr=false&service=travis&build=32.5&slug=Code0x58%2Fpython-stripzip&package=py2.0.22
    Pinging Codecov...
    Uploading to S3...
Error: 400 Client Error: Bad Request for url: https://storage.googleapis.com/codecov/v4/raw/2020-05-17/2AA085FFEB8AE9A7F6F779A7CD3EACDA/38b91d3db01c8298941661b260b3d3b3faf1c756/678a8d1c-bd5d-486a-a6ed-786d0c0e97c2.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20200517%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20200517T124659Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=7ca9e41d70298434c624156494848493efc953343a8185e5c1051204f9258239
1 Like
2

Same issue here for my builds at Travis CI - Test and Deploy Your Code with Confidence

3

the same here - http://35.192.60.23/PyTorchLightning/pytorch-lightning/2962/1/2
it started about 17h ago and holds… :[
see the commit upload history - https://codecov.io/gh/PyTorchLightning/pytorch-lightning/branch/master/commits

4

The same issue on AppVeyor AppVeyor

5

Same with Github actions: https://github.com/diofant/diofant/actions/runs/107179945

6

Hi everyone, thanks for letting us know about this issue. This seems to be happening for many third-party uploaders and we are patching them to resolve the problem. Please let me know which uploader you are using, so we can properly follow up.

7

Hi everyone, if you are using the python uploader, please upgrade to 2.1.0

If you are using the node uploader, please upgrade to 3.7.0

1 Like
8

Thanks! I can confirm the latest python codecov is succeeding

9

I’m also seeing HTTP 400 from our Azure Pipeline using the Chocolatey package

10

@TimHess, not sure what I’m supposed to be seeing here, looks like it went through fine.

https://codecov.io/gh/SteeltoeOSS/Steeltoe/commit/3b25bd87e26c5b23fc2513010934bbf4c30722ef/build

11

Yeah, it is a bit weird that it says the step succeeded, but these are the logs:

2020-05-27 14:36:48 INF Uploading Reports.
2020-05-27 14:36:49 INF url: https://codecov.io
2020-05-27 14:36:49 INF query: https://codecov.io/upload/v4?branch=refactor&commit=3b25bd87e26c5b23fc2513010934bbf4c30722ef&build=3.0.0-2755&tag=&pr=&name=&flags=&slug=SteeltoeOSS%2Fsteeltoe&package=exe-1.10.0&build_url=https://dev.azure.com/SteeltoeOSS/Steeltoe/_build/results?buildId=2755&yaml=codecov.yml&job=2755&service=azure_pipelines&project=Steeltoe&server_uri=https://dev.azure.com/SteeltoeOSS/
2020-05-27 14:36:49 INF Pinging Codecov
2020-05-27 14:36:50 INF Uploading
2020-05-27 14:36:51 WRN Unable to upload coverage report to Codecov. Server returned: (400) Bad Request
2020-05-27 14:36:51 WRN Unknown reason. Possible reason being invalid parameters.
2020-05-27 14:36:51 WRN Failed to upload the report with CodecovUploader.
12

@TimHess, can you provide a more recent commit of this happening?

13

Sure, here’s one from yesterday

2020-06-02 20:16:57 INF Reading reports.
2020-06-02 20:16:57 INF d:\a\1\s\CodeCoverage\Cobertura.xml
2020-06-02 20:16:57 INF Uploading Reports.
2020-06-02 20:16:58 INF url: https://codecov.io
2020-06-02 20:16:58 INF query: https://codecov.io/upload/v4?branch=2.x&commit=a4fc76d891399f1fd8d723b4a45dd255b4dd1716&build=2.5.0-ci2800&tag=&pr=345&name=&flags=&slug=SteeltoeOSS%2Fsteeltoe&package=exe-1.10.0&build_url=https://dev.azure.com/SteeltoeOSS/Steeltoe/_build/results?buildId=2800&yaml=codecov.yml&job=2800&service=azure_pipelines&project=Steeltoe&server_uri=https://dev.azure.com/SteeltoeOSS/
2020-06-02 20:16:58 INF Pinging Codecov
2020-06-02 20:16:59 INF Uploading
2020-06-02 20:17:00 WRN Unable to upload coverage report to Codecov. Server returned: (400) Bad Request
2020-06-02 20:17:00 WRN Unknown reason. Possible reason being invalid parameters.
2020-06-02 20:17:00 WRN Failed to upload the report with CodecovUploader.
14

@TimHess, would it be possible to try to upload with the bash uploader?

One thing that seems strange to me is that the commits in the query don’t match with the codecov output.

For the output you submitted above, what should have been the commit SHA?
a4fc76d891399f1fd8d723b4a45dd255b4dd1716 or
b74026f72f30990111405f123ce118d9138703d1

15

That build should be for b74026f72f30990111405f123ce118d9138703d1
I suppose builds off of force-pushes could make this look a bit more complicated than it needs to - it isn’t limited to those for us though, that was just the most recent one I had at the time. This build has similar logs on a PR with a single commit.

I’m running a build right now with the bash uploader

16

I see, might be an issue with how we are pulling the commit SHA which is why you get the 400. Let me take another look there.

As for the bash uploader, I see it failing here.

17

Yeah, that was the first time I’d used bash in there, take 2 is here https://dev-azure-com/SteeltoeOSS/Steeltoe/_build/results?buildId=2817&view=logs&j=15d433b5-8b20-5a20-f833-1cba5bd05c3c&t=b08a86a5-fc9c-50dc-b5f9-dad7aab61bb6 and looks successful

Sorry, I had to mangle the url because I can’t post a link for that hostname anymore ?? change the dashes to dots to fix

1 Like
18

Amazing @TimHess. I’ll continue to dig into the wrong SHA in the other uploader, but does using the bash uploader happen to be a working solution for you?

19

I’m not attached to the mechanism that does the uploading. This appears to be functional and results in shaving about a minute off the build times so it should be fine for us, thanks

1 Like
20

@TimHess, looks like this was an issue with the codecov-exe uploader. I’ve made a pull request here