Upload Timeouts

We are still experiencing the Codecov upload timeouts, which as the screenshot shows, can add up to 5 minutes to our entire CI workflow.

3 Likes

We experiment the same issue when uploading the report through the bash uploader. This adds us more than 12 mins to our CI workflow… :slightly_frowning_face:

This is a still an issue.

Hi @DannyNemer, @PhilippeBoisney

Sorry for the delay in responding, I thought I had. Gonna try and dig into this today to see what’s causing these upload failures, will keep you folks posted.

Hi @drazisil
Thanks for your response! Hope it will be fixed soon :crossed_fingers:

Likewise, thank you for your response!

Think we have a handle on the cause here, and are working to fix it. Appreciate your patience.

@PhilippeBoisney @DannyNemer

I hesitate to share the workaround here because it’s not preferred and will put more strain on our servers (and we know Google will find it and share it as the top result :stuck_out_tongue:) but adding -X s3 to your bash upload command will bypass the presigned PUT that is timing out and get you back in business until we hunt down the cause of the lag.

1 Like

We also experience these timeouts. We sequentially upload coverage from twelve projects in a row. I wonder if it’s some sort of rate limiting.

I’m not sure if it is related, but we sometimes even fail to download the bash uploader:

curl: (7) Failed to connect to codecov.io port 443: Connection timed out

We’re downloading it with the following command:

curl https://codecov.io/bash

I hope this information is helpful. I am happy to turn on extra debugging and report back, if that’s possible.

Any updates on resolving the Codecov-upload timeout issues?

I believe we made some changes over the weekend end that will help, I’m not sure if that’s the full fix though.

Still an issue.
Having this problem, multiple times.

@lsmar, can you open a new ticket for this? Unfortunately, every situation for timeouts is a little different.

I’m still seeing this. As mentioned in the corresponding issue Fixing timeouts · Issue #37 · codecov/codecov-action · GitHub I have a larger number of concurrent builds Fix upload by Midnighter · Pull Request #9 · Midnighter/structurizr-python · GitHub

It seems to happen much more often than when I used bash <(...) but could be due to software changes in the meantime, of course.

Hi @Midnighter thanks here, we’ll take a look to see what this is happening to you.

I was also trying to find a possibility to retry failed steps in GitHub workflows. So far I couldn’t find anything but maybe retrying the upload could be included in your action?

1 Like

Thanks for the suggestion! We’d like to get some stability here, but I’ll look into adding some retry logic.

Can confirm we’re also having the issue and we’re not running parallel builds or anything like that. Every few runs we get timeouts.

Hi all, we made some updates to the bash script that should hopefully help. Let me know if you are still running into issues.

Timeouts continue to pop up:
https://github.com/apereo/cas/actions/runs/194270944