Github PR comment often not up to date

Support
, ,
1

Description

We have Travis CI configured to run python tests of 3 jobs.
Codecov seems to merge these just fine.

We have the comment bot enabled like so:

comment:
  layout: "header, diff"
  behavior: default
  require_changes: no

We occasionaly see PR comments reporting an unexpected drop in coverage, but when we look at codecov.io on the web, the coverage looks as expected. By looking at the edit timestamps on the github PR comment, and comparing to job build times shown on Travis, it appears that the codecov comment bot edits the comment after the first job finishes, edits it again after the second job finishes, and then (sometimes) does NOT edit it after the last job finishes. This would explain the reported drop in coverage.

Question is … is there any way to configure the comment bot to only trigger after all the jobs have finished? Similar to the after_n_builds option for chat notifications?
I don’t see that in the docs.

I don’t feel like it’s useful to us for the bot to comment before all the jobs finish, it’s just misinformation.

Repository

private

1 Like
2

The codecov.require_ci_to_pass setting seems relevant (About the Codecov YAML), but should be on by default.
Might be just a bug after all?
I came across codecov.notify settings in Discussion at https://github.com/codecov/support/issues/105 · GitHub, but they do not appear to be documented.