It would be nice to infer coverage for partial builds from branch master.
We run a mono repo and we don’t always run coverage for every sub repo on every PR to save CI time.
We upload reports with -F <topic> to specify a topic. It would be nice if codecov would infer coverage for the other -F <topic> as the coverage of the branch commit if nothing is reported before the timeout.
This would make the coverage accurate with respect to a merge to master.
Great idea @nickpalmer!! We are calling this “carryforward” flags where we carryforward any coverage not updated from wherever it was most recently run. Looking at building this in the next several sprint cycles.
Awesome! Looking forward to seeing it roll out.
If you need a beta tester, let me know. I would be happy to turn it on and let you know how it is working on our repository! 
Thanks @nickpalmer – still building this. I’ll let you know when we get to beta!!
Any updates to this feature request? We tried using codecov in a monorepo and not having this made it unreliable.
I’m working on a project using dynamic build pipelines (via Buildkite), and we are tagging all of our coverage submissions using -F <flags>. This would be incredibly helpful to us, as currently have had to turn off coverage diff messages since they don’t contain accurate data, and it would be a very useful thing to be able to alert the team on coverage regressions!
Latest update is “coming very soon!” Jerrod’s last update stands, but work is coming along nicely. Really looking forward to sharing when done.
@ccarpita – getting very close! Definitely by end of March. Here’s a small sample for y’all
Hey @nickpalmer @JCMais @ccarpita – this feature for partial test runs “Carryforward Flags” is now in production in beta, would any of you like to test it out?
Invite open to rest of community as well
I would be interested in trying it out this week, please let me know how
I would love to be able to test it. Thanks for your work so far!
I would love to! I am just about to setup another mono repo project that could use this feature.
Hi everyone, thanks for all your patience. We released Carryforward Flags last week )https://blog.codecov.io/2020/04/21/new-test-only-what-you-change-with-carryforward-flags/) and you can find docs here (Carryforward Flags). Closing this out, but would love to get feedback and bug reports.
Thanks @tom.
We are not getting what I would expect with carry forward turned on in the repository I am attempting to use it on.
On a recent change that swapped 3 ‘let’ statements for ‘const’ statements across two flags in areas which are uncovered, we do see that it correctly identified the delta of zero in those three files. However, the coverage diff does not seem to be carrying forward, and shows -12.19% as an overall coverage drop.
I believe we have setup carry forward correctly and are certainly uploading with flags set. Our codecov.yml looks like this:
codecov:
notify:
require_ci_to_pass: yes
coverage:
status:
project:
defaut:
target: auto
common:
target: auto
flags:
- common
agent:
target: auto
flags:
- agent
client:
target: auto
flags:
- client
messages:
target: auto
flags:
- messages
api:
target: auto
flags:
- api
patch:
default:
only_pulls: true
changes: no
precision: 2
if_not_found: success
round: down
range: '0...100'
comment:
layout: 'reach, diff, flags, files'
behavior: default
require_changes: false
require_base: no
require_head: yes
branches: !staging
flags:
common:
paths: packages/common
carryforward: true
agent:
paths: packages/agent
carryforward: true
client:
paths: packages/client
carryforward: true
messages:
paths: packages/lambda/messages
carryforward: true
api:
paths: packages/api
carryforward: true
parsers:
javascript:
enable_partials: yes
We would love to get the correct coverage numbers on the status report to make them mandatory passing checks on Github.
As an example of another branch with this, when we go look at this in the UI, it shows no changes in overview, but “Coverage Changes” shows 99.
Hi @nickpalmer, looks like there are some issues with your yaml
I have removed comment -> branches: !staging because there is currently a known issue with negative expressions.
if_not_found is also misplaced in the yaml. It should go under coverage -> status -> project -> <project-name>
Finally, I have pasted a valid YAML without the above two issues. Let me know if this works for you!
codecov:
notify:
require_ci_to_pass: yes
coverage:
status:
project:
defaut:
target: auto
common:
target: auto
flags:
- common
agent:
target: auto
flags:
- agent
client:
target: auto
flags:
- client
messages:
target: auto
flags:
- messages
api:
target: auto
flags:
- api
patch:
default:
only_pulls: true
changes: no
precision: 2
round: down
range: '0...100'
comment:
layout: 'reach, diff, flags, files'
behavior: default
require_changes: false
require_base: no
require_head: yes
flags:
common:
paths:
- packages/common
carryforward: true
agent:
paths:
- packages/agent
carryforward: true
client:
paths:
- packages/client
carryforward: true
messages:
paths:
- packages/lambda/messages
carryforward: true
api:
paths:
- packages/api
carryforward: true
parsers:
javascript:
enable_partials: yes
Thanks you! That fixed it for us. Is there a linter for codecov.yml that would reveal these issues, or a place to see parse errors in the UI? I searched docs for that but didn’t find it.
Hi @nickpalmer, we are working on getting more of these issues user-facing, but as of now there isn’t. We are planning to get a better yaml validator out soon.
@tom Awesome thanks. I did find the validator we can post to, but it just seems to check if the yaml is valid and return it as json, not validate it against a schema.
I posted a new support issue around multiple comments on our PRs which I am not sure if that is related to carry forward, but it might be.