Before submitting a topic, please confirm the following
I have searched for similar issues before creating this topic.
I have verified that my repository is using the Codecov GitHub app, if using GitHub
I have validated my codecov.yaml configuration file.
I have filled out the below sections to the best of my ability.
I am looking to find out how far back the carry forward flags look with regards to what it should carry forward. We have this setup and I can see it working but sometimes it does not carry forward flags and my assumption is that if they weren’t part of the last CI run, they can’t be carried forward.
An example I have is as follows:
All five actions runs and upload coverage succesfully.
Only three actions run and the other two are carried forward successfully.
The same as PR 1 but this time, the two that were previously carried forward are not. Codecov thinks only three actions were run and so the diff reporting is wrong.
As per my example, tests from PR 0 are carried forward all the way through to PR 2 and beyond until these CI actions are actually run again.
Hi Tom. I think that makes sense but I have tried to lay out what seems to be happening regarding my latest deployments. I hope this is helpful!
- 0fd246 is opened to merge from Staging into Master - All of the CI is run and all five reports are uploaded with full and correct reporting.
- e0c34a is showing as the last commit to Master on Codecov. This is the commit before 0fd246 (HEAD~1) but is the last commit that had reports run against it.
- 0a1ace is the current head of Staging and is three commits ahead of master.
- 0fd246 is the current head of Master.
- A PR is opened to merge staging into master. Only 3/5 reports are run. Coverage data is based on HEAD 0a1ace3 compared to PARENT e0c34a6. This means that the coverage is based on the head of Staging and HEAD~1 of Master. I assume this is because it was the last SHA that recieved reports?
- Codecov says that is has carried forward the necessary missing 2 reports, but the PR coverage for these shows "?".
For now, I am just manually dispatching the missing workflows against Staging to force new reports into Codecov but this isn’t ideal.
Another point that might be worth noting is that we have a workflow action that will automatically back merge Master into Staging when anything is merged into Master.
You can see the CI and reporting was run against e0c34a6 but the merge committed 0fd246f into Master. I guess this makes sense as e0c34a6 was the PARENT in the example above. Still not sure why it never carried forward though, feeling very confused now!
Ok, so my suspicion is that you are running any uploads consistently on the staging branch. 0fd246 and 0a1ace do not have any uploads, and the last known commit on this branch is 831ef804 (except for the commit that was made earlier).
So I’m a little confused when you say that coverage data is based on HEAD 0a1ace as that doesn’t exist in our system. Is this making sense, is that not what you were asking for?