Coverage not updated when CI failed

Description

I have two github workflow jobs. One for windows and one for ubuntu.
As long as one job failed, the code coverage isn’t updated, but this is what I want.
The repo contains a mod for a game and it should be valid for each OS.
If ubuntu isn’t working, but windows is, then I expect the code coverage to be updated.

CI/CD URL

Ubuntu: Actions · Ismoh/NoitaMP · GitHub
Windows: Actions · Ismoh/NoitaMP · GitHub

Codecov Output

For some reason there is no output for the succeeded one.

Expected Results

I’ve expected the shown commit above would have one build and updated code coverage.

Actual Results

“There was an error processing coverage reports.”

Additional Information

Here is a commit where both jobs succeeded:

You can see both builds.
In addition to this: Merge seems not to be working, but let’s fix this issue first.
Did I do anything wrong?

@Ismoh, what looks like is happening is that the Windows build (always) doesn’t have a coverage report that is getting processed by Codecov. So when your Ubuntu build fails, it shows There was an error processing coverage reports, because the Windows one won’t be processed.

So the issue with the Windows build is that the file directory on upload has files presented as

noita-mp/files/lib/external/init_package_loading.lua
noita-mp/files/lib/external/luaunit.lua
noita-mp/files/lib/external/os_name.lua

in the coverage report, but the coverage files show them as

noita-mp\files\lib\external\init_package_loading.lua
noita-mp\files\lib\external\luaunit.lua
noita-mp\files\lib\external\os_name.lua
noita-mp\files\scripts\util\file_util.lua 

Not the switch from / to \.

I would suggest changing your Windows workflow to

    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v2
      with:
        directory: ${{ github.workspace }}
        flags: unittesting
        functionalities: network
        name: Windows Lua Unit Testing
        verbose: true

Not 100% sure that will work, but let me know.

Thank you for your fast reply!

Where do you see the differences? Can I see this as well?

Are the path separators a problem? I unfortunatelly don’t understand the quoted answer, sorry!

Changed it as you suggested, let’s see if this helps :slight_smile:

Sorry, I meant note not not, my mistake. You can view the raw reports in the build tab of a commit, like the link you posted.

I think the separators are a problem, but I’m not 100% sure. I figure adding the functionalities might fix it.

Thank you!
Looks like the suggestion you made, does not help, because there is no merge of two coverage reports. I added ‘shell: bash’ on githubs step to fix the path separator problem. Let’s see if it helps.

Github doesn’t like that yml property.

I added this workaround, but I even don’t see any changes on codecov, therefore I don’t know if this fixes anything.

I still need some assit!
Thank you in advantage!

The property doesn’t have to do with merging reports. Can you show me what you tried?

Sure thing. See below:

This is what I’ve done:

This didn’t help, because the workflow didn’t started. Error message:
https://github.com/Ismoh/NoitaMP/actions/runs/1636837564

Besides that, this is a workaround I tried, but codecov coverage percentage on codecov is still different.

This is a workaround, which I don’t like, because there is no different on codecov atm.

Besides that I need to fix the windows workflow in total. It’s broken, because of infinite loading. I’ll be back if it is working again, to see changes on codecov.

A workflow run where both builds succeeded:

Comparison of summary:
windows:

==============================================================================
Summary
==============================================================================

File                                                                            Hits Missed Coverage
----------------------------------------------------------------------------------------------------
D:/a/NoitaMP/NoitaMP/.testing/tests/files/scripts/util/file_util_test.lua       135  1      99.26%
D:/a/NoitaMP/NoitaMP/.testing/tests/files/scripts/util/guid_test.lua            23   0      100.00%
D:/a/NoitaMP/NoitaMP/.testing/tests/files/scripts/util/tabe_extensions_test.lua 30   0      100.00%
noita-mp/files/lib/external/logging.lua                                         116  71     62.03%
noita-mp/files/lib/external/luaunit.lua                                         690  815    45.85%
noita-mp/files/lib/external/os_name.lua                                         22   12     64.71%
noita-mp/files/scripts/init/init_.lua                                           3    0      100.00%
noita-mp/files/scripts/init/init_logger.lua                                     9    0      100.00%
noita-mp/files/scripts/init/init_package_loading.lua                            56   16     77.78%
noita-mp/files/scripts/util/file_util.lua                                       151  72     67.71%
noita-mp/files/scripts/util/guid.lua                                            58   8      87.88%
noita-mp/files/scripts/util/table_extensions.lua                                13   0      100.00%
noita-mp/files/scripts/util/util.lua                                            11   2      84.62%
----------------------------------------------------------------------------------------------------
Total                                                                           1317 997    56.91%

ubuntu:

==============================================================================
Summary
==============================================================================

File                                                                                         Hits Missed Coverage
-----------------------------------------------------------------------------------------------------------------
/home/runner/work/NoitaMP/NoitaMP/.testing/tests/files/scripts/util/file_util_test.lua       124  11     91.85%
/home/runner/work/NoitaMP/NoitaMP/.testing/tests/files/scripts/util/guid_test.lua            23   0      100.00%
/home/runner/work/NoitaMP/NoitaMP/.testing/tests/files/scripts/util/tabe_extensions_test.lua 30   0      100.00%
noita-mp/files/lib/external/logging.lua                                                      116  71     62.03%
noita-mp/files/lib/external/luaunit.lua                                                      702  806    46.55%
noita-mp/files/lib/external/os_name.lua                                                      22   12     64.71%
noita-mp/files/scripts/init/init_.lua                                                        3    0      100.00%
noita-mp/files/scripts/init/init_logger.lua                                                  9    0      100.00%
noita-mp/files/scripts/init/init_package_loading.lua                                         56   16     77.78%
noita-mp/files/scripts/util/file_util.lua                                                    132  88     60.00%
noita-mp/files/scripts/util/guid.lua                                                         58   8      87.88%
noita-mp/files/scripts/util/table_extensions.lua                                             13   0      100.00%
noita-mp/files/scripts/util/util.lua                                                         11   2      84.62%
-----------------------------------------------------------------------------------------------------------------
Total                                                                                        1299 1014   56.16%

What I expect:
codecov merge both results into the overview:

Only one example file for coverage comparison:

It is 91,85% which is only ubuntu coverage report value.

Is the merge broken?

@Ismoh, the merge is not the issue, the fact that we cannot process the Windows file due to the backslashes is.

I don’t think I saw it, can you post the CI link to where you added

functionalities: network

Do you need more info?

@Ismoh apologies, I think this got lost in my queue.

I tried opening the logs for the links you provided, and I’m seeing Error: We are currently unable to download the log. Please try again later. which isn’t super helpful. Would you be able to re-run or make the change again?