Thanks again @drazisil. There are several things I don’t understand about how to use the codecov. We build and run the model in a different directory than where the source code lives. Then we copy the gcno,gcda/gcov files into some part of the source code directory before running the codecov uploader. Mostly this works, but in this case it seems it doesn’t. I do not understand the requirements for this tool. It’s possible that fixing-paths would help us, I have read thru almost all the codecov documentation in the last week or two, but because many of the basics are not explicitly stated, the benefit of things like fixing-paths is not entirely clear.
My sense is that most of your users are shooting for 100% coverage as part of their CI testing and that codecov addresses that need and is tightly linked into that workflow (git + CI). We are a very different kind of application. We use github, but we are a sea ice science code written mostly in Fortran, running on high performance computing platforms. We do use automated travisCI testing but only for a tiny set of testing when a PR is created. We run our full test suite on our HPC platforms with MPI and OpenMP and that requires some different strategies (like storing our code in one directory but building and running from a different directory). We also have to run many different cases, each with different input, to support our test coverage. We are also not expert python/SE tools types. So we’re struggling a bit to understand exactly what we need to do to get things to work.
What we want is to generate a code coverage report occasionally and as needed to see how we’re doing and where we can add new tests to improve coverage. It’s not going to be part of our standard model development workflow. So we are willing to deal with some work-arounds as needed. I have looked for other tools and codecov seems like just what we need, if we can get it to work reliably.
So far, we’re struggled with
- uploads timings out
- problems with stalled reporting (it seems others are seeing this as well)
- inability to show reports for source code associated with git submodules
- this issue where hits and misses are not reported in source code reports
I think we are making progress and hope that we get to the point where the tool is robust for us. I would be happy to provide feedback somewhere about our experience once everything is working to help improve documentation for other users like us. Any help we can get is appreciated. I will play with the fixing-paths next. Thanks!