I have noticed over the past few days, that the merged reports for my repository seem to be excluding some reports. The excluded reports have been uploaded (I can see the raw data), but the information they contain is not reflected in the merged report (lines that are covered in the excluded report are not covered in the merged report).
Repository
Steps to Reproduce
I am not sure if this occurs for anyone else, but you can see lines 9 & 10 are marked missing from the merged report for the yggdrasil/serialize/init.py file here, but there are hits for those lines in the raw report for build/job 2941.8 here.
Expected behavior: All of the reports are merged.
Actual behavior: Only some of the reports are included resulting in lines marked missing that are only covered in the excluded reports.
Flakiness? This has been constant for the past few days, unsure when it began exactly, but definitely started within the past two weeks.
Thank you for the detailed report. I agree, this seems very odd, since we can confirm those lines are indeed covered in the report. Let me investigate and get back to you.
@drazisil, I’m sorry to say that no, it isn’t related at all.
I quickly checked and astropy has all the necessary __init__.py, as yggdrasil does (great name, btw!)
One thing that I noticed, @langmm, is that the code that uses astropy.table appears not to be tested either.
Could that be related? If you test that if statement at line #682, does the library import get tested as well?
@langmm I paired with an engineer and we went though the code. It looks like those reports never got generated after upload. Really not sure why, when we went looked for a newer commit to check logs it looks like this is working again?
@drazisil Sorry for the delay in my reply. Everything does appear to be working now I will let you know if I see this happen again.
@smoia The missing lines for astropy.table on that job were intentional, I was in the process of refactoring my tests to limit the number of jobs. Those lines are once again covered.
@drazisil It’s has been a while, but I have encountered this issue again (for example, the report for this commit)). There are lines covered in some of the uploaded reports that are not reflected in the merged report information displayed. I can also open a new topic if that is preferred.
An example of what I’m seeing is line 445 in this file shows up as uncovered, but this build report shows that line with 1 hit.
Good catch! I hadn’t noticed there were two services.py files being tracked. I am covering that particular module by running coverage.py on a CLI in a subprocess so it must be picking up the site-packages file or logging it differently due to a missing configuration option.
Thank you for pointing me in the right direction and sorry for the noise!