Reports for some jobs missing from merged report, but were successfully uploaded

Description

I have noticed over the past few days, that the merged reports for my repository seem to be excluding some reports. The excluded reports have been uploaded (I can see the raw data), but the information they contain is not reflected in the merged report (lines that are covered in the excluded report are not covered in the merged report).

Repository

Steps to Reproduce

I am not sure if this occurs for anyone else, but you can see lines 9 & 10 are marked missing from the merged report for the yggdrasil/serialize/init.py file here, but there are hits for those lines in the raw report for build/job 2941.8 here.

Expected behavior: All of the reports are merged.

Actual behavior: Only some of the reports are included resulting in lines marked missing that are only covered in the excluded reports.

Flakiness? This has been constant for the past few days, unsure when it began exactly, but definitely started within the past two weeks.

Versions

Chrome Version 78.0.3904.108 (Official Build) (64-bit)
OS: Travis CI misc Linux & Mac (see https://travis-ci.org/cropsinsilico/yggdrasil/builds/647791953)

Additional Information

The excluded reports are not for the same jobs (i.e. different jobs are excluded each time I trigger a new build).

Hi @langmm

Thank you for the detailed report. I agree, this seems very odd, since we can confirm those lines are indeed covered in the report. Let me investigate and get back to you.

@smoia this isn’t related to your init.py findings at all, is it?

@drazisil, I’m sorry to say that no, it isn’t related at all.
I quickly checked and astropy has all the necessary __init__.py, as yggdrasil does (great name, btw!)

One thing that I noticed, @langmm, is that the code that uses astropy.table appears not to be tested either.
Could that be related? If you test that if statement at line #682, does the library import get tested as well?

@smoia thank you for looking, appreciate the assist.

@langmm if their suggestion does’t help, I have time schedule with an engineer next week to investigate this as a possible bug. I’ll keep you updated.

@langmm I paired with an engineer and we went though the code. It looks like those reports never got generated after upload. Really not sure why, when we went looked for a newer commit to check logs it looks like this is working again?

Can you let me know?

@drazisil Sorry for the delay in my reply. Everything does appear to be working now I will let you know if I see this happen again.

@smoia The missing lines for astropy.table on that job were intentional, I was in the process of refactoring my tests to limit the number of jobs. Those lines are once again covered.

@drazisil It’s has been a while, but I have encountered this issue again (for example, the report for this commit)). There are lines covered in some of the uploaded reports that are not reflected in the merged report information displayed. I can also open a new topic if that is preferred.

An example of what I’m seeing is line 445 in this file shows up as uncovered, but this build report shows that line with 1 hit.

@langmm I believe that might be because the build report is showing the file that hits line 445 as

filename="/usr/share/miniconda/envs/test/lib/python3.6/site-packages/yggdrasil/services.py"

However,

filename="yggdrasil/services.py"

shows it as uncovered. Are you expecting the site-packages one to be covered? If so, is that a different file altogether?

Good catch! I hadn’t noticed there were two services.py files being tracked. I am covering that particular module by running coverage.py on a CLI in a subprocess so it must be picking up the site-packages file or logging it differently due to a missing configuration option.

Thank you for pointing me in the right direction and sorry for the noise!

1 Like