Invalid links added to PR on GitHub, plus inconsistent results

Support
1

Description

The links added to both the codecov entries in the “checks” section are invalid.

Repository

Versions

N/A for this issue

Additional Information

This is a public repo. You can see an example of this in [DO NOT MERGE] Change that doesn't do anything by texasaggie97-zz · Pull Request #1251 · ni/nimi-python · GitHub

I can then go to the repo overview page and see the report for this PR.

These links are from the initial creation of the PR:

  • The invalid links are /gh/ni/nimi-python/compare/ffe7399289302397b2aa01bc465a5dc54dde31b4…b8fd403b92ebf4107a66105f1adea84338e01a2f
  • The link to the actual report is /gh/ni/nimi-python/commit/b8fd403b92ebf4107a66105f1adea84338e01a2f
  • As of right now, the percentage of coverage on the report on codecov doesn’t match the percentage coverage reported to GitHub: 87.62% on codecov, 91.92% reported to GitHub

I made another change that has no impact on anything (add/remove a blank line in a file) and here is what I get on the same PR:

  • Links in “Check” section still bad: /gh/ni/nimi-python/compare/ffe7399289302397b2aa01bc465a5dc54dde31b4…72ad54ecc4b90f838610a34d71f03f8e9ddc0216
  • Actual link to report /gh/ni/nimi-python/commit/72ad54ecc4b90f838610a34d71f03f8e9ddc0216/build
  • Percentage covered now matches between report on codecov and what is reported to PR: 90.18%

All tests completed successfully and uploaded their coverage results for both. None of the changes in this PR have had any impact on actual code.

Occasionally, though not on this PR, I will see two different reports for the same PR. Because which ever of the reports has updated the “checks” section doesn’t include all the coverage results, it will mark the PR as failed since the coverage is too low.

What other information would be useful for tracking this down?

Replace any with the base codecov url since there were too many links

2

Hi @texasaggie97

Could you clarify what links you think are invalid? The status ones? Or ones on the PR comment? For PRs, the status ones should be compares so you can see what we are calculating from.

You should be able to click the links on the PR comment to visit the PR view.

3

The links that have compare. When I click on the link from GitHub I consistently get an error 500, as do my colleagues:

image
image1016×807 135 KB

Both links (codecov/project and codecov/patch) have the same URL.

4

For the comment added by codecov, it says coverage will increase by 4.1%, but I have made no substantial changes in this PR. All I have done is add or remove a line from .codecov.yml.
image

5

Opened issue #1275 to track this problem.

6

Hi @texasaggie97, would you mind pushing a new commit onto that PR? It’s a little outdated and we’ve made a ton of changes to improve our parent and base commit picking. I can go in after and inspect why coverage dropped the way it did.

7

Closing this out due to the resolution here: Inconsistent results from codecov · Issue #1275 · ni/nimi-python · GitHub