Github Checks api and annotation not working

Support Bug Fixes
1

Suddenly I find coverage status in checks tab and annotations not working in github. Is there any change in the behaviour ?

Screen Shot 2020-10-09 at 4.47.07 PM
Screen Shot 2020-10-09 at 4.47.07 PM1468×1518 183 KB

2

@jahan01, what repository is this for? Also can you paste your codecov.yml here?

3

Hi @tom

It is a private repo and below is my codecov.yml

coverage:
  round: up
  range: "50...80"
  status:
    project:
      default:
        threshold: 5%
      framework:
        threshold: 5%
        paths:
          - src/main/*/com/framework
      money:
        threshold: 5%
        paths:
          - src/main/*/com/money
    patch:
      default:
        threshold: 5%
4

@jahan01, would you be able to supply a commit SHA so we can dig into this? Is this happening for all commits?

5

@tom Yes, this is happening for all commits since 4/5 days.

Few commits:

  1. a68ccd4e1d6a7a6de2e9beb7fae8c00b540396c3
  2. c0f6f307395d2766e0accf5dfafb22c870608b59
  3. cb5f4aa26a685b928fa7c0668c1a651ec605b7eb

Last known commit to have checks tab populated:

  1. cf3b16ded272a31965f96247d022ca675c8cf6a4
  2. 9d2364a8b6177ef3f9eaca43f1f869be897dabec
6

@jahan01, this is indeed strange. Are you getting the same behavior if you click the Details section of the status check as when you go to the Checks tab?

8

Hi, we are observing a similar behaviour, where github-checks (resulting in annotations) have stopped working recently.

Here is a ~2 weeks old PR on which the checks were correctly run. However on recent PRs, there’s no Codecov checks being run.

There were no changes to our codecov.yml and according to documentation the checks are enabled by default: GitHub Checks are enabled by default on the patch status.

Thanks!

9

Hi @tom, as shown in my initial screenshot Details section looks fine. It displays all the checks performed by Codecov.

10

We are having the same problem. See:

I turned off GitHub annotations for codecov 1,5 months ago and everything worked perfectly. See this PR:

Now it has stopped working and annotations are shown again despite being turned off in codecov.yml. This is very frustrating since it makes PRs much more difficult to review.

11

@danielb987, can you be more specific? It looks like you just disabled annotations, but what is wrong when annotations are enabled.

12

Annotations destroys the flow of the code. When I review a PR, I need to understand what the code does. So when I do my review, I care about the source code, and not about whenether the code is tested or not. The attached image shows an example there the annotation splits the code in half, making the code much more difficult to read. And for PRs with lots of changed files, these annotations heavily reduces the quality of my reviews since they make the code so much more difficult to read.

When I review a PR, I need to quickly see what the code does and I need to be able to spot errors in the source code. Anything that gets in the way of that is really bad. And the Codecov annotations are big culprits in this regard.

I like Codecov. It’s good software. But it needs to stay out of the review process.

CodecovAnnotations
CodecovAnnotations853×433 19.4 KB

13

@danielb987 can you link us to a PR where annotations are still turned on? I agree they should be turned off based on your YAML.

Jerrod

14

Hey @danielb987 – it appears your YAML is invalid. You can see how here:

https://codecov.io/validate

  in "<byte string>", line 3, column 7:
        - !appveyor # ignore CI builds by  ... 
          ^
15

@danielb987, to clarify @jerrod’s response, I don’t believe ! are permissible in that section of the yaml. You should not get annotations with the removal of that character.

You can always validate using

curl --data-binary @codecov.yml https://codecov.io/validate
16

Thank you! I have created a new PR with the changed codecov.yml file.

1 Like