Unable to run codecov with upload locally or on CircleCI

Description

I am attempting to add my first repository into Codecov. Eventually, I want to have the code coverage reports generated on CircleCI and uploaded after each build to Codecov. I have also tried running tox with codecov locally. In both instances it has failed, and I’m hoping someone here can clue me in as to what I’ve done wrong.

My project is in Python and my pytest-based tests run via tox.

I have added the repository to my account via the codecov web application and I have copied the upload key.

On my local machine, I’ve set the upload key as an environment variable and then I ran tox:

$ export CODECOV_TOKEN="MY-TOKEN-STRING"
$ tox
py37 inst-nodeps: /home/james/git/cvdata/.tox/.tmp/package/1/cvdata-0.0.6.tar.gz
py37 installed: absl-py==0.9.0,astor==0.8.1,attrs==19.3.0,boto3==1.11.9,botocore==1.14.9,cachetools==4.0.0,certifi==2019.11.28,chardet==3.0.4,codecov==2.0.15,contextlib2==0.6.0.post1,coverage==5.0.3,cvdata==0.0.6,cycler==0.10.0,decorator==4.4.1,docutils==0.15.2,gast==0.2.2,google-auth==1.11.0,google-auth-oauthlib==0.4.1,google-pasta==0.1.8,grpcio==1.26.0,h5py==2.10.0,idna==2.8,ImageHash==4.0,imageio==2.6.1,importlib-metadata==1.5.0,jmespath==0.9.4,Keras-Applications==1.0.8,Keras-Preprocessing==1.1.0,kiwisolver==1.1.0,lxml==4.5.0,Markdown==3.1.1,matplotlib==3.1.2,more-itertools==8.2.0,networkx==2.4,numpy==1.18.1,oauthlib==3.1.0,opencv-python==4.1.2.30,opt-einsum==3.1.0,packaging==20.1,pandas==1.0.0,Pillow==7.0.0,pluggy==0.13.1,protobuf==3.11.2,py==1.8.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pyparsing==2.4.6,pytest==5.3.5,python-dateutil==2.8.1,pytz==2019.3,PyWavelets==1.1.1,requests==2.22.0,requests-oauthlib==1.3.0,rsa==4.0,s3transfer==0.3.2,scikit-image==0.16.2,scipy==1.4.1,six==1.14.0,tensorboard==2.1.0,tensorflow==2.1.0,tensorflow-estimator==2.1.0,termcolor==1.1.0,tqdm==4.42.0,urllib3==1.25.8,wcwidth==0.1.8,Werkzeug==0.16.1,wrapt==1.11.2,zipp==2.1.0
py37 run-test-pre: PYTHONHASHSEED='750695532'
py37 run-test: commands[0] | python setup.py check -m -s
running check
py37 run-test: commands[1] | py.test tests
============================= test session starts ==============================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
cachedir: .tox/py37/.pytest_cache
rootdir: /home/james/git/cvdata
collected 9 items                                                              

tests/test_analyze.py .                                                  [ 11%]
tests/test_clean.py ..                                                   [ 33%]
tests/test_convert.py .                                                  [ 44%]
tests/test_exclude.py .                                                  [ 55%]
tests/test_relabel.py ..                                                 [ 77%]
tests/test_resize.py ..                                                  [100%]

=============================== warnings summary ===============================
.tox/py37/lib/python3.7/site-packages/tensorflow_core/python/pywrap_tensorflow_internal.py:15
  /home/james/git/cvdata/.tox/py37/lib/python3.7/site-packages/tensorflow_core/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================= 9 passed, 1 warning in 2.31s =========================
py37 run-test: commands[2] | codecov

      _____          _
     / ____|        | |
    | |     ___   __| | ___  ___ _____   __
    | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
    | |___| (_) | (_| |  __/ (_| (_) \ V /
     \_____\___/ \____|\___|\___\___/ \_/
                                    v2.0.15

==> Detecting CI provider
  -> Got branch from git/hg
  -> Got sha from git/hg
==> Preparing upload
Error: Missing repository upload token

Tip: See all example repositories: https://github.com/codecov?query=example
Support channels:
  Email:   hello@codecov.io
  IRC:     #codecov
  Gitter:  https://gitter.im/codecov/support
  Twitter: @codecov

___________________________________ summary ____________________________________
  py37: commands succeeded
  congratulations :)

Notice the error above:

Error: Missing repository upload token

Apparently I need to set the token somehow other than as an environment variable. Can any here advise?

Also, the Gitter link listed in the support channels list seems to be defunct, at least I was unable to reach it when I tried.

When I run a build on CircleCI I get a different error:

#!/bin/bash -eo pipefail
. venv/bin/activate
tox
.package create: /home/circleci/repo/.tox/.package
.package installdeps: setuptools>=40.8.0, wheel
py37 create: /home/circleci/repo/.tox/py37
py37 installdeps: codecov, opencv-python, pytest, scikit-image
py37 inst: /home/circleci/repo/.tox/.tmp/package/1/cvdata-0.0.6.tar.gz
py37 installed: absl-py==0.9.0,astor==0.8.1,attrs==19.3.0,boto3==1.11.9,botocore==1.14.9,cachetools==4.0.0,certifi==2019.11.28,chardet==3.0.4,codecov==2.0.15,contextlib2==0.6.0.post1,coverage==5.0.3,cvdata==0.0.6,cycler==0.10.0,decorator==4.4.1,docutils==0.15.2,gast==0.2.2,google-auth==1.11.0,google-auth-oauthlib==0.4.1,google-pasta==0.1.8,grpcio==1.26.0,h5py==2.10.0,idna==2.8,ImageHash==4.0,imageio==2.6.1,importlib-metadata==1.5.0,jmespath==0.9.4,Keras-Applications==1.0.8,Keras-Preprocessing==1.1.0,kiwisolver==1.1.0,lxml==4.5.0,Markdown==3.1.1,matplotlib==3.1.2,more-itertools==8.2.0,networkx==2.4,numpy==1.18.1,oauthlib==3.1.0,opencv-python==4.1.2.30,opt-einsum==3.1.0,packaging==20.1,pandas==1.0.0,Pillow==7.0.0,pluggy==0.13.1,protobuf==3.11.2,py==1.8.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pyparsing==2.4.6,pytest==5.3.5,python-dateutil==2.8.1,pytz==2019.3,PyWavelets==1.1.1,requests==2.22.0,requests-oauthlib==1.3.0,rsa==4.0,s3transfer==0.3.2,scikit-image==0.16.2,scipy==1.4.1,six==1.14.0,tensorboard==2.1.0,tensorflow==2.1.0,tensorflow-estimator==2.1.0,termcolor==1.1.0,tqdm==4.42.0,urllib3==1.25.8,wcwidth==0.1.8,Werkzeug==0.16.1,wrapt==1.11.2,zipp==2.1.0
py37 run-test-pre: PYTHONHASHSEED='2312670270'
py37 run-test: commands[0] | python setup.py check -m -s
running check
py37 run-test: commands[1] | py.test tests
============================= test session starts ==============================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
cachedir: .tox/py37/.pytest_cache
rootdir: /home/circleci/repo
collecting ... collecting 3 items                                                             collected 9 items                                                              

tests/test_analyze.py .                                                  [ 11%]
tests/test_clean.py ..                                                   [ 33%]
tests/test_convert.py .                                                  [ 44%]
tests/test_exclude.py .                                                  [ 55%]
tests/test_relabel.py ..                                                 [ 77%]
tests/test_resize.py ..                                                  [100%]

=============================== warnings summary ===============================
.tox/py37/lib/python3.7/site-packages/tensorflow_core/python/pywrap_tensorflow_internal.py:15
  /home/circleci/repo/.tox/py37/lib/python3.7/site-packages/tensorflow_core/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================= 9 passed, 1 warning in 2.40s =========================
py37 run-test: commands[2] | codecov

      _____          _
     / ____|        | |
    | |     ___   __| | ___  ___ _____   __
    | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
    | |___| (_) | (_| |  __/ (_| (_) \ V /
     \_____\___/ \____|\___|\___\___/ \_/
                                    v2.0.15


==> Detecting CI provider
    Circle CI Detected
==> Preparing upload
==> Processing gcov (disable by -X gcov)
    Executing gcov (find /home/circleci/repo -not -path './bower_components/**' -not -path './node_modules/**' -not -path './vendor/**' -type f -name '*.gcno'  -exec gcov -pb  {} +)
==> Collecting reports
Error: No coverage report found

Tip: See all example repositories: https://github.com/codecov?query=example
Support channels:
  Email:   hello@codecov.io
  IRC:     #codecov
  Gitter:  https://gitter.im/codecov/support
  Twitter: @codecov

___________________________________ summary ____________________________________
  py37: commands succeeded
  congratulations :)
CircleCI received exit code 0

This is my tox.ini:

[tox]
envlist = py37
minversion = 3.3.0
isolated_build = true

[testenv]
passenv = CI CIRCLECI CIRCLE_*
deps =
    codecov
    opencv-python
    pytest
    scikit-image
commands =
    python setup.py check -m -s
    py.test tests {posargs}
    codecov

This is the CircleCI config.yaml:

jobs:
    build:
        docker:
            - image: circleci/python:3.7

        working_directory: ~/repo

        steps:
            - checkout

            # Download and cache dependencies
            - restore_cache:
                keys:
                    - v1-dependencies-{{ checksum "requirements.txt" }}
                    # fallback to using the latest cache if no exact match is found
                    - v1-dependencies-

            - run:
                name: install dependencies
                command: |
                    python3 -m venv venv
                    . venv/bin/activate
                    pip install -e .
                    pip install tox

            - save_cache:
                paths:
                    - ./venv
                key: v1-dependencies-{{ checksum "requirements.txt" }}

            # run tests with tox
            - run:
                name: run tests
                command: |
                    . venv/bin/activate
                    tox

            - store_artifacts:
                path: test-reports

            - store_test_results:
                path: test-results

Thanks in advance for any suggestions or insight!

Repository

Versions

Linux Ubuntu 18.04, Python 3.7

Hi @monocongo

The token does work as an ENV, I wonder if the quotes are upsetting our regex parser. Can you try without them?

Also, you mentioned CircleCI. They have a very nice environmental variables section you can add the token too (as do most CI providers), that may be a cleaner way of passing the token into the job.

1 Like

Thanks, @drazisil.

On my local machine I reset the environment variable without the quotation marks. Unfortunately this had no effect, same error.

On CircleCI I added the variable to my project. Again, same error. I don’t think the environment variable is at play here, or at least the information on the page displayed after adding the project to Codecov mentions that this is not necessary for CI services such as CircleCI.

It’s not needed for public builds, still required for private.

Since you are using tox, do you need to add it here? passenv = CI CIRCLECI CIRCLE_*

1 Like

Yes, that helped. Good eye!

Now I am facing the same issue as what I am seeing on CircleCI (no coverage report found):

py37 run-test: commands[2] | codecov

      _____          _
     / ____|        | |
    | |     ___   __| | ___  ___ _____   __
    | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
    | |___| (_) | (_| |  __/ (_| (_) \ V /
     \_____\___/ \____|\___|\___\___/ \_/
                                    v2.0.15

==> Detecting CI provider
  -> Got branch from git/hg
  -> Got sha from git/hg
==> Preparing upload
==> Processing gcov (disable by -X gcov)
    Executing gcov (find /home/james/git/cvdata -not -path './bower_components/**' -not -path './node_modules/**' -not -path './vendor/**' -type f -name '*.gcno'  -exec gcov -pb  {} +)
==> Collecting reports
Error: No coverage report found

Does codecov run coverage internally to produce the reports?

On my machine I have coverage version 5.0.3. Maybe this version is not supported, causing this issue? This issue makes me wonder if this is what’s going on here.

The python uploader is not as maintained as the bash one. Are you able to test with the bash uploader to rule this out?

Thanks for the suggestion, @drazisil.

I have managed to get a successful completion on my local machine when I run tox:

$ conda create -n codecovtest python=3
$ conda activate codecovtest
$ conda install tox
$ export CODECOV_TOKEN=MY-TOKEN-STRING
$ tox
py37 inst-nodeps: /home/james/git/cvdata/.tox/.tmp/package/1/cvdata-0.0.6.tar.gz
py37 installed: absl-py==0.9.0,astor==0.8.1,attrs==19.3.0,boto3==1.11.9,botocore==1.14.9,cachetools==4.0.0,certifi==2019.11.28,chardet==3.0.4,codecov==2.0.15,contextlib2==0.6.0.post1,coverage==5.0.3,cvdata==0.0.6,cycler==0.10.0,decorator==4.4.1,docutils==0.15.2,gast==0.2.2,google-auth==1.11.0,google-auth-oauthlib==0.4.1,google-pasta==0.1.8,grpcio==1.26.0,h5py==2.10.0,idna==2.8,ImageHash==4.0,imageio==2.6.1,importlib-metadata==1.5.0,jmespath==0.9.4,Keras-Applications==1.0.8,Keras-Preprocessing==1.1.0,kiwisolver==1.1.0,lxml==4.5.0,Markdown==3.1.1,matplotlib==3.1.2,more-itertools==8.2.0,networkx==2.4,numpy==1.18.1,oauthlib==3.1.0,opencv-python==4.1.2.30,opt-einsum==3.1.0,packaging==20.1,pandas==1.0.0,Pillow==7.0.0,pluggy==0.13.1,protobuf==3.11.2,py==1.8.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pyparsing==2.4.6,pytest==5.3.5,python-dateutil==2.8.1,pytz==2019.3,PyWavelets==1.1.1,requests==2.22.0,requests-oauthlib==1.3.0,rsa==4.0,s3transfer==0.3.2,scikit-image==0.16.2,scipy==1.4.1,six==1.14.0,tensorboard==2.1.0,tensorflow==2.1.0,tensorflow-estimator==2.1.0,termcolor==1.1.0,tqdm==4.42.0,urllib3==1.25.8,wcwidth==0.1.8,Werkzeug==0.16.1,wrapt==1.11.2,zipp==2.1.0
py37 run-test-pre: PYTHONHASHSEED='915968560'
py37 run-test: commands[0] | python setup.py check -m -s
running check
py37 run-test: commands[1] | py.test tests
============================= test session starts ==============================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
cachedir: .tox/py37/.pytest_cache
rootdir: /home/james/git/cvdata
collected 9 items                                                              

tests/test_analyze.py .                                                  [ 11%]
tests/test_clean.py ..                                                   [ 33%]
tests/test_convert.py .                                                  [ 44%]
tests/test_exclude.py .                                                  [ 55%]
tests/test_relabel.py ..                                                 [ 77%]
tests/test_resize.py ..                                                  [100%]

=============================== warnings summary ===============================
.tox/py37/lib/python3.7/site-packages/tensorflow_core/python/pywrap_tensorflow_internal.py:15
  /home/james/git/cvdata/.tox/py37/lib/python3.7/site-packages/tensorflow_core/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================= 9 passed, 1 warning in 2.33s =========================
py37 run-test: commands[2] | coverage xml -i
py37 run-test: commands[3] | codecov

      _____          _
     / ____|        | |
    | |     ___   __| | ___  ___ _____   __
    | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
    | |___| (_) | (_| |  __/ (_| (_) \ V /
     \_____\___/ \____|\___|\___\___/ \_/
                                    v2.0.15

==> Detecting CI provider
  -> Got branch from git/hg
  -> Got sha from git/hg
==> Preparing upload
==> Processing gcov (disable by -X gcov)
    Executing gcov (find /home/james/git/cvdata -not -path './bower_components/**' -not -path './node_modules/**' -not -path './vendor/**' -type f -name '*.gcno'  -exec gcov -pb  {} +)
==> Collecting reports
    + /home/james/git/cvdata/coverage.xml bytes=1705
==> Uploading
    .url https://codecov.io
    .query commit=29a5aa41d783bbdbac2b15dd6f511aed06951d96&branch=issue_53_circleci&token=<secret>&package=py2.0.15
    Pinging Codecov...
    Uploading to S3...
    https://codecov.io/github/monocongo/cvdata/commit/29a5aa41d783bbdbac2b15dd6f511aed06951d96
___________________________________ summary ____________________________________
  py37: commands succeeded
  congratulations :)

I’m pretty sure what fixed it was adding a coverage command into my tox.ini:

[tox]
envlist = py37
minversion = 3.3.0
isolated_build = true

[testenv]
passenv = CI CIRCLECI CIRCLE_* CODECOV_TOKEN
deps =
    codecov
    coverage
    opencv-python
    pytest
    scikit-image
commands =
    python setup.py check -m -s
    py.test tests {posargs}
    coverage xml -i
    codecov

However when I look at my project on the codecov site I don’t see any of the files from my project other than a utility module used by my test case codes, whereas I expected to see a list of my package’s files. In any event it’s a start…

When I commit this code and push the branch the CircleCI build that is triggered fails at the coverage command step:

py37 run-test: commands[2] | coverage xml -i
No data to report.
ERROR: InvocationError for command /home/circleci/repo/.tox/py37/bin/coverage xml -i (exited with code 1)

It’s not clear why things are not working the same on CircleCI as they do on my local machine, not why I am seeing a report on the codecov page for my project showing only a utility file used for testing (tests/assert_utils.py) rather than the package’s module files. Any ideas?

In case it helps shed some light on the issue, here is the project’s page showing the latest result from running tox on my local machine: https://codecov.io/gh/monocongo/cvdata/branch/issue_53_circleci

I seem to have worked it out using the tox and CircleCI configuration files shown below. It seems that what really fixed things was having tox run the pytest-based tests with a coverage command, i.e.

coverage run -m pytest tests {posargs}

I assume that by doing this it correctly makes the test coverage reports available so that codecov can then do its thing.

tox.ini:

[tox]
envlist = py37
minversion = 3.3.0
isolated_build = true

[testenv]
passenv = CI CIRCLECI CIRCLE_* CODECOV_TOKEN
deps =
    codecov
    coverage
    opencv-python
    pytest
    scikit-image
commands =
    python setup.py check -m -s
    coverage run -m pytest tests {posargs}
    codecov

.circleci/config.yml:

jobs:
    build:
        docker:
            - image: circleci/python:3.7

        working_directory: ~/repo

        steps:
            - checkout

            - restore_cache:
                keys:
                    - v1-dependencies-{{ checksum "requirements.txt" }}
                    # fallback to using the latest cache if no exact match is found
                    - v1-dependencies-

            - run:
                name: install dependencies
                command: |
                    python3 -m venv venv
                    . venv/bin/activate
                    pip install -e .
                    pip install codecov
                    pip install coverage
                    pip install tox

            - save_cache:
                paths:
                    - ./venv
                key: v1-dependencies-{{ checksum "requirements.txt" }}

            - run:
                name: run tests
                command: |
                    . venv/bin/activate
                    tox

            - store_artifacts:
                path: test-reports
1 Like