Codecov issue with CircleCI parallel builds

Support
1

Description

I’m having an incredibly hard to time configuring the tool with CircleCI. We are using parallel CircleCI containers and for some reason, it seems like Codecov isn’t dealing with multiple uploads properly. All my builds show with seemingly random coverage results.

17

Here’s my CircleCI file:

# Cicle version

version: 2.1

orbs:

  codecov: [codecov/codecov@1.0.4]

jobs:

  build:

    working_directory: ~/snappea

    parallelism: 10

    docker:

      - image: circleci/ruby:2.5

        environment:

          BUNDLE_JOBS: 3

          BUNDLE_RETRY: 3

          BUNDLE_PATH: ./vendor/bundle

          PGHOST: 127.0.0.1

          PGUSER: postgres

          RAILS_ENV: test

      - image: docker.io/foodadev/elasticsearch-icu:1.0.0

      - image: circleci/postgres:9.6.6-alpine-ram

        environment:

          POSTGRES_USER: postgres

          POSTGRES_PASSWORD: secretpass

          POSTGRES_DB: snappea_test

      - image: redis:4.0

      - image: node

      - image: rabbitmq:3

    steps:

      - checkout

 

      # This is required for tests that require ImageMagick

      - run: sudo apt-get install ghostscript

 

      - run: cp config/database.yml.ci config/database.yml

      - run: mkdir -p /tmp/artifacts

 

      # Restore bundle cache

      - restore_cache:

          keys:

            - snappea-gem-cache-{{ checksum "Gemfile.lock" }}

            - snappea-gem-cache-

 

      - run:

          name: Bundle Install

          command: bundle install --path ./vendor/bundle

 

      # Store bundle cache

      - save_cache:

          key: snappea-gem-cache-{{ checksum "Gemfile.lock" }}

          paths:

            - ./vendor/bundle

 

      - run:

          name: Wait for elasticsearch

          command: dockerize -wait tcp://localhost:9200 -timeout 1m

 

      - run:

          name: Wait for database

          command: dockerize -wait tcp://localhost:5432 -timeout 1m

 

      - run:

          name: Database setup

          command: bin/rake db:structure:load db:migrate db:seed db:fixtures:load --trace

 

      - run:

          name: Rspec

          command: |

            RAILS_ENV=test bundle exec rspec \

              --no-color \

              --profile 10 \

              --format RspecJunitFormatter \

              --out test_results/rspec.xml \

              --format documentation \

              $(circleci tests glob \

                "spec/api/**/*_spec.rb" \

                "spec/controllers/**/*_spec.rb" \

                "spec/exchanges/**/*_spec.rb" \

                "spec/graphql/**/*_spec.rb" \

                "spec/helpers/**/*_spec.rb" \

                "spec/lib/**/*_spec.rb" \

                "spec/mailers/**/*_spec.rb" \

                "spec/models/**/*_spec.rb" \

                "spec/policies/**/*_spec.rb" \

                "spec/presenters/**/*_spec.rb" \

                "spec/serializers/**/*_spec.rb" \

               "spec/services/**/*_spec.rb" \

                "spec/validators/**/*_spec.rb" \

                "spec/workers/**/*_spec.rb" \

                "spec/workflows/**/*_spec.rb" \

                | circleci tests split --split-by=timings)

 

      - store_artifacts:

          path: /tmp/artifacts

 

      - store_artifacts:

          path: coverage

 

      - store_test_results:

          path: test_results

 

      - codecov/upload:

          file: coverage/.resultset.json

          token: 5f80ab98-3b5a-4a03-943e-e1443983730b

Repository

Private

Additional Information

Any additional information, configuration or data that might be necessary to reproduce the issue.

2

Hi @sanjitsaluja, can you give a little more details about the seemingly random coverage results? We pull in together all the coverage results from those builds into one. Are you seeing discrepancies across builds or across commits?