Needed help to upload coverage reports with google cloud build

Support
1

Description

Can’t upload from google cloud build.
(I’ve just posted this on stackoverflow before I found out this comunity. I’ve pasted this question from my SO question here: docker - Upload jest code coverage results to codecov during google cloud build - Stack Overflow)

Repository

A link to the repository in question (or private)

Versions

Please include the browser and OS and what versions you’re running.

Additional Information

I’m trying to figure out how to upload my jest code coverage reports to codecov.
From there documentation:

bash <(curl -s https://codecov.io/bash) -t token

So I tried to run the bash script from a cloud build step witth the following cloudbuild.yaml

steps:
  - name: node:10.15.1
    entrypoint: npm
    args: ["install"]
  - name: node:10.15.1
    entrypoint: npm
    args: ["test", "--", "--coverage"]
  - name: 'gcr.io/cloud-builders/curl'
    entrypoint: bash
    args: ['<(curl -s https://codecov.io/bash)', '-t', '$_CODECOV_TOKEN']
  - name: node:10.15.1
    entrypoint: npm
    args: ["run", "build:production"]

I get the following error:

Step #2: bash: <(curl -s https://codecov.io/bash): No such file or directory

Obviously because the <(curl -s https://codecov.io/bash) is interpreted as string while I want it to be executed.

Edit:

I’ve changed my build step to the following:

  - name: "gcr.io/cloud-builders/curl"
    entrypoint: bash
    args: ["./scripts/codecov-upload.bash", "$_CODECOV_TOKEN"]

And added a file codecov-upload.bash

bash <(curl -s https://codecov.io/bash) -t $1

When running my cloud build the codecov bash uploader succesfully starts. However, I doesn’t manage to upload to reports to clodecov.

Here’s the logs from codecov bash uploader:

Step #2: Test Suites: 1 passed, 1 total
Step #2: Tests:       1 passed, 1 total
Step #2: Snapshots:   1 passed, 1 total
Step #2: Time:        28.981s
Step #2: Ran all test suites.
Finished Step #2
Starting Step #3
Step #3: Already have image (with digest): gcr.io/cloud-builders/curl
Step #3: /dev/fd/63: option requires an argument -- t
Step #3: 
Step #3:   _____          _
Step #3:  / ____|        | |
Step #3: | |     ___   __| | ___  ___ _____   __
Step #3: | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
Step #3: | |___| (_) | (_| |  __/ (_| (_) \ V /
Step #3:  \_____\___/ \__,_|\___|\___\___/ \_/
Step #3:                               Bash-tbd
Step #3: 
Step #3: 
Step #3: x> No CI provider detected.
Step #3:     Testing inside Docker? http://docs.codecov.io/docs/testing-with-docker
Step #3:     Testing with Tox? https://docs.codecov.io/docs/python#section-testing-with-tox
Step #3:     project root: .
Step #3: /dev/fd/63: line 897: git: command not found
Step #3: /dev/fd/63: line 897: hg: command not found
Step #3:     Yaml not found, that's ok! Learn more at http://docs.codecov.io/docs/codecov-yaml
Step #3: ==> Running gcov in . (disable via -X gcov)
Step #3: ==> Python coveragepy not found
Step #3: ==> Searching for coverage reports in:
Step #3:     + .
Step #3:     -> Found 3 reports
Step #3: ==> Detecting git/mercurial file structure
Step #3: ==> Reading reports
Step #3:     + ./coverage/clover.xml bytes=163786
Step #3:     + ./coverage/coverage-final.json bytes=444241
Step #3:     + ./coverage/lcov.info bytes=71582
Step #3: ==> Appending adjustments
Step #3:     http://docs.codecov.io/docs/fixing-reports
Step #3:     + Found adjustments
Step #3: ==> Gzipping contents
Step #3: ==> Uploading reports
Step #3:     url: https://codecov.io
Step #3:     query: branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Uploading to Codecov
Step #3:     HTTP 400
Step #3: missing required properties: [&#39;commit&#39;]
Finished Step #3
Starting Step #4
Step #4: Already have image: node:10.15.1
Step #4:

I’ve noticed two things in the logs:

1. Step #3: /dev/fd/63: option requires an argument -- t
2. Step #3: missing required properties: [&#39;commit&#39;]

When searching to fix number 2, I found that I might had to install git in my container.

So I’ve tried to make a custom container image with docker:

Dockerfile:

FROM gcr.io/cloud-builders/curl
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y git

So I build the image:

build -t "gcr.io/[PROJECT_ID]/builder .

And update my build step to use this image instead:

But using the image created with that dockerfile returns the same errors.

Maybe the Dockerfile for that custom image isn’t correct? Or I’m missing something else?

2

Hi Thomas, my guess is that Codecov isn’t able to access environment variables (e.g. commit hash). These are necessary to pair coverage reports to a commit. I think you can take a look at https://docs.codecov.io/v4.3.0/docs/testing-with-docker (if you haven’t already) which might solve your problem.

1 Like
3

HI Tom, Thanks for you message, I’ve seen the testing-with-docker page but that didn’t help me much. However, your note regarding environment variables is definitely something I’ll look into. I’ve found the necessary variables in the source code of the bash uploader:

commit="$VCS_COMMIT_ID"
branch="$VCS_BRANCH_NAME"
pr="$VCS_PULL_REQUEST"
slug="$VCS_SLUG"
tag="$VCS_TAG"
build_url="$CI_BUILD_URL"
build="$CI_BUILD_ID"
job="$CI_JOB_ID"

But they can be passed as arguments as well, so I’ll try that and let you know here what the results are.

1 Like
4

So I’ve modified my bash script as follows:
#!/bin/bash

echo "Commit $2 on branch $3"

bash <(curl -s https://codecov.io/bash) -t $1 -C $2 -B $3

And is nicely prints:

Step #3: Commit b72ccd13b6d23ae930302d02da9143adc07cb348 on branch feat/112-1
Howerver it returns the same message.

And I still see empty values in the query:
Step #3: ==> Uploading reports

Step #3:     url: https://codecov.io

Step #3:     query: branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=

So my guess is there is something wrong on how I pass my arguments to the bash uploader
bash <(curl -s https://codecov.io/bash) -t $1 -C $2 -B $3

Any ideas?

5

Thanks to Tom for pointing out the required environment variables, I’ve adjust my build step as follows:

- name: 'gcr.io/cloud-builders/curl'
  entrypoint: bash
  args: ['-c', 'bash <(curl -s https://codecov.io/bash)']
  env:
  - 'VCS_COMMIT_ID=$COMMIT_SHA'
  - 'VCS_BRANCH_NAME=$BRANCH_NAME'
  - 'VCS_PULL_REQUEST=$_PR_NUMBER'
  - 'CI_BUILD_ID=$BUILD_ID'
  - 'CODECOV_TOKEN=$_CODECOV_TOKEN' # _CODECOV_TOKEN is user user substituion variable specified in my cloud build trigger

I’ve explained my steps in more detail on my original question on stackoverflow here.

1 Like