Bash uploader checksum includes 'env' file in version 1.0.2

Support
1

Description

Hello. I’m following the instructions to validate the Bash uploader script before uploading my coverage reports, however the sample validation script is failing when using version 1.0.2 of the CodeCov bash uploader script as the checksum file now include the hash of an ‘env’ file.

The checksum files for version 1.0.1 of the uploader script only contained a single hash for the codecov file and I didn’t have any issue validating the bash script and uploading my coverage reports. I don’t know if the inclusion of the env file is an error or if the sample validation script should additionally create/download an env file for verification.

CI/CD or Build URL

Use CircleCI, tried both macOS and Android nodes

Uploader

Here is the script I’m using to download the bash script, validate its checksum files, and then upload my coverage reports.

curl -s https://codecov.io/bash > codecov;
            VERSION=$(grep 'VERSION=\".*\"' codecov | cut -d'"' -f2);
            echo "Using CodeCov version '$VERSION'"
            for i in 1 256 512
            do
              shasum -a $i -c --ignore-missing <(curl -s "https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM") ||
              shasum -a $i -c <(curl -s "https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM")
            done
            bash ./codecov -v -X s3 -c -D "./ci/unit-test/build/reports" -F unit-tests

Codecov Output

Using CodeCov version ‘1.0.2’
codecov: OK
shasum: env: No such file or directory
env: FAILED open or read
shasum: WARNING: 1 listed file could not be read

Expected Results

Expect the checksum files to only include hashes for codecov

Output from https://raw.githubusercontent.com/codecov/codecov-bash/1.0.1/SHA1SUM

0ddc61a9408418c73b19a1375f63bb460dc947a8  codecov

Actual Results

The checksum files contain hashes for an additional env file.
Output from https://raw.githubusercontent.com/codecov/codecov-bash/1.0.2/SHA1SUM

537069158a6f72b145cfe5f782dceb608d9ef594  codecov
af0dd19ee59f977bf5793cffafe116d7c248aa62  env

The sample checksum validation does not include guidance on how to handle this env file. [Deprecating] Bash Uploader

2

@kevinlind what version of shasum are you running? You can find that out by running shasum -v

3

@tom

Thanks for the reply. It appears the validation of the checksums is working on CircleCI’s Android image, but not on a macOS image. I have to apologize, I didn’t initially have --ignore-missing added to my Android build. After adding the flag the upload is working. The macOS version of shasum doesn’t have this this flag.

Android is using shasum 6.02 (passing)
macOS is using shasum 5.84 (failing)

4

I can use head to just read the first line of the checksum file. Is it guaranteed that the first line of the checksum hosted by CodeCov will be the hash of the bash uploader script? Is there anything that customers need to do with the checksum hash for env? Is the below code sufficient to guard against the concerns raised by recent bash uploader security issue?

curl -s https://codecov.io/bash > codecov;
VERSION=$(grep 'VERSION=\"[0-9\.]*\"' codecov | cut -d'"' -f2);
for i in 1 256 512
do
  shasum -a $i -c --ignore-missing <(curl -s "https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM") ||
  shasum -a $i -c <(curl -s "https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM" | head -n 1)
done
5

@kevinlind you should be able to use head to read the first line, but I’m curious, does the documented version not work for the macOS builds? Do you mind sharing a link to CI if possible?

6

@tom here’s a link to the CI build which failed. The CI step is named “Upload Code Coverage Report” and is running on a macOS image. Later PRs pass after I added the head command to get the first line.
https://app.circleci.com/pipelines/github/adobe/aepsdk-edgeidentity-ios/135/workflows/f86bb6d3-d880-42af-94e4-4868d25b6627/jobs/137

From the output, you can see shasum fails to read the env file:

Unknown option: ignore-missing
Type shasum -h for help
(23) Failed writing body
codecov: OK
shasum: env: 
env: FAILED open or read
shasum: WARNING: 1 listed file could not be read

Also, just running the documented version of the validation script fails on my local Mac.

Another question, what is the “env” file referenced in the checksum file and do I need to verify against it? How do I get or generate the “env” file to calculate the SHA hash?

7

@kevinlind thanks for this, what happens if you switch the lines to

  shasum -a $i -c <(curl -s "https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM" | head -n 1) ||
  shasum -a $i -c --ignore-missing <(curl -s "https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM")
8

@tom What you provided would cause the first line to always execute on macOS and Linux CI images as the provided shasum options are a subset of the second line. The command using --ignore-missing will never get called. You could simplify this to

shasum -a $i -c <(curl -s "https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM" | grep -w "codecov")

Note I changed to using grep -w "codecov" as there’s no guarantee that the “codecov” checksum hash will appear first in the file. If the checksum file does not contain a “codecov” entry, shasum will fail as well, which is what we’d want to happen.

9

Thanks @kevinlind, I have updated the documentation with that addition. I realized I didn’t answer your question about the env file. We publish this file at https://codecov.io/env to be used when running Codecov inside of Docker.

Is there anything outstanding here that needs to be addressed?

10

Thanks for all your help with this @tom . I don’t have anymore questions or concerns. Thanks for updating your documentation.

1 Like