Combination of stuck in processing and " Unable to find report content in the storage archive."

Description

Over the past two months, a gradually increasing percentage of commits have gotten stuck with the “processing” status. For many, when I click on the “Processing” button I get an “Unable to find report content in the storage archive” error. For others I can see data but there’s a " Reports are queued for processing… Please review report with caution, it may change." message at the top.

Repository

CI/CD

Travis CI

Uploader

Bash uploader: curl -s https://codecov.io/bash | bash

Commit SHAs

First commit where this happened: ce7e4e51067e61a192bc8d9ad1a0ef96f27a97e5
It’s now happens on all commits.

Codecov YAML

Default YAML

Codecov Output

  _____          _

 / ____|        | |

| |     ___   __| | ___  ___ _____   __

| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /

| |___| (_) | (_| |  __/ (_| (_) \ V /

 \_____\___/ \__,_|\___|\___\___/ \_/

                              Bash-20200707-353aa93

==> Travis CI detected.

    project root: .

    Yaml not found, that's ok! Learn more at http://docs.codecov.io/docs/codecov-yaml

==> Running gcov in . (disable via -X gcov)

==> Python coveragepy not found

==> Searching for coverage reports in:

    + .

    -> Found 195 reports

==> Detecting git/mercurial file structure

==> Appending build variables

    + TRAVIS_OS_NAME

==> Reading reports

    + ./third-party/emsdk/upstream/emscripten/system/lib/compiler-rt/lib/sanitizer_common/sanitizer_coverage_libcdep_new.cc bytes=7448

    + ./third-party/emsdk/upstream/emscripten/system/lib/compiler-rt/lib/sanitizer_common/sanitizer_coverage_win_dynamic_runtime_thunk.cc bytes=1189

    + ./third-party/emsdk/upstream/emscripten/system/lib/compiler-rt/lib/sanitizer_common/sanitizer_coverage_fuchsia.cc bytes=9899

    + ./third-party/emsdk/upstream/emscripten/system/lib/compiler-rt/lib/sanitizer_common/sanitizer_coverage_win_weak_interception.cc bytes=1198

    + ./third-party/emsdk/upstream/emscripten/system/lib/compiler-rt/lib/sanitizer_common/sanitizer_coverage_win_sections.cc bytes=3503

    + ./third-party/emsdk/upstream/emscripten/system/lib/compiler-rt/lib/sanitizer_common/sanitizer_coverage_interface.inc bytes=1711

    + ./third-party/emsdk/upstream/emscripten/system/lib/compiler-rt/lib/sanitizer_common/sanitizer_coverage_win_dll_thunk.cc bytes=1055

    + ./tests/base/coverage-array.out bytes=12831488

    + ./tests/base/coverage-Ptr.out bytes=12779320

    + ./tests/base/coverage_coverage-array.txt bytes=225172

    + ./tests/base/coverage_coverage-macro_math.txt bytes=10750

    + ./tests/base/coverage_coverage-errors.txt bytes=17691

    + ./tests/base/coverage_coverage-Ptr.txt bytes=123537

    + ./tests/base/coverage-vector.out bytes=12481968

    + ./tests/base/coverage-macros.out bytes=12520016

    + ./tests/base/coverage_coverage-macros.txt bytes=9767

    + ./tests/base/coverage_coverage-assert.txt bytes=16184

    + ./tests/base/coverage_coverage-vector.txt bytes=52572

    + ./tests/base/coverage-errors.out bytes=12458024

    + ./tests/base/coverage-assert.out bytes=12224584

    + ./tests/base/coverage-macro_math.out bytes=12422000

    + ./tests/constexpr/coverage_coverage-ce_array.txt bytes=294

    + ./tests/constexpr/coverage-ce_random.out bytes=12200544

    + ./tests/constexpr/coverage-ce_array.out bytes=12197912

    + ./tests/constexpr/coverage_coverage-ce_random.txt bytes=296

    + ./tests/constexpr/coverage_coverage-ce_string.txt bytes=345904

    + ./tests/constexpr/coverage-ce_string.out bytes=12298464

    + ./tests/hardware/coverage-avida_gp.out bytes=14236616

    + ./tests/hardware/coverage-event_driven_gp.out bytes=30204664

    + ./tests/hardware/coverage_coverage-avida_gp.txt bytes=440422

    + ./tests/hardware/coverage_coverage-event_driven_gp.txt bytes=2559147

    + ./tests/data/coverage-DataInterface.out bytes=13169216

    + ./tests/data/coverage-DataFile.out bytes=13639984

    + ./tests/data/coverage-DataManager.out bytes=13100640

    + ./tests/data/coverage_coverage-DataNode.txt bytes=521223

    + ./tests/data/coverage_coverage-DataInterface.txt bytes=455519

    + ./tests/data/coverage_coverage-DataFile.txt bytes=481568

    + ./tests/data/coverage-DataNode.out bytes=13939592

    + ./tests/data/coverage_coverage-DataManager.txt bytes=459610

    + ./tests/games/coverage-Mancala.out bytes=12671520

    + ./tests/games/coverage_coverage-Othello8.txt bytes=315797

    + ./tests/games/coverage-Othello8.out bytes=12678112

    + ./tests/games/coverage-Othello.out bytes=13043848

    + ./tests/games/coverage_coverage-Othello.txt bytes=301861

    + ./tests/games/coverage_coverage-PayoffMatrix.txt bytes=210244

    + ./tests/games/coverage-PayoffMatrix.out bytes=12679136

    + ./tests/games/coverage_coverage-Mancala.txt bytes=275869

    + ./tests/geometry/coverage-Point2D.out bytes=12291592

    + ./tests/geometry/coverage_coverage-Body2D.txt bytes=418193

    + ./tests/geometry/coverage_coverage-Circle2D.txt bytes=97978

    + ./tests/geometry/coverage-Body2D.out bytes=13138984

    + ./tests/geometry/coverage-Circle2D.out bytes=12243056

    + ./tests/geometry/coverage_coverage-Point2D.txt bytes=94258

    + ./tests/meta/coverage-ConceptWrapper.out bytes=12214008

    + ./tests/meta/coverage_coverage-reflection.txt bytes=249209

    + ./tests/meta/coverage_coverage-ConceptWrapper.txt bytes=24386

    + ./tests/meta/coverage_coverage-ValPack.txt bytes=16358

    + ./tests/meta/coverage-reflection.out bytes=12700848

    + ./tests/meta/coverage-TypePack.out bytes=13314248

    + ./tests/meta/coverage-ValPack.out bytes=12195760

    + ./tests/meta/coverage-TypeID.out bytes=12830360

    + ./tests/meta/coverage_coverage-StringType.txt bytes=215237

    + ./tests/meta/coverage_coverage-TypePack.txt bytes=809329

    + ./tests/meta/coverage_coverage-meta.txt bytes=263696

    + ./tests/meta/coverage-type_traits.out bytes=12213112

    + ./tests/meta/coverage_coverage-TypeID.txt bytes=249318

    + ./tests/meta/coverage-meta.out bytes=12782824

    + ./tests/meta/coverage-StringType.out bytes=12673088

    + ./tests/meta/coverage_coverage-type_traits.txt bytes=92712

    + ./tests/tools/coverage-map_utils.out bytes=12605520

    + ./tests/tools/coverage_coverage-info_theory.txt bytes=120612

    + ./tests/tools/coverage_coverage-random_utils.txt bytes=282250

    + ./tests/tools/coverage_coverage-BitMatrix.txt bytes=455057

    + ./tests/tools/coverage-BitVector.out bytes=12887584

    + ./tests/tools/coverage-tuple_utils.out bytes=12380464

    + ./tests/tools/coverage-File.out bytes=13173664

    + ./tests/tools/coverage-set_utils.out bytes=12627344

    + ./tests/tools/coverage_coverage-map_utils.txt bytes=6769

    + ./tests/tools/coverage-math.out bytes=13026744

    + ./tests/tools/coverage_coverage-combos.txt bytes=47342

    + ./tests/tools/coverage_coverage-Range.txt bytes=48274

    + ./tests/tools/coverage_coverage-NFA.txt bytes=378561

    + ./tests/tools/coverage-MatchBin_logging.out bytes=20351536

    + ./tests/tools/coverage_coverage-lexer_utils.txt bytes=652627

    + ./tests/tools/coverage_coverage-Cache.txt bytes=5349

    + ./tests/tools/coverage-MatchBin.out bytes=26553416

    + ./tests/tools/coverage-graph_utils.out bytes=13172200

    + ./tests/tools/coverage_coverage-functions.txt bytes=118587

    + ./tests/tools/coverage-RegEx.out bytes=14390760

    + ./tests/tools/coverage-QueueCache.out bytes=13091912

    + ./tests/tools/coverage-info_theory.out bytes=12352168

    + ./tests/tools/coverage_coverage-SolveState.txt bytes=267015

    + ./tests/tools/coverage-mem_track.out bytes=12926504

    + ./tests/tools/coverage_coverage-FunctionSet.txt bytes=58010

    + ./tests/tools/coverage_coverage-MatchBin.txt bytes=2595985

    + ./tests/tools/coverage_coverage-bitset_utils.txt bytes=124357

    + ./tests/tools/coverage_coverage-flex_function.txt bytes=28572

    + ./tests/tools/coverage-random_utils.out bytes=13115944

    + ./tests/tools/coverage_coverage-stats.txt bytes=270172

    + ./tests/tools/coverage-stats.out bytes=12912808

    + ./tests/tools/coverage-NullStream.out bytes=12209928

    + ./tests/tools/coverage-Cache.out bytes=12433600

    + ./tests/tools/coverage-Graph.out bytes=13106816

    + ./tests/tools/coverage-Lexer.out bytes=14754912

    + ./tests/tools/coverage_coverage-Distribution.txt bytes=88644

    + ./tests/tools/coverage-DFA.out bytes=12912184

    + ./tests/tools/coverage_coverage-string_utils.txt bytes=329708

    + ./tests/tools/coverage-vector_utils.out bytes=12855288

    + ./tests/tools/coverage-sequence_utils.out bytes=12440632

    + ./tests/tools/coverage-SolveState.out bytes=12813616

    + ./tests/tools/coverage-flex_function.out bytes=12363344

    + ./tests/tools/coverage-functions.out bytes=12287704

    + ./tests/tools/coverage_coverage-Random.txt bytes=1621793

    + ./tests/tools/coverage_coverage-BitVector.txt bytes=261994

    + ./tests/tools/coverage_coverage-ContiguousStream.txt bytes=80606

    + ./tests/tools/coverage-alert.out bytes=12680592

    + ./tests/tools/coverage_coverage-Lexer.txt bytes=705355

    + ./tests/tools/coverage-DynamicString.out bytes=12501072

    + ./tests/tools/coverage-Random.out bytes=14321880

    + ./tests/tools/coverage_coverage-keyname_utils.txt bytes=284279

    + ./tests/tools/coverage-hash_utils.out bytes=12459056

    + ./tests/tools/coverage-BitSet.out bytes=34170440

    + ./tests/tools/coverage_coverage-graph_utils.txt bytes=319298

    + ./tests/tools/coverage-keyname_utils.out bytes=13758064

    + ./tests/tools/coverage_coverage-math.txt bytes=414856

    + ./tests/tools/coverage_coverage-RegEx.txt bytes=665438

    + ./tests/tools/coverage-Distribution.out bytes=12391480

    + ./tests/tools/coverage-NFA.out bytes=12805872

    + ./tests/tools/coverage_coverage-QueueCache.txt bytes=83142

    + ./tests/tools/coverage_coverage-IndexMap.txt bytes=58543

    + ./tests/tools/coverage_coverage-tuple_utils.txt bytes=65150

    + ./tests/tools/coverage_coverage-hash_utils.txt bytes=57598

    + ./tests/tools/coverage_coverage-sequence_utils.txt bytes=164226

    + ./tests/tools/coverage_coverage-matchbin_utils.txt bytes=1830367

    + ./tests/tools/coverage-matchbin_utils.out bytes=19327968

    + ./tests/tools/coverage-ContiguousStream.out bytes=12415656

    + ./tests/tools/coverage-BitMatrix.out bytes=12680592

    + ./tests/tools/coverage_coverage-MatchBin_logging.txt bytes=1989950

    + ./tests/tools/coverage-TypeTracker.out bytes=13774424

    + ./tests/tools/coverage-bitset_utils.out bytes=12244936

    + ./tests/tools/coverage-Range.out bytes=12329456

    + ./tests/tools/coverage_coverage-File.txt bytes=301302

    + ./tests/tools/coverage-string_utils.out bytes=13875952

    + ./tests/tools/coverage-UnorderedIndexMap.out bytes=12454720

    + ./tests/tools/coverage-FunctionSet.out bytes=12650320

    + ./tests/tools/coverage_coverage-BitSet.txt bytes=11554747

    + ./tests/tools/coverage_coverage-genericfunction.txt bytes=21981

    + ./tests/tools/coverage-lexer_utils.out bytes=14474728

    + ./tests/tools/coverage_coverage-Graph.txt bytes=295456

    + ./tests/tools/coverage_coverage-alert.txt bytes=270755

    + ./tests/tools/coverage_coverage-mem_track.txt bytes=303702

    + ./tests/tools/coverage-memo_function.out bytes=13628728

    + ./tests/tools/coverage-combos.out bytes=12326120

    + ./tests/tools/coverage-IndexMap.out bytes=12497456

    + ./tests/tools/coverage_coverage-DFA.txt bytes=297547

    + ./tests/tools/coverage_coverage-TypeTracker.txt bytes=496287

    + ./tests/tools/coverage_coverage-NullStream.txt bytes=1985

    + ./tests/tools/coverage_coverage-UnorderedIndexMap.txt bytes=52965

    + ./tests/tools/coverage-genericfunction.out bytes=12267032

    + ./tests/tools/coverage_coverage-memo_function.txt bytes=86096

    + ./tests/tools/coverage_coverage-DynamicString.txt bytes=44313

    + ./tests/tools/coverage_coverage-set_utils.txt bytes=53920

    + ./tests/tools/coverage_coverage-vector_utils.txt bytes=86066

    + ./tests/scholar/coverage_coverage-Bibliography.txt bytes=317

    + ./tests/scholar/coverage-Citation.out bytes=12197912

    + ./tests/scholar/coverage_coverage-Author.txt bytes=664218

    + ./tests/scholar/coverage_coverage-Citation.txt bytes=309

    + ./tests/scholar/coverage-Author.out bytes=13965096

    + ./tests/scholar/coverage-Bibliography.out bytes=12208472

    + ./tests/config/coverage_coverage-SettingCombos.txt bytes=356240

    + ./tests/config/coverage-config_utils.out bytes=12707664

    + ./tests/config/coverage_coverage-ConfigManager.txt bytes=279813

    + ./tests/config/coverage-ArgManager.out bytes=16153632

    + ./tests/config/coverage-ConfigManager.out bytes=12692328

    + ./tests/config/coverage-SettingCombos.out bytes=12702152

    + ./tests/config/coverage-command_line.out bytes=12725904

    + ./tests/config/coverage_coverage-command_line.txt bytes=273854

    + ./tests/config/coverage_coverage-ArgManager.txt bytes=715822

    + ./tests/config/coverage-config.out bytes=14352896

    + ./tests/config/coverage_coverage-config.txt bytes=682311

    + ./tests/config/coverage_coverage-config_utils.txt bytes=446989

    + ./tests/Evolve/coverage_coverage-World_reflect.txt bytes=962763

    + ./tests/Evolve/coverage_coverage-World.txt bytes=1095029

    + ./tests/Evolve/coverage_coverage-World_select.txt bytes=968470

    + ./tests/Evolve/coverage-World_reflect.out bytes=16118896

    + ./tests/Evolve/coverage_coverage-NK.txt bytes=309838

    + ./tests/Evolve/coverage-NK.out bytes=13139048

    + ./tests/Evolve/coverage-World_select.out bytes=16004512

    + ./tests/Evolve/coverage-World_structure.out bytes=16263064

    + ./tests/Evolve/coverage_coverage-Resource.txt bytes=1040874

    + ./tests/Evolve/coverage_coverage-World_structure.txt bytes=948554

    + ./tests/Evolve/coverage_coverage-Systematics.txt bytes=1642955

    + ./tests/Evolve/coverage-World.out bytes=18966616

    + ./tests/Evolve/coverage-Systematics.out bytes=27939952

    + ./tests/Evolve/coverage-Resource.out bytes=15891440

==> Appending adjustments

    https://docs.codecov.io/docs/fixing-reports

    + Found adjustments

==> Gzipping contents

==> Uploading reports

    url: https://codecov.io

    query: branch=d3-wrapper&commit=88e0cd93a2d1bfae048a60b7f26735650c475789&build=2852.14&build_url=&name=&tag=&slug=devosoft%2FEmpirical&service=travis&flags=&pr=false&job=711278400&cmd_args=

->  Pinging Codecov

https://codecov.io/upload/v4?package=bash-20200707-353aa93&token=secret&branch=d3-wrapper&commit=88e0cd93a2d1bfae048a60b7f26735650c475789&build=2852.14&build_url=&name=&tag=&slug=devosoft%2FEmpirical&service=travis&flags=&pr=false&job=711278400&cmd_args=

->  Uploading to

https://storage.googleapis.com/codecov/v4/raw/2020-07-24/D1707B2FE71EBA691927B15F32B99003/88e0cd93a2d1bfae048a60b7f26735650c475789/851ce9ca-4573-4310-86fc-7d2ceee503fa.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EQX6OZVJGHKK3633AAFGLBUCOOATRACRQRQF6HMSMLYUP6EAD6XSWAAY%2F20200724%2FUS%2Fs3%2Faws4_request&X-Amz-Date=20200724T003224Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=e61db0ef1fd06e1fba2de36b6479008d2f954ba698972d8ab52cfff624b15cf8

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

100  236M    0     0  100  236M      0   106M  0:00:02  0:00:02 --:--:--  106M

    -> View reports at https://codecov.io/github/devosoft/Empirical/commit/88e0cd93a2d1bfae048a60b7f26735650c475789

Done. Your build exited with 0.

Additional Information

Any additional information, configuration or data that might be necessary to reproduce the issue.

I think I figured it out! We recently switched to naming the executables containing the compiled test code with the pattern coverage-[test_name].out, which matched the pattern that the uploader was looking for. I think whatever program interprets the coverage reports was very confused by receiving a file full of machine code and failed.

Oh that makes a lot fo sense. Has this been fixed for you @seaotternerd?

Yes! We renamed the executables and that fixed it. Thanks!

FYI, I also tried to fix it by setting an ignore pattern in my YAML and that didn’t seem to work (despite it definitely finding the correct YAML and the YAML passing validation).

1 Like