CodeCov uploads from Azure Pipeline CI runs on the GitHub ActiveDirectoryDsc repo have been failing for the last two weeks with the error ‘Build numbers do not match. Please upload with the Codecov repository upload token to resolve issue.’. They were working previously.
Hi @X-Guardian
Let me check over at the bash uploader repo GitHub - codecov/codecov-bash: Global coverage report uploader for Codecov and see if we recently caused something that could cause this.
Yes, afraid so. This one is from a new PR: https://dev.azure.com/dsccommunity/ActiveDirectoryDsc/_build/results?buildId=1229&view=logs&jobId=f83e3f09-a299-5b31-26ad-d8f96216e5fb&j=f83e3f09-a299-5b31-26ad-d8f96216e5fb&t=a4cfa163-c668-5abb-1e34-ccd4fe91a617
This is strange. When I call the API for that build: https://dev.azure.com/dsccommunity/ActiveDirectoryDsc/_apis/build/builds/1229?api-version=5.0
It does indeed return a different SHA:
sourceVersion "467ebb76f54fc5fd2a5c4156d2878abf35612b77"
Any idea why that may be? It looks like the SHA you are passing us is 28934a337b6ac8b60ba101deec6e3278f41f5c7e
We are not explicitly specifying the SHA. We just call the codecov bash script like so:
bash <(curl -s https://codecov.io/bash) -f "./$(buildArtifactName)/$(codecovTestArtifactName)/JaCoCo_coverage.xml"
Right, sorry for being unclear. 28934a337b6ac8b60ba101deec6e3278f41f5c7e is the SHA being picked up by the uploader from env variables.
467ebb76f54fc5fd2a5c4156d2878abf35612b77 is the SHA the Azure API says it is. Which is correct and can you tell me the relationship between the two SHAs?
OK,
28934a3 is the commit from the remote forked branch that is the source of the PR: ADGroup: Add MSA Member Support · dsccommunity/ActiveDirectoryDsc@28934a3 · GitHub.
I’m not sure what 467ebb76 is: Merge 28934a337b6ac8b60ba101deec6e3278f41f5c7e into 811eef557ad5841d6… · dsccommunity/ActiveDirectoryDsc@467ebb7 · GitHub. Description is “Merge 28934a3 into 811eef5”. 811eef5 is the latest commit in the master branch.
Both of these SHA’s appear in the response from https://dev.azure.com/dsccommunity/ActiveDirectoryDsc/_apis/build/builds/1229?api-version=5.0.
28934a3 is the “triggerInfo.pr.sourceSha”
467ebb76 is the “sourceVersion”
Ok.
sourceVersion is what we check for tokenless.
commit="$BUILD_SOURCEVERSION" is what the bash uploader grabs for the commit.
@ibrahim0814, any thoughts here?
We are encountering the same issue. In case it helps, here is some info that may help debug the issue.
PR arviz#1123 tries to merge commit 5ec1c2c3802382d4be2847053ebb1e57ef0c76cb into master (at time of writing and running CI at commit e5208d14ec89a1970b0d3be9f55e1147d97fd1e4). It looks like Azure creates an extra commit from merging 5ec... into e52...: commit 051f6d88e4d980c72cef76b0e1e6cde1758b4d7f.
The commit message from 051... is Merge 5ec1c2c3802382d4be2847053ebb1e57ef0c76cb into e5208d14ec89a1970b0d3be9f55e1147d97fd1e4. All environment variables for the Azure build are shown here.
SYSTEM_PULLREQUEST_SOURCECOMMITID and “triggerInfo.pr.sourceSha” are 5ec...
BUILD_SOURCEVERSION is 051...
We were able to fix this by manually defining the sha
bash <(curl -s https://codecov.io/bash) -n "$(NAME)" -C $BUILD_SOURCEVERSION
I think something wrong happens here (in codecov.io/bash)
if [ "$commit_o" = "" ];
then
# merge commit -> actual commit
mc=
if [ -n "$pr" ] && [ "$pr" != false ];
then
mc=$(git show --no-patch --format="%P" 2>/dev/null || echo "")
fi
if [[ "$mc" =~ ^[a-z0-9]{40}[[:space:]][a-z0-9]{40}$ ]];
then
say " Fixing merge commit SHA"
commit=$(echo "$mc" | cut -d' ' -f2)
elif [ "$GIT_COMMIT" != "" ];
then
commit="$GIT_COMMIT"
elif [ "$commit" = "" ];
then
commit=$(git log -1 --format="%H" 2>/dev/null || hg id -i --debug 2>/dev/null | tr -d '+' || echo "")
fi
else
commit="$commit_o"
fi
Here -C skips this if by using commit="$commit_o"
Using -C does not work for the scenario in the DSC Community repos (we have this issue over all repos in the org that we are enable this for). This DID work before. So something has change somewhere. 
NOTE: This example below is from a branch, not a PR. So the error occurs for branch commits too. Testing on a fork of xFailOverCluster repo.
Using argument -C is the same as the script already does. At line 790 it does
commit="$BUILD_SOURCEVERSION"
Which when the block that is mentioned above it hits which is skipped since commit was set at line 790.
elif [ "$commit" = "" ];
Running the bash script with debug messages (a copy with added say lines) I see that both returns the same commit at the end. First run is without -C, and the second run is with -C.
==> END merge commit -> actual commit. COMMIT: 616ceb491c568b110b5b9f78af9f33fa61c734d3
And from the API the source version is
https://dev.azure.com/viscalyx/xFailOverCluster/_apis/build/builds/524?api-version=5.0
"sourceBranch": "refs/heads/debug-codecov",
"sourceVersion": "616ceb491c568b110b5b9f78af9f33fa61c734d3",
@drazisil what does the error message “Build numbers do not match.” mean? It can’t have to do with the commits. Is something else going on? 
I am also hitting this issue, I had just created an issue when I found this thread
PM from Azure Pipelines here. I’m suspicious of the timing and code change in this PR: [CE-1318] Update to system format for Azure by drazisil · Pull Request #256 · codecov/codecov-bash · GitHub
Previously, $pr would always have been empty, because we never had environment variables called $PULL_REQUEST_NUMBER or $PULL_REQUEST_ID. I bet that correcting that assignment to $SYSTEM_PULLREQUEST_PULLREQUEST* broke the logic further down the page.
We’re also investigating on our side, but we believe we haven’t changed the contents/semantics of any of the variables the Bash uploader looks for.
Hi everyone, thanks for reaching out here. I’m not exactly sure what happened, but I’m no longer seeing this issue on the repositories above. I would guess that maybe there was an issue with the bash uploader that has since been resolved. Please let me know if this issue has not been resolved for you.
Also thanks @vtbassmatt for reaching out here, we always appreciate the help.
This issue still exist Azure DevOps Services | Sign In
Since there was no response on this issue we had to start workaround it by adding the upload token to a few repos that really needed this. We rather see a correct fix to this issue so we can remove the upload token variable again.
Did you look into the suggestion by @vtbassmatt?
@tom can you give us what you are seeing on your end? What value is Codecov API expecting in the example PR I linked to above? That would help narrowing down if the wrong value is sent or if your end is expecting the wrong value.
Yes, I think I misspoke last night. Thanks @johlju for the logs, this is extremely helpful. I need to add some logging on our side and play around to see why this is happening.