Build should not fail if there is a merge to dotnet/versions

[riarenas]

Currently, the official production build fails to check out the dotnet/versions repository if any commit happens to the repo while a build is ongoing.

See https://dev.azure.com/dnceng/internal/_build/results?buildId=2108561&view=logs&j=e27bb03b-8ee4-51d2-159a-afc23d6ec395&t=711edc8a-9bea-5e54-c831-e6c141e593fc&l=405 for an example of how this manifests:

From https://github.com/dotnet/versions
 * [new branch]      MattGal-patch-1         -> origin/MattGal-patch-1
 * [new branch]      add-prereqs-docker-prod -> origin/add-prereqs-docker-prod
 * [new branch]      dependabot/nuget/Maestro/src/Microsoft.DotNet.Maestro.WebApi/Newtonsoft.Json-13.0.1 -> origin/dependabot/nuget/Maestro/src/Microsoft.DotNet.Maestro.WebApi/Newtonsoft.Json-13.0.1
 * [new branch]      dependabot/nuget/Maestro/src/Microsoft.DotNet.Maestro.WebJob/Newtonsoft.Json-13.0.1 -> origin/dependabot/nuget/Maestro/src/Microsoft.DotNet.Maestro.WebJob/Newtonsoft.Json-13.0.1
 * [new branch]      dependabot/nuget/Maestro/src/Microsoft.DotNet.Maestro/Newtonsoft.Json-13.0.1 -> origin/dependabot/nuget/Maestro/src/Microsoft.DotNet.Maestro/Newtonsoft.Json-13.0.1
 * [new branch]      main                    -> origin/main
 * [new branch]      mmitche-patch-1         -> origin/mmitche-patch-1
 * [new branch]      mmitche-patch-2         -> origin/mmitche-patch-2
 * [new branch]      mmitche-patch-6         -> origin/mmitche-patch-6
 * [new branch]      revert-814-add-ubuntu-mirror-test -> origin/revert-814-add-ubuntu-mirror-test
 * [new tag]         v1.0.3                  -> v1.0.3
 * [new tag]         v1.1.0                  -> v1.1.0
git checkout --progress --force a9b450269d3f9c1f28af30086022980efa1d8753
fatal: reference is not a tree: a9b450269d3f9c1f28af30086022980efa1d8753
##[warning]Git checkout failed on shallow repository, this might because of git fetch with depth '1' doesn't include the checkout commit 'a9b450269d3f9c1f28af30086022980efa1d8753'. Please reference documentation (http://go.microsoft.com/fwlink/?LinkId=829603)
##[error]Git checkout failed with exit code: 128

This happened because dotnet/versions@a9b4502 was the tip of the main branch when the build was queued, but the dotnet-docker-bot merged dotnet/versions@abff130 while the build was running. Causing the commit to no longer be available with a depth 1 fetch of the repo.

Once this failure mode is introduced, a whole new build needs to be started to recover from the failed state.

[ChadNedzlek]

Yeah, I think, realistically, we need to ditch the "depth 1" here. Anything else is just a huge headache for a short term problem.

[mthalman]

What I don't understand is that this is the only repo that has this issue, and only for the production branch, and only for the arm machines. Seems very strange.

[riarenas]

Another hit today: https://dnceng.visualstudio.com/internal/_build/results?buildId=2114784&view=results, caused by dotnet/versions@b25461d

[mthalman]

There is a note about the shallow fetch behavior and how it can lead to this issue at https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml#shallow-fetch

[ChadNedzlek]

Where is it configured? I've been searching the repo and can't find it... is it in the pipeline configuration? We should be able to set it to something like 50 really easily.

[mthalman]

It's in the AzDO configuration (internal link). I've set it to 20 which should be plenty. Hopefully that takes care of it.