Microsoft.NET.Build.Containers.IntegrationTests are pulling from Docker Hub

[akoeplinger]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=733130
Build error leg or test failing: Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.WriteToPrivateBasicRegistry
Pull request: https://github.com/dotnet/sdk.git/pull/42019

We need to stop pulling from upstream Docker Hub since we're hitting rate limits.

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "You have reached your pull rate limit",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=733130
Error message validated: [You have reached your pull rate limit]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/8/2024 7:20:41 PM UTC

Report

Build	Definition	Test	Pull Request
821983	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43787
821584	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43783
821577	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43416
821221	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.ApiEndToEndWithLocalLoad	#43772
821219	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43771
821214	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43659
820734	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43762
820575	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43762
820337	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43753
820330	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43751
820325	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43662
820020	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43730
819890	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43239
819821	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43736
819603	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43733
819129	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43719
819114	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43662
819115	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43455
819107	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43422
819079	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.ApiEndToEndWithLocalLoad	#43453
819073	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43416
819060	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43658
819071	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.CreateNewImage_Baseline	#43659
819046	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.EndToEnd_MultiProjectSolution	#43717
819027	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43713
819030	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43715
819013	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43712
818993	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.ApiEndToEndWithLocalLoad	#43085
817822	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43683
816792	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43652
816735	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43648
816519	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43614
816517	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43598
816512	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43643
816500	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43624
816422	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43571
815729	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43547
815723	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43623
814819	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43602
814816	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.EndToEnd_MultiProjectSolution	#43422
814811	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.ApiEndToEndWithLocalLoad	#43416
814791	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43598
814789	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43614
814787	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43612
814010	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43602
814006	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43601
813964	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43598
813720	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43588
813210	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43157
813184	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43581
813019	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.ApiEndToEndWithLocalLoad	#43543
813059	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43239
813014	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43456
812988	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43579
812985	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43578
812613	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43568
812015	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43510
811979	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43545
811974	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43544
811969	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43543
808685	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43455
811480	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43493
811550	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43488
810782	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.EndToEnd_MultiProjectSolution	#43503
810762	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43501
810766	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43490
810360	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43489
810300	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43484
810158	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43436
809925	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.EndToEnd_MultiProjectSolution	#43045
810064	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43478
810110	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation
810040	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43472
810082	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43479
810059	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43477
809730	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43471
808713	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43458
808737	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43460
809686	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43464
809678	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry
808664	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43453
808680	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43454
808701	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43457
808699	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43456
808639	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43452
808625	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43255
808588	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43450
2538257	dotnet-sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#42849
808353	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation
808352	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.CreateNewImage_RootlessBaseImage
808349	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43443
808326	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43438
808236	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43440
808109	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43436
807773	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43425
807771	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43424
807755	dotnet/sdk	Microsoft.NET.Build.Containers.Tasks.IntegrationTests.CreateNewImageTests.Tasks_EndToEnd_With_EnvironmentVariable_Validation	#43422
807754	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43421
807587	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.GetFromRegistry	#43413
807312	dotnet/sdk	Microsoft.NET.Build.Containers.IntegrationTests.EndToEndTests.EndToEnd_MultiProjectSolution	#43402
Displaying 100 of 128 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	36	128

[akoeplinger]

@marcpopMSFT who owns Microsoft.NET.Build.Containers?

[baronfel]

@dotnet/sdk-container-builds-maintainers

[baronfel]

@akoeplinger the image in question is registry:2, which we need/use to validate our logic for communicating with container registries via the container registry APIs. Do you know if this image is mirrored anywhere that we can pull from? It's not currently on mcr.microsoft.com.

[nagilson]

Bumping this thread since it's affecting more PRs

[nagilson]

We don't have a path forward to fix this yet - Aspire moved to using their own ACR for storing the images they use, but we need an engineering-systems-managed solution for our entire stack. There's an issue tracking this request but I can't find it at the moment.

[baronfel]

Could be fixed at a root cause level by dotnet/dnceng#3389

[MichaelSimons]

@akoeplinger the image in question is registry:2, which we need/use to validate our logic for communicating with container registries via the container registry APIs. Do you know if this image is mirrored anywhere that we can pull from? It's not currently on mcr.microsoft.com.

@baronfel - It sounds like the test has this dependency and is what is pulling the image. One possible option to workaround scenarios where tests are pulling images directly from DockerHub would be to create a wrapper buildtools image. The Dockerfile would just be a FROM statement. The buildtools images are stored on MCR and therefore you would be able to get around this rate limiting issue.