NameResolutionPalTest failing intermittently on Linux Arm64

[weshaggard]

https://mc.dot.net/#/user/weshaggard/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/fdfc526a83091617be7978de875969f48d2666bf/workItem/System.Net.NameResolution.Pal.Tests/analysis/xunit/System.Net.NameResolution.PalTests.NameResolutionPalTests~2FTryGetAddrInfo_HostName

Message :
Assert.Equal() Failure
Expected: Success
Actual:   TryAgain
Stack Trace :
   at System.Net.NameResolution.PalTests.NameResolutionPalTests.TryGetAddrInfo_HostName() in /mnt/j/workspace/dotnet_corefx/master/linux-TGroup_netcoreapp+CGroup_Release+AGroup_arm64+TestOuter_false_prtest/src/System.Net.NameResolution/tests/PalTests/NameResolutionPalTests.cs:line 54

https://mc.dot.net/#/user/weshaggard/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/fdfc526a83091617be7978de875969f48d2666bf/workItem/System.Net.NameResolution.Pal.Tests/analysis/xunit/System.Net.NameResolution.PalTests.NameResolutionPalTests~2FTryGetAddrInfo_HostName_TryGetNameInfo

Message :
Assert.Equal() Failure
Expected: Success
Actual:   TryAgain
Stack Trace :
   at System.Net.NameResolution.PalTests.NameResolutionPalTests.TryGetAddrInfo_HostName_TryGetNameInfo() in /mnt/j/workspace/dotnet_corefx/master/linux-TGroup_netcoreapp+CGroup_Release+AGroup_arm64+TestOuter_false_prtest/src/System.Net.NameResolution/tests/PalTests/NameResolutionPalTests.cs:line 112

[danmoseley]

@karelz broke CI again
https://mc.dot.net/#/user/danmosemsft/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/5651b5c2b7134c8ab662c5594b10f86db7f3cb9c

Can we please make it outer loop or disable for ARM until someone has time to robustify it?

[karelz]

@wfurt @rmkerr how often does it fail?
@wfurt is in process to enable us debug test failures ...

[rmkerr]

It doesn't appear to be failing at all in the daily runs, but it is failing regularly on PRs. It's failing ten to fifteen times a day, so we should fix it or move it to outerloop ASAP.

[wfurt]

note that OSX has similar issues. For one, when OS returns EAGAIN do not retry.
But I have seen cases when OS simply returns "host not found".

[danmoseley]

Failing PR again
https://mc.dot.net/#/user/dotnet-maestro-bot/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/22c651c43edc1bda66ff93535eef073530f6b941/workItem/System.Net.NameResolution.Pal.Tests

@rmkerr can you please "move it to outerloop ASAP." 😺

[rmkerr]

Yep, I'll do that now.

[danmoseley]

@wfurt we re-enabled these tests on ARM in dotnet/corefx#34962, unfortunately this and another failed again there.

https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F35056~2Fmerge/test~2Ffunctional~2Fcli~2F/20190204.11/workItem/System.Net.NameResolution.Pal.Tests

Ubuntu.1604.Arm64.Open-arm64-Release
Get Repro environment
Unhandled Exception of Type Xunit.Sdk.EqualException
Message :
Assert.Equal() Failure
Expected: Success
Actual:   TryAgain
Stack Trace :
   at System.Net.NameResolution.PalTests.NameResolutionPalTests.TryGetAddrInfo_HostName_TryGetNameInfo() in /__w/1/s/src/System.Net.NameResolution/tests/PalTests/NameResolutionPalTests.cs:line 132

same for TryGetAddrInfo_HostName

[wfurt]

yes, I know. I've been trying to work out with @ulisesh to make some infrastructure changes. I was hoping we can get it done quickly to avoid another disable/enable cycle.
(see notes in dotnet/corefx#34934)
If too noisy, I can disable them again and retry to enable them later.

[danmoseley]

Is it possible to make them outer loop so they don't break CI jobs? Unless you have a reason for them to be inner loop. You can still request outer loop in CI for testing, of course...

[BruceForstall]

Another failure:
https://ci.dot.net/job/dotnet_coreclr/job/master/view/arm64/job/jitstress/job/arm64_cross_checked_ubuntu16.04_corefx_jitstress1_tst/113/consoleFull

[wfurt]

Changes to container configuration was made. Last failure was at 2019-02-06 04:18:16 (~2days ago)
keeping fingers crossed.

[wfurt]

After 5 day passes it failed again today for pr/dotnet/corefx/refs/pull/34931/merge

Are we sure we updated all machines @ulisesh ?
This is Ubuntu.1604.Arm64.Open vs Ubuntu.1604.Arm64 used for official builds.

[ulisesh]

Right now, Ubuntu.1604.Arm64.Open doesn't run on containers, it runs on Centriq servers. We will move Ubuntu.1604.Arm64.Open to run on containers early next week

[wfurt]

I did not see failure for almost month after infrastructure changes. Closing for now.

[stephentoub]

I did not see failure for almost month after infrastructure changes. Closing for now.

That's because the test is disabled.

[karelz]

Seems like test issue, no need to have it in 3.0. cc @wfurt