Test System.Net.Sockets.Tests.*.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: True) failed in CI. #1479

Sunny-pu · 2018-05-23T03:16:05Z

Edit by @antonfirsov: recent failures seem to happen on Windows only, see #1479 (comment).

Test: System.Net.Sockets.Tests.SendReceiveMemoryNativeTask.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: True) has failed.

MESSAGE:
Assert.Equal() Failure\nExpected: 123456\nActual: 123455
+++++++++++++++++++
STACK TRACE:
at System.Net.Sockets.Tests.SendReceive`1.SendRecvPollSync_TcpListener_Socket(IPAddress listenAt, Boolean pollBeforeOperation) in /mnt/j/workspace/dotnet_corefx/master/outerloop_netcoreapp_ubuntu16.04_debug/src/System.Net.Sockets/tests/FunctionalTests/SendReceive.cs:line 630 --- End of stack trace from previous location where exception was thrown ---

Details:
https://ci.dot.net/job/dotnet_corefx/job/master/job/outerloop_netcoreapp_ubuntu16.04_debug/403/testReport/System.Net.Sockets.Tests/SendReceiveMemoryNativeTask/SendRecvPollSync_TcpListener_Socket_listenAt____1__pollBeforeOperation__True_/

karelz · 2018-05-23T15:50:44Z

@rmkerr can you please check how common is the failure?

rmkerr · 2018-05-23T16:23:03Z

@karelz it's failed five times in the last two weeks. That isn't including the recent Nano failures, or a catastrophic failure that briefly caused all tests to fail on RHEL 6.9.

With the failure today it's jumped up to the second most common networking failure over the last two weeks, after dotnet/corefx#29742.

karelz · 2018-05-23T16:24:47Z

Sounds like worth deeper investigation, especially if it started failing this often recently-ish.

wfurt · 2019-04-25T19:34:02Z

This test is still failing on Windows10RS4 and Nano.

https://mc.dot.net/#/product/netcore/30/source/official~2Fdotnet~2Fcorefx~2Frefs~2Fheads~2Fmaster/type/test~2Ffunctional~2Fcli~2F/build/20190423.6/workItem/System.Net.Sockets.Tests/wilogs

    System.Net.Sockets.Tests.SendReceiveSync.SendRecvPollSync_TcpListener_Socket(listenAt: 127.0.0.1, pollBeforeOperation: False) [FAIL]
      Read poll after completion should have succeeded
      Expected: True
      Actual:   False
      Stack Trace:
        F:\workspace\_work\1\s\src\System.Net.Sockets\tests\FunctionalTests\SendReceive.cs(598,0): at System.Net.Sockets.Tests.SendReceive`1.<>c__DisplayClass8_1.<<SendRecvPollSync_TcpListener_Socket>b__0>d.MoveNext()
        --- End of stack trace from previous location where exception was thrown ---
        F:\workspace\_work\1\s\src\Common\tests\System\Threading\Tasks\TaskTimeoutExtensions.cs(83,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        F:\workspace\_work\1\s\src\Common\tests\System\Threading\Tasks\TaskTimeoutExtensions.cs(111,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        F:\workspace\_work\1\s\src\Common\tests\System\Threading\Tasks\TaskTimeoutExtensions.cs(71,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
        F:\workspace\_work\1\s\src\System.Net.Sockets\tests\FunctionalTests\SendReceive.cs(645,0): at System.Net.Sockets.Tests.SendReceive`1.SendRecvPollSync_TcpListener_Socket(IPAddress listenAt, Boolean pollBeforeOperation)
        --- End of stack trace from previous location where exception was thrown ---
      Output:
        4/23/2019 5:08:32 PM Starting listener at 127.0.0.1:53456

Documentation onn Poll() says:

true if Listen(Int32) has been called and a connection is pending; -or- true if data is available for reading; -or- true if the connection has been closed, reset, or terminated; otherwise, returns false.

Do you have any recommendation @davidsh what additional instrumentation should we add?

davidsh · 2019-04-25T19:45:41Z

All the tests in this file are historically flaky. You'll notice that many of them are Outerloop and yet don't contact external servers. That's because they were so unstable at the time that we moved them to Outerloop.

poll() operations of sockets in Windows especially loopback can be problematic because of interesting optimizations for fastpath I/O that Windows does in the loopback adapter.

Unless we want to spend a lot of time making these tests more stable, I would disable this particular test with an ActiveIssue for now.

v-haren · 2020-09-10T05:52:56Z

failed again in job: runtime-libraries-coreclr outerloop 20200909.3

failed test: System.Net.Sockets.Tests.SendReceiveApm.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: False)

net5.0-Windows_NT-Release-x64-CoreCLR_release-Windows.81.Amd64.Open

Error message

Read poll after completion should have succeeded
Expected: True
Actual:   False


Stack trace
   at System.Net.Sockets.Tests.SendReceive`1.<>c__DisplayClass10_0.<<SendRecvPollSync_TcpListener_Socket>b__0>d.MoveNext() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive.cs:line 605
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 96
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 124
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 84
   at System.Net.Sockets.Tests.SendReceive`1.SendRecvPollSync_TcpListener_Socket(IPAddress listenAt, Boolean pollBeforeOperation) in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive.cs:line 645
--- End of stack trace from previous location ---

v-haren · 2020-12-24T01:41:17Z

failed again in job: runtime-libraries-coreclr outerloop 20201223.2

failed test: System.Net.Sockets.Tests.SendReceive_Sync.SendRecvPollSync_TcpListener_Socket(listenAt: 127.0.0.1, pollBeforeOperation: False)

net6.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504

Error message

Read poll after completion should have succeeded
Expected: True
Actual:   False


Stack trace
   at System.Net.Sockets.Tests.SendReceive`1.<>c__DisplayClass7_0.<<SendRecvPollSync_TcpListener_Socket>b__0>d.MoveNext() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive/SendReceive.cs:line 521
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 96
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 124
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 90
   at System.Net.Sockets.Tests.SendReceive`1.SendRecvPollSync_TcpListener_Socket(IPAddress listenAt, Boolean pollBeforeOperation) in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive/SendReceive.cs:line 575
--- End of stack trace from previous location ---

antonfirsov · 2021-01-27T18:56:05Z

Removed os-linux label, since all recent failures seem to happen on Windows according to Kusto.

antonfirsov · 2021-03-10T10:50:54Z

Recent failures are happening in sync variants.

VincentBu · 2021-04-02T06:22:38Z

Similar test failed in runtime-libraries-coreclr outerloop 20210401.4

Failed test:

net6.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open
 -System.Net.Sockets.Tests.SendReceive_SpanSyncForceNonBlocking.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: False)

Error message:

Read poll after completion should have succeeded
Expected: True
Actual:   False


Stack trace
   at System.Net.Sockets.Tests.SendReceive`1.<>c__DisplayClass7_0.<<SendRecvPollSync_TcpListener_Socket>b__0>d.MoveNext() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive/SendReceive.cs:line 521
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 63
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 82
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 49
   at System.Net.Sockets.Tests.SendReceive`1.SendRecvPollSync_TcpListener_Socket(IPAddress listenAt, Boolean pollBeforeOperation) in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive/SendReceive.cs:line 575
--- End of stack trace from previous location ---

VincentBu · 2021-04-12T06:40:23Z

Failed again in runtime-libraries-coreclr outerloop 20210411.4

Faield test:

net6.0-windows-Release-x64-CoreCLR_release-Windows.81.Amd64.Open
 -System.Net.Sockets.Tests.SendReceive_SpanSync.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: False)

Error message:

Read poll after completion should have succeeded
Expected: True
Actual:   False


Stack trace
   at System.Net.Sockets.Tests.SendReceive`1.<>c__DisplayClass7_0.<<SendRecvPollSync_TcpListener_Socket>b__0>d.MoveNext() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive/SendReceive.cs:line 521
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 63
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 82
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 49
   at System.Net.Sockets.Tests.SendReceive`1.SendRecvPollSync_TcpListener_Socket(IPAddress listenAt, Boolean pollBeforeOperation) in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/SendReceive/SendReceive.cs:line 575
--- End of stack trace from previous location ---

MihaZupan · 2021-04-14T15:58:02Z

Disabled for Windows 8 in #51186

Updating my ZeroSharp samples and this is getting in the way. The fix we did in dotnet/runtimelab#743 was insufficient (I want things to be debuggable in ZeroSharp).

wfurt · 2022-11-30T17:58:02Z

triage: we should re-enable the test and fix it if we see failures.

karelz · 2023-06-09T15:05:14Z

Fixed in #79145

karelz transferred this issue from dotnet/corefx Jan 9, 2020

Dotnet-GitSync-Bot added area-System.Net.Sockets untriaged New issue has not been triaged by the area owner labels Jan 9, 2020

karelz added os-linux Linux OS (any supported distro) test-run-core Test failures in .NET Core test runs and removed untriaged New issue has not been triaged by the area owner labels Jan 9, 2020

karelz added this to the 5.0 milestone Jan 9, 2020

karelz modified the milestones: 5.0, Future Jun 5, 2020

antonfirsov removed the os-linux Linux OS (any supported distro) label Jan 27, 2021

antonfirsov mentioned this issue Jan 27, 2021

Refactor Socket.SendFile tests and extend coverage #47479

Merged

antonfirsov mentioned this issue Mar 10, 2021

WinHttpHandler: Read HTTP/2 trailing headers (replacement PR) #48704

Merged

antonfirsov mentioned this issue Apr 6, 2021

Test failure System.Net.Sockets.Tests.SendReceive_Sync.SendRecvPollSync_TcpListener_Socket #50769

Closed

MihaZupan mentioned this issue Apr 13, 2021

Disable SendRecvPollSync_TcpListener_Socket on Windows 8 #51186

Merged

ghost added in-pr There is an active PR which will close this issue when it is merged and removed in-pr There is an active PR which will close this issue when it is merged labels Apr 13, 2021

MihaZupan added the disabled-test The test is disabled in source code against the issue label Apr 14, 2021

wfurt modified the milestones: Future, 8.0.0 Nov 30, 2022

wfurt assigned liveans Nov 30, 2022

liveans mentioned this issue Dec 2, 2022

Re-Enable SendRecvPollSync_TcpListener_Socket Test #79145

Merged

karelz closed this as completed Jun 9, 2023

ghost locked as resolved and limited conversation to collaborators Jul 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test System.Net.Sockets.Tests.*.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: True) failed in CI. #1479

Test System.Net.Sockets.Tests.*.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: True) failed in CI. #1479

Sunny-pu commented May 23, 2018 •

edited by antonfirsov

Loading

karelz commented May 23, 2018

rmkerr commented May 23, 2018

karelz commented May 23, 2018

wfurt commented Apr 25, 2019

davidsh commented Apr 25, 2019

v-haren commented Sep 10, 2020

v-haren commented Dec 24, 2020

antonfirsov commented Jan 27, 2021

antonfirsov commented Mar 10, 2021

VincentBu commented Apr 2, 2021

VincentBu commented Apr 12, 2021

MihaZupan commented Apr 14, 2021

wfurt commented Nov 30, 2022

karelz commented Jun 9, 2023

Test System.Net.Sockets.Tests.*.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: True) failed in CI. #1479

Test System.Net.Sockets.Tests.*.SendRecvPollSync_TcpListener_Socket(listenAt: ::1, pollBeforeOperation: True) failed in CI. #1479

Comments

Sunny-pu commented May 23, 2018 • edited by antonfirsov Loading

karelz commented May 23, 2018

rmkerr commented May 23, 2018

karelz commented May 23, 2018

wfurt commented Apr 25, 2019

davidsh commented Apr 25, 2019

v-haren commented Sep 10, 2020

v-haren commented Dec 24, 2020

antonfirsov commented Jan 27, 2021

antonfirsov commented Mar 10, 2021

VincentBu commented Apr 2, 2021

VincentBu commented Apr 12, 2021

MihaZupan commented Apr 14, 2021

wfurt commented Nov 30, 2022

karelz commented Jun 9, 2023

Sunny-pu commented May 23, 2018 •

edited by antonfirsov

Loading