Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.
/ corefx Public archive

Rework ExclusiveAddressUse and ReuseAddress on non-Windows platforms #11509

Merged
merged 2 commits into from
Sep 26, 2016
Merged

Rework ExclusiveAddressUse and ReuseAddress on non-Windows platforms #11509

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dca9ceb
Rework ExclusiveAddressUse and ReuseAddress on non-Windows platforms
ericeil Sep 1, 2016
8e85830
Fix error checking
Sep 23, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions src/Native/Unix/System.Native/pal_networking.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2071,6 +2071,51 @@ extern "C" Error SystemNative_GetSockOpt(

int fd = ToFileDescriptor(socket);

//
// Handle some special cases for compatibility with Windows
//
if (socketOptionLevel == PAL_SOL_SOCKET)
{
if (socketOptionName == PAL_SO_EXCLUSIVEADDRUSE)
{
//
// SO_EXCLUSIVEADDRUSE makes Windows behave like Unix platforms do WRT the SO_REUSEADDR option.
// So, for non-Windows platforms, we act as if SO_EXCLUSIVEADDRUSE is always enabled.
//
if (*optionLen != sizeof(int32_t))
{
return PAL_EINVAL;
}

*reinterpret_cast<int32_t*>(optionValue) = 1;
return PAL_SUCCESS;
}
else if (socketOptionName == PAL_SO_REUSEADDR)
{
//
// On Windows, SO_REUSEADDR allows the address *and* port to be reused. It's equivalent to
// SO_REUSEADDR + SO_REUSEPORT other systems. Se we only return "true" if both of those options are true.
//
auto optLen = static_cast<socklen_t>(*optionLen);

int err = getsockopt(fd, SOL_SOCKET, SO_REUSEADDR, optionValue, &optLen);

if (err == 0 && *reinterpret_cast<uint32_t*>(optionValue) != 0)
{
err = getsockopt(fd, SOL_SOCKET, SO_REUSEPORT, optionValue, &optLen);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As pointed out in the design discussion, this could be confusing: what if on *NIX I only want to set SO_REUSEPORT but not SO_REUSEADDR?
Is that a a valid *NIX scenario? If yes, is there a way to achieve that somehow?

Copy link
Contributor Author

@ericeil ericeil Sep 7, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a a valid *NIX scenario? If yes, is there a way to achieve that somehow?

I'm not sure if that's a valid scenario. It's not one anyone's asking for right now. 😃

The premise of this change is that we have an already-established semantic for ReuseAddress in managed code, and there's more value in this semantic being portable than having the managed socket option names map 1:1 with native socket options. I agree that breaking that 1:1 mapping may be confusing for folks who are specifically targeting Unix socket options, but keeping the 1:1 mapping is also confusing, to those who are trying to get consistent behavior across platforms.

Copy link

@christianhuening christianhuening Sep 7, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I've done a bit of research and found that SO_REUSEPORT seems to be a bit exotic yet useful in extreme load scenarios (i.e. at Google: https://lwn.net/Articles/542629/).
My guess is that no one would ask for this in the near future, since it hasn't been available in .NET Framework so far. However can it hurt to add an option to allow to possibly tweak high performance socket code in special conditions, when you know you're running on *NIX?

EDIT: Here is the original motivation for this feature (source: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=c617f398edd4db2b8567a28e899a88f8f574798d):

The motivating case for so_resuseport in TCP would be something like
a web server binding to port 80 running with multiple threads, where
each thread might have it's own listener socket. This could be done
as an alternative to other models: 1) have one listener thread which
dispatches completed connections to workers. 2) accept on a single
listener socket from multiple threads. In case #1 the listener thread
can easily become the bottleneck with high connection turn-over rate.
In case #2, the proportion of connections accepted per thread tends
to be uneven under high connection load (assuming simple event loop:
while (1) { accept(); process() }, wakeup does not promote fairness
among the sockets. We have seen the disproportion to be as high
as 3:1 ratio between thread accepting most connections and the one
accepting the fewest. With so_reusport the distribution is
uniform.

Nginx uses it as well: https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/
And Apache does so too: https://httpd.apache.org/docs/trunk/en/mod/mpm_common.html#listencoresbucketsratio

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @christianhuening for the investigation!

Maybe properly supporting this and documenting/adding a new property/method would be a better long-term plan.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However can it hurt to add an option to allow to possibly tweak high performance socket code in special conditions, when you know you're running on *NIX?

Once Socket has its public Handle property added back, this could be done via a P/Invoke that's only invoked when on Unix. That could serve as a stop-gap until such time as it was determined to be important to add .NET APIs for.

}

if (err != 0)
{
return SystemNative_ConvertErrorPlatformToPal(errno);
}

assert(optLen <= static_cast<socklen_t>(*optionLen));
*optionLen = static_cast<int32_t>(optLen);
return PAL_SUCCESS;
}
}

int optLevel, optName;
if (!TryGetPlatformSocketOption(socketOptionLevel, socketOptionName, optLevel, optName))
{
Expand Down Expand Up @@ -2099,6 +2144,47 @@ SystemNative_SetSockOpt(intptr_t socket, int32_t socketOptionLevel, int32_t sock

int fd = ToFileDescriptor(socket);

//
// Handle some special cases for compatibility with Windows
//
if (socketOptionLevel == PAL_SOL_SOCKET)
{
if (socketOptionName == PAL_SO_EXCLUSIVEADDRUSE)
{
//
// SO_EXCLUSIVEADDRUSE makes Windows behave like Unix platforms do WRT the SO_REUSEADDR option.
// So, on Unix platforms, we consider SO_EXCLUSIVEADDRUSE to always be set. We allow manually setting this
// to "true", but not "false."
//
if (optionLen != sizeof(int32_t))
{
return PAL_EINVAL;
}

if (*reinterpret_cast<int32_t*>(optionValue) == 0)
{
return PAL_ENOTSUP;
}
else
{
return PAL_SUCCESS;
}
}
else if (socketOptionName == PAL_SO_REUSEADDR)
{
//
// On Windows, SO_REUSEADDR allows the address *and* port to be reused. It's equivalent to
// SO_REUSEADDR + SO_REUSEPORT other systems.
//
int err = setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, optionValue, static_cast<socklen_t>(optionLen));
if (err == 0)
{
err = setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, optionValue, static_cast<socklen_t>(optionLen));
}
return err == 0 ? PAL_SUCCESS : SystemNative_ConvertErrorPlatformToPal(errno);
}
}

int optLevel, optName;
if (!TryGetPlatformSocketOption(socketOptionLevel, socketOptionName, optLevel, optName))
{
Expand Down
2 changes: 1 addition & 1 deletion src/Native/Unix/System.Native/pal_networking.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ enum SocketOptionName : int32_t
PAL_SO_LINGER = 0x0080,
PAL_SO_OOBINLINE = 0x0100,
// PAL_SO_DONTLINGER = ~PAL_SO_LINGER,
// PAL_SO_EXCLUSIVEADDRUSE = ~PAL_SO_REUSEADDR,
PAL_SO_EXCLUSIVEADDRUSE = ~PAL_SO_REUSEADDR,
PAL_SO_SNDBUF = 0x1001,
PAL_SO_RCVBUF = 0x1002,
PAL_SO_SNDLOWAT = 0x1003,
Expand Down
60 changes: 60 additions & 0 deletions src/System.Net.Sockets/tests/FunctionalTests/SocketOptionNameTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,5 +233,65 @@ private static Socket CreateBoundUdpSocket(out int localPort)
localPort = (receiveSocket.LocalEndPoint as IPEndPoint).Port;
return receiveSocket;
}

[Theory]
[InlineData(null, null, null, true)]
[InlineData(null, null, false, true)]
[InlineData(null, false, false, true)]
[InlineData(null, true, false, true)]
[InlineData(null, true, true, false)]
[InlineData(true, null, null, true)]
[InlineData(true, null, false, true)]
[InlineData(true, null, true, true)]
[InlineData(true, false, null, true)]
[InlineData(true, false, false, true)]
[InlineData(true, false, true, true)]
public void ReuseAddress(bool? exclusiveAddressUse, bool? firstSocketReuseAddress, bool? secondSocketReuseAddress, bool expectFailure)
{
using (Socket a = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
{
if (exclusiveAddressUse.HasValue)
{
a.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ExclusiveAddressUse, exclusiveAddressUse.Value);
}
if (firstSocketReuseAddress.HasValue)
{
a.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, firstSocketReuseAddress.Value);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have existing tests verifying the roundtripping behavior for SocketOptionName.ReuseAddress with GetSocketOption?

}

a.Bind(new IPEndPoint(IPAddress.Loopback, 0));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any need to test both IPv4 and IPv6, or is just IPv4 sufficient for this purpose?


using (Socket b = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
{
if (secondSocketReuseAddress.HasValue)
{
b.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, secondSocketReuseAddress.Value);
}

if (expectFailure)
{
Assert.ThrowsAny<SocketException>(() => b.Bind(a.LocalEndPoint));
}
else
{
b.Bind(a.LocalEndPoint);
}
}
}
}

[Theory]
[PlatformSpecific(PlatformID.Windows)]
[InlineData(false, null, null, true)]
[InlineData(false, null, false, true)]
[InlineData(false, false, null, true)]
[InlineData(false, false, false, true)]
[InlineData(false, true, null, true)]
[InlineData(false, true, false, true)]
[InlineData(false, true, true, false)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the expected behavior of these cases on Unix? Should we verify those failures via a test that's Unix-only?

public void ReuseAddress_Windows(bool? exclusiveAddressUse, bool? firstSocketReuseAddress, bool? secondSocketReuseAddress, bool expectFailure)
{
ReuseAddress(exclusiveAddressUse, firstSocketReuseAddress, secondSocketReuseAddress, expectFailure);
}
}
}
31 changes: 25 additions & 6 deletions src/System.Net.Sockets/tests/FunctionalTests/TcpClientTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ public void ConnectedAvailable_NullClient()
[OuterLoop] // TODO: Issue #11345
[Fact]
[PlatformSpecific(PlatformID.Windows)]
public void ExclusiveAddressUse_NullClient()
public void ExclusiveAddressUse_NullClient_Windows()
{
using (TcpClient client = new TcpClient())
{
Expand All @@ -100,13 +100,33 @@ public void ExclusiveAddressUse_NullClient()

[OuterLoop] // TODO: Issue #11345
[Fact]
[PlatformSpecific(PlatformID.Windows)]
public void Roundtrip_ExclusiveAddressUse_GetEqualsSet()
[PlatformSpecific(~PlatformID.Windows)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically we've used PlatformID.AnyUnix for this rather than ~Windows.

public void ExclusiveAddressUse_NullClient_NonWindows()
{
using (TcpClient client = new TcpClient())
{
client.Client = null;

Assert.True(client.ExclusiveAddressUse);
}
}

[Fact]
public void Roundtrip_ExclusiveAddressUse_GetEqualsSet_True()
{
using (TcpClient client = new TcpClient())
{
client.ExclusiveAddressUse = true;
Assert.True(client.ExclusiveAddressUse);
}
}

[Fact]
[PlatformSpecific(PlatformID.Windows)]
public void Roundtrip_ExclusiveAddressUse_GetEqualsSet_False()
{
using (TcpClient client = new TcpClient())
{
client.ExclusiveAddressUse = false;
Assert.False(client.ExclusiveAddressUse);
}
Expand All @@ -115,14 +135,13 @@ public void Roundtrip_ExclusiveAddressUse_GetEqualsSet()
[OuterLoop] // TODO: Issue #11345
[Fact]
[PlatformSpecific(PlatformID.AnyUnix)]
public void ExclusiveAddressUse_NotSupported()
public void ExclusiveAddressUse_Set_False_NotSupported()
{
using (TcpClient client = new TcpClient())
{
Assert.Throws<SocketException>(() => client.ExclusiveAddressUse);
Assert.Throws<SocketException>(() =>
{
client.ExclusiveAddressUse = true;
client.ExclusiveAddressUse = false;
});
}
}
Expand Down