Sync+error complete combined

[benaadams]

The two PRs had merge conflicts, so this is them combined

SocketOutput exception completes all waiting tasks #567
Write chunks async; unblock sync-waits directly #586

Resolves #572
Resolves #565
Resolves #566
Resolves #590

Closes #567
Closes #586

[benaadams]

Throughput check still looking good (large async chunked requests) coreclr x64

private readonly byte[] bytes = Encoding.UTF8.GetBytes(new string('a', 1024));
public async Task Invoke(HttpContext httpContext)
{
    // We check Ordinal explicitly first because it's faster than OrdinalIgnoreCase
    if (httpContext.Request.Path.StartsWithSegments(_path, StringComparison.Ordinal))
    {
        for (var i = 0; i < 300; i++)
        {
            await httpContext.Response.Body.WriteAsync(bytes, 0, bytes.Length);
        }
        return;
    }

    await _next(httpContext);
}

Running 10s test @ http://.../plaintext
  32 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   200.80ms  183.95ms   2.00s    76.91%
    Req/Sec   146.90     62.94   400.00     67.41%
  46299 requests in 10.10s, 13.36GB read
  Socket errors: connect 35, read 0, write 0, timeout 47
Requests/sec:   4583.76
Transfer/sec:      1.32GB

Some occasional dips over a longer run (GC); though @halter73 had something for that.

[benaadams]

Sync now just has regular threadpool scaling issues (to be expected)

private Task _completedTask = Task.FromResult(0);
private readonly byte[] bytes = Encoding.UTF8.GetBytes(new string('a', 1024));
public Task Invoke(HttpContext httpContext)
{
    // We check Ordinal explicitly first because it's faster than OrdinalIgnoreCase
    if (httpContext.Request.Path.StartsWithSegments(_path, StringComparison.Ordinal))
    {
        for (var i = 0; i < 300; i++)
        {
            httpContext.Response.Body.Write(bytes, 0, bytes.Length);
        }
        return _completedTask;
    }

    return _next(httpContext);
}

Reducing the concurrent connections and doing it a couple times to warm up the threadpool count

Running 10s test @ http://.../plaintext
  32 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    63.36ms   72.80ms 211.48ms   76.16%
    Req/Sec    56.73    156.25   666.00     91.18%
  10928 requests in 10.10s, 3.15GB read
Requests/sec:   1082.04
Transfer/sec:    319.28MB

Its unhappy if you have a high number of concurrent connections; i.e. beyond threadpool count; but it does make forward progress, unlike previously.

[benaadams]

Probably additionally needs some way to bump up minThreads for ThreadPool for people that do a lot of sync calls - but that's a hosting thing? (e.g. dnx, dotnet cli, etc)

[halter73]

@benaadams I think increasing the ThreadPool's minimum threads is something that users will have to do themselves if they have to make a lot of sync calls.

[benaadams]

@halter73 yeah, don't think there is a facility to do it - will have a quick scan of https://github.com/dotnet/cli and raise issue if not. I think that's the right place?

[benaadams]

Raised issue for threadpool size https://github.com/dotnet/cli/issues/889

[benaadams]

Code change feedback on the original issues so will be changing them rather than this. @halter73 feel free to close - though it has more info, like the code I ran for the tests and the results.

[benaadams]

@halter73 threadpool setting currently by switching the environment variable

set ComPlus_ThreadPool_ForceMinWorkerThreads=1000