Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration of the proxied HTTP request version. #512

Merged
merged 15 commits into from
Nov 10, 2020
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -353,3 +353,9 @@ MigrationBackup/

# Ionide (cross platform F# VS Code tools) working folder
.ionide/

# Rider
.idea/

# nohup
nohup.out
35 changes: 32 additions & 3 deletions docs/docfx/articles/proxyhttpclientconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,18 @@
Introduced: preview5

## Introduction

Each [Cluster](xref:Microsoft.ReverseProxy.Abstractions.Cluster) has a dedicated [HttpMessageInvoker](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpmessageinvoker?view=netcore-3.1) instance used to proxy requests to its [Destination](xref:Microsoft.ReverseProxy.Abstractions.Destination)s. The configuration is defined per cluster. On YARP startup, all `Clusters` get new `HttpMessageInvoker` instances, however if later the `Cluster` configuration gets changed the [IProxyHttpClientFactory](xref:Microsoft.ReverseProxy.Service.Proxy.Infrastructure.IProxyHttpClientFactory) will re-run and decide if it should create a new `HttpMessageInvoker` or keep using the existing one. The default `IProxyHttpClientFactory` implementation creates a new `HttpMessageInvoker` when there are changes to the [ProxyHttpClientOptions](xref:Microsoft.ReverseProxy.Abstractions.ProxyHttpClientOptions).

Properties of outgoing requests for a given cluster can be configured as well. They are defined in [ProxyHttpRequestOptions](xref:Microsoft.ReverseProxy.Abstractions.ProxyHttpRequestOptions).

The configuration is represented differently if you're using the [IConfiguration](https://docs.microsoft.com/en-us/dotnet/api/microsoft.extensions.configuration.iconfiguration?view=dotnet-plat-ext-3.1) model or the code-first model.

## IConfiguration
HTTP client configuration contract consists of [ProxyHttpClientData](xref:Microsoft.ReverseProxy.Configuration.Contract.ProxyHttpClientData) and [CertificateConfigData](xref:Microsoft.ReverseProxy.Configuration.Contract.CertificateConfigData) types defining the following configuration schema. These types are focused on defining serializable configuration. The code based HTTP client configuration model is described below in the "Code Configuration" section.
These types are focused on defining serializable configuration. The code based configuration model is described below in the "Code Configuration" section.

### HttpClient
HTTP client configuration contract consists of [ProxyHttpClientData](xref:Microsoft.ReverseProxy.Configuration.Contract.ProxyHttpClientData) and [CertificateConfigData](xref:Microsoft.ReverseProxy.Configuration.Contract.CertificateConfigData) types defining the following configuration schema.
```JSON
"HttpClient": {
"SslProtocols": [ "<protocol-names>" ],
Expand Down Expand Up @@ -70,8 +75,25 @@ Configuration settings:
}

```

### HttpRequest
HTTP request configuration contract is defined by [ProxyHttpRequestData](xref:Microsoft.ReverseProxy.Configuration.Contract.ProxyHttpRequestData) type defining the following configuration schema.
```JSON
"HttpRequest": {
"RequestTimeout": "<timespan>",
"Version": "<string>",
"VersionPolicy": ["RequestVersionOrLower", "RequestVersionOrHigher", "RequestVersionExact"]
}
```

Configuration settings:
- RequestTimeout - the timeout for the outgoing request sent by [HttpMessageInvoker.SendAsync](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpmessageinvoker.sendasync?view=netcore-3.1). If not specified, 100 seconds is used.
- Version - outgoing request [version](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httprequestmessage.version?view=netcore-3.1). The supported values at the moment are 1.0, 1.1 and 2. Default value is 2.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Version - outgoing request [version](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httprequestmessage.version?view=netcore-3.1). The supported values at the moment are 1.0, 1.1 and 2. Default value is 2.
- Version - outgoing request [version](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httprequestmessage.version?view=netcore-3.1). The supported values at the moment are `1.0`, `1.1` and `2`. Default value is `2`.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

- VersionPolicy - defines how the final version is selected for the outgoing requests. This feature is available from .NET 5.0, see [HttpRequestMessage.VersionPolicy](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httprequestmessage.versionpolicy?view=net-5.0). The default value is `RequestVersionOrLower`.


## Configuration example
The below example shows 2 samples of HTTP client configurations for `cluster1` and `cluster2`.
The below example shows 2 samples of HTTP client and request configurations for `cluster1` and `cluster2`.

```JSON
{
Expand All @@ -88,6 +110,9 @@ The below example shows 2 samples of HTTP client configurations for `cluster1` a
"MaxConnectionsPerServer": "10",
"DangerousAcceptAnyServerCertificate": "true"
},
"HttpRequest": {
"RequestTimeout": "00:00:30"
},
"Destinations": {
"cluster1/destination1": {
"Address": "https://localhost:10000/"
Expand All @@ -108,6 +133,10 @@ The below example shows 2 samples of HTTP client configurations for `cluster1` a
"Password": "1234abc"
}
},
"HttpRequest": {
"Version": "1.1",
"VersionPolicy": "RequestVersionExact"
},
"Destinations": {
"cluster2/destination1": {
"Address": "https://localhost:10001/"
Expand All @@ -119,7 +148,7 @@ The below example shows 2 samples of HTTP client configurations for `cluster1` a
```

## Code Configuration
HTTP client configuration abstraction constists of the only type [ProxyHttpClientOptions](xref:Microsoft.ReverseProxy.Abstractions.ProxyHttpClientOptions) defined as follows.
HTTP client configuration abstraction consists of the only type [ProxyHttpClientOptions](xref:Microsoft.ReverseProxy.Abstractions.ProxyHttpClientOptions) defined as follows.

```C#
public sealed class ProxyHttpClientOptions
Expand Down
5 changes: 3 additions & 2 deletions samples/ReverseProxy.Code.Sample/CustomConfigFilter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,16 @@ public class CustomConfigFilter : IProxyConfigFilter
public Task ConfigureClusterAsync(Cluster cluster, CancellationToken cancel)
{
// How to use custom metadata to configure clusters
if (cluster.Metadata?.TryGetValue("CustomHealth", out var customHealth) ?? false
if (cluster.Metadata != null
&& cluster.Metadata.TryGetValue("CustomHealth", out var customHealth)
&& string.Equals(customHealth, "true", StringComparison.OrdinalIgnoreCase))
{
cluster.HealthCheck ??= new HealthCheckOptions { Active = new ActiveHealthCheckOptions() };
cluster.HealthCheck.Active.Enabled = true;
cluster.HealthCheck.Active.Policy = HealthCheckConstants.ActivePolicy.ConsecutiveFailures;
}

// Or wrap the meatadata in config sugar
// Or wrap the metadata in config sugar
var config = new ConfigurationBuilder().AddInMemoryCollection(cluster.Metadata).Build();
if (config.GetValue<bool>("CustomHealth"))
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,11 @@ public sealed class Cluster : IDeepCloneable<Cluster>
/// </summary>
public ProxyHttpClientOptions HttpClient { get; set; }

/// <summary>
/// Options of an outgoing HTTP request.
/// </summary>
public ProxyHttpRequestOptions HttpRequest { get; set; }

/// <summary>
/// The set of destinations associated with this cluster.
/// </summary>
Expand All @@ -78,6 +83,7 @@ Cluster IDeepCloneable<Cluster>.DeepClone()
SessionAffinity = SessionAffinity?.DeepClone(),
HealthCheck = HealthCheck?.DeepClone(),
HttpClient = HttpClient?.DeepClone(),
HttpRequest = HttpRequest?.DeepClone(),
Destinations = Destinations.DeepClone(StringComparer.OrdinalIgnoreCase),
Metadata = Metadata?.DeepClone(StringComparer.OrdinalIgnoreCase),
};
Expand All @@ -103,6 +109,7 @@ internal static bool Equals(Cluster cluster1, Cluster cluster2)
&& SessionAffinityOptions.Equals(cluster1.SessionAffinity, cluster2.SessionAffinity)
&& HealthCheckOptions.Equals(cluster1.HealthCheck, cluster2.HealthCheck)
&& ProxyHttpClientOptions.Equals(cluster1.HttpClient, cluster2.HttpClient)
&& ProxyHttpRequestOptions.Equals(cluster1.HttpRequest, cluster2.HttpRequest)
&& CaseInsensitiveEqualHelper.Equals(cluster1.Destinations, cluster2.Destinations, Destination.Equals)
&& CaseInsensitiveEqualHelper.Equals(cluster1.Metadata, cluster2.Metadata);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
using System;
using System.Net.Http;

namespace Microsoft.ReverseProxy.Abstractions
{
/// <summary>
/// Outgoing request configuration.
/// </summary>
public sealed class ProxyHttpRequestOptions
Tratcher marked this conversation as resolved.
Show resolved Hide resolved
Tratcher marked this conversation as resolved.
Show resolved Hide resolved
{
/// <summary>
/// Timeout for the outgoing request.
/// Default is 100 seconds.
/// </summary>
public TimeSpan? RequestTimeout { get; set; }
Tratcher marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// HTTP version for the outgoing request.
/// Default is HTTP/2.
/// </summary>
public Version Version { get; set; }

#if NET
/// <summary>
/// Version policy for the outgoing request.
/// Defines whether to upgrade or downgrade HTTP version if possible.
/// Default is <c>RequestVersionOrLower</c>.
/// </summary>
public HttpVersionPolicy? VersionPolicy { get; set; }
#endif

internal ProxyHttpRequestOptions DeepClone()
{
return new ProxyHttpRequestOptions
{
RequestTimeout = RequestTimeout,
Version = Version,
#if NET
VersionPolicy = VersionPolicy,
#endif
};
}

internal static bool Equals(ProxyHttpRequestOptions options1, ProxyHttpRequestOptions options2)
{
if (options1 == null && options2 == null)
{
return true;
}

if (options1 == null || options2 == null)
{
return false;
}

return options1.RequestTimeout == options2.RequestTimeout
&& options1.Version == options2.Version
#if NET
&& options1.VersionPolicy == options2.VersionPolicy
#endif
;
}
}
}
25 changes: 25 additions & 0 deletions src/ReverseProxy/Configuration/ConfigurationConfigProvider.cs
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ private Cluster Convert(string clusterId, ClusterData data)
SessionAffinity = Convert(data.SessionAffinity),
HealthCheck = Convert(data.HealthCheck),
HttpClient = Convert(data.HttpClient),
HttpRequest = Convert(data.HttpRequest),
Metadata = data.Metadata?.DeepClone(StringComparer.OrdinalIgnoreCase)
};
foreach(var destination in data.Destinations)
Expand Down Expand Up @@ -353,6 +354,30 @@ private ProxyHttpClientOptions Convert(ProxyHttpClientData data)
};
}

private ProxyHttpRequestOptions Convert(ProxyHttpRequestData data)
{
if (data == null)
{
return null;
}

// Parse version only if it contains any characters; otherwise, leave it null.
Version version = null;
if (!string.IsNullOrEmpty(data.Version))
{
version = Version.Parse(data.Version + (data.Version.Contains('.') ? "" : ".0"));
}

return new ProxyHttpRequestOptions
{
RequestTimeout = data.RequestTimeout,
Version = version,
#if NET
VersionPolicy = data.VersionPolicy,
#endif
};
}

private static Destination Convert(DestinationData data)
{
if (data == null)
Expand Down
5 changes: 5 additions & 0 deletions src/ReverseProxy/Configuration/Contract/ClusterData.cs
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ public sealed class ClusterData
/// </summary>
public ProxyHttpClientData HttpClient { get; set; }

/// <summary>
/// Options of an outgoing HTTP request.
/// </summary>
public ProxyHttpRequestData HttpRequest { get; set; }

/// <summary>
/// The set of destinations associated with this cluster.
/// </summary>
Expand Down
32 changes: 32 additions & 0 deletions src/ReverseProxy/Configuration/Contract/ProxyHttpRequestData.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
using System;
using System.Net.Http;

namespace Microsoft.ReverseProxy.Configuration.Contract
{
/// <summary>
/// Outgoing request configuration.
/// </summary>
public class ProxyHttpRequestData
{
/// <summary>
/// Timeout for the outgoing request.
/// Default is 100 seconds.
/// </summary>
public TimeSpan? RequestTimeout { get; set; }

/// <summary>
/// HTTP version for the outgoing request.
/// Default is HTTP/2.
/// </summary>
public string Version { get; set; }

#if NET
/// <summary>
/// Version policy for the outgoing request.
/// Defines whether to upgrade or downgrade HTTP version if possible.
/// Default is <c>RequestVersionOrLower</c>.
/// </summary>
public HttpVersionPolicy? VersionPolicy { get; set; }
#endif
}
}
17 changes: 17 additions & 0 deletions src/ReverseProxy/Middleware/ProxyInvokerMiddleware.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.ReverseProxy.RuntimeModel;
using Microsoft.ReverseProxy.Service.Proxy;
using Microsoft.ReverseProxy.Telemetry;
using Microsoft.ReverseProxy.Utilities;
Expand Down Expand Up @@ -75,6 +76,22 @@ public async Task Invoke(HttpContext context)
Transforms = routeConfig.Transforms,
};

var requestOptions = reverseProxyFeature.ClusterConfig.HttpRequestOptions;
Tratcher marked this conversation as resolved.
Show resolved Hide resolved
if (requestOptions.RequestTimeout.HasValue)
{
proxyOptions.RequestTimeout = requestOptions.RequestTimeout.Value;
}
if (requestOptions.Version != null)
{
proxyOptions.Version = requestOptions.Version;
alnikola marked this conversation as resolved.
Show resolved Hide resolved
}
#if NET
if (requestOptions.VersionPolicy.HasValue)
{
proxyOptions.VersionPolicy = requestOptions.VersionPolicy.Value;
}
#endif

try
{
cluster.ConcurrencyCounter.Increment();
Expand Down
19 changes: 19 additions & 0 deletions src/ReverseProxy/Service/Config/ConfigValidator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authorization;
Expand Down Expand Up @@ -118,6 +119,7 @@ public ValueTask<IList<Exception>> ValidateClusterAsync(Cluster cluster)

ValidateSessionAffinity(errors, cluster);
ValidateProxyHttpClient(errors, cluster);
ValidateProxyHttpRequest(errors, cluster);
ValidateActiveHealthCheck(errors, cluster);
ValidatePassiveHealthCheck(errors, cluster);

Expand Down Expand Up @@ -322,6 +324,23 @@ private void ValidateProxyHttpClient(IList<Exception> errors, Cluster cluster)
}
}

private void ValidateProxyHttpRequest(IList<Exception> errors, Cluster cluster)
alnikola marked this conversation as resolved.
Show resolved Hide resolved
{
if (cluster.HttpRequest == null)
{
// Proxy http request options are not set.
return;
}

if (cluster.HttpRequest.Version != null &&
cluster.HttpRequest.Version != HttpVersion.Version10 &&
cluster.HttpRequest.Version != HttpVersion.Version11 &&
cluster.HttpRequest.Version != HttpVersion.Version20)
{
errors.Add(new ArgumentException($"Outgoing request version '{cluster.HttpRequest.Version}' is not any of supported HTTP versions (1.0, 1.1 and 2)."));
}
}

private void ValidateActiveHealthCheck(IList<Exception> errors, Cluster cluster)
{
if (cluster.HealthCheck == null || cluster.HealthCheck.Active == null || !cluster.HealthCheck.Active.Enabled)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using System.Net;
using System.Net.Http;
using Microsoft.AspNetCore.Http.Extensions;
using Microsoft.ReverseProxy.RuntimeModel;
Expand All @@ -15,7 +16,13 @@ public HttpRequestMessage CreateRequest(ClusterConfig clusterConfig, Destination
var probePath = clusterConfig.HealthCheckOptions.Active.Path;
UriHelper.FromAbsolute(probeAddress, out var destinationScheme, out var destinationHost, out var destinationPathBase, out _, out _);
var probeUri = UriHelper.BuildAbsolute(destinationScheme, destinationHost, destinationPathBase, probePath, default);
return new HttpRequestMessage(HttpMethod.Get, probeUri) { Version = ProtocolHelper.Http2Version };
return new HttpRequestMessage(HttpMethod.Get, probeUri)
{
Version = clusterConfig.HttpRequestOptions.Version ?? HttpVersion.Version20,
#if NET
VersionPolicy = clusterConfig.HttpRequestOptions.VersionPolicy ?? HttpVersionPolicy.RequestVersionOrLower
#endif
};
}
}
}
9 changes: 9 additions & 0 deletions src/ReverseProxy/Service/Management/ProxyConfigManager.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
Expand Down Expand Up @@ -311,6 +313,13 @@ private void UpdateRuntimeClusters(IList<Cluster> newClusters)
settings: newCluster.SessionAffinity?.Settings as IReadOnlyDictionary<string, string>),
httpClient,
newClusterHttpClientOptions,
new ClusterProxyHttpRequestOptions(
requestTimeout: newCluster.HttpRequest?.RequestTimeout,
version: newCluster.HttpRequest?.Version
#if NET
, versionPolicy: newCluster.HttpRequest?.VersionPolicy
#endif
),
(IReadOnlyDictionary<string, string>)newCluster.Metadata);

if (currentClusterConfig == null ||
Expand Down
Loading