Sending Full Client Certificate Chain

[leobasilio]

Hi. I need to send a SOAP request to a server, and a client certificate is required for authentication. The client class was automatically generated from a wsdl file. I have a pfx file, which contains the private key and three certificates (client, intermediate CA, root CA). Here's what I'm doing:

var binding = new BasicHttpsBinding();

binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Certificate;

var client = new MyClient(binding, new EndpointAddress("..."));

client.ClientCredentials.ClientCertificate.Certificate = new X509Certificate2(pfxRawData, pfxPassword);

await client.SomeServiceAsync();

Nothing unusual here, however I keep getting this error:

System.Private.ServiceModel: The SSL connection could not be established, see inner exception.
System.Net.Http: The SSL connection could not be established, see inner exception.
System.Net.Sockets: Unable to read data from the transport connection: Connection reset by peer.

The same request works fine on Postman, using the same pfx file. After trying every other possible code variation, I decided to inspect the connection using Wireshark.

Here's what I got when using Postman:

And here's what I got using WCF:

As you can see, Postman presents all 3 certificates to the server, while WCF presents only the one at the end of the chain. That's the only difference I could find and I believe that's the reason why the server is rejecting my request.

I have been reading the source code for the last couple of hours and it seems to me that's a limitation of X509Certificate2. I have considered writing my own SecurityCredentialsManager, but I couldn't figure out yet whether that's the right path.

This seems to be related to dotnet/wcf#3791.

Any suggestions?

[mconnew]

All we do is set the X509Certificate2 instance that you provide onto the property HttpClientHandler.ClientCertificates without doing anything special. Transferring this issue to the runtime repo as this is ultimately a question on how to do this with HttpClientHandler and we're just the middle man setting a property.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Hi. I need to send a SOAP request to a server, and a client certificate is required for authentication. The client class was automatically generated from a wsdl file. I have a pfx file, which contains the private key and three certificates (client, intermediate CA, root CA). Here's what I'm doing:

var binding = new BasicHttpsBinding();

binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Certificate;

var client = new MyClient(binding, new EndpointAddress("..."));

client.ClientCredentials.ClientCertificate.Certificate = new X509Certificate2(pfxRawData, pfxPassword);

await client.SomeServiceAsync();

Nothing unusual here, however I keep getting this error:

System.Private.ServiceModel: The SSL connection could not be established, see inner exception.
System.Net.Http: The SSL connection could not be established, see inner exception.
System.Net.Sockets: Unable to read data from the transport connection: Connection reset by peer.

The same request works fine on Postman, using the same pfx file. After trying every other possible code variation, I decided to inspect the connection using Wireshark.

Here's what I got when using Postman:

And here's what I got using WCF:

As you can see, Postman presents all 3 certificates to the server, while WCF presents only the one at the end of the chain. That's the only difference I could find and I believe that's the reason why the server is rejecting my request.

I have been reading the source code for the last couple of hours and it seems to me that's a limitation of X509Certificate2. I have considered writing my own SecurityCredentialsManager, but I couldn't figure out yet whether that's the right path.

This seems to be related to dotnet/wcf#3791.

Any suggestions?

Author:	leobasilio
Assignees:	-
Labels:	area-System.Net.Http, untriaged
Milestone:	-

[leobasilio]

Today I found the issue #26323 and did some further investigation. I'll leave my findings here, in case anyone else face the same problem.

TLDR: It's not possible. You should use a proxy or an external library.
TLDR: See next comment.

The BasicHttpsBinding relies on a HttpClient to send the requests.

public async Task SendRequestAsync(Message message, TimeoutHelper timeoutHelper)
{
   //.....
    _httpClient = await _channel.GetHttpClientAsync(_to, _via, _timeoutHelper);

https://github.com/dotnet/wcf/blob/v3.2.1-rtm/src/System.Private.ServiceModel/src/System/ServiceModel/Channels/HttpChannelFactory.cs#L989

The certificate itself is added to a HttpClientHandler, which is the object that effectively manipulates the HTTP message.

internal override HttpClientHandler GetHttpClientHandler(EndpointAddress to, SecurityTokenContainer clientCertificateToken)
{
    HttpClientHandler handler = base.GetHttpClientHandler(to, clientCertificateToken);
    if (RequireClientCertificate)
    {
        SetCertificate(handler, clientCertificateToken);
    }

    AddServerCertMappingOrSetRemoteCertificateValidationCallback(handler, to);
    return handler;
}

private static void SetCertificate(HttpClientHandler handler, SecurityTokenContainer clientCertificateToken)
{
    if (clientCertificateToken != null)
    {
        X509SecurityToken x509Token = (X509SecurityToken)clientCertificateToken.Token;
        ValidateClientCertificate(x509Token.Certificate);
        handler.ClientCertificateOptions = ClientCertificateOption.Manual;
        handler.ClientCertificates.Add(x509Token.Certificate);
    }
}

https://github.com/dotnet/wcf/blob/v3.2.1-rtm/src/System.Private.ServiceModel/src/System/ServiceModel/Channels/HttpsChannelFactory.cs#L232-L255

The HttpClient takes that handler in its constructor.

internal async Task<HttpClient> GetHttpClientAsync(EndpointAddress to, Uri via,
    SecurityTokenProviderContainer tokenProvider, SecurityTokenProviderContainer proxyTokenProvider,
    SecurityTokenContainer clientCertificateToken, TimeSpan timeout)
{
    // ......
        var clientHandler = GetHttpClientHandler(to, clientCertificateToken);
    // ......
    HttpMessageHandler handler = clientHandler;
    if (_httpMessageHandlerFactory != null)
    {
        handler = _httpMessageHandlerFactory(clientHandler);
    }
    httpClient = new HttpClient(handler);

https://github.com/dotnet/wcf/blob/v3.2.1-rtm/src/System.Private.ServiceModel/src/System/ServiceModel/Channels/HttpChannelFactory.cs#L345

We know that X509Certificate2 can only handle one certificate, but the following line shows us that the handler can take multiple certificates.

handler.ClientCertificates.Add(x509Token.Certificate);

Two snippets above we see _httpMessageHandlerFactory. It is defined here:

_httpMessageHandlerFactory = context.BindingParameters.Find<Func<HttpClientHandler, HttpMessageHandler>>();

https://github.com/dotnet/wcf/blob/v3.2.1-rtm/src/System.Private.ServiceModel/src/System/ServiceModel/Channels/HttpChannelFactory.cs#L129

So it's a sort of callback that allows us to manipulate the handler. That sounds like a plan. I tried creating a custom EndpointBehavior that would add all certificates to the handler.

private class FullChainEndpointBehavior : IEndpointBehavior
{
    private readonly X509Certificate2Collection _certificateCollection;

    public FullChainEndpointBehavior(string certificateBase64, string certificatePassword)
    {
        _certificateCollection = new X509Certificate2Collection();

        _certificateCollection.Import(
            Convert.FromBase64String(certificateBase64),
            certificatePassword,
            X509KeyStorageFlags.EphemeralKeySet
        );
    }

    public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
    {
        bindingParameters.Add(
            new Func<HttpClientHandler, HttpMessageHandler>(
                client =>
                {
                    client.ClientCertificates.AddRange(_certificateCollection);
                    return client;
                }
            )
        );
    }
    // ....
}

The function was called, the certificates were added, but there's still only one certificate on Wireshark. There must be something deeper going on. So I decided to investigate the internals of HttpClient.

I couldn't find the source code for .NET Core 3.1 (the version I'm using), so I took a look at the current .NET 5 code.

The only reference to the ClientCertificates property seen above is here:

runtime/src/libraries/System.Net.Http/src/System/Net/Http/HttpClientHandler.cs

Line 213 in 4c92aef

_underlyingHandler.SslOptions.LocalCertificateSelectionCallback = (sender, targetHost, localCertificates, remoteCertificate, acceptableIssuers) => CertificateHelper.GetEligibleClientCertificate(ClientCertificates)!;

So HttpClientHandler relies on another handler of type SocketsHttpHandler and there's a callback that selects a certificate. If we take a look at GetEligibleClientCertificate we see that it basically returns the first valid certificate.

runtime/src/libraries/Common/src/System/Net/Security/CertificateHelper.cs

Lines 29 to 62 in 4c92aef

	internal static X509Certificate2? GetEligibleClientCertificate(X509Certificate2Collection candidateCerts)
	{
	if (candidateCerts.Count == 0)
	{
	return null;
	}
	foreach (X509Certificate2 cert in candidateCerts)
	{
	if (!cert.HasPrivateKey)
	{
	if (NetEventSource.Log.IsEnabled())
	{
	NetEventSource.Info(candidateCerts, $"Skipping current X509Certificate2 {cert.GetHashCode()} since it doesn't have private key. Certificate Subject: {cert.Subject}, Thumbprint: {cert.Thumbprint}.");
	}
	continue;
	}
	if (IsValidClientCertificate(cert))
	{
	if (NetEventSource.Log.IsEnabled())
	{
	NetEventSource.Info(candidateCerts, $"Choosing X509Certificate2 {cert.GetHashCode()} as the Client Certificate. Certificate Subject: {cert.Subject}, Thumbprint: {cert.Thumbprint}.");
	}
	return cert;
	}
	}
	if (NetEventSource.Log.IsEnabled())
	{
	NetEventSource.Info(candidateCerts, "No eligible client certificate found.");
	}
	return null;
	}

Finally, I wanted to know it who calls that LocalCertificateSelectionCallback, and I found that SslStream itself does it.

runtime/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.cs

Lines 91 to 101 in 4c92aef

	public SslStream(Stream innerStream, bool leaveInnerStreamOpen, RemoteCertificateValidationCallback? userCertificateValidationCallback,
	LocalCertificateSelectionCallback? userCertificateSelectionCallback, EncryptionPolicy encryptionPolicy)
	: base(innerStream, leaveInnerStreamOpen)
	{
	if (encryptionPolicy != EncryptionPolicy.RequireEncryption && encryptionPolicy != EncryptionPolicy.AllowNoEncryption && encryptionPolicy != EncryptionPolicy.NoEncryption)
	{
	throw new ArgumentException(SR.Format(SR.net_invalid_enum, "EncryptionPolicy"), nameof(encryptionPolicy));
	}
	_userCertificateValidationCallback = userCertificateValidationCallback;
	_userCertificateSelectionCallback = userCertificateSelectionCallback;

runtime/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.cs

Lines 146 to 149 in 4c92aef

	private X509Certificate UserCertSelectionCallbackWrapper(string targetHost, X509CertificateCollection localCertificates, X509Certificate? remoteCertificate, string[] acceptableIssuers)
	{
	return _userCertificateSelectionCallback!(this, targetHost, localCertificates, remoteCertificate, acceptableIssuers);
	}

And that brings me back to the issue I mentioned at the beginning. Apparently it's yet to be fixed.

[wfurt]

There is really no way how to pass explicit certificates to Schannel. The fix/workaround is #47680

var store =  new X509Store(StoreName.CertificateAuthority, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadWrite);
var certs = X509Certificate2Collection.Import(fxRawData, pfxPassword,  X509ContentType.Pfx)
foreach (var cert in certs)
{
   if (!cert.HasPrivateKey and cert.Subject != cert.Subject)
    {
        // add intermediates e.g. not leaf and not root
        store.Add(cert);
    }
}

note that the CertificateAuthority is for intermediate CAs not for the trusted one.
This is not fixable directly on Windows unless OS exposes API to control the certificates.
Until then making the intermediates to OS is only known way how to get to working.

[leobasilio]

Thank you! Now all but the root CA certificate are being sent and that seems to be enough. Finally I got a successful response from the server.

[ManickaP]

@leobasilio from your comment I assume the problem has been solved, questions answered and we can close this issue. Please feel free to reopen if you still have more problems.