[Bug] IDX10503: Signature validation failed. Token does not have a kid. (System.NotImplementedException: The method or operation is not implemented. at Microsoft.IdentityModel.Tokens.SignatureProvider.Verify...)

[KelDazan]

Which version of Microsoft.IdentityModel are you using?
Issue is reproduced at latest (atm) version - 6.25.0
Issue is NOT reproducing at 6.22.1 version

Where is the issue?

• M.IM.JsonWebTokens

Is this a new or an existing app?

a. The app is in production and I have upgraded to a new version of Microsoft.IdentityModel.*

Exactly the issue - everything was working on 6.22.1, but since 6.23.0 i am failing a JWT signature check. I am using https://nsec.rocks/ for custom JWT signing and validating. I am using custom TokenValidationParameters.CryptoProviderFactory that points to custom ICryptoProvider provider, that creates custom SignatureProvider for signing an validating JWT. Simplified code is provided below

Repro

Custom classes that are used:

    public record KeyPair
    {
        public string Public { get; }

        public string Private { get; }

        public byte[] PublicKey => Convert.FromBase64String(Public);

        public byte[] PrivateKey => Convert.FromBase64String(Private);

        public KeyPair(
            string publicKey,
            string privateKey)
        {
            Public = string.IsNullOrWhiteSpace(publicKey) ?
                throw new ArgumentNullException(nameof(publicKey)) : publicKey;

            Private = string.IsNullOrWhiteSpace(privateKey) ?
                throw new ArgumentNullException(nameof(privateKey)) : privateKey;
        }

        public KeyPair(string publicKey)
        {
            Public = string.IsNullOrWhiteSpace(publicKey) ?
                throw new ArgumentNullException(nameof(publicKey)) : publicKey;

            Private = string.Empty;
        }
    }

    public class EdDsaSecurityKey : AsymmetricSecurityKey
    {
        public KeyPair KeyPair { get; }

        public EdDsaSecurityKey(
            KeyPair keyPair)
        {
            KeyPair = keyPair ?? throw new ArgumentNullException(nameof(keyPair));
        }

        [Obsolete("HasPrivateKey method is deprecated, please use PrivateKeyStatus.")]
        public override bool HasPrivateKey => !string.IsNullOrWhiteSpace(KeyPair.Private);

        public override PrivateKeyStatus PrivateKeyStatus =>
            !string.IsNullOrWhiteSpace(KeyPair.Private) ? PrivateKeyStatus.Exists  :  PrivateKeyStatus.DoesNotExist;

        public override int KeySize => KeyPair?.PrivateKey?.Length ?? 0;
    }

    public class EdDsaSignatureProvider : SignatureProvider
    {
        private readonly EdDsaSecurityKey _edDsaSecurityKey;

        public EdDsaSignatureProvider(EdDsaSecurityKey key, string algorithm)
            : base(key, algorithm)
        {
            _edDsaSecurityKey = key ?? throw new ArgumentNullException(nameof(key));
        }

        public override byte[] Sign(byte[] input)
        {
            Ed25519 algorithm = new();
            ReadOnlySpan<byte> blob = new(_edDsaSecurityKey.KeyPair.PrivateKey);

            bool wasImported = NSec.Cryptography.Key.TryImport(
                algorithm,
                blob,
                KeyBlobFormat.NSecPrivateKey,
                out Key? key,
                new KeyCreationParameters
                {
                    ExportPolicy = KeyExportPolicies.AllowPlaintextExport
                });

            if (!wasImported || key == null)
                throw new InvalidOperationException("Private key was not imported successfully");

            return algorithm.Sign(key, input);
        }

        public override bool Verify(byte[] input, byte[] signature)
        {
            Ed25519 algorithm = new();
            ReadOnlySpan<byte> blob = new(_edDsaSecurityKey.KeyPair.PublicKey);

            bool wasImported = PublicKey.TryImport(
                algorithm,
                blob,
                KeyBlobFormat.NSecPublicKey,
                out PublicKey? key);

            if(!wasImported || key == null)
                throw new InvalidOperationException("Public key was not imported successfully");

            return algorithm.Verify(key, input, signature);
        }

        protected override void Dispose(bool disposing)
        {

        }
    }

    public class EdDsaCryptoProvider : ICryptoProvider
    {
        public bool IsSupportedAlgorithm(string algorithm, params object[] args)
            => algorithm.Equals(ExtendedSecurityAlgorithms.EdDsa);

        public object Create(string algorithm, params object[] args)
        {
            if (!algorithm.Equals(ExtendedSecurityAlgorithms.EdDsa))
                throw new NotSupportedException($"Only '{ExtendedSecurityAlgorithms.EdDsa}' algorithm is supported");

            EdDsaSecurityKey? key = args.OfType<EdDsaSecurityKey>().FirstOrDefault();
            if (key == null)
                throw new ArgumentException($"Arguments are expected to contain key of type '{nameof(EdDsaSecurityKey)}'");

            return new EdDsaSignatureProvider(key, algorithm);
        }

        public void Release(object cryptoInstance)
        {
            if (cryptoInstance is IDisposable disposableObject)
                disposableObject.Dispose();
        }
    }

Calling the validation:

            TokenValidationParameters validationParameters = new()
            {
                ValidateIssuer = false,
                ValidateAudience = false,
                IssuerSigningKey = new EdDsaSecurityKey(
                    new KeyPair(
                        publicKey: "*public key generated by NSec lib*",
                        privateKey: "*empty - no private key - check only*")),
                ClockSkew = TimeSpan.Zero,
                CryptoProviderFactory = new CryptoProviderFactory()
                {
                    CustomCryptoProvider = new EdDsaCryptoProvider(),
                }
            };

            TokenValidationResult validationResult = new JsonWebTokenHandler().ValidateToken(
                "*a well-formed and signed JWT*",
                validationParameters);

Expected behavior
validationResult.IsValid should return True (if signature is valid, that is)

Actual behavior
An exception in validationResult.Exception, with message:

IDX10503: Signature validation failed. Token does not have a kid. Keys tried: 'Cryptography.Core.EdDsa.EdDsaSecurityKey, KeyId: '3aee9357-d10f-48e1-a993-046c9d639726', InternalId: ''. , KeyId: 3aee9357-d10f-48e1-a993-046c9d639726
'. Number of keys in TokenValidationParameters: '1'. 
Number of keys in Configuration: '0'. 
Exceptions caught:
 'System.NotImplementedException: The method or operation is not implemented.
   at Microsoft.IdentityModel.Tokens.SignatureProvider.Verify(Byte[] input, Int32 inputOffset, Int32 inputLength, Byte[] signature, Int32 signatureOffset, Int32 signatureLength)
   at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.IsSignatureValid(Byte[] signatureBytes, Int32 signatureBytesLength, SignatureProvider signatureProvider, Byte[] dataToVerify, Int32 dataToVerifyLength)
   at Microsoft.IdentityModel.Tokens.Base64UrlEncoding.Decode[T,TX,TY,TZ](String input, Int32 offset, Int32 length, TX argx, TY argy, TZ argz, Func`6 action)
   at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateSignature(Byte[] bytes, Int32 len, String stringWithSignature, Int32 signatureStartIndex, SignatureProvider signatureProvider)
   at Microsoft.IdentityModel.Tokens.EncodingUtils.PerformEncodingDependentOperation[T,TX,TY,TZ](String input, Int32 offset, Int32 length, Encoding encoding, TX argx, TY argy, TZ argz, Func`6 action)
   at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateSignature(JsonWebToken jsonWebToken, SecurityKey key, TokenValidationParameters validationParameters)
   at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateSignature(JsonWebToken jwtToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
'.
token: 'Microsoft.IdentityModel.JsonWebTokens.JsonWebToken'.

Possible solution
Use 6.22.1 version and don't upgrade. I believe i am missing some kind of a setting, or an injection, but i do not understand what exactly. Much obliged for any assistance.

[brentschmaltz]

@KelDazan our error message is not informative enough.
Can you try implementing: Verify

[KelDazan]

@brentschmaltz

@KelDazan our error message is not informative enough. Can you try implementing: Verify

Thank you very much for the reply!
I was inattentive to full exception message, and didn't realize that it was another Verify method - my mistake.
I have overridden the Verify method that you provided link to and everything worked like a charm.

[brentschmaltz]

@KelDazan super, sorry for the hassle and this is clearly NOT your issue but ours.
I will improve the message in the exception.
Would you be willing to review the exception message in the hope others would understand what to do?

[KelDazan]

@brentschmaltz
If i will be of any help - yes, sure.

[brentschmaltz]

@KelDazan finally got to this: #2694

[KelDazan]

@KelDazan finally got to this: #2694

Much obliged!

[brentschmaltz]

@KelDazan thanks for bringing this to our attention we have made plans that will help avoid breaks in the future.

1. We only modify internals on major releases, this addresses 'method not found' issues as our packages all require the same major version.
2. When adding a new method to a call graph, we will make sure the old call graph still works, it may not be as performant, but if will be non-breaking.
   3 If for some reason we can't make the old call graph work, produce a meaningful error message for NotImplementedExceptions.
3. Consider postponing the modification until a major release.
4. Add Roslyn warnings to inform users of the new API(s) to improve performance, reduce allocations, address a bug, etc.