Fix 3.0->3.1 regression in JSON serializing nested concurrent dictionaries

[layomia]

Description

Fixes dotnet/runtime#670, where there was a regression between 3.0 and 3.1 where serializing nested concurrent dictionaries went from supported to unsupported. The issue also exists in master/5.0. dotnet/runtime#784 fixes it in master, and this PR ports the fix to 3.0.

In addition to the fix described above, this PR also also has a deserialization fix for an InvalidCastException thrown when a dictionary element type has a converter that returns a type different from the declared type, e.g. when you're deserializing Dictionary<string, MyClass>, where

private interface IClass { }

private class MyClass : IClass { }

private class MyFactory : JsonConverterFactory
{
    public override bool CanConvert(Type typeToConvert)
    {
        return typeToConvert == typeof(IClass) || typeToConvert == typeof(MyClass);
    }

    public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
    {
        return new MyStuffConverter();
    }
}

private class MyStuffConverter : JsonConverter<IClass>
{
    public override IClass Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        return new MyClass();
    }

    public override void Write(Utf8JsonWriter writer, IClass value, JsonSerializerOptions options)
    {
        writer.WriteNumberValue(1);
    }
}

In this case, the read method of the converter for typeof(MyClass) returns a type IClass (not the declared type, MyClass) which causes an InvalidCastException later in the deserialization flow.

Customer Impact

The ability to serialize nested concurrent dictionaries is restored: dotnet/runtime#670.

The deserialization scenario is an edge case, but the fix defends against leaking exceptions were it to occur.

Regression

No. The serialization change's tests are wide-ranging and covers various permutations of nested dictionaries, so a regression is unlikely.

The deserialization change adds more support without rescinding support. A regression is unlikely.

Risk

Low, per the regression section above.

[danmoseley]

Can you link to the change you’re back porting

[layomia]

dotnet/runtime#784 fixes dotnet/runtime#670 for runtime/master (5.0), but doesn't contain the change on deserialization described above. We can consider this PR a back-port + the deserialization change.

[ericstj]

LGTM: @ahsonkhan or @steveharter can you also review?

Note that this cannot merge until it is approved for servicing. Please add the servicing template to the pull request description.

[danmoseley]

@ericstj is this ready for tactics consideration now? just noticed it's not tagged but I see the template above

[ericstj]

Yeah this is good, @steveharter signed off on same code change in master.

[jamshedd]

Approved for March.

[danmoseley]

Moved back to 3.1.2 per tactics.