Use MetadataUpdateHandlerAttribute to clear Mono's reflection cache

[lambdageek]

Related CoreCLR work #50938 (the attribute was added as part of #50954).

The slightly interesting thing for Mono is that we'll need to call down to reflection.c to clear to the unmanaged hash tables that map unmanaged metadata method representations to their managed counterparts.

Also need to clear the Cache in RuntimeType added by #78840

Enable the tests in #50954

Part of #44806 Part of #57365

[ghost]

Tagging subscribers to this area: @CoffeeFlux
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Related CoreCLR work #50938 (the attribute was added as part of #50954).

The slightly interesting thing for Mono is that we'll need to call down to reflection.c to clear to the unmanaged hash tables that map unmanaged metadata method representations to their managed counterparts.

Enable the tests in #50954

Part of #44806

Author:	lambdageek
Assignees:	-
Labels:	area-VM-meta-mono
Milestone:	6.0.0

[lambdageek]

We actually also run into this trying to support the UpdateParameters capability.

I have kind of a bad plan to fix it:

I think what we need to do is to change

typedef struct {
    gpointer item;
    MonoClass *refclass;
} ReflectedEntry;

in `reflection-cache.h` to

```c 
typedef struct {
    gpointer item;
    MonoClass *refclass;
    uint32_t generation; /* 0 is normal; hot reload may change it */
} ReflectedEntry;

and then change check_object_handle to use mono_conc_g_hash_table_lookup_extended (and mono_weak_hash_table_lookup_extended) to get back the original key. and if the original key generation is less than mono_metadata_update_get_thread_generation() then clear the cached entry with mono_conc_g_hash_table_remove (or mono_weak_hash_table_remove) and then return null.

Then the rest of the caching machinery will create the new entry.
and in cache_object and cache_object_handle, when they call alloc_reflected_entry, they should set e->generation = mono_metadata_update_get_thread_generation()

So this will be correct, but it's gonna be calling mono_metadata_update_get_thread_generation a lot, which is gonna be slow. so we will need to stash a way some has_updates boolean for the normal case of no updates. probably on the MonoMemoryManager

---

So if we do this, then we don't actually need the metadata update handler to do anything - all the work will be done by the reflection object lookup code. But the cost is an extra generation counter on every reflection cache entry, even in release builds that don't use hot reload.

---

An alternative is to stop using the MonoMemoryManager:refobject_hash (and weak_refobject_hash) directly from reflection-cache.h and instead to make some getters. Still store a has_updates flag on the MonoMemoryManager, and in the normal case (no updates) just return the fields. In the "has updates" case, call into hot_reload.c and have it return a new hash table for the generation - keyed on the baseline refobject_hash.

I don't think we want to ever change what MonoMemoryManager:refobject_hash points to - that could lead to crashes since our code accesses that field without locking in many codepaths.

I think it might be possible, however, to sometimes clear all the entries in the outdated concurrent hash (the weak hash we can probably just leave alone and let the GC clear the managed objects) when we discover it's no longer the latest one.

In this case too, the metadata update handler won't be clearing anything. The issue is that (especially with generics) it's really hard to find all the relevant MonoMemoryManager structs if we decide to invalidate. if a generic instance comes from multiple MonoImages (ie the gtd is from assembly A and the instance types are from assemblies, B, C, D, ...) I don't know if we have a nice way to find all the relevant MonoMemoryManagers.

But making sure we always do the lookup in an up to date hash table seems doable...