Metadata decoding complex and leaks memory

[jeromekelleher]

As observed in #2428, and directly here:

#2428 (comment)_

[jeromekelleher]

In #2432 I've made a pass at solving this by making metadata decoding fully "eager", (instead of lazy). Specifically, this decodes the metadata when an object (row or entity) is created, rather than when the .metadata attribute is accessed. This works and does simplify the code a lot, but it may be a step too far in terms of changing the current behaviour.

I've had another go at tweaking things, but there's a complicated web of semi-conflicting requirements that it's very hard to disentangle. We're treating XTableRow and the entity X classes in the same way as dataclasses. These have both the metadata attribute, but that's somehow accessed dynamically behind the scenes. Some of the dataclasses are specified as frozen, and some aren't.

Any thoughts here @benjeffery? Can we simplify the design a bit here, or should we just try to get to the bottom of the circular references problem in #2428?

[jeromekelleher]

Decoding leak fixed in #2439

We should still probably discuss the possibility of simplifying things a bit.

[benjeffery]

I think with #2441 and #2439 I think we can close this? The metadata decoding is still complex due to needing the decoding to be lazy in a dataclass, but there the perf tradeoff is more than worth it.

[jeromekelleher]

Yeah, agreed. Be nice to get some data on the perf tradeoffs here, but lets close.

[benjeffery]

Yeah, agreed. Be nice to get some data on the perf tradeoffs here, but lets close.

We'll get this in #2444