contracts: Cache compiled modules

[athei]

As of right now we unconditionally load and compile each contract whenever it is called by another contract. This makes the system easy to benchmark and argue about. However, this naive approach slows down use cases where the same code is used multiple times within the same call stack. Implementing a DEX would be such a use case where trading pairs use the same code.

We should implement the following optimization: Store the compiled wasm Module of each contract in its Frame. Whenever a cross contract happens we search the Stack if there is a contract already using the same code hash. If yes we instantiate a new Instance from this Module. This will skip loading from storage and compilation.

We have to check how that interacts with lazy compilation @Robbepop .

We also need to think about how to benchmark this.

[Robbepop]

We have to check how that interacts with lazy compilation @Robbepop .

I think if we enable lazy compilation via Wasmi it should already do a good enough job here even without caching since it would only compile those parts of a contract call that are actually needed for execution.

Vanilla caching of Wasmi compiled Wasm modules is not (yet) possible and given that we now have lazy compilation I doubt that it will help much on top of lazy compilation if enabled. I have not implemented Wasmi bytecode (de)serialization so we cannot quickly try it out. Also even with serialized Wasmi bytecode we'd still need structured encoding for it which needs to be parsed (in parts) etc. just like Wasm itself.
Compiled Wasmi bytecode is stored in the Wasmi Engine and not within the respective Wasmi Module.

tldr; I don't think we'd gain much by caching compiled Wasmi bytecode over just using the already available Wasmi lazy option.

[deuszx]

Hey @Robbepop , @athei - thanks for following up on this. Is lazy compilation something that can be quickly enabled and tried out on a branch to see how much it improves the situation?

[Robbepop]

@deuszx You need to update Wasmi in pallet-contracts from v0.31 to v0.32 (most recent is v0.32.0-beta.7) which I experimentally did in this PR: #2941
Note that most of the diff in the PR is due to automatically generated weights info.

Then enabling the lazy compilation is a one-liner: config.compilation_mode(wasmi::CompilationMode::Lazy):
https://docs.rs/wasmi/0.32.0-beta.7/wasmi/struct.Config.html#method.compilation_mode

[athei]

Vanilla caching of Wasmi compiled Wasm modules is not (yet) possible

Compiled Wasmi bytecode is stored in the Wasmi Engine and not within the respective Wasmi Module.

Why not? If we keep the Engine and Module in memory for the duration of a call stack we can just instantiate a new Instance from it?

[Robbepop]

Vanilla caching of Wasmi compiled Wasm modules is not (yet) possible

Compiled Wasmi bytecode is stored in the Wasmi Engine and not within the respective Wasmi Module.

Why not? If we keep the Engine and Module in memory for the duration of a call stack we can just instantiate a new Instance from it?

Yes if that is a possibility in contracts-pallet then this should work fine.

[athei]

Setting it to blocked for now. We should check if lazy compilation will help enough before adding a fair amount of complexity.

[pgherveou]

@deuszx You need to update Wasmi in pallet-contracts from v0.31 to v0.32 (most recent is v0.32.0-beta.7) which I experimentally did in this PR: #2941 Note that most of the diff in the PR is due to automatically generated weights info.

Then enabling the lazy compilation is a one-liner: config.compilation_mode(wasmi::CompilationMode::Lazy): https://docs.rs/wasmi/0.32.0-beta.7/wasmi/struct.Config.html#method.compilation_mode

Took over @Robbepop previous PR here
#3679

[deuszx]

Thanks @pgherveou .

While I have you here - @Robbepop and @athei - I've been going through the changes in wasmi@0.32.X and I found this wasmi-labs/wasmi#829 . Does it make sense to use this new API instead in pallet_contracts? The code is changed, hence needs validation, only when it's upload and upgraded (set_code). After that, its API doesn't change so it doesn't need to be fully validated on every load.

[Robbepop]

Thanks @pgherveou .

While I have you here - @Robbepop and @athei - I've been going through the changes in wasmi@0.32.X and I found this wasmi-labs/wasmi#829 . Does it make sense to use this new API instead in pallet_contracts? The code is changed, hence needs validation, only when it's upload and upgraded (set_code). After that, its API doesn't change so it doesn't need to be fully validated on every load.

Yes we could use this API technically and it would further improve the startup performance for eager compilation and also to some extend the startup and execution performance in lazy mode since validation in lazy mode is no longer done when lazily compiling a function during its first use.

However, the API is unsafe for a reason. If the caller (read contracts-pallet) does not 100% make sure that the Wasm modules it hands over to this API are purposefully pre-validated this is UB, so anything might happen which is a huge attack vector.

[deuszx]

I understand. I'm not the expert on pallet_contracts of course but it seems like it should be possible to create and maintain the invariant.

Thanks for the input. (And the awesome, continuous work on wasmi 🙏 💪 ).

[Robbepop]

The invariant is already established since the contracts_pallet already pre-validates Wasm input in the upload_code phase.
Funnily, it just occurred to me that avoiding Wasm validation in smart contract execution (due to pre-validation) also has some benefits against malicious actors since they are no longer able to attack the Wasm validation but only the translation parts.

@athei What are your thoughts on that?

[athei]

I am closing here because after much discussion that this is not really possible without many changes to the wasmi API as we cannot selectively cache modules. We would need to cache all of the modules for a whole call stack. We don't have the runtime memory for that.

We will instead rely on lazy compilation to reduce the costs of cross contract calls.