Remove MemoryPlan from VM passes

[mbrookhart]

@jroesch @masahi

As discussed, this disables MemoryPlan in the VM until we can rewrite it to do full reuse planning. The current pass slows down compilation a lot without providing a strong benefit for runtime performance.

This PR also removes a debug print that snuck into another part of the VM

[kevinthesun]

Can we have some benchmark data from dynamic models, such as tf ssd/rcnn, to show the performance impact of disabling this pass?

[icemelon]

Based on my experience, the performance difference with and without this pass is not evident, at least for the BERT case.

[monklof]

@icemelon9 Great to see you working on Bert. I've recently been trying to optimize bert with TVM, especially dynamic-batched case. I found that the Relay VM solution introduced a lot of small pieces of PackedFunc related to alloc_storage's size calculation, and it makes vm slower. Do you have any relevant experience or idea to share on this? Thanks. More detailed discussion: https://discuss.tvm.apache.org/t/guideline-relay-aot/5977/17?u=monklof

[kevinthesun]

cc @zhiics @icemelon9

[masahi]

Can we simply use the pass infra to disable it, like

with tvm.transform.PassContext(opt_level=3, disabled_pass=["MemoryPlan"]):
     ...

[mbrookhart]

We could, but we've tested a few onnx and pytorch models and don't see any performance differences, and @jroesch tells me the purpose of the pass was the first half of plan to do graph-runtime like memory reuse, but the second half was never implemented. Unless we can find a usecase where it helps, I think it makes more sense to disable it entirely until we can get the full feature working.

[kevinthesun]

@mbrookhart Thanks. It would be great if you can try tf ssd and fasterrcnn so that we can ensure there is no regression for tf models as well.

[icemelon]

LGTM. I'll leave the merge decision to @kevinthesun.

[mbrookhart]

@kevinthesun I'd be happy to do some testing. Do you have scripts for running those models? I'm not finding them in the tutorials.

[kevinthesun]

@mbrookhart Sure. You can refer to tf ssd integration test.

[mbrookhart]

Okay for TF-SSD, I timed 10 calls to vm.invoke after all of the compilation happened and averaged them, running on a Ryzen 5950x

main:
34.41 ms/iteration
439s to execute the whole test (import/compile/execution) as measured by pytest
this:
34.57 ms/iteration
266s to execute the whole test

[kevinthesun]

LGTM

[altanh]

I'm not sure we have the infra for this, but going forward it would also be interesting to see the memory usage differences with and without, since this pass would affect allocation behavior. I imagine our peak usage would actually decrease atm without the pass since the liveness analysis phase hasn't been implemented, so we are keeping memory around longer than needed.

[kevinthesun]

Thanks @mbrookhart @masahi @icemelon9 @zhiics