🐞 Add device flag

[ashwinvaidya17]

Description

• Provide a summary of the modification as well as the issue that has been resolved. List any dependencies that this modification necessitates.

• Fixes ONNX inference and TENSORRT optimisation #600

Changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which refactors the code base)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist

• My code follows the pre-commit style and check guidelines of this project.
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing tests pass locally with my changes

[samet-akcay]

Only a single comment...

[samet-akcay]

@ashwinvaidya17, #600 points out that device didn't make any impact on the inference. Would you be able to check the inference speed with and without device set to gpu?

[djdameln]

Don't we need to pass the map_location here? It might fail otherwise if we load a gpu model on cpu.
https://pytorch.org/tutorials/recipes/recipes/save_load_across_devices.html

[ashwinvaidya17]

Good point

[ashwinvaidya17]

I am now getting the same FPS from torch_inference script (without visualization) and benchmarking script. I guess this is ready to be merged.