convert to tensorrt with static batch_size > 1

[FHarhammer]

Hi, i want to deploy my mask rcnn model to tensorrt for optimal performance at batchsize 2.
I am using the following config for this:

    '../_base_/base_instance-seg_static.py',
    '../../_base_/backends/tensorrt-int8.py'
]

onnx_config = dict(
    input_shape=[1344, 768],)

backend_config = dict(
    common_config=dict(max_workspace_size=1 << 33),
    model_inputs=[
        dict(
            input_shapes=dict(
                input=dict(
                    min_shape=[2, 3, 768, 1344],
                    opt_shape=[2, 3, 768, 1344],
                    max_shape=[2, 3, 768, 1344])))
    ])

when i start the deploy.py script i get the following error:
Error Code 4: Internal Error (input: kMIN dimensions in profile 0 are [2,3,768,1344] but input has static dimensions [1,3,768,1344].)

So i guess the input batch size of my images is still 1 while tensorrt needs the batchsize 2 to optimize for it.
I just have no idea how to change the input batchsize.
Could you give me a hint

[FHarhammer]

Any ideas what i could try?

[FHarhammer]

Any updates on this topic @lvhan028

[lvhan028]

1. add "is_batched: true" in

{
                "name": "maskrcnn",
                "type": "Task",
                "module": "Net",
                "input": [
                    "prep_output"
                ],
                "output": [
                    "infer_output"
                ],
                "input_map": {
                    "img": "input"
                },
                "output_map": {}
            },

2. There is a deploy.json file. Change the batch_size to 2
3. update to the latest mmdeploy and rebuild it.

[FHarhammer]

This worked! Thanks a lot for your help! @lvhan028

[mydhui]

@FHarhammer @lvhan028 Can you share where to change to parameters? because json files are dumped info not input config

[Harhammer]

@mydhui if i remember it correctly the is_batched parameter is in pipeline.json and the batch_size as Ivhan028 already said is in deploy.json. I dont really get your last sentence but i hope this helps already