YOLOv8-NAS 3D

[imdsafi09]

Hi,
I was recently exploring YOLO-NAS models and came across your GitHub yolonas_ros, which you already integrated with ROS (highly appreciated). If I have to use YOLO-NAS in this 3D object detection framework, would I need to change the weight files and configuration files from yolov8 to YOLO-NAS, or would a more comprehensive modification be required?

Thanks

[mgonzs13]

Hi @imdsafi09, YOLO-NAS is supported since 3.3.0. Here you have an example of launching YOLO-NAS. You can use a modified version of the 3D launch by changing the model_tye to NAS and the model If you want to use YOLO-NAS with the 3D version.

[imdsafi09]

Thank you, @mgonzs13; I highly appreciated your quick response.

[imdsafi09]

hi @mgonzs13

I ran successfully with YOLO-NAS, too; thank you again. However, as you can see in the screenshot below, the 3D bounding boxes' shape, especially height, increases according to the 2D bounding boxes. It makes sense just for visualization, but when I integrate it with the SLAM, the bounding box size goes out of the map. I tried to put a constant value of z-axis when creating 3D bound boxes detect_3d_node.py, it works somehow, but I am just wondering this the right way or there is any better solution?

Note: This is not related to YOLO-NAS

[mgonzs13]

This may happen because the 2D bounding boxes include sections of the background of the detected object. You can try instance segmentation to create the 3D bounding boxes using YOLOv8 ros2 launch yolov8_bringup yolov8_3d.launch.py model:=yolov8m-seg.pt.