Realsense D400 series Cameras support?

[HappySamuel]

Hi

Can this rmf_obstacle_detectors work with any Realsense D400 series Cameras? If i wanna use D400 series cameras, what are the steps to integrate them up?

Best,
Samuel

[Yadunund]

There are several modalities of obstacle detection. Apart from rmf_human_detector_oakd, the other packages are hardware agnostic and perform inference based on ROS 2 sensor msg data. The realsense-ros driver can help publish RGB images which you can pass to rmf_human_detector. You may even be able to convert the depth pt cloud into a laser scan msg and pass that data to rmf_obstacle_detector_laserscan.

If you're willing to contribute code, I can also guide you on enhancing rmf_human_detector to
a) make it a more generalized detector (not just humans)
b) Incorporate (optionally) a depth stream to obtain 3D pose data of the inferences. Right now some heuristics are used to estimate depth from 2D RGB image.

[HappySamuel]

Hi @Yadunund

That would be great to have rmf_human_detector to be
a) more generalized detector
b) incorporate with depth stream

I am willing to contribute the code under your guidance on achieving this.

Best,
Samuel

[Yadunund]

Hi @HappySamuel sorry for the overdue response. I was occupied with other efforts. I should have more time for this towards the end of this month. Will put a plan together. Feel free to ping here again if you don't hear from me.

[HappySamuel]

Hi @Yadunund
Good to hear from you again. Looking forward to work with you together on this.

Best,
Samuel

[Yadunund]

@HappySamuel sorry for the long silence here.

Open-RMF is part of GSoC 2024 and one of the projects is to cleanup and improve the obstacle detection pipeline. You may want to checkout https://discourse.ros.org/t/attention-students-open-robotics-google-summer-of-code-2024-projects/

I've also opened #22 with a list of items to work on. The goal will be to support a wide range of sensors and detection algorithms via a plugin based system. Let me know if you're still keen on contributing.