-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Understand how to handle Eigen::Ref conversions of yarp Images #110
Comments
cc @traversaro @GiulioRomualdi @S-Dafarra would you have some pointers? |
What would be a classic usage of these objects? What would you do once you have a |
I use
I do so by dynamic casting the Now, in order to pass this image to the user through the ISensorBridge, the user has to call getRGBImage or getDepthImage which accepts Eigen ref as an input argument. In this case, I might have to do a necessary conversion from yarp::sig::Image to Eigen::ref. I wouldn't have to do any other manipulations.
With the current implementation of the YarpSensorBridge, nothing but only to hold it in an internal buffer until a getXXX() is called, when the conversion described above must be done. |
I see. I am not really familiar with those conversions. I guess that if the chain of conversions |
I gave them a superficial look. I will try to report them here soon. |
There is something I am missing in general about using However, sorry for being a bit late for #87 (comment), but I have a bit of a feeling that the design tradeoff for passing images around are quite different from the vector and matrices we are used in control. As an example, at least for the realsense the type of "channels" in the rgb image is not fixed, but can change at runtime https://github.com/robotology/yarp/blob/46c51b8872b984d2d587b20238d3e819dac1596e/src/devices/realsense2/realsense2Driver.cpp#L1010. An additional interesting observation that even in Drake, that as BLF tends to express as much as possible quantities using Eigen vectors, introduced a dedicate image class similar to the one used in YARP: https://drake.mit.edu/doxygen_cxx/classdrake_1_1systems_1_1sensors_1_1_image.html and https://github.com/RobotLocomotion/drake/blob/8e36e2e7bb17de3869432e77a07d39fb7a46adc6/systems/sensors/image.h . I wonder if there is any specific reason for which we need to handle images and similar data structures using the |
I think you made this point earlier. I had not understood it properly, or at least I thought we were talking about point clouds datatype only. But it does make sense to have a separate interface for images as you suggest. I shall refactor out handling the images out of the ISensorBridge. About the Eigen ref over |
Note that it may be worth also to have some prototype code that does not use an interface at all (i.e. use directly the source of images) and once that works, the requirement for the interface will be more clear. |
Within the ISensoBridge interface, we decided to pass image data using Eigen refs given the availability of suitable conversions to OpenCV that can be used by the client applications. (Please see #87 (comment)) for the discussion. In short, OpenCV offers to and fro conversions with Eigen Ref (https://github.com/opencv/opencv/blob/e3d502310f6ae0cad79f3474b9ea2b25705985f9/modules/core/include/opencv2/core/eigen.hpp).
Within the YARP implementations, we handle the images using
yarp::sig::Image
and YARP's Eigen helpers do not offer suitable Eigen ref conversions for Images, see https://www.yarp.it/namespaceyarp_1_1eigen.html.(YARP offers OpenCV conversions if the usage of OpenCV is enabled. However, we want to circumvent the dependency of OpenCV in any case, so this is not a suitable option)
We need to understand these conversions to complete the missing implementation for getting images in YarpSensorBridge. #106
The text was updated successfully, but these errors were encountered: