Image visualization / LED Strip

[rafal-gorecki]

Hello,
Is there currently an option to visualize camera data like in gazebo classic?

[azeey]

I know there's an "Image Display" plugin you can use. Is that sufficient?

[rafal-gorecki]

Hello @azeey,

Thank's for replay.

What I really care about is simulating an LED strip.
I was planning to make a clever workaround that instead of writing the entire functionality from scratch, I would take a camera and display the image from the camera on the robot, but i notice this function is not available for gazebo ignition.

If you have any advice on how to start your adventure with such a task, I can try to do it. However, I have little experience when it comes to writing the gazebo code itself, so I would like to ask for a little mentoring and if you have and idea how to start this project what to base on.

[azeey]

What I really care about is simulating an LED strip.

I'm not quite sure what you want to do. Maybe a Projector would help?

[rafal-gorecki]

Thank you @azeey, Projector looks interesting.

I would like to explain even better what I have in mind when they write an LED strip. On the robot, I would like to add a link that represents the LED strip. The difference between a projector and an LED strip is that the projector sends an image onto the space it encounters. However, the LED strip would be a permanent element that can be attached to the robot and which displays the image from the topic.

[bperseghetti]

@rafal-gorecki Don't think that would be a great way to show LEDs IMO, you can actually "control LEDs" easily if you are on harmonic.

Here is one example showing spotlights from various object:

MaterialsLights.mp4

Here is how we control LEDs one our ground based vehicles for status lighting (seamlessly through the posix of our Zephyr RTOS based autopilot the same way the real LEDs are controlled):

vehicle_status_lights.mp4

[rafal-gorecki]

Hello @bperseghetti and thank you,
That's look great! Could you share a demo that can be inspired or what's the name of this plugin?

[bperseghetti]

Hello @bperseghetti and thank you, That's look great! Could you share a demo that can be inspired or what's the name of this plugin?

I sorta hate "one off" plugins, so I extended the use of material color and lights to make it far more versatile than a plugin IMO (it also works with ros_gz bridge):

• LEDs defined in model: https://github.com/CogniPilot/b3rb_simulator/blob/7830685f9b9c3aca2105ee9e149969d2db696350/b3rb_gz_resource/models/b3rb/model.sdf#L111-L601
• Control of LEDs from SIL side of RTOS controller: https://github.com/CogniPilot/cerebri/blob/5e9375ea18a6b0b9cccaea2551c1916d8ef89d07/lib/dream/sil/native_main.c#L225-L240
• Converted into gz-transport to set lights/material color from led_array:
  https://github.com/CogniPilot/synapse_gz/blob/d726b0c83d43ef246f7a0f82fcc1be11248057a2/src/link/udp_link.cpp#L149-L209

[rafal-gorecki]

Thanks a lot @bperseghetti. I will try this and let you know about results. ;)

[bperseghetti]

Also if you want you can run this in an ipython notebook for ros_gz based control, just make sure to change world name in it (from default to whatever your world name is) and change the name of the entities you want to control:

import rclpy
import numpy as np
from rclpy.node import Node
from ros_gz_interfaces.msg import MaterialColor, Light
from rclpy.qos import QoSProfile
from rclpy.qos import qos_profile_sensor_data
# ros2 run ros_gz_bridge parameter_bridge /world/default/light_config@ros_gz_interfaces/msg/Light@gz.msgs.Light
# ros2 run ros_gz_bridge parameter_bridge /world/default/material_color@ros_gz_interfaces/msg/MaterialColor@gz.msgs.MaterialColor
class lmTester(Node):

    def __init__(self):
        super().__init__('lm_tester_node')
        
        self.MaterialColorTopic = '/world/default/material_color'
        self.lightTopic = '/world/default/light_config'
        timer_period = 0.05
        self.timer = self.create_timer(timer_period, self.timerCallback)
        self.i=0
        self.delta=0.75
        self.ranges2=np.hstack((np.linspace(1-self.delta,1+self.delta,25),np.flip(np.linspace(1-self.delta,1+self.delta,25))))
        self.MaterialColorPub = self.create_publisher(MaterialColor,'{:s}'.format(self.MaterialColorTopic), 1)
        self.lightPub = self.create_publisher(Light,'{:s}'.format(self.lightTopic), 1)
        self.last_button = 0
    
    def timerCallback(self):

        if self.i < 49:
            self.i+=1
        elif self.i >= 49:
            self.i=0
        val2=self.ranges2[self.i]

        msgM=MaterialColor()
        msgM.entity_match=1
        msgM.entity.name="led_0"
        msgM.diffuse.r = 1.0
        msgM.diffuse.g = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgM.diffuse.b = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgM.diffuse.a = 1.0
        msgM.emissive=msgM.specular=msgM.ambient=msgM.diffuse
        self.MaterialColorPub.publish(msgM)
        msgM.entity.name="led_1"
        self.MaterialColorPub.publish(msgM)
        msgM.entity.name="led_2"
        self.MaterialColorPub.publish(msgM)
        
        msgL=Light()
        msgL.name="led_0"
        msgL.type=1
        msgL.diffuse.r = 1.0
        msgL.diffuse.g = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgL.diffuse.b = .1-.1*np.sin(2*np.pi*self.i/49+np.pi/2)
        msgL.diffuse.a = 1.0
        msgL.specular=msgL.diffuse
        msgL.spot_inner_angle = 1.25+np.sin(2*np.pi*self.i/49)
        msgL.spot_outer_angle = 1.5+np.sin(2*np.pi*self.i/49)
        msgL.spot_falloff = 0.0
        msgL.attenuation_constant = 1.0
        msgL.attenuation_linear = 1.0
        msgL.attenuation_quadratic = 0.0
        msgL.intensity=5.0
        msgL.range=100.0
        self.lightPub.publish(msgL)
        msgL.name="led_1"
        self.lightPub.publish(msgL)
        msgL.name="led_2"
        self.lightPub.publish(msgL)

        return 
            
if __name__ == '__main__':
    rclpy.init()
    lmMT = lmTester()
    rclpy.spin(lmMT)
    lmMT.destroy_node()
    rclpy.shutdown()

[rafal-gorecki]

Hi @bperseghetti,

What version of ROS are you running?
Currently I am using Humble to which I have installed the gz_sim in the Harmonic version. I have a problem uploading an image using ros_gz_bridge, the data is not transferred to the simulator.

Did you build ros_gz_bridge from source or do you run your project some other way?

[bperseghetti]

I'm running Humble with Harmonic, look at using these to install with apt and make sure to get rid of any old ros_gz otherwise that would be for fortress or garden potentially:

sudo apt-get install ros-humble-ros-gzharmonic*

[bperseghetti]

But also I don't project an image to gz from ros only transport images from gz to ros.

[rafal-gorecki]

Hi @azeey I already checked markers. What do you think about creating such carmera image from markers?

[rafal-gorecki]

I use markers and it's work, but it seems that the solution is not the best, except for the computational costs.
The results look like this :)

Thank you @azeey and @bperseghetti for your support.

[scpeters]

related: a feature request for camera frustum visualization gazebosim/gz-sim#2401

[bperseghetti]

I use markers and it's work, but it seems that the solution is not the best, except for the computational costs. The results look like this :)

Thank you @azeey and @bperseghetti for your support.

Not sure if you want it on or off, but you can remove the green pyramidal bounding box on that visualization if you want to (as a heads up).

[azeey]

Closing in favor of gazebosim/gz-sim#2401