issue with wait_for_frames

[TheNemo05]

Hello there,

I ve attached my python code. The thing is i can observer that the camera is working but the error is displayed RuntimeError: Frame didn't arrive within 5000

import open3d as o3d
import pyrealsense2 as rs
import numpy as np

#configure realsense camera
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.accel, rs.format.motion_xyz32f, 100)
config.enable_stream(rs.stream.gyro, rs.format.motion_xyz32f, 200)

#Start Streaming
pipeline.start(config)

#create Open3D visualizer
vis = o3d.visualization.Visualizer()
vis.create_window()

#Create Open3D pcd
pcd = o3d.geometry.PointCloud()

try:
while True:
#Wait for frames
frames = pipeline.wait_for_frames(timeout_ms=10000)

    #Get RGB & Depth frames
    color_frame = frames.get_color_frame()
    depth_frame = frames.get_depth_frame()

    #Get IMU data
    imu_frame = frames.first_or_default(rs.stream.accel)

    #convert Realsense frames to open3d 
    color_image = o3d.geometry.Image(np.array(color_frame.get_data()))
    depth_image = o3d.geometry.Image(np.array(depth_frame.get_data()))
    rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(color_image,depth_image)

    #convert IMU data to open3d format
    imu_data = np.asarray(imu_frame.as_motion_frame().get_motion_data())

    #update pcd with rgbd and imu data
    pcd = pcd.create_from_rgbd_image(rgbd_image, o3d.camera.PinholeCameraIntrinsic(o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
    pcd.transform(np.linalg.inv(imu_data))

    #update open3d visualizer
    vis.clear_geometries()
    vis.add_geometry(pcd)
    vis.update_geometry(pcd)
    vis.poll_events()
    vis.update_renderer()

    print("Events polled and renderer updated")

except KeyboardInterrupt:
pass

#stop Streaming
pipeline.stop()

#save pcd to file
o3d.io.write_point_cloud("output.pcd",pcd)

[MartyG-RealSense]

If Depth, Color and IMU are enabled simultaneously then this can cause no frames to be received on one of the streams, often the color stream. The trigger for the issue is the enabling of the IMU stream.

A workaround for this problem in Python is to create two separate pipelines and put the IMU on its own on one pipeline and depth + color on the other pipeline. An excellent example of code for doing so is at #5628 (comment)

[MartyG-RealSense]

Hi @TheNemo05 Do you require further assistance with this case, please? Thanks!

[jrecasens]

I was having a similar issue in Windows with Pycharm. When using "Execute selection in Python console", when depth stream was enables the IMU reception failed. The same RuntimeError: Frame didn't arrive within 5000

Running the entire script works (Run your script.py) because it ensures proper initialization in a clean, isolated environment. For hardware tasks, run the full script. This is super strange, it is the only way I can actually run code when enabling gyro (or accelerometer)

    conf.enable_stream(stream_type=rs.stream.gyro,
                       format=rs.format.motion_xyz32f,
                       framerate=200)

Now, I if I also enable gyro AND depth stream, I dont get past the frames = pipeline.wait_for_frames(timeout_ms=10000) .

[MartyG-RealSense]

Hi @jrecasens Whilst depth only and IMU only can work well, combining depth and IMU streams in the same script can result in problems unless depth and IMU are put on two separate pipelines, like in the Python script at #5628 (comment)