Camera stops sending IMU frames python

[a7u7a]

---

Required Info
Camera Model	{ D435i }
Firmware Version	5.12.9.0
Operating System & Version	Raspbian Buster
Kernel Version (Linux Only)	5.4.72-v7l
Platform	Raspberry Pi
SDK Version	v2.38.1
Language	python
Segment	others

Issue Description

I'm experiencing an issue with the camera no longer sending IMU frames when Depth + Color frames are also enabled. This happens after about 20 seconds to 5 minutes of operation.
I think its the same issue that is happening here except my use case is in python.

This is the code that I am using:

import pyrealsense2 as rs
from signal import pause

def callback(frame):
    frame_name = frame.get_profile().stream_name()
    print(frame_name)

p = rs.pipeline()
conf = rs.config()
conf.enable_stream(rs.stream.accel)
conf.enable_stream(rs.stream.gyro)
conf.enable_stream(rs.stream.color, 1280 , 720, rs.format.bgr8, 30)
conf.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 30)
prof = p.start(conf, callback)

pause()

Things I have tried so far. (problem persists in all cases):

• IMU + color only
• IMU + depth only
• Two separate pipelines (I get: Could not open device failed to set power state)
• No imports (except pyrealsense2)
• Different USB cables

Some obs:

• I followed these installation instructions.
• Same issue happens when I use realsense-viewer: I get No frames received! warning after a few moments
• Measured the ASIC temperature after a few minutes of operation, never goes above 38ºc
• I have not yet tried the rs-multicam example
• Probably unrelated but also attempted to run rs-imu-calibration.py and upon following all the steps I get:

Would you like to write the results to the camera? (Y/N)y
Writing calibration to device.

Error: failed to set power state

[MartyG-RealSense]

Hi @a7u7a The issue with problems occurring when all three stream types are active (depth, color and IMU) affects both Python and C++. It is more common for the result to be 'No Frames Received' on RGB, though I have seen a small number of cases where depth or IMU frames were affected instead. The effects of the phenomenon are summed up in the case that you linked to.

#7391 (comment)

On Python, separating the pipelines into two (IMU on one, depth / color on the other) typically works well. The case that you linked to already has what I would consider the best reference for this though.

#5628 (comment)

Separating the pipelines has not worked for every Python user in this situation though. There is a Python callback script created by a RealSense Python user that was tested successfully by another Python user who was having problems streaming depth-color-IMU.

#5417

Edit: I see that you have already been to that script though.

#6370 (comment)

[MartyG-RealSense]

There was a case similar to your circumstances (Pi, D435i, Python, 'Failed to set power state' when using 2 pipelines) that was solved by adding a 500ms pause between the start of the pipelines.

#6120 (comment)

[a7u7a]

Many thanks for all the info! I will try your suggestion asap and let you know.

[a7u7a]

Hi MartyG,
I was able to start two separate pipelines by adding the 500ms delay. However, the issue of IMU frames not received after a while still persists.
So far the alternatives seem to be:

• Switch to a Jetson
• Use external IMU unit
  Both of these are non ideal so if you think there could be anything else worth trying I would really appreciate it!

[MartyG-RealSense]

The issue of No Frames Received when three stream types (IMU, depth, color) are enabled is a software related issue, so changing the hardware to a different computing device or IMU would likely not make a difference.

Could you try explicityly defining the IMU configuration please (format and frequency). I recall past cases where problems with IMU frame streaming disappeared when using lower frequencies (63 accel / 200 gyro instead of 250 accel / 400 gyro).

conf.enable_stream(rs.stream.accel, rs.format.motion_xyz32f, 63)
conf.enable_stream(rs.stream.gyro, rs.format.motion_xyz32f, 200)

[a7u7a]

Hi @MartyG-RealSense
Just attempted setting a lower sampling rate for the motion streams as you suggested and the problem still persists.

I understand the problem is in the software. Do you have an idea where it might be localized? Is it related to the camera's firmware, the SDK or the linux kernel? Might it be possible that using different cmake flags could solve it?

[MartyG-RealSense]

The Acrobotic guide is usually a reliable one for a Pi 4 installation of librealsense and Python, so it is unlikely that there is an error in that guide's instructions.

Its CMake instructions look fine and it builds with -DFORCE_LIBUVC=true (i.e the 'backend' method that bypasses the kernel) and so it is likely not a kernel related issue.

I have not seen the header instruction from signal import pause before in a pyrealsense2 script. I believe you have already tried removing this instruction though and the results have not changed?

If you also receive the No frames received message in RealSense Viewer when trying to stream IMU, depth and color together then this would indicate that the issue is not Python-related, since the Python wrapper has no involvement in the Viewer (which is C++ based).

I note that you have tried IMU + color and IMU + depth (i.e only two stream types at a time) and still had the problem. Normally the IMU would only cause the frame receival problem if three streams were activated (in any order). This makes me think that your problem might be different from the one that separate pipelines is usually a fix for.

Can you confirm please that the IMU is only ever on its own pipeline, and that depth and color are always put on the other pipeline (never on the same pipeline as the IMU).

[a7u7a]

Hi, I can confirm that all the separate pipeline attempts have been made with IMU on pipeline1 and Depth+Color on pipeline2.
As for the import header I am only using it to keep code to a minimum in the example when using callbacks. I have attempted without using signal many times.

[MartyG-RealSense]

I researched your case again from the beginning but it looks as though we have tried just about everything and exhausted all research leads, with just this one left:

#3460 (comment)

[MartyG-RealSense]

Hi @a7u7a Do you require further assistance with this case, please? Thanks!

[a7u7a]

Hi @MartyG-RealSense, we have dealt with the issue by using an external IMU unit. Many thanks for your help. I will close this issue now.

[MartyG-RealSense]

Thanks very much for the update @a7u7a - I'm pleased to hear that you achieved a solution :)