D415: pyrealsense2 is giving out stale frames or repeated frames

[redM0nk]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	D415
Firmware Version	05.12.02.100
Operating System & Version	Linux (Ubuntu 16)
Kernel Version (Linux Only)	4.15.0-88-generic
Platform	PC
SDK Version	2.32.1
Language	python
Segment	Robot

Issue Description

Hello realsense community

I'm facing an issue where a D415 realsense camera is giving out stale frames (frames with an old timestamp).

The following code captures the frames in 5 seconds and prints the summary on the terminal:

pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.color, 1280, 720, rs.format.rgb8, 30)
config.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 30)
pipeline.start(config)
start_time = time.time()
num_frames_captured, num_stale_frames = 0, 0
color_ts, depth_ts = None, None
while (time.time() - start_time) < 5:
    frames = pipeline.wait_for_frames()
    color_frame, depth_frame = frames.get_color_frame(), frames.get_depth_frame()
    if color_ts == color_frame.timestamp or depth_ts == depth_frame.timestamp:
        num_stale_frames += 1
    num_frames_captured += 1
    color_ts, depth_ts = color_frame.timestamp, depth_frame.timestamp

print('Summary:\n# Total frames: {}'.format(num_frames_captured))
print('# Stale frames: {}'.format(num_stale_frames))

Output:

Summary:
# Total frames: 115
# Stale frames: 51

Any help is appreciated.

[dorodnic]

Hi @redM0nk

There could be two possible reasons for this:

1. The RGB camera by default is controlled by auto-exposure algorithm and can decide to reduce its FPS (when the environment is sufficiently dark). You can enforce the FPS you requested by setting RS2_OPTION_AUTO_EXPOSURE_PRIORITY to 0 on the second sensor.

p = pipe.start(cfg)
p.get_device().query_sensors()[1].set_option(rs.option.auto_exposure_priority, 0.0)

2. If RGB FPS is fixed, pipeline will provide stale frames in case of frame drops. This can sometimes happen due to high CPU load, but should not be widespread

[RealSenseCustomerSupport]

---

@redM0nk Any other questions about this with the reply from dorodnic? Looking forward to your update. Thanks!

[redM0nk]

@dorodnic Thank you for the reponse.
I tried the above mentioned solution and it is giving me some error:
RuntimeError: object doesn't support option #30

The current camera configuration is as follows:

Resolution: 1280x720
Fps: 30
RGB Exposure: 32ms
Depth Exposure: auto

Upgrading the camera to the latest firmware 5.12.3.000 and installing pyrealsense2 using the source code definitely helps.

At present, I'm noticing another issue with timestamp delta (system timestamp - frame timestamp) of 40-70 ms.
Any help with the issue is appreciated.

[RealSenseCustomerSupport]

---

@redM0nk The error "RuntimeError: object doesn't support option #30" is due to the sensor you queried is not RGB sensor. Please add the query for RGB sensor.
And please create another one to track the timestamp issue, logs will be appreciated. Thanks!

[RealSenseCustomerSupport]

---

@redM0nk Any other question about this? Looking forward to your update. Thanks!

[redM0nk]

thanks. i was able to set the RS2_OPTION_AUTO_EXPOSURE_PRIORITY option on the RGB sensor.
as of now I'm not noticing any more issues with stale frames.
summary:

1. move to the latest firmware,
2. compile realsense library from source code,
3. use appropriate settings (mentioned above)