Old frames returned from wait_for_frames

[kogan-alex]

---

Required Info
Camera Model	D415
Firmware Version	05.12.06.00
Operating System & Version	Ubuntu 18.04
Platform	PC
Language	Python
Segment	Robot

Issue Description

I'm recording frames from a D415 using "pipeline.wait_for_frames()". It seems as though the returned frames are older than the time the "wait_for_frames" command was issued by the Python code. This is inconsistent with the documentation that states that "Device frames, which were produced while the function wasn't called, are dropped". Did I misunderstand the documentation? Do I need to insert a manual sleep to avoid this? I need for the frame to be taken after the command was issued.

[MartyG-RealSense]

Hi @kogan-alex I wonder if what you are describing is a phenomenon that has been referred to as stale frames.

#5885

[kogan-alex]

Hi! Thanks for the reply.

1. FW version is up to date: 05.12.06.00
2. Auto exposure is turned off.
3. How can I check for frame drops?

[MartyG-RealSense]

The link below discusses FPS rates and has some Python scripting for checking it.

https://support.intelrealsense.com/hc/en-us/community/posts/360037286634-Strange-FPS-when-enable-both-color-and-depth-streams

I believe that the relevant line in the script for printing the FPS is:

print('Frame num: %d, fps: %.2f' %(frame_count, frame_count/total_elapsed))

[MartyG-RealSense]

Hi @kogan-alex Do you still require assistance with this case, please? Thanks!

[kogan-alex]

Currently implemented a workaround, when I'll get around to this issue I would appreciate your help

[MartyG-RealSense]

Thanks very much for the update. I will keep this case open for a further time period.

[ev-mp]

@kogan-alex hello, the timestamps of the acquired frames shall and will be older than the time wait_for_frames call was made. This is actually the envisaged behavior of the system.

This is inconsistent with the documentation that states that "Device frames, which were produced while the function wasn't called, are dropped". Did I misunderstand the documentation? - the frames are processed internally in asynchronous mode. I.e. the call wait_for_frames does not activate the processing pipeline in Host/Device but is merely a consumer asking for the latest frame(set) generated by the library.
See similar #8112

[MartyG-RealSense]

Hi @kogan-alex Do you require further assistance with this case, please? Thanks!

[kogan-alex]

I don't understand, why then, does the documentation state that "old frames are dropped"?

[MartyG-RealSense]

Hi @kogan-alex You are referring to the quote from pyrs_pipeline.cpp, yes?

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/pyrs_pipeline.cpp#L141

[kogan-alex]

I was referencing the documentation of wait_for_frames in: https://intelrealsense.github.io/librealsense/doxygen/classrs2_1_1pipeline.html#ac77cdd3d2f4ce1e85ea30d475743cec8
But your link seems to have the same quote.

[MartyG-RealSense]

@ev-mp further explains the difference between syncronous vs asynchronous in regard to wait_for_frames() the link below,

#3817 (comment)

I believe the gist of it is that callbacks are Asynchronous and wait_for_frames() is Synchronous, and the two cannot be mixed together interchangeably.

This subject can be best explained by @ev-mp rather than myself.

[MartyG-RealSense]

Hi @kogan-alex Do you still require assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.

[YotamIshay]

hi, lately we've seen this phenomenon again (Alex and I are on the same team), but this time our workaround didn't help.
we've received a frame that is more than 1 second old, meaning - wait_for_frames() returned a frame representing a scene that happened >1 second prior to our call for wait_for_frames().
this is a very big problem for us. can you pls help? we don't fully understand how this is possible. does it mean that no new frame was streamed for more than 1 second? (we're working in 6 FPS)

[MartyG-RealSense]

Hi @YotamIshay I would first recommend trying 15 FPS to see whether the problem still occurs. This will help to confirm whether or not the frame delivery is timing-out at 6 FPS due to new frames not arriving fast enough.

[YotamIshay]

hi @MartyG-RealSense, thx for the response. unfortunately we can't try 15FPS due to USB bandwith issues (we're connecting several cameras to a USB hub). I must say I'm not sure I understand how this might help though - isn't there a way to directly check the root cause here? is there an upper bound for the delay the camera might have from the desired FPS? I'd assume an error would be raised in case of a very large frame delivery timeout, but I guess this isn't the case :/
would appreciate any input you may have regarding this.

[MartyG-RealSense]

You may be able to get around the bandwidth issue by connecting another USB hub and putting two of the cameras on it, as the USB standard allows up to 5 hubs to be linked together on the same computer.

If it was a prolonged timeout then you would likely experience an error such as 'Frames didn't arrive within 5000' or 'Frames didn't arrive within 15000'. A short FPS lag and then recovery may not show up as an exception though.

As a rough guide to performance, you could launch the RealSense Viewer, open the debug console with the upward triangle icon in the bottom corner of the Viewer, and then enable depth streaming and view the Frame Drops Per Second graph beside the log.

You can also click the 'i' icon above the stream panel to overlay a real-time display of the current FPS to see whether it is noticably varying.

If the FPS is varying and you have auto-exposure enabled then you could try disabling the RGB option Auto-Exposure Priority to enforce a constant FPS rate instead of allowing the FPS to vary.