Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add example showing streaming of large data over datagrams performantly #190

Open
jan-ivar opened this issue Feb 4, 2021 · 4 comments
Open
Assignees

Comments

@jan-ivar
Copy link
Member

jan-ivar commented Feb 4, 2021

Datagrams are the lowest-level building blocks we expose. But are they as performant as streams?

From #105: "imagine implementing QUIC streams (or TCP) on top of DatagramTransport."

If someone wants to send megabytes of data over datagrams, using their own homerolled framing scheme and back-channel for dealing with packet loss, can we support that?

If so, the examples on sending datagrams seem lacking here, not illustrating the necessary chunking of large data into transport.maxDatagramSize (with some custom framing needed obviously) and piping to datagrams using e.g. a pull-based ReadableStream to send at the max rate the user agent can do.

I did a comparison in Canary, of a mock file upload over a bidirectionalStream using a tiny chunk size matching datagrams vs. a mock file upload over datagrams without any kind of framing or packet loss recovery.

Uploading a 4.4 megabyte file took 2.3 seconds with the former and 17 seconds with the latter. 🤔

Should we add this use case to ensure this will be performant? Or do datagrams have some inherent performance limitation?

@jan-ivar
Copy link
Member Author

jan-ivar commented Feb 4, 2021

Heh, turned out it was an output problem. Removing the server output gives me comparable speed: 2.3 seconds. 😊

Phew, good news. We should add the use-case then I think.

@jan-ivar
Copy link
Member Author

jan-ivar commented Feb 4, 2021

I should add this is still much lower performance than a mock file upload over bidirectionalStream with 16k chunks at 0.2 seconds. But there are likely JS reasons for that. E.g. I should probably use something more performant than blob.slice.

@aboba
Copy link
Collaborator

aboba commented Feb 4, 2021

Jan-Ivar said:

"If someone wants to send megabytes of data over datagrams, using their own homerolled framing scheme and back-channel for dealing with packet loss, can we support that?"

[BA] I hope so, because realtime communications use cases will depend on this (e.g. game streaming can consume upwards of 20 megabits/second).

@jan-ivar jan-ivar self-assigned this Feb 16, 2021
@jan-ivar
Copy link
Member Author

I'll try to provide a PR to discuss for next meeting.

@wilaw wilaw added the Discuss at next meeting Flags an issue to be discussed at the next WG working label Feb 17, 2021
@jan-ivar jan-ivar removed the Discuss at next meeting Flags an issue to be discussed at the next WG working label Mar 10, 2021
@yutakahirano yutakahirano added this to the No mileston milestone Jun 30, 2021
@jan-ivar jan-ivar changed the title Use case: stream large data over datagrams your own way with high throughput Add example showing streaming of large data over datagrams performantly Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants