Warn about aggregate possibly consuming a lot of RAM?

[vorner]

Hello

I'm wondering about one thing. Let's say I have a web server and it wants to get the whole body sent by the user in POST. The aggregate would be a good candidate for that. However, I think there's no way of protecting against the client sending excessive amounts of data and consuming all my RAM that way (at least, I've found none).

I could check the Content-Length header first, before calling this function. However, if the client is sending with chunked encoding, the header is optional. That makes the function (and its sibling to_bytes) potential DoS hazard.

Therefore I wanted to ask if there should be some kind of aggregate_with_limit function, or if the documentation should at least include a warning and point out the fact it shouldn't be used when exposed to untrusted clients.

[seanmonstar]

Yes, it's possible to use without protection, and that could be problematic. I see it kind of similar to Read::read_to_end. That doesn't include a warning. That said, I see nothing wrong with including a warning in the documentation.

My personal view is that an aggregate_with_limit is less useful. When creating servers, I've always required that data/uploads have a content-length header, with a maximum, so that I can know before having to buffer any of the body whether I'd want to accept it. If I was accepting chunked request payloads, I wouldn't be buffering it fully, but rather streaming it somewhere else.

So, I'd be fine with adding a warning, but otherwise I'll just close this.

[vorner]

I see it kind of similar to Read::read_to_end

I'd say situations where I can trust local files are much more common than trusting something on the other side of the Internet. And sometimes, one has no option than just support arbitrary clients.

So, I'd be fine with adding a warning, but otherwise I'll just close this.

Thanks :-).

[seanmonstar]

I can trust local files

Yes, certainly. Though, Read isn't only implemented for File, but TcpStream and who knows what else. 🤷