Using Isahc in a multi-threaded web server

[kathampy]

I understand that curl's multi interface executes all requests in a single thread. So even though Isahc's HttpClient can be called from multiple threads, is it ultimately CPU-bound to single-threaded performance?
I'm going to make outbound API calls using HttpClient asynchronously within each request hitting my web server. Should I create an instance of HttpClient for each worker thread in actix-web so that curl can use all CPU cores?

[sagebind]

Hmm, good question. It is true that curl is unable to take advantage of multiple cores, so it might be possible to squeeze more performance out by creating 1-2 clients per core. Though in practice I am not sure, it might be worth benchmarking the difference first. Note that a client uses things like connection pooling and HTTP/2 multiplexing to improve performance for multiple requests to the same destination, so it might be possible that under certain circumstances using a separate core reduces the CPU time, but increases the network time since there's less of the aformentioned optimizations taking place.

[kathampy]

In an asynchronous crate like reqwest, there is no agent thread per client, and all events are handled on the executor thread already being used by the application (actix-web worker in my case). Is the agent thread absolutely required for the curl multi interface, or is it technically possible to write isahc in a way that only uses the executor thread it's called from? According to the curl multi documentation, the only thing that will block in HTTP is DNS resolution when not using C-ARES, which isn't the case here.

[sagebind]

While libcurl is Isahc's greatest strength, it is often its greatest weakness as well. Using a background thread was a compromise that seemed like the only way to interop with futures considering curl's multi interface without extereme hacks.

The core problem is that a curl multi handle is essentially !Send. All easy handles attached to the multi handle must not be used except through the attached multi handle to drive the transfer. Since curl_multi_perform does all the CPU work for all handles in one call, it isn't possible to parallelize this work across multiple threads, and it isn't possible to split each request into its own future that an executor can poll. Either all the requests make progress, or none of them.

Now you could assign every request its own multi handle, allowing you to make incremental progress for each request separately (and could ultimately tie into something like a Tokio executor), you would have no sharing of resources between each request, and would lose connection pooling and reuse, DNS caching, etc. (It may be worth noting that this is how Isahc 0.1 worked.)

Even so, you could still get these things back by sharing data between all the handles, but then you have to introduce mandatory mutexes for these things, which would be locked by any one request at any one time, potentially lowering the throughput that you sought to gain.

A second, totally separate problem is that curl is incompatible with vanilla IOCP on Windows, which mio uses. Practically every futures executor I've seen that uses an I/O selector loop uses mio, and part of mio's contract is that you must use mio's versions of read() and write() in order to maintain correctness. Now it so happens that generally on UNIX platforms these just defer to the normal read() and write() functions, but when using IOCP you must use WSASend() and WSARecv() (if I recall), and curl does not allow you to overload which read/write functions to use.

In other words, curl is currently incompatible with mio on Windows, and in turn, incompatible with 99% of reactors out there (like Tokio) that use mio.

All this to say that, yeah, having a separate thread kinda sucks for some scenarios, but it's really the best option with respect to compatability, safety, and complexity. Really the only downsides are: (a) doesn't work on systems without thread support, and (b) can't parallelize any CPU-bound work that curl does.

It might be worth looking at the tokio-curl crate for another approach to integrating curl. Note that it is specifically tied to Tokio (can't use it with an arbitrary executor) and that the Windows implementation basically does what Isahc does.

[sagebind]

Closing this, as I think the question has been answered. Feel free to re-open if you have follow-up questions!