Add a yield primitive for computation-heavy tasks

[Ekleog]

https://tokio.rs/docs/getting-started/tasks/ mentions “Tasks must not perform computation heavy logic or they will prevent other tasks from executing.”

However, sometimes heavy computation must be performed for a task (to take the example from said web page, a server that returns the n-th fibonacci sequence number). Spawning a new thread for each such request wouldn't be efficient, and handling a separate thread pool of workers wouldn't be simple.

So I think maybe a primitive like yield!(), that would return Pending and immediately re-enqueue the future for further progress, might make sense: it could be called at times during the computation-intensive task, so that other tasks aren't blocked from progress. Obviously it requires that the executor be fair for this to work, ie. a yield-ed task should be enqueued at the end of the to-be-run task queue, not at the beginning.

What do you think about this?

[tobz]

This sounds like an interesting idea!

Just thinking it through, we can obviously write a macro for this, and I think it could be as simple as:

macro yield! {
    task::current().notify();
    return Ok(Async::NotReady);
}

That's obviously not syntactically accurate, but, I think it would work?

Random thoughts:

• if a task is running and is notified via its task handle, is that notify a no-op or is it still queued up for a notification on the next tick of the reactor?
• what does the notification queue look like for notified tasks? could it already be FIFO ordering? is it randomized?

[tobz]

And, just in case others stumble on to this: this would clearly not be a true coroutine with stack resumption, and all of that. A future would still need to maintain state, but as long as it did that, it could co-operatively yield itself to the runtime.

[Ekleog]

Hmm, I'm not sure about the possibility to return directly Ok(Async::NotReady) (because it'd need to return NotReady only once, so would need to store a bit of state). Also, it'd need to resume execution to where it was currently running, in order to not have a way too complex control flow?

With await! it becomes easy, though: just do await!(NotReadyOnceThenValidate()), where NotReadyOnceThenValidate is a future that does the context.notify(); if self.alreadyCalled { Ok(Async::Done(())) } else { self.alreadyCalled = true; Ok(Async::Pending) }. And when not having await!, just do NotReadyOnceThenValidate().and_then(|_| …).

Though, thinking of it… maybe that's actually something that should be implemented in futures-rs?

[tobz]

From a pure coroutine perspective, yeah, this wouldn't be (easily) possible.

I do think there's a valuable, and approachable, technical outcome from your original problem state, though. A future is just a normal type that implements the Future trait. There's no reason it can't internally store state, and do incremental work.

I can totally imagine a future with a list of work items, and every time it polls, it just pulls off items from the list, and after some condition, conditionally yields itself.

[tobz]

Also, regarding your comment about only returning Async::NotReady once, maybe you're working off of some futures 0.2/0.3 documentation, but... that is exactly how futures today signal that they aren't yet complete, and they can return it over multiple poll calls.

[Ekleog]

Oh yeah, so I was thinking about the conditional yield as being in a more coroutine-like fashion within async/await, hence the need to yield once and then not yield (in order to actually execute the code that comes after). I guess these are two different designs, mine being like yield_once and yours being like yield_unconditional.

[tobz]

Yeah, I think I see what you mean now.

I'm certainly not the dictator of design vision in Tokio, but I do think this would be an interesting tool to have in the toolbox, so to speak. Could be useful in situations where you're doing heavy work, but aren't looking to actually run an entirely separate thread pool, etc.

Up to you whether you want to close it or not. I think it'd be great to flesh out an example of the thing, one way or another, to build a more meaningful discussion.

[Ekleog]

Leaving open to track the idea, then :)

[carllerche]

Both are useful. The macro is useful when implementing futures by hand. And yes, one would have to track extra state.

For await! Having a future variant also does seem useful.

The question where should these two helpers live and how should they be differentiated.

[carllerche]

task::yield_now().await is on master.