Sorting threads by most recently responded to

[clokep]

It is currently difficult for clients to sort threads by the most recently responded to. Clients use the /messages API with a filter of "relation_types": ["m.thread"] to fetch the list of threads in a room. This returns the root thread events in topological order of those events (either forwards or backwards depending on the dir parameter). Note that each event also includes bundled aggregation, which will include the latest event in each thread.

In order to sort threads by the latest event this requires clients to paginate all of the threads in the room, inspect the latest event from the bundled aggregation and attempt to sort by something. There's two issues with this approach:

1. Using information available to clients (e.g. age or origin_server_ts) doesn't result in the proper ordering (?)
2. It requires pulling all threads in a room, which can result in many round trips to the server as it requires paginating the entire results.

Ideally the /messages API could return the results in the expected order, by sorting by the latest thread relation message before returning the events.

Potential work breakdown

1. Write an MSC describing this behavior.
2. Implement the MSC in Synapse.
3. Implement the MSC in clients.

[clokep]

From some discussion elsewhere -- there aren't any APIs in Matrix where a client can modify the sort order of the results (except reversing the result via backwards/forwards flags). This makes one wonder if it would be better to provide a separate API for this.

Technically this may be interesting as the server will need to track room ID -> thread roots -> latest event in thread. Hopefully that can be pulled out from the existing data, however!

[t3chguy]

I wonder how this will work given the order will be constantly changing, due to redactions and new threads responses coming in, so backpaginating this custom ordered mixed timeline may race with /sync bringing in new events causing order confusion

[clokep]

Another meta issue was filed for this, see #195. I'm closing this and opening a separate Synapse one: matrix-org/synapse#13921.