Sliding sync: The Two Sync Loops

[poljar]

Push notifications

Push notifications are used to deliver information about messages relevant to the user in a timely manner as a OS notification.

The mobile device receives a room ID and event ID from the push service. The Client on the device then determines whether to convert this data into a visible OS notification.

Since the room ID and event ID are not enough to display a notification, more data needs to be fetched from the homeserver before doing so. To display all the relevant data we'll need to sync with the server. Push notifications on Apple devices are handled by a separate, resource limited, process. From now on called the "Notification process". It cannot wake up the main process and must independently retrieve any additional data.

In cases where an encrypted event triggers the notification, the presence of room keys may be necessary for event decryption. Room keys are delivered as to-device events to each recipient device.

To fulfill this requirement, the Notification process must initiate a /sync request. This request may fetch more data, including the room display name calculated by the SS proxy, the member event of the sender, and the event itself.

Client setup for multiple process sync support

At present, there is no support for multiple processes making changes to the Client database simultaneously. The following flowchart models modifications to our Client and its sync loop setup which aim to allow this.

flowchart TD
    homeserver[Homeserver]

    subgraph client[Main process]
     direction LR
     main[Main sync]
     to-device[Extension sync]
    end

    nse[Notification process]

    main <==> homeserver
    to-device <==> homeserver
    nse <==> homeserver

Loading

The main process will split out its sync loop into two parts. The main part, which is responsible to sync room data and events the client wants to display right away, and the extensions part (or only to-device) part, where we mainly sync to-device events.

This solves two problems:

1. The main sync, getting reset by changing sync settings because of user interaction, is not blocking to-device and room key delivery anymore.
2. If another process is doing the extensions sync, the main sync does not need to wait for the process to be done.
3. Prevent the state where two sync loops are fighting for to-device events, since those are supposed to be delivered only once.

Main process client flow

The following flow chart models how the main process should behave when setting up a sync loop.

flowchart TD
    A[Main process] --> B(Start the sync streams)
    B --> C(Main Sync stream)
    C --> E(Sync once)
    E --> E
    B --> F(Extension sync stream)
    F --> G{Acquire the extensions sync lock}
    G -->|fa:fa-lock-open Lock acquired| H(Sync once - extensions)
    G -->|fa:fa-lock Can't acquire lock| I(Wait)
    I --> F
    H --> H

Loading

It would probably be nice, or even required, to stop the sync to release the "Extensions sync lock" when the main process gets suspended by the OS.

Notification client flow

The notification client on the other hand should behave a bit differently. It mainly should not set up a long running sync loop. It should sync only to fetch the required data which is necessary to display the notification.

flowchart TD
    A[Notification process] --> B(Received a push)
    B --> C(Start up the Client)
    C --> D{Acquire the extensions sync lock}
    D -->|fa:fa-lock-open Lock acquired| E(Sync once with to-device enabled)
    D -->|fa:fa-lock Can't acquire lock| F(Sync once with to-device disabled)

Loading

Tasks

• Split out the sliding sync loop into two loops #1961
• Create a "Notification service" specific client which supports only a subset of the whole sync logic #1962
• sliding sync: Always infer the storage key from a sliding sync loop identifier #2007
• sliding sync: coordinate shutdown across instances #2014
• Add a value-based lock in the CryptoStores #2049
• feat: make OlmMachine resettable #2091
• tests: add unit tests for the CryptoStoreLock #2167
• feat: implement time lease based locks for the CryptoStore #2140
• feat: implement a generation counter for the CryptoStore lock #2155

Related but not mandatory for this

• Group Session Cache inconsistency between IOS main app and Share Extension #1960
• Change the store traits so they expose a transactional API #2000