Replies: 3 comments 7 replies
-
Thanks for jumping in this in the project. I really appreciate the help here. So first let me clarify (and introduce some jargon) to make sure that all the WP contributors are aligned and understand here. When you say save document, for WordPress that corresponds to the different "entities" that can be saved through the REST API using the editor. Here's a non-exhaustive list of document "types" or entities:
Most of these entities are defined here in the frontend gutenberg/packages/core-data/src/entities.js Lines 23 to 232 in 3a310c4 Please correct me if I'm wrong but my understanding here is that for each one of these entities, the proposal above requires 1- updating the database, to attach a list of YJS document to it. Some of the questions that needs answering are:
I would love as much feedback and discussion here as possible in order for us to make the right decisions as it seems that it's going to be an invasive changes regardless of the approach. cc @WordPress/gutenberg-core It seems that the work here involves a lot of backend work. It would be ideal if @dmonad can pair with an experienced WP contributor to help with all the service side changes. Anyone available to help there? |
Beta Was this translation helpful? Give feedback.
-
Welcome, @dmonad! Excited to see where this goes. Undo/redo: It'd be great to improve this regardless of real-time collaboration. We have some long-standing issues about undo management, its connection to block attributes and third-party blocks — #8119. Revisions: we have a whole roadmap for post revisions that would be great to align architecturally — #61161 Making Yjs a first-class citizen in the editor makes a lot of sense. I'm a lot more worried about server modifications. We've gone to great lengths to introduce blocks in WordPress without modifying the preexisting server representation of data. There are backwards and forwards-compatibility reasons for that, as well as ecosystem integration, long-term survival of the content, data sync issues with duplication, db size concerns, etc. There's a myriad ways in which WP content is manipulated server side by hooks and filters, sometimes overwriting source data in the process. Our own explorations have shown that pure client indeed doesn't seem very feasible, even with the server orchestrating webrtc handshakes, but I'm curious how we can reduce the server responsibility to the minimum necessary to avoid bloating the representations we need to deal with and to ensure the foundation can be widely deployed. It stands to reason that a websockets configuration, for servers capable of offering it, would provide the best concurrent experience on top of it. Also for context, some of the "sync engine" work traced here — #52593 |
Beta Was this translation helpful? Give feedback.
-
Thank you for working on this one @dmonad! I'm a bit concerned about all the potentially breaking changes that need to happen in order to make this work. Supporting multiple authors per revision (and therefore per entity) may be a breaking change for WordPress, and we'll need to devise a good plan to approach it. Will it be possible for the entities to continue having 1 author but multiple collaborators? In other words, it may make sense to introduce the concept of collaborator as a separate abstraction from the entity author. Would that make sense in the Yjs world? Also, what happens with the linear revision history once we have multiple collaborators? I'm also concerned about the backend implementation. The Heartbeat API and post locking were introduced in WordPress 3.6 because many data loss problems were caused by multiple users editing the same post simultaneously. This has historically been a limitation due to the technology used (PHP and flavors of SQL) that don't support concurrent editing. Autosaving is also reliant on these limitations, FWIW. Considering that we may still need to have a lightweight version of concurrent editing (potentially without a dedicated WebRTC server), I'm concerned about how it will work and what limitations from the user's perspective we might have to impose to account for those pre-existing limitations, and if those won't hinder the overall collaboration experience. Do you have any experience with such limitations before? Is it possible that the most basic Apache shared server with just PHP / MySQL will be limited and unable to use any collaborative features? My biggest concern is that from this proposal, it feels like Yjs will become the centerpiece, and WordPress will have to bend its database, structure, and flow around the Yjs as a single source of truth. Backward compatibility is a significant limitation - fundamental architectural change in WordPress can be challenging to achieve in a 20+-year-old ecosystem with hundreds of thousands of plugins, themes, custom setups, etc. I'm worried about the number of architectural changes we might have to make to WordPress to support Yjs as a first-class citizen and the high chances of breaking backward compatibility in the meantime. What are your thoughts or prior experience with integrating Yjs into legacy technologies where backward compatibility was required? |
Beta Was this translation helpful? Give feedback.
-
Hello Gutenberg community 👋,
In the coming months, I want to help to make the Gutenberg editor more collaborative. Automattic is kind enough to sponsor my work.
A little about me: I'm Kevin, the author of Yjs - a popular framework for building collaborative applications. I'm also the author of several collab extensions for other popular editors like ProseMirror, Quill, and CodeMirror.
I learned from experience that collaborative editing is quite an invasive feature that touches many aspects of the application.
Ideally, I'd like to make Yjs a first-class citizen of the Gutenberg editor, which custom blocks can use to offer their own collaborative experience (e.g., a drawing app using tldraw).
I'm currently meeting with people in this space to learn more about Gutenberg.
I opened this discussion for three reasons:
WebRTC
The current collaborative implementation uses WebRTC (using y-webrtc) to connect peers directly with each other. The idea is that the backend doesn't need to understand "real-time collaborative updates", the clients can simply sync directly with each other.
I advise against this because..
Ideally, the WordPress instance would be responsible for distributing document updates (every keystroke is synced to the backend, like in Google Docs). However, the backend is currently limited to simple REST calls, and it doesn't understand Yjs documents.
I got a lot of pushback when I initially suggested that we should simply put a (nodejs) server in-between WordPress and the client for the collaborative experience. We want to offer collaborative editing to as many users as possible.
Milestone 1 - collaborative editing using the existing backend.
My first milestone is to enhance the existing backend to understand that there can be conflicts that will be resolved eventually.
We could teach the backend that every document is associated to a list of Yjs documents.
When a user "saves" a document (through autosave), it will send the Yjs document alongside the JSON representation to the backend. In case of a conflict, we might end up with multiple "conflicting" Yjs documents on the backend. The client will be able to merge these Yjs documents.
The server will publish changes to the document through the existing autosave functionality. This progressive enhance will give all users a stable (albeit slow) collaborative experience. We may still put a websocket server in-between WordPress and the client to get a faster, more scalable experience that doesn't require autosave.
Protocol
Beta Was this translation helpful? Give feedback.
All reactions