Plugin: PHP wrapper to add new blocks

[folletto]

Following at distance trying to keep the perspective as a developer "new to Gutenberg", I was pointed out today that blocks API seems to be at a good stable stage (ref). With the idea of "I want to add a third-party block" I looked for a ticket with the discussion, and there seem to be none, so here's a little contribution with some ideas for how to do the Blocks Plugin trying to get a good code usability.

I'm fairly sure a lot of this things have been already discussed, or at least thought, so this thread is mostly to make them explicit at the time seems ripe. I'll start with a few options.

---

Option 1: Function

A plugin could declare one or many blocks in a declarative way similar to add_theme_support(), maybe along the lines of:

add_editor_block( 'newblock' );

Where the parameter is the sub-folder of the plugin that contains the js+css pair.

This is very flexible (can be called anywhere in the plugin, and can be conditional), and lightweight enough, but has a major drawback in not making possible to know if a plugin is going to instance blocks or not, which makes harder to do plugin management (i.e. filtering only plugins with blocks, showing them in a plugin directory, etc) which is very important for admin and discovery.

• Declarative / Parseable ❌
• Multiple blocks ✅
• Conditional loading ✅

Option 2: Metadata

Currently the bare minimum for a plugin to be detected by WordPress is a single PHP file with some metadata:

<?php
   /*
   Plugin Name: Block Lorem
   Description: A plugin to implement the lorem editor block
   Type: Editor
   Version: 1.0
   ...
   */
?>

The idea is to provide an extra line in the metadata. If the line is present, then it loads the index.js file contained in the folder, as needed.

This approach has a big upside from Option 1: Function as it makes the metadata parseable without running the PHP file, which is fairly important for plugin directories and fast indexing in general.
The downside is that it restrict the option of one block to one plugin, which might not be ideal for rich plugins. To solve that, the declaration might be slightly different (to specify more items) but that adds some complexity probably not worth the metadata approach.

• Declarative / Parseable ✅
• Multiple blocks ❌
• Conditional loading ❌

Option 3: Folder Conventions

Using the power of default and conventions, we can simply declare a standard folder structure:

/plugin
/plugin/blocks
/plugin/blocks/lorem1
/plugin/blocks/lorem1/index.js
/plugin/blocks/lorem2
/plugin/blocks/lorem2/index.js

This would allow a parser to easily identify if a plugin loads block, and how many of them. Further convention in the future could even have readmes in each folder to be automatically loaded and indexed.

• Declarative / Parseable ✅
• Multiple blocks ✅
• Conditional loading ❌

Option N: What else?

As mentioned above, I think this was already in the mind of everyone. Let's comment below. ;)

---

A note on conditional loading: this in theory could still be possible even in option 2 and 3, as the declarative approaches means that we provide a "default", but then that default could be still programmatically overridden with a line in PHP, effectively providing easy to build blocks (i.e. using Option 3) but also the flexibility of disabling blocks under specific conditions (in that case the runtime function could act to deactivate, as by default blocks are loaded active, i.e. disable_editor_block() or similar).

[aduth]

Related: #422, #586

There's already a register_block function implemented, but it doesn't do much aside from the content replacement with a custom rendered substitute when presenting the block on the front-end.

When this is called could be an important consideration to deciding how the JavaScript is enqueued. We may want an editor-specific action to fire at the time we expect block implementers to register their block types, or block type registrations could be passive in a way similar to wp_register_script (registered at init) and "enabled" at an action. Associating the block's script could be either simply wp_enqueue_script or specifying a script handle or path in $settings passed to register_block. I'm not sold on discovery conventions.

As noted at #422 (comment), styling can be complex because a block's base styles, editor styles, and front-end styles aren't always shared. Similar to the JavaScript enqueuing dilemma, we could either leave this up to the developer to enqueue the correct style(s) depending on context, or give specific settings options for e.g. stylesheet_base, stylesheet_editor, stylesheet_content.

I've historically not been fond of CSS in JavaScript approaches, but to this specific need it could certainly alleviate some of the issues fumbling with context-specific stylesheet enqueues if styles are instead tied to the JavaScript's edit and save implementations. I'm not suggesting we pursue it, but is worth evaluating as an option. I starred a few earlier today which piqued my interest (styled-jsx, glamor, csjs).

[westonruter]

There's already a register_block function implemented, but it doesn't do much aside from the content replacement with a custom rendered substitute when presenting the block on the front-end.

@aduth has there been discussion about extending register_block() with a props schema suitable for the REST API?

[aduth]

@westonruter Closest thing to a discussion might be #104 (comment) , the "Block Schemas" heading, where defining schemas server-side could have enabled REST API endpoints serving structure of the blocks. This was largely tangled with the idea that we'd store attribute values of a block outside the markup in post meta. Its usefulness is diminished if the server (or any other context aside from the browser) can't know how to generate the markup a block, notably problematic if we'd expected to support block updating via REST API.

[westonruter]

@aduth I've been giving some more thought to the value of defining a server-side schema for blocks. To me it seems there is a hole right now with blocks in that they lack server-side sanitization and validation. Similar to shortcodes, you can throw whatever attributes at a block as you want (e.g. when editing the content via as serialized Text or via REST API) and it's up to the block renderer on the server (or the shortcode handler) to validate and sanitize the props. But shouldn't there be some enforcement of valid data being written into blocks?

It's one thing to silently sanitize and enforce proper attributes when the post_content is written to the wp_posts table (similar to what widgets do presently), but it's another thing when creating a post via the REST API and you include blocks, should not the request be rejected if you provide invalid block attributes? Otherwise, how would a REST API client be able to determine if a block in post content is valid or not? Though post_content is serialized, it does have a structure and there should be server-side constraints to enforce valid block.

Beyond this error handling case then there is the benefit of being able to expose REST API endpoints for blocks, and for widgets if they are indeed migrated over to blocks.

I guess this is just a fleshed-out version of what I wrote a week ago in #780 (comment)

[aduth]

See also #391, #914

I find I have a hard time discussing schemas and validation, because they seem so obviously a good thing: why wouldn't we want to protect against invalid block content? I think it's harder to argue on behalf of the implementer against a miserable developer experience. Here and in #914 we're already talking about schemas implemented both in the client and on the server. What compelling reason are we communicating to developers for why they should have specify these schemas? "Trust us, it'll be for the better good for all those times people mess with content directly". Not so compelling. It puts me in a state of conflict; the same conflict I've found myself in my few experiences of writing REST API controllers, finding those schemas to be unpleasant to manage but understanding the benefit they provide.

Where then can we strike a good balance and compel block implementers that they should want to provide a schema? Do we make it optional, or at least only implemented on either the client or server? In the case of #914, I see some future benefits in perhaps eliminating the need for save altogether, or providing a clearer experience for mapping saved content back to its object attributes form. These are compelling in that it ends up that a block is overall simpler to implement, assuming we can pull it off. Server-side, we could expose these schemas through the REST API, but for what benefit? We still can't generate the content anywhere other than in a JavaScript runtime.

I don't have any answers here now. Just expressing my own internal strife 😄

[westonruter]

@aduth The parallels you draw with schemas for REST API endpoints is a good one because for them, as well, schemas are optional. I recall this being a longstanding debate when the REST API was being implemented as well (WP-API/WP-API#759). Ultimately I think the benefit does come down to being able to manage attributes externally, and for there to be discoverability of the blocks in other environments than the Gutenberg client app.

We still can't generate the content anywhere other than in a JavaScript runtime.

I won't pretend to be fully up to speed yet on how blocks are created and the long discussions that have happened, but I recall that there are multiple ways that a given block's attributes can be encoded, either as key/value pairs in the block comment or via semantic HTML. So if currently an image block gets encoded as:

<!-- wp:core/image -->
<img src="https://cldup.com/uuUqE_dXzy.jpg" class="alignnone" />
<!-- /wp:core/image -->

Then conceivably it might also be able to be serialized as:

<!-- wp:core/image src="https://cldup.com/uuUqE_dXzy.jpg" class="alignnone" /-->

When Gutenberg loads up such post content into the interface, when it parses each of the blocks all it would have to do is run them through the save method in order to generate the alternate semantic HTML serialization. When calling the edit method, the attributes passed in could be a merge of the block comment attributes along with the attributes obtained via querying the nodes. This would allow for blocks to be fully managed programmatically as just kay/value attributes.

Having said this, this probably would then result in there having to be PHP implementations of the save logic for each block, to be able to do server-side rendering of each block. Maybe that is not desirable. Or, maybe it would. The hydration of attributes into their rendered representation could be done as part of the save_post process. But… then it would seem for the sake of REST API clients to be able to read out the attributes as they initially wrote them in, that the save serialization as HTML would need to get parsed back into attributes, which is again, currently handled only in JS. And this brings me back to what you raised in the first place.

Maybe the schemas would only be relevant to “dynamic blocks” which may not have semantic HTML serializations. If we use blocks as the next generation of widgets, then in order for there to REST API endpoints for widgets then there would need to be structure for these blocks. But I suppose the same goes for the post content. If external REST API clients aren't able to create blocks since they lack the Gutenberg runtime, then would this not be a severe restriction to apps that need to manage content?

Ref. #780 (comment)

[notnownikki]

Do we make it optional, or at least only implemented on either the client or server?

My first thoughts are, make it optional, and make it possible to define validation rules that both client and server can use.

If we define a schema and validation that looks like this...

  {
      'url': {
          'validate:' 'isUrl'
      },
     ...
  }

...it could go in a .json file in the block module. That could be read on the server and client side to perform validation. We'd have to implement each validator type on both the client and server, but are we really going to do anything more complex than regex and data type stuff?

[westonruter]

This is partially blocked by having a PHP-based post content parser that has parity with the JS parser. We won't be able to obtain the block attributes server-side to do sanitization/validation in the first place until that is there. See #1086.

[westonruter]

Let's also include in the scope here the fleshing out of the PHP classes that should be used for manipulating blocks on the server. See conversation starting at #1217 (comment)

[jasonbahl]

We use WordPress as the entry point for content not only displayed by our WordPress themes, but also our native mobile apps ANDprint publications.

Our native apps already have a horrible experience having to parse through the unpredictable mess that is post_content so blockifying the content is great, where each block can have attributes that certain consumers can legibly read through and determine if there's value for their use, but if the "blockification" really only happens on the client, it doesn't benefit the other consumers of the content.

I'm working on WPGraphQL (GraphQL API for WordPress) and I quickly experimented with integrating Gutenberg and got it working via the do_blocks method, but the knowledge of what blocks exist is only available after query time. Having the knowledge of what blocks are capable to be returned, what attributes and fields each of those block can have, etc is where the real power comes in.

In my head, the block registration API should be similar to registering a post_type, where certain fields are required and others are optional.

This way, we can do discovery on the available blocks and expose them to consumers.

The way we're planning on using the blocks with GraphQL and our Mobile clients is to give the consumer the power to query just the blocks their platform supports.

For example, our WordPress theme might support a certain Video embed, but maybe our mobile team doesn't, so in a GraphQL query, each consumer could decide what they want returned from the blocks at the query layer.

getPostWithVideo {
   post( id: "..." ) {
      contentBlocks {
          ...on Video {
             videoUrl
             autoPlay
          }
          ...on Freeform {
            rawContent
          }
      }
   } 
}

But our mobile consumer could query like so:

getPostWithoutVideo {
   post( id: "..." ) {
      contentBlocks {
          ...on Freeform {
            rawContent
          }
      }
   } 
}

If the Block schema only lives in client side code, then this rich data is only useful inside WordPress, but in today's world of decoupled WordPress, the server side schema isn't just nice to have, it's a requirement for Gutenberg to succeed and be widely useful.

I'd be happy to elaborate more if needed, but in short, I'm arguing for a required server-side Schema for Block registration.

[aduth]

@jasonbahl There have been some revisions to attributes definition in #1905, with one goal being a step toward interoperability between client and server attributes definition using JSON-schema-like definitions. A further step I could imagine is allowing these schemas to be defined server-side and "hydrated" into the client automatically. Whether we take that to the extreme of "all blocks must register schemas server-side" is not certain yet.

See also some previous related discussions: #391, #104, aduth/wp-block#1

[aduth]

See also #2529 for efforts toward supporting server-registered block attributes.

[jasmussen]

Also wp-cli/scaffold-command#88 in case relevant.

[itibook]

context: I am still wrapping my head around blocks and how they work, so feel free to correct me if I misunderstood some basic stuff...

I share @jasonbahl concerns and the initial issue raised by @folletto
the same goes for issue #2751

I think it's very safe to assume that we will see an explosion of custom blocks by plugin & theme authors as soon as Gutenberg hits core. My estimate is 10,000+ custom blocks (e.g. top 1000 themes/plugins create 10 custom blocks each - very conservative estimate).

Issues

• The more custom blocks are added by plugin/theme authors, the more cluttered the block selection UI will become (considering the above estimates, it will just be a big mess).
• from my understanding it's impossible for the end user to hide/remove certain blocks, as soon as a block is registered it will appear in the list by a plugin/theme
• currently there's a limited list of categories which will only increase the first issue about clutter

Ideas

• Ideally each plugin that registers a block should have its own category/sub-category from which the user can choose blocks. e.g. Yoast blocks, Jetpack blocks, etc etc

• all these blocks should be available at server level just like what happens with re-usable blocks

• As a end user I need to be able to have control on blocks and how they will be available for selection / usage, including based on roles admin/editor/author (maybe this is already the case, but I did not see anything about it).

hope it helps...

[danielbachhuber]

@folletto Any other actionable items out of this issue, or is it safe to close?

[folletto]

I'm not sure I can reply that question: some things have been implemented, some were linked to this thread. I think it's done, but would be nice if before closing someone that worked in the implementations could recap what was done. :)

[youknowriad]

There's a lot of overlap with this issue and #2751

A lot of these APIs are already available

• PHP wrapper to register blocks
• scripts/styles of blocks are already attached to the register_block_type call
• Possibility to set Block properties (title, icon, description, attributes...) server side is possible but not enforced at the moment.
• It's also possible to add custom categories server side

I'm closing this issue for now and let's consolidate in #2751 unless I missed something.