Mandatory non-interactive defaults

[rootulp]

Discussion

Non-interactive default rules are currently framed as optional.

However, parts of the current implementation assume that non-interactive defaults are mandatory (e.g. inclusion/paths.go). If desired, we can modify the current implementation to support optional non-interactive defaults. Before we do so, I'm wondering why non-interactive defaults are optional rather than mandatory (i.e. a block validity rule)?

[evan-forbes]

The original reasoning behind allowing for non-deterministic square layouts was to allow for tighter packing of the square via allowing some form of interaction between the block producer and the users submitting blobs. With recent changes specified in ADR009, and the potential to reduce padding even further with other mechanisms #1161, this might not be needed any more.

This would mean that if we wanted to increase the packing efficiency, then it is up to us, and the community, to build such tooling. It also has the benefit of forcing all validators to use an efficient packing algorithm, instead of only the sophisticated ones.

Deterministic block layout would allow us to simplify the implementation by not having to wrap PFB transactions with the index when we gossip them. As pointed out by @cmwaters, doing so would significantly simplify the compact blocks implementation, because we would be able to reference the transactions gossiped via its hash. Not only that, but it would also stop block producers from arbitrarily setting the max square size.

Notably, deterministic block layouts might limit other features that we have discussed, such as adding priority metadata to specific blobs within the same namespace, or allowing for custom ordering of blobs in the same namespace.

Ideally, we would make a decision for this before incentivized testnet, but we might not have the bandwidth. From an implementation point of view, it should be pretty straight forward.

[nashqueue]

I would be in favour of mandatory non-interactive default rules, but I do not know the consequences and possible limitations.

Custom ordering within the same namespace could still be possible if the deterministic block layout has an ordered set of transactions as an input. The order can be specified in the input. The only downside is that it would follow the non-interactive default rules if that is one.

[cmwaters]

I would like to elaborate on some of these points:

• When we're talking about deterministic construction, we're talking about the square and not the block (these are two distinct abstractions to me). A block producer still has all the sovereignty to dictate what transactions go in and in what order. This means that there is still a market for MEV infrastructure and block producers can think of ways of maximising their return.
• I think the square should be an internal abstraction that both block producers and users are unaware about. How all this works is rather complex. It's far simpler for users to have the conventional mental model of "I post blob, I get blob". Non-interactive submission is simpler to use. Similarly, the block producers shouldn't be subject to a system of constraints where they need to think of novel methods of packing.
• I think it's important that celestia's core function (i.e. publishing blobs) works seamlessly out of the box. Other areas we may want to leave the design space open for third party infra and services but I think relying on it here may impact a users experience of Celestia. What this means is that if we want interactive PFB submission, we should properly support a form of that flow and not rely on others. That way we're directly responsible for users positive experience.
• Padding is a computational cost, not a storage cost or a bandwidth cost. Computational costs are generally the cheapest (and the quickest). While there are undoubtedly gains to be made for more efficient packing of the square. These are negligible and I don't think they should be our top priority.
• As mentioned, it leads to a far simpler implementation of compact blocks.
• We are still able to improve the efficiency of the constructing algorithm over time.
• We should still be wary of the consequences of human error. If we implement a non-deterministic deterministic construction algorithm this can be consequential to the liveness of the chain.

[nashqueue]

If we have mandatory non-interactive default rules, then we will get a nice side effect that the square can be recomputed given an ordered list of transactions. This could enable more compact storage requirements for full nodes, in particular archival nodes. Because computation is less expensive than storage.

[evan-forbes]

Per recent sync discussion, we are moving forward with this! and it will be incorporated before mainnet.