Custom libp2p sub-protocols

[mxinden]

Libp2p enables users to define custom protocols by implementing the NetworkBehaviour trait. These custom protocols use a substream via whatever multiplexing solution libp2p uses for a connection.

It would be useful to pass the ability to define custom protocols through Substrate up to code within Polkadot.

Related work having the gossip system open custom substreams: paritytech/substrate#4125

[tomaka]

I would say this is more of a Substrate issue, and we want to be able to add support for custom protocols for things outside of Polkadot as well.

[rphmeier]

@tomaka Is there any more comprehensive writeup on what the work items are for this and what the API will look like?

[tomaka]

To go a bit more into details:

For now the plan would be for Polkadot to use the register_notifications_protocol/write_notifications architecture that is now in sc-network (either directly, or through the sc-network-gossip crate which is based on top of it). It's a pragmatic solution and this API isn't hard to figure out.

I think that in the long term we should change that, and instead make Polkadot inject a NetworkBehaviour (or something similar) into Substrate. In other words, when initializing Substrate, you'd pass a NetworkBehaviour, similar to how you pass a NetworkSpecialization at the moment. The NetworkBehaviour trait is aware of all the concepts specific to libp2p and has full control over what's happening on the network.

Injecting a NetworkBehaviour is in fact what I'm planning to do for substrate-lfs, because we need it there.

However I don't think it's a good idea to straight-up do the NetworkBehaviour thing in Polkadot, because:

• We can't maintain backwards compatibility with the current network this way. (The NetworkBehaviour trait enforces a split between protocols, whereas right now all protocol messages go through a single substream.)

• Most importantly, the NetworkBehaviour trait is way too hard to implement. I really want to land a lot of quality-of-life changes into libp2p, which were blocked on the stable futures refactoring.

Once rust-libp2p is easier to use, we could eventually transition from a "write_notification-based code" to a "NetworkBehaviour-based" code and maintain backwards compatibility.

[rphmeier]

(this is done)