LDK-node server-side

[ariard]

Opening this issue to track the interest and design considerations for ldk-node for enterprise purpose. The end goal is to have a high-available, high-perfomant, modularity and secure entreprise Lightning node.

One design consideration - the ldk-node should be configurable to match the advanced Bitcoin hobbyist eager to provide peripheral liquidity to the network to large-scale entreprise with a fully dedicated Bitcoin infrastructure team.

What are the entreprise use-cases ?

With the growth of the Lightning ecosystem, one can expect specialization of the Lightning nodes for different class of Lightning services, level of integration with application logic and types of nodes

• Lightning routing nodes: providing reliable payments paths across the network and liquidity services (splicing, swaps), care about availability, performance and security.
• Lightning gateway nodes: providing access to the Lightning network to "spokes" users in exchange of markup fees for the liquidity lockup ; care about security and mobile features support (e.g async payment).
• Lightning merchant nodes: providing virtual/physical goods to the network.
• Lightning custom packets nodes: providing routing services and liquidity for e.g Lightning escrow packets ; care about security and modularity
• Lightning mining pools nodes: providing integration with Stratum v2 for custom mining protocols to send payouts or buy shares over Lightning ; care about security and modularity

Note it is expected in practice that many of those types of nodes to be deployed in one physical deployment.

What types of component a Lightning entreprise nodes should encompass ?

Rather to have one wallet for each type of layer and protocol spoken (L1 wallet, L2 wallet, swap wallet) it is expected coin selection, key and descriptors management and UTXO control to be under a unified component.

There can be an expectation than mempool and fee-estimation to be offered under a single API as this is consumed both for fees management for anchor support, and in the future for automated liquidity management.

There can be an expectation than all the keys managements is handled by a single component with a generic interface with its own recovery and storage handling, something like VLS).

There can be an expectation than the routing stack and all the invoice/offers logic can be separated in its own component to be reused more freely and in a parallel fashion by merchant logic.

What else ? There are few more obvious like watchtower integration and watchdog integration.

What should be the level of runtime modularity ?

I think service-oriented architecture should be favored (without falling in the trap of the micro-services) as it matches well the intrinsic requirements of Lightning and Bitcoin protocol (run multiple watchtower backend and run multiple transaction-relay endpoints).

The LDK-node for entreprise could be splitted in multiple process: channel-management, on-chain monitoring, routing,
p2p stack and one for wallet/signing (though maybe combined with channel-management ?). The exact separation of concerns should match what make sense in term of availability, perfomance and security. The process should expose simplified API (e.g gRPCs) and access to the raw data structure/methods.

If the process interface can be made compatible with the ongoing work in Core's about multiprocess this makes a lot of sense on the long-term to integrate directly the mempool, transaction-relay and any monitoring infrastructure.

[adoerr]

The README says it all

LDK Node is a non-custodial Lightning node in library form. Its central goal is to provide a small, simple, and straightforward interface that enables users to easily set up and run a Lightning node with an integrated on-chain wallet. While minimalism is at its core, LDK Node aims to be sufficiently modular and configurable to be useful for a variety of use cases.

Any bias should be towards "easy to set up and run", i.e. a Lightning node which could be run by individuals without much fuss. By definition this means a quite limited set of configuration (and hence miss-configuration) options and extension points.

[ariard]

Any bias should be towards "easy to set up and run", i.e. a Lightning node which could be run by individuals without much > fuss. By definition this means a quite limited set of configuration (and hence miss-configuration) options and extension points.

Do you intent to use LDK-node for your usage and if yes can you detail more what set of features, options and extensions are you expecting for the early versions ? By a Lightning node which could be run by individuals what level of knowledge and technical skills are you assuming from those individuals and for what types of deployment (mobile/LSP/routing hops/merchants) ? For the "easy to set up and run" can you qualify in term of time and what level of state backup in case of mainnet deployment are you expecting ?

[adoerr]

Glad you ask. IMO, LDK-node should be the go-to option for individuals hosting a self-custody Lightning infrastructure. As such it might / should become part of the various RasPi based deployments, for example. Basically a Lightning node I can connect my LN Wallet to. A mobile deployment could be seen as an extension / variation of the above mentioned use case, I guess.

Such a well focused and sufficiently simple LDK node implementation could also serve as a kind of "reference" or "template" for a canonical LDK node implementation. Due to it's focus and simplicity such a canonical LDK node would also foster adaption of new Lightning features / protocols.

Last but not least, a LDK node with a focus towards individual use is an open source project part-time Lightning developers will be attracted to contribute to, because it is something they will / could use themselves.

[TheBlueMatt]

I don't know/think there needs to be one "ldk-node-server", certainly once we're talking about building a freestanding node we can/should have many. But I definitely agree that there should be at least one, or maybe the "main one" that is....just a lightning node. Something designed to be stable and robust and performant enough for an enterprise don't-think-too-hard deployment, but also that you can fit in an RPi and run as a pleb (assuming you can afford transaction fees lol). In both of those cases you really don't want a hell of a lot of knobs and don't need complexity and hooks everywhere. You do maybe if you want a "big metal" enterprise deployment that runs on many servers and does failover and such, but I'm not sure that needs to be an initial goal.

[ariard]

I don't know/think there needs to be one "ldk-node-server", certainly once we're talking about building a freestanding node we can/should have many. But I definitely agree that there should be at least one, or maybe the "main one" that is....just a lightning node. Something designed to be stable and robust and performant enough for an enterprise don't-think-too-hard deployment, but also that you can fit in an RPi and run as a pleb (assuming you can afford transaction fees lol).

We might need to dissociate "ldk-node-server" in "lower half" and "upper half". "Lower half" can be distributed on-chain monitoring, VLS integration, fee-bumping reserves management and transaction-relay integration, like everything which is security-sensitive and that I don't expect many teams to have the level of knowledge to get it correctly. Ideally we can have this robust "lower half" and then people can build more easily their custom "upper half", i.e swap service, routing or payment nodes with maybe standard process maintained in other repository.

For sure the enterprise don't think too-hard deployment is a laudable goal and hopefully it can cover RPI and run as a pleb (assuming you're an advanced hobbyist). Here packaging with docker or nix can simplify the task, it's like OS you have thousands of knobs under the hood, though in practice 90% use wizards configuration and don't touch the default.

I mind the warning on the targeting a too-complex "big metal" enterprise deployment for first steps. This is not realistic as an initial goal. However, if the ball roll forward, I sense "multi-process" and modularisation matching the Lightning separation of concerns should come early on to avoid the functional interdependency nightmare than Bitcoin Core constitutes.

Look, I'll start from LDK-node as a foundation then we can split into crates or modules and move in different repositories, like we did in the past when it made sense.

[ariard]

I'll move a ldk-node-server or whatever we end up call in its own repository to move faster.