-
Notifications
You must be signed in to change notification settings - Fork 895
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set custom onchain fees for channel closing #2797
Comments
This is not an easy feature to make. For a unilateral close, the feerate is controlled by the funder of the channel. In current C-Lightning (and other implementations), this feerate is updated slowly, because we often get coordination problems with other implementations around this feerate if we update them too quickly (leading to channel closures when onchain feerates change). The last feerate proposed by the channel funder is the one that is used, and a unilateral close command cannot change the feerate (since it uses a stored commitment transaction that is updated only when the channel funder says to update the feerate). So if you do a unilateral close while we are slowly moving the feerates (which we have to do slowly because coordination between various implementations about how tight the feerates should be is so fraught) then you can only get a not-very-updated feerate. For a mutual close, if I remember correctly the funder begins at its latest feerate, and both sides negotiate a feerate by providing initial feerate estimates, then providing proposals to each other until they converge on an agreed feerate. So in this case, you can affect the feerate (by starting your initial feerate at your desired feerate), but the other side can still play hardball and only advance by 1 satoshi at a time. Note that the funder is the one that indicates the desired feerate first, so the funder can't know if the other peer will bid higher or lower, so it can't actually overbid in an attempt to "pull" the other side to the proper feerate. Was this a mutual close or a unilateral close? If it was unilateral then nothing can be done about feerates at all: you're bound to our slow update (which we are bound to because otherwise channels will get mass-disconnected between C-Lightning, lnd, and Eclair). If it was mutual then it is possible to affect but not set the feerate. |
Thanks a lot for the detailed explanation. It was mutual close and I was the funder. One thing that I still don't get, when you say that it is possible to "affect the feerate (by starting your initial feerate at your desired feerate)", the initial feerate is the one I used with fundchannel? Or something I can set before closing the channel? |
It is something that is transmitted during the mutual close protocol. The mutual close protocol starts with the two sides giving their initial desired fee for the mutual close, then as long as they are different, they keep proposing fees (which must move closer, by at least one satoshi, towards the other side) until they both propose the same fee. Currently we just use the current slowly-moving feerate as the initial feerate to compute the initial fee in the mutual close protocol. We could add an option to a mutual close to change the feerate that will be used to compute the initial fee for the mutual close ceremony, but do note the protocol means the other side will still affect the final feerate and you will never achieve the actual feerate you indicate in a mutual close. |
@ZmnSCPxj: Is it possible to abort the mutual close protocol (leaving the channel open and unaffected) if the negotiated fee is still too high? Sometimes I might want to close a channel opportunistically but could put it off until later if the peer still wants to use a higher fee than I would prefer. |
The mutual close protocol is started by a Further, once we are in This can be done if we have a new message like |
@ZmnSCPxj: Is it possible to query what the peer thinks would be a good mutual closing fee prior to sending a My node's |
Currently not possible without a protocol change. |
I noticed my node did a unilateral close and paid 1,313.5 sat/vB – 236,758 sat ($38.17) in onchain fees for a 2.8 mil sat channel. This wasn't even a period of high onchain fees. You can see this tx at the top in this block Is there any setting I can amend to relax this calculation and tell lightningd "Relax, we're not in a rush, use economy fees" ? Why are channel closing fees an order of magnitude higher on average than channel opening fees? |
@hosiawak: Anchor outputs are supposedly going to mostly remedy this problem by letting your node choose a fee rate at the time of channel closing rather than at the time of commitment signing. In theory it should be possible to bump the transaction that spends the anchor output too, though I haven't seen whether C-Lightning is going to implement that. (It does no RBF anywhere yet that I've seen.) |
@whitslack Thanks but I don't really understand why my node chose to use the highest fees in a block if this was a unilateral close? Why was it in such a hurry to close the channel ? I'm trying to understand the logic behind this behaviour and find a workaround. Either the current behaviour is completely unreasonable and pathological or I'm missing something. |
@hosiawak: Both: you're missing something and it's completely unreasonable. What you're missing is that the fee rate for unilateral channel closures isn't decided at the time of the closure; it's decided at the time your node was last in communication with your peer. If that was at a time when the mempool was extremely congested, then the commitment transactions (one of which is what your node broadcasts to unilaterally close the channel) would use a much higher fee rate than might be necessary at the time your node actually broadcasts one of them. It's not ideal (okay, completely unreasonable), but Lightning is a work in progress, and this will get better. |
@whitslack Thank you for the explanation, I'm trying to wrap my head around the fee estimation and what it's based on. Sorry for asking daft questions but I still don't get why a unilateral close uses I changed lightningd code to use The modified node used 4 sats/vbyte for the opening transaction. Here is
And here from the current, unmodified node:
The site I always use when trying to manually estimate a fee before sending transactions without overpaying currently recommends 3 sats/vbyte for the fastest fee and 1 sat/vbyte as normal. So even though I used the Is there any risk to running a node using such |
@hosiawak: If your fee estimates differ too much from those of other peers in the network, then you won't be able to maintain open channels, as nodes are programmed to unilaterally close channels when the two ends disagree by too much on what the fees should be. Opening channels has never been an issue. The |
@whitslack So the only risk to using slow fee estimates is increased chances of triggering a unilateral close ? Notice that If that's the case and this is the only risk then I still don't understand why unilateral close uses I closed the channel from my modified node to my unmodified node and the closing fee was 4 sats/vbyte. If I didn't use the modified node the agreement would settle at the current Would it be possible to introduce |
@hosiawak: I don't know what all the risks are, but I assume there must be significant risks in having a commitment transaction delayed for too long since it is constructed with urgent fee rates by design. I agree with you that Bitcoin Core's fee estimation logic is total garbage. I've never understood why it produces such horrible estimates or why the Core devs think it's passable as is. I've even had it return a higher fee rate for longer block targets than it does for shorter ones. That's just broken. I also agree that running a Lightning node is a quick way to get poor. Sometimes I wonder if the designers of Lightning actually run big nodes themselves or are somewhat disconnected from the expensive realities of their theoretical ideas. |
Pluggable feerate estimators would be awesome. I could write an estimator based on mempool.space for example (or any other you might find useful) and forget about this issue. |
@ZmnSCPxj Do you know why unilateral close uses |
Because of this theft vector:
The above is chancy and not 100% assured of success but we think it can happen often enough to be lucrative. The point at which I marked "<<<<<" is where the mitigation we use is inserted: even if the current fees are low, we still make unilateral closes high-fee, because the attacker can go offline and prevent unilateral closes from being updated to use higher fees later. The reason why we use an even higher fee for unilateral closes vs. penalty is that penalties are one-sided (do not require signatures from both parties) while unilateral closes need to be presigned by both parties. The penalty transaction can use the next-block estimate for right now, the unilateral close has to use the next-block estimate for what if suddenly the fees go high right after the peer goes offline. The general philosophy here is "a million bitcoins in defense, and not a sat in tribute", i.e. we are willing to lose money in general operation rather than let even a few satoshi get stolen by the above (or any) attack. Re: pluggable estimators: this risks mass closure if your estimator diverges from the estimators others use. This is because, even if we are not the ones paying for the close (it is the one opening the channel that has to pay for the close, pre-anchor-commitments) we need to ensure that the unilateral close transaction can be confirmed in a timely manner even if the above attack is attempted (and the victim is not the initiator of the channel --- nothing in the above requires the victim to be the initiator, the attack only works better if the victim is also the one paying for fees (i.e. is the channel initiator) for pouring even more salt on the wound, but does not require them to be the one paying for fees). Thus, every node will check that the fees you set on channels you opened "looks reasonable" to them. If they think it is unreasonable, they unilaterally close right now rather than risk time passing too much and possibly making your low fee estimate even worse for them. If everyone uses drastically different fee estimators, then every onchain fee movement is going to cause a lot of channel closures. This can turn into a positive feedback loop: sudden fee spike causes some (but not all) fee estimators to suddenly increase their fee estimates while others are still thinking "low fees" to disagree and cause channel closure, which further congests blockchain for further fee spike so that estimators that were barely agreeing get even further into disagreement and trigger more closures, and wham Lightning Network shuts down and everybody cries because their favorite blockchain has gigabytes of mempool. C-Lightning has an option to skip this check |
That's a great philosophy to hold against attackers. I am solidly opposed to giving a mouse a cookie, paying ransoms, negotiating with terrorists, etc. But in my real-world experience, unilateral channel closures overwhelmingly occur due to bugs either in my node's software or in my peers' software. Are we just punting for now, admitting that using Lightning is going to be a losing proposition versus the blockchain for the foreseeable future? |
It is difficult to judge if a bug comes into a codebase from incompetence or malice. Just as bug fixes can be secretly inserted into regular refactorings to fix CVE-level bugs, CVE-level bugs can also be secretly inserted into regular refactorings. Worse, anyone can add this to their own node software without publishing it anywhere else, so you will not know if an arbitrary node you make a channel with is buggy or is trying to attack you.
No. There is a reason why anchor commitments is being worked on, and that is to allow unilateral closes to be at minimum allowed relay feerate, with extra outputs (the "anchors" in "anchor commitments") to allow either node to CPFP (or CPFP-RBF) either commitment at the next-block feerate at the time it is dropped onchain, instead of the current situation where we have to speculatively overprice every commitment transaction because it is hard to predict the future and whether fees will be high or low later. There are reasons why we have to move to anchor commitments and not just add workarounds to the current commitment scheme, I described one of them. |
@ZmnSCPxj Thanks for a thorough explanation. It makes more sense now. I think I understand the risk better and can make better decisions wrt setting custom fees. |
Yes, onchain fees have been a headache since massive deployment of Lightning. Anchor commitments seems the best way forward, but of course there is the question of "what about existing channels that were created pre-anchor commitments?" There is discussion about whether we can upgrade "seamlessly" a pre-anchor-commitment channel to post-anchor-commitment channel: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002763.html |
@ZmnSCPxj Are you aware of any risks of changing just the normal opening fee estimates from Context: for the past week my node opened ~30 channels and for 90% of funding transactions overpaid between 13x and 41x in channel opening fees (eg. paying 13sats/vbyte when 1sats/vbyte were being confirmed in the next block). This is what 34.7 sats/vbyte in block 656522 (confirming 1 sat/vbyte txs) = 34x overpaid I know I could have used a custom fee when opening but I use clboss on this node which uses the default feerates in c-lightning when opening channels (I think, correct me if I'm wrong here please). Could we add a knob for the target block for channel opening fees or will this cause issues when coordinating with other nodes as well ? It could default to the current value of 4 but allow node operators to relax their channel opening time preference. I don't know what algorithm core uses to estimate the fee but it doesn't seem to be able to estimate the actual, lowest fee that gets you into the next X block (at least not in the current, low fee market) but rather tries to play it safe and overshoots by an order of magnitude even with its "ECONOMICAL" setting. If there's no risk in using low end fees for channel opening then perhaps defaulting to 12, 24 or even 100 blocks would be better (it doesn't matter much if I have to wait an hour or 10 or 72 hours for a channel to open) It's a long lived thing anyway. |
The risk is that if a feerate spike occurs just after we broadcast, the low-fee funding transaction could remain in the mempool. In the worst case, the funding transaction could remain in the mempool longer than two weeks. Now, let us move our attention to the acceptor of the channel. In general, we accept anyone opening a channel to us, especially in the current situation of single-funding, where it is the initiator of the channel open that is the only one who puts funds in the channel, which removes a lot of risk (particularly, since only the initiator funds it, the acceptor has no risk even if the channel funding is delayed in confirmation, as it has no funds locked in a low-fee transaction). However, it can be an annoyance if a bunch of nodes propose channels to our node, completing the opening sequence, but ultimately not getting their funding transactions confirmed in the end, this would be a trivial way to cheaply load our node with useless channels in its database (create a new random pubkey, connect, propose channel, give a random number as funding txid, disconnect, repeat). So the acceptor of the channel forgets about the channel after two weeks. This is the function which tells an acceptor when to forget: lightning/lightningd/channel_control.c Lines 782 to 789 in cd7d5cd
The default So if we underestimate channel funding fees and end up having a funding transaction that fails to confirm after two weeks, the acceptor of the channel will forget it anyway, so once it confirms, you can only unilateral close in your side and be unable to use the channel. (this holds even if we rebroadcasted the funding tx, by the time it confirms, the acceptor will say "wut?" when we try to bring up the channel with them, and we still cannot do anything than unilaterally close it). What we should really do is support some kind of CPFP+RBF as proposed in #475 , or RBF as proposed in #668 (but note that this latter requires either a protocol change (that is now available with dual-funding protocol, but might not be easy to use/access over the current CL RPC yet) or C-Lightning has to support multiple channels per peer). Start with a low-fee funding tx, but if that fails to confirm quickly, bump up its fee (by either RBFing an CPFP of the funding tx, or RBFing the funding tx directly). Maybe CLBOSS can increase its onchain remainder, and start with a low-fee funding tx (it always leaves a small amount of funds onchain anyway, so funding txes would always have a change output it can CPFP onto, and the CPFP tx can be RBFed later), but that would require reimplementing a good part of |
@hosiawak this PR also describes an actual problem encountered by @darosior with too-low feerates, which is another reason why CLBOSS prefers the "normal" feerate: #3910 In particular, the minimum relay feerate of a |
@ZmnSCPxj Thanks for another thorough explanation. In the meantime I changed the channel open fee to use |
LND is working on it too. |
I may have glanced over it being mentioned, but I still see an argument for adding the ability to manually set a fee manually for the close clommand on a per-tx basis. Consider the following scenario:
I now need to do a unilateral close. I don't see why I couldn't explicitly set a specific feerate for this unilateral close-tx only, without changing the general close-fee advertised to other peers. FWIW, lnd does allow doing this through |
@3nprob: A unilateral closure uses a transaction that is signed by your peer. If your peer is permanently offline, then you can't obtain a signature from them on a new transaction with a new fee rate; you have to use whatever you already have from them. That's one reason unilateral closes are so expensive: they have to work when you need them, and there's no possibility to set the fee rate at that time. That should change with anchor outputs. |
I hope I'm not coming off as obtuse, but how come LND does it then? Also, is there a way to preview the fee that would be used for the specific channel before executing it? EDIT: looking in the database, am I corect in understanding that |
It doesn't. |
Makes sense! Looking in the database, am I corect in understanding that channel_feerates.feerate_per_kw is the fee that will be used for a forced close? |
We are already able to set a feerate with
fundchannel
butclose
is missing this feature. According to blockstream's explorer I just overpaid by 743% the fees to close a channel.How could I have estimated what fee will be paid? In the end it was 7943sats (58sat/B).
The text was updated successfully, but these errors were encountered: