[Custom Transactions] Refactor channel validation #3921
Conversation
|
👋 Thanks for assigning @TheBlueMatt as a reviewer! |
|
We discussed earlier passing the entire list of HTLCs to So we prefer to add a single method that surfaces this limit to channel, and then let channel sort HTLCs depending on whether their value sits above or below that amount. As a result, builders of custom transactions will only have a say on where the dust limit sits, and won't be able to choose arbitrary subsets for dust and non-dust HTLCs. |
Is the main motivation for this to get all of the commitment-related logic out of
The key questions we're looking to answer seems to be "can I afford a commitment with this theoretical dust/nondust htlc"? This does seem to be something we could move into Sadly here's no getting around needing to know the dust limit if we want to clamp our capacity to that value, so we'd still need to surface
Tempting to suggest just adding an so tl;dr: I'd be interested in seeing what trying to pull more of the dust logic out into Meta note: This has got a lot of overlap with 3bb0586, so I think we should either:
|
I would say we focus on the latter for this PR, and let the former be the overarching goal :)
Let me know what you think of this new direction here. Still have some clunkiness to resolve, but that's what it's looking like right now.
|
Definitely let me rebase on top of your PR, you've rebased once already :) |
tankyleo
force-pushed
the
dust-limit
branch
2 times, most recently
from
12fd6c3 to
067bfa0
Compare
|
🔔 1st Reminder Hey @carlaKC! This PR has been waiting for your review. |
I like this approach! Definitely prefer being able to move a lot of the dust / second stage tx reasoning into the builder 👍 |
|
🔔 2nd Reminder Hey @carlaKC! This PR has been waiting for your review. |
tankyleo
changed the title
|
I'm aware I still owe a follow-up PR from the previous PR in this project, will push that soon in a separate PR :) |
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
lightning/src/sign/tx_builder.rs
Outdated
| fn on_holder_tx_dust_exposure_msat( | ||
| &self, dust_buffer_feerate: u32, holder_dust_limit_satoshis: u64, | ||
| channel_type: &ChannelTypeFeatures, htlcs: &[HTLCAmountDirection], | ||
| ) -> u64; | ||
| fn on_counterparty_tx_dust_exposure_msat( |
There was a problem hiding this comment.
Not really sure I understand the point of these two methods if we have htlc_success_timeout_dust_limit as a method - if the API includes a method to say "hey, for an HTLC of type X what is the threshold where its dust", why bother with a method to say "given these HTLCs, how many are dust?", it seems the second can be calculated from the first, no? The same applies for passing the HTLC list to commit_tx_fee_sat.
Alternatively, we could drop the htlc_success_timeout_dust_limits method (if we want to not require there be some strict threshold to indicate when an HTLC is dust, though I think its a fine assumption to bake into the API, I dunno why someone would want to have some HTLCs be dust and others at the same value not be) and have a more generic "is HTLC dust on the next counterparty/local commitment transaction" call, but even there it seems like we don't need all of these methods.
There was a problem hiding this comment.
Sounds good yes I initially had a version of the API that matches what you describe. We wanted to see what it would look like if we moved more logic out of channel into TxBuilder, but I agree we only really need the dust limits.
I will clean up those methods, and add TxBuilder methods for 1) the dust limit 2) the htlc endogenous fees (to calculate counterparty dust exposure).
There was a problem hiding this comment.
I mean don't get me wrong, I'd love to move more logic out of channel.rs into TxBuilder, that would be great, but we also don't want to have an overlapping API where we have two ways to do things over the API. Also, we want to eventually make TxBuilder public, so it would be nice to avoid adding too much complexity (in the form of a ton of different methods).
There was a problem hiding this comment.
FWIW the real blocker on the more ambitious / complex API was the get_available_balances method - at the moment this method requires an exact dust limit above which all HTLCs are non-dust.
I have some code written that moves most of get_available_balances behind TxBuilder, but the complexity is not worth it, especially if we can make the assumption you describe above about the strict dust-vs-nondust threshold.
|
👋 The first review has been submitted! Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer. |
tankyleo
changed the title
tankyleo
changed the title
TxBuilder set the HTLC dust limit
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3921 +/- ##
==========================================
+ Coverage 88.93% 88.98% +0.04%
==========================================
Files 174 174
Lines 123876 124105 +229
Branches 123876 124105 +229
==========================================
+ Hits 110173 110429 +256
+ Misses 11250 11227 -23
+ Partials 2453 2449 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
for what it can be worth it I have reviewed 1eb1c9e and it looks correct to me.
As for my understanding on the reasoning for the change, motivation seems to be to move usage of second_stage_tx_fees_sat and htlc_tx_fees_sat to TxBuilder in order to abstract away usage of weight of htlc transactions away from channel.rs.
There was a problem hiding this comment.
Discussed it offline, but I'm somewhat skeptical of a design that adds too many more methods on TxBuilder. We should consider exploring options that keep TxBuilder much smaller with more logic in larger methods (that we're willing to call and throw away some of the result of, eg relying more on something like that looks like build_commitment_stats to figure out dust exposures and whether an htlc is acceptable).
|
@TheBlueMatt would appreciate a look here :)
|
tankyleo
changed the title
TxBuilder set the HTLC dust limit
| let htlc_candidate = HTLCCandidate::new(msg.amount_msat, HTLCInitiator::RemoteOffered); | ||
| let local_commit_tx_fee_msat = self.next_local_commit_tx_fee_msat(funding, htlc_candidate, None); | ||
| if local_balance_before_fee_msat < funding.counterparty_selected_channel_reserve_satoshis.unwrap() * 1000 + local_commit_tx_fee_msat { | ||
| if next_commitment_stats.holder_balance_msat.unwrap() < funding.counterparty_selected_channel_reserve_satoshis.unwrap() * 1000 + next_commitment_stats.holder_commit_tx_fee_sat * 1000 { |
There was a problem hiding this comment.
A couple of differences to note here:
local_commit_tx_fee_msatpreviously accounted for outbound HTLCs in the holding cell,next_commitment_stats.holder_commit_tx_fee_satdoes not account for HTLCs in the holding cell.local_balance_before_fee_msatwas previouslyfunding.value_to_self_msat - 2 * 330'000msats,next_commitment_stats.holder_balance_msatis nowfunding.value_to_self_msat - pending outbound HTLCs - anchors(note we do not count holding cell HTLCs here, the remote does not know about them, and we expect to fail these if we can no longer afford these when freeing the holding cell)
| if remote_balance_before_fee_msat < msg.amount_msat { | ||
| return Err(ChannelError::close("Remote HTLC add would overdraw remaining funds".to_owned())); | ||
| } | ||
| let remote_balance_before_fee_msat = next_commitment_stats.counterparty_balance_msat.ok_or(ChannelError::close("Remote HTLC add would overdraw remaining funds".to_owned()))?; |
There was a problem hiding this comment.
Here we expect TxBuilder to set counterparty_balance_msat to None if the HTLCs and the anchors the counterparty has to pay for would drain its balance. It includes the candidate HTLC we are considering here.
| let remote_commit_tx_fee_msat = if funding.is_outbound() { 0 } else { | ||
| let htlc_candidate = HTLCCandidate::new(msg.amount_msat, HTLCInitiator::RemoteOffered); | ||
| self.next_remote_commit_tx_fee_msat(funding, Some(htlc_candidate), None) // Don't include the extra fee spike buffer HTLC in calculations | ||
| next_commitment_stats.counterparty_commit_tx_fee_sat * 1000 |
There was a problem hiding this comment.
remote_commit_tx_fee_msat remains the same here; next_commitment_stats.counterparty_commit_tx_fee_sat * 1000 == self.next_remote_commit_tx_fee_msat
bring back useful comment
| } | ||
| } | ||
| } | ||
| let PredictedNextFee { predicted_feerate, predicted_htlcs, predicted_fee_sat } = funding.next_local_fee.lock().unwrap().clone(); | ||
| let mut actual_nondust_htlcs: Vec<_> = commitment_data.tx.nondust_htlcs().iter().map(|htlc| HTLCAmountDirection { outbound: htlc.offered, amount_msat: htlc.amount_msat }).collect(); | ||
| actual_nondust_htlcs.sort_unstable(); | ||
| if predicted_feerate == commitment_data.tx.feerate_per_kw() && predicted_htlcs == actual_nondust_htlcs { | ||
| assert_eq!(predicted_fee_sat, commitment_data.stats.commit_tx_fee_sat); | ||
| } |
There was a problem hiding this comment.
Note the equivalent assert_eq!(commitment_data.stats.commit_tx_fee_sat, info.fee / 1000); statement above here is never hit when running the current test suite... This statement here and its sibling in build_commitment are each hit in ~200 tests, even though they assume no dust HTLCs on the commitment transaction.
| if htlc_stats.on_holder_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { | ||
| if next_commitment_stats.on_holder_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { | ||
| return Err(ChannelError::close(format!("Peer sent update_fee with a feerate ({}) which may over-expose us to dust-in-flight on our own transactions (totaling {} msat)", | ||
| msg.feerate_per_kw, htlc_stats.on_holder_tx_dust_exposure_msat))); | ||
| msg.feerate_per_kw, next_commitment_stats.on_holder_tx_dust_exposure_msat))); | ||
| } | ||
| if htlc_stats.on_counterparty_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { | ||
| if next_commitment_stats.on_counterparty_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { |
There was a problem hiding this comment.
Unlike previously, these now do not include any HTLCs in the holding cell, and any outbound HTLCs in states where we are sure they will not be in any upcoming commitment transaction.
- these are
AwaitingRemoteRevokeToRemoveandAwaitingRemovedRemoteRevokeoutbound HTLCs
| let htlc_stats = self.get_pending_htlc_stats(funding, None, dust_exposure_limiting_feerate); | ||
| let next_commitment_htlcs = self.next_commitment_htlcs(None); | ||
| let value_to_self_msat = self.get_next_commitment_value_to_self_msat(funding); | ||
| let next_commitment_stats = SpecTxBuilder {}.get_builder_stats(funding.is_outbound(), funding.get_value_satoshis(), value_to_self_msat, &next_commitment_htlcs, 0, msg.feerate_per_kw, dust_exposure_limiting_feerate, funding.get_channel_type(), self.holder_dust_limit_satoshis, self.counterparty_dust_limit_satoshis); |
There was a problem hiding this comment.
SpecTxBuilder will calculate dust exposure using a buffer offset from msg.feerate_per_kw, not from cmp::max(msg.feerate_per_kw, self.feerate_per_kw)
| // Note that `stats.commit_tx_fee_sat` accounts for any HTLCs that transition from non-dust to dust under a higher feerate (in the case where HTLC-transactions pay endogenous fees). | ||
| if holder_balance_msat < stats.commit_tx_fee_sat * 1000 + funding.counterparty_selected_channel_reserve_satoshis.unwrap() * 1000 { |
There was a problem hiding this comment.
to obtain next_commitment_stats.holder_balance_msat
- we do not subtract holding cell HTLCs (previously these were subtracted, and would have resulted in a lower
holder_balance_msat) - we now subtract
OutboundHTLCState::AwaitingRemoteRevokeToRemove(..)HTLCs only if they were successful - these were previouslyincluded_in_commitmentand hence always subtracted fromvalue_to_self_msat, including in the failure case. - like before, we subtract
OutboundHTLCState::AwaitingRemovedRemoteRevoke(..)only in case of success.
| // Note that `stats.commit_tx_fee_sat` accounts for any HTLCs that transition from non-dust to dust under a higher feerate (in the case where HTLC-transactions pay endogenous fees). | ||
| if holder_balance_msat < stats.commit_tx_fee_sat * 1000 + funding.counterparty_selected_channel_reserve_satoshis.unwrap() * 1000 { | ||
| if next_commitment_stats.holder_balance_msat.unwrap() < next_commitment_stats.holder_commit_tx_fee_sat * 1000 + funding.counterparty_selected_channel_reserve_satoshis.unwrap() * 1000 { |
There was a problem hiding this comment.
next_commitment_stats.holder_commit_tx_fee_sat
- like before, we do not include holding cell HTLCs
- unlike before, we do not include
OutboundHTLCState::AwaitingRemoteRevokeToRemoveHTLCs (these were previouslyincluded_in_commitment) - we now include all pending inbound HTLCs, whereas we previously would include only
AwaitingAnnouncedRemoteRevokeandCommittedHTLCs
| let holder_balance_msat = stats.local_balance_before_fee_msat - htlc_stats.outbound_holding_cell_msat; | ||
| let next_commitment_htlcs = self.next_commitment_htlcs(None); | ||
| let value_to_self_msat = self.get_next_commitment_value_to_self_msat(funding); | ||
| let next_commitment_stats = SpecTxBuilder {}.get_builder_stats(funding.is_outbound(), funding.get_value_satoshis(), value_to_self_msat, &next_commitment_htlcs, CONCURRENT_INBOUND_HTLC_FEE_BUFFER as usize, feerate_per_kw, dust_exposure_limiting_feerate, funding.get_channel_type(), self.holder_dust_limit_satoshis, self.counterparty_dust_limit_satoshis); |
There was a problem hiding this comment.
SpecTxBuilder will calculate dust exposure using a buffer offset from feerate_per_kw, not from cmp::max(feerate_per_kw, self.feerate_per_kw, self.pending_update_fee)
| if next_commitment_stats.on_holder_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { | ||
| log_debug!(logger, "Cannot afford to send new feerate at {} without infringing max dust htlc exposure", feerate_per_kw); | ||
| return false; | ||
| } | ||
| if htlc_stats.on_counterparty_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { | ||
| if next_commitment_stats.on_counterparty_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { |
There was a problem hiding this comment.
These now do not include any outbound HTLCs in the holding cell, and any outbound HTLCs with states AwaitingRemoteRevokeToRemove and AwaitingRemovedRemoteRevoke
| }; | ||
| let next_commitment_htlcs = self.next_commitment_htlcs(None); | ||
| let value_to_self_msat = self.get_next_commitment_value_to_self_msat(funding); | ||
| let next_commitment_stats = SpecTxBuilder {}.get_builder_stats(funding.is_outbound(), funding.get_value_satoshis(), value_to_self_msat, &next_commitment_htlcs, fee_spike_buffer_htlc, self.feerate_per_kw, dust_exposure_limiting_feerate, funding.get_channel_type(), self.holder_dust_limit_satoshis, self.counterparty_dust_limit_satoshis); |
There was a problem hiding this comment.
SpecTxBuilder will now calculate dust exposure using a buffer offset from self.feerate_per_kw, not from cmp::max(self.feerate_per_kw, self.pending_update_fee)
| if on_counterparty_tx_dust_htlc_exposure_msat > max_dust_htlc_exposure_msat { | ||
| if next_commitment_stats.on_counterparty_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { | ||
| // Note that the total dust exposure includes both the dust HTLCs and the excess mining fees of the counterparty commitment transaction | ||
| log_info!(logger, "Cannot accept value that would put our total dust exposure at {} over the limit {} on counterparty commitment tx", | ||
| on_counterparty_tx_dust_htlc_exposure_msat, max_dust_htlc_exposure_msat); | ||
| next_commitment_stats.on_counterparty_tx_dust_exposure_msat, max_dust_htlc_exposure_msat); | ||
| return Err(LocalHTLCFailureReason::DustLimitCounterparty) | ||
| } | ||
| let dust_buffer_feerate = self.get_dust_buffer_feerate(None); | ||
| let (htlc_success_tx_fee_sat, _) = second_stage_tx_fees_sat( | ||
| &funding.get_channel_type(), dust_buffer_feerate, | ||
| ); | ||
| let exposure_dust_limit_success_sats = htlc_success_tx_fee_sat + self.holder_dust_limit_satoshis; | ||
| if msg.amount_msat / 1000 < exposure_dust_limit_success_sats { | ||
| let on_holder_tx_dust_htlc_exposure_msat = htlc_stats.on_holder_tx_dust_exposure_msat; | ||
| if on_holder_tx_dust_htlc_exposure_msat > max_dust_htlc_exposure_msat { | ||
| log_info!(logger, "Cannot accept value that would put our exposure to dust HTLCs at {} over the limit {} on holder commitment tx", | ||
| on_holder_tx_dust_htlc_exposure_msat, max_dust_htlc_exposure_msat); | ||
| return Err(LocalHTLCFailureReason::DustLimitHolder) | ||
| } | ||
| if next_commitment_stats.on_holder_tx_dust_exposure_msat > max_dust_htlc_exposure_msat { |
There was a problem hiding this comment.
These now do not include outbound holding cell HTLCs, and outbound AwaitingRemoteRevokeToRemove and AwaitingRemovedRemoteRevoke HTLCs
| let mut remote_fee_cost_incl_stuck_buffer_msat = self.next_remote_commit_tx_fee_msat( | ||
| funding, None, fee_spike_buffer_htlc, | ||
| ); | ||
| let mut remote_fee_cost_incl_stuck_buffer_msat = next_commitment_stats.counterparty_commit_tx_fee_sat * 1000; |
There was a problem hiding this comment.
As before, includes all inbound HTLCs, all Committed | RemoteRemoved | LocalAnnounced pending outbound HTLCs, and no holding cell HTLCs
| if !funding.get_channel_type().supports_anchors_zero_fee_htlc_tx() { | ||
| remote_fee_cost_incl_stuck_buffer_msat *= FEE_SPIKE_BUFFER_FEE_INCREASE_MULTIPLE; | ||
| } | ||
| let remote_balance_before_fee_msat = next_commitment_stats.counterparty_balance_msat.unwrap_or(0); |
There was a problem hiding this comment.
This value remains the same as before: channel_value_satoshis - value_to_self_msat - pending_inbound_htlcs + removed_outbound_htlcs
| // Note: We now always check holder dust exposure, whereas we previously would only | ||
| // do it if the incoming HTLC was dust on our own commitment transaction |
There was a problem hiding this comment.
Highlighting this comment here as well
clarify comments
| htlcs | ||
| .iter() | ||
| .filter_map(|htlc| { | ||
| htlc.is_dust(true, dust_buffer_feerate, holder_dust_limit_satoshis, channel_type) |
There was a problem hiding this comment.
Shouldn't this only count sent HTLCs? If our counterparty adds a dust HTLC it doesnt impact how much of our money might be burned to dust
There was a problem hiding this comment.
hmmm inbound HTLCs are your money once the preimage is revealed, so you do care how many of them go to dust ?
| use OutboundHTLCState::*; | ||
| let pending_outbound_htlcs = self.pending_outbound_htlcs.iter().filter_map( | ||
| |OutboundHTLCOutput { ref state, amount_msat, .. }| { | ||
| matches!(state, LocalAnnounced { .. } | Committed | RemoteRemoved { .. }) |
There was a problem hiding this comment.
This needs to depend on who generated the commitment we're talking about - if we're being called in, eg, validate_update_add_htlc, we're looking at a commitment the counterparty generated, which shouldn't include any LocalAnnounced or RemoteRemoved HTLCs, whereas if we're being called from can_send_update_fee we can't assume we'll wait until the counterparty sends a CS, so we need those.
There was a problem hiding this comment.
For LocalAnnounced HTLCs, we could do
(1) -> update_add_htlc `LocalAnnounced`
(2) <- update_add_htlc `validate_update_add`
-> commitment_signed
<- revoke_and_ack
(3) <- commitment_signed
-> revoke_and_ack
-> commitment_signed
<- revoke_and_ack
At (3), (1) is present on the local commitment transaction (generated by the counterparty), so seems reasonable to count it on the commitments at (2) ?
As for RemoteRemoved HTLCs, these for sure won't be on the next local commitment transaction, but might still be present on the next remote commitment transaction depending on the ordering of the CS...
So far, I have attempted to imitate fn next_local_commit_tx_fee_msat and fn next_remote_commit_tx_fee_msat as much as possible, will review all this tomorrow thank you.
| self.pending_outbound_htlcs | ||
| .iter() | ||
| .filter_map(|htlc| { | ||
| matches!( |
TBD