Add Multi-RFQ Send

[GeorgeTsagk]

Description

Allows payments to use multiple quotes across multiple peers in order to pay an invoice. Previously we had to define a specific peer to negotiate a quote with, with this PR this is no longer required (but still supported) as Tapd will automatically scan our peers and establish quotes with all valid ones for the asset/amount of this payment.

The signature of ProduceHtlcExtraData had to be changed, as it's not possible to distinguish which of the quotes in the rfqmsg.Htlc should be used. We now provide the pubkey of the peer this HTLC is being sent to, in order to help Tapd extract the corresponding quote and calculate the correct amount of asset units.

Closes #1358

Depends on: lightningnetwork/lnd#9980

[guggero]

Looks pretty good! Have a couple of questions and suggestions, nothing major though.

[guggero]

We allocate memory from a number we received over the wire. Need to check and error out the length before to make sure we limit the number of bytes we allocate.

[GeorgeTsagk]

Good point, so far the total length of available RFQ IDs is open ended.

We should introduce a static limit (which would also limit the max number of quotes we acquire) and enforce it everywhere.

[coveralls]

Pull Request Test Coverage Report for Build 16603360913

Details

• 76 of 406 (18.72%) changed or added relevant lines in 7 files are covered.
• 69 unchanged lines in 14 files lost coverage.
• Overall coverage decreased (-0.09%) to 56.444%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
server.go	0	2	0.0%
fn/send.go	0	6	0.0%
rfqmsg/records.go	64	81	79.01%
taprpc/tapchannelrpc/tapchannel.pb.go	6	70	8.57%
tapchannel/aux_traffic_shaper.go	0	82	0.0%
rpcserver.go	3	162	1.85%

Files with Coverage Reduction	New Missed Lines	%
fn/context_guard.go	1	88.71%
taprpc/tapchannelrpc/tapchannel.pb.go	1	16.62%
asset/group_key.go	2	72.15%
rpcserver.go	2	61.41%
tapdb/sqlc/transfers.sql.go	2	82.87%
tapchannel/aux_leaf_signer.go	3	43.43%
tapdb/assets_store.go	3	79.18%
universe/archive.go	3	80.0%
rpcperms/interceptor.go	4	79.57%
tapchannel/aux_traffic_shaper.go	5	6.33%

Totals
Change from base Build 16599218187:	-0.09%
Covered Lines:	59034
Relevant Lines:	104588

---

💛 - Coveralls

[ZZiigguurraatt]

Please update

taproot-assets/taprpc/tapchannelrpc/tapchannel.proto

Lines 125 to 130 in df8ce52

	// The node identity public key of the peer to ask for a quote for sending
	// out the assets and converting them to satoshis. This must be specified if
	// there are multiple channels with the given asset ID.
	bytes peer_pubkey = 3;

to reflect this new capability. Also, I think we need to be more specific to include group_key in addition to asset_id in there.

[ZZiigguurraatt]

What is this expected to do with an invoice with no amount specified?

[ZZiigguurraatt]

rfq_id is of type bytes

taproot-assets/taprpc/tapchannelrpc/tapchannel.proto

Lines 139 to 144 in df8ce52

	// The rfq id to use for this payment. If the user sets this value then the
	// payment will immediately be dispatched, skipping the rfq negotiation
	// phase, and using the following rfq id instead.
	bytes rfq_id = 5;

but with multi-rfq send, I would expect the ability to provide an array. Not sure that you want to fix that in this PR, but maybe it should be a separate issue? I did not make this comment for multi-rfq receive because of #1442 .

[guggero]

Did another pass, getting very close here too.

[ffranr]

I'm up-to-date with this PR. I see that some of Oli's comments are unresolved. I don't have anything to add otherwise. I can review quickly once Oli's comments are addressed.

[ffranr]

We should consider adding a comment here to guide the user on the appropriate alternative action.

[GeorgeTsagk]

I'm not sure there even is an alternative action? We're deprecating this because as far as we know nobody is using this anymore

[ffranr]

Ah, was this RPC endpoint only used internally? And since we don’t think it’s useful to users, we’re removing it—is that the idea?

[guggero]

I think streaming multiple quotes to the user is probably not the best idea, since there is no way for the user to know when they're done reading (other than the stream being closed).
So my idea was to add a second field to this RPC message, AcceptedSellOrders that is a slice of quotes. Then we can deprecate the old field (and update its documentation to say it will only ever contain the first quote and that clients must update to receive the full list).

[ZZiigguurraatt]

I think streaming multiple quotes to the user is probably not the best idea, since there is no way for the user to know when they're done reading (other than the stream being closed). So my idea was to add a second field to this RPC message, AcceptedSellOrders that is a slice of quotes. Then we can deprecate the old field (and update its documentation to say it will only ever contain the first quote and that clients must update to receive the full list).

yeah, summary was here #1613 (comment) . not sure if @GeorgeTsagk was part of that conversation though??

[GeorgeTsagk]

I think the clients know when to terminate the stream once they receive the payment updates (since the RFQ responses precede the payment update)

Regardless, I went ahead with this change and it's added in the new diff

[guggero]

Ah, you're right, if you want all errors then the errgroup.Group isn't useful.
But I think we already have a helper for exactly what you need, that does all the mutex locking and stuff: fn.ParSliceErrCollect().

[ZZiigguurraatt]

Would like to see some release notes for this PR similar to a74f887 in #1587

[GeorgeTsagk]

Changed the RPC response stream to now return all the quotes in an array.
Also added a release note for the introduced multi-rfq feature.

[guggero]

LGTM 🎉

[guggero]

Yes, perhaps something like this?

// ResultFunc is a type def for a function that takes a context (to allow early
// cancellation) and a series of value returning a Result. This is typically
// used a closure to perform concurrent work over a homogeneous slice of
// values.
type ResultFunc[V any, R fn.Result[any]] func(context.Context, V) R

// ParSliceResultCollect can be used to execute a function on each element of a
// slice in parallel. This function is fully blocking and will wait for all
// goroutines to finish (subject to context cancellation/timeout). Any results
// will be collected and returned as a map of slice element index to Result.
// Active goroutines limited with number of CPU.
func ParSliceResultCollect[V any, R fn.Result[any]](ctx context.Context, s []V,
	f ResultFunc[V, R]) (map[int]R, error) {

	errGroup, ctx := errgroup.WithContext(ctx)
	errGroup.SetLimit(runtime.GOMAXPROCS(0))

	var instanceResultsMutex sync.Mutex
	instanceResults := make(map[int]R, len(s))

	for idx := range s {
		errGroup.Go(func() error {
			result := f(ctx, s[idx])
			instanceResultsMutex.Lock()
			instanceResults[idx] = result
			instanceResultsMutex.Unlock()

			// Avoid returning an error here, as that would cancel
			// the errGroup and terminate all slice element
			// processing instances. Instead, collect the error and
			// return it later.
			return nil
		})
	}

	// Now we will wait/block for all goroutines to finish.
	//
	// The goroutines that are executing in parallel should not return an
	// error, but the Wait call may return an error if the context is
	// canceled or timed out.
	err := errGroup.Wait()
	if err != nil {
		return nil, fmt.Errorf("failed to wait on error group in "+
			"ParSliceErrorCollect: %w", err)
	}

	return instanceResults, nil
}

[guggero]

But can do in a follow-up PR.

[guggero]

👍 looks good. This will require a small adjustment in litcli as well, just as a reminder.

[ffranr]

Took a look at this from the beginning once more. Noticed a few things we might want to change. But all trivial. I can approve quickly in a followup review.