garbage collect zero-value UTXOs #1832

darioAnongba · 2025-10-01T15:04:36Z

Garbage collect the residue orphaned UTXOs when creating transactions. Orphaned UTXOs occur when creating tombstones or full burns.
Currently, these UTXOs accumulate in the DB and are never cleaned. This PR introduces a garbage collection mechanism to collect these UTXOs and use them as inputs of transactions initiated by tapd:

Transfers
Burns

The PR adds a new swept flag to the managed_utxo table because UTXOs are not removed from the table when spent. This flag is also returned by the ListUtxos RPC endpoint.

The mechanism preserves the liveness and safety properties, ensuring that zero-value UTXOs can never accumulate in the DB. Adding garbage collection to Mint transactions is not necessary to ensure these properties.

Fixes #514

Note to reviewers

UTXO meaning unspent, keeping the spent outputs in a table called manage_utxos seems like a contradiction, same logic applies for the need of a swept flag in that table. We should either rename the table or store spent utxos somewhere else?

coveralls · 2025-10-01T15:30:55Z

Pull Request Test Coverage Report for Build 19360804740

Details

295 of 380 (77.63%) changed or added relevant lines in 16 files are covered.
57 unchanged lines in 10 files lost coverage.
Overall coverage increased (+7.0%) to 56.584%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
asset/asset.go	3	5	60.0%
itest/assertions.go	0	3	0.0%
rpcserver.go	10	15	66.67%
tapfreighter/chain_porter.go	40	45	88.89%
tapfreighter/coin_select.go	23	29	79.31%
tapfreighter/wallet.go	55	76	72.37%
tapdb/assets_store.go	104	147	70.75%

Files with Coverage Reduction	New Missed Lines	%
tapdb/multiverse.go	2	80.85%
tapdb/sqlc/mssmt.sql.go	2	48.34%
tapdb/universe.go	2	80.81%
rpcserver.go	4	61.36%
tapdb/mssmt.go	4	90.45%
mssmt/compacted_tree.go	6	78.57%
tapfreighter/chain_porter.go	6	83.31%
tapgarden/custodian.go	8	77.02%
tapgarden/re-org_watcher.go	11	71.65%
tapdb/assets_store.go	12	79.03%

Totals
Change from base Build 19360481010:	7.0%
Covered Lines:	64603
Relevant Lines:	114172

💛 - Coveralls

darioAnongba · 2025-10-28T17:42:54Z

I like SweepOrphanUtxos better as well. I renamed it everywhere at the config level, let me know if you think we should rename all the way down to the DB layer. In the context of assets, zero-value is correct though, but more verbose and uglier.

GeorgeTsagk

last nits, 99% there

itest/burn_test.go

itest/integration_test.go

tapfreighter/wallet.go

rpcserver.go

itest/test_list_on_test.go

itest/zero_value_anchor_test.go

GeorgeTsagk

LGTM

docs/release-notes/release-notes-0.8.0.md

tapdb/assets_store.go

This includes the addition of a "swept" field in the "managed_utxos" table with the corresponding migration and the "MarkManagedUTXOAsSwept" function.

…ng family and index

lightninglabs-deploy · 2025-11-18T10:48:17Z

@ffranr: review reminder

ffranr

I think we should double check our logic around coin release but otherwise just nits.

ffranr · 2025-11-17T15:23:57Z

tapdb/assets_store.go

+				LeaseExpiry: sql.NullTime{
+					Time:  finalLeaseExpiry.UTC(),
+					Valid: true,
+				},


Could be a helper like sqlInt16

ffranr · 2025-11-17T15:24:25Z

tapdb/assets_store.go

+			err := readOutPoint(
+				bytes.NewReader(u.Outpoint), 0, 0, &anchorPoint)


trailing ) should be on a new line

ffranr · 2025-11-17T15:24:47Z

tapdb/assets_store.go

+			err = q.MarkManagedUTXOAsSwept(ctx,
+				MarkManagedUTXOAsSweptParams{
+					Outpoint:     outpointBytes,
+					SweepingTxid: conf.AnchorTXID[:],
+				})


trailing ) should be on a new line

ffranr · 2025-11-18T17:53:41Z

tapfreighter/chain_porter.go

-		len(pkg.InputCommitments) > 0 {
+	// Also unlock any zero-value UTXOs that were leased for this package.
+	if pkg.SendState < SendStateStorePreBroadcast {
+		// Gather all outpoints to unlock in a single array


Punctuation missing for comments here and below.

ffranr · 2025-11-18T17:59:11Z

tapfreighter/coin_select.go

+	// We now need to lock/lease/reserve those selected coins so
+	// that they can't be used by other processes.
+	if len(zeroValueInputs) > 0 {
+		expiry := time.Now().Add(defaultCoinLeaseDuration)


Worth using time.Now().UTC() here IMO so every time as much as possible is UTC at every point.

ffranr · 2025-11-18T18:15:26Z

tapfreighter/wallet.go

+	zeroValueInputs, err := f.cfg.CoinSelector.SelectOrphanCoins(ctx)
+	if err != nil {
+		return nil, fmt.Errorf("unable to select zero-value "+
+			"UTXOs: %w", err)
+	}


Same here. If we return an error at this point then we wont release all coins.

ffranr · 2025-11-18T18:26:58Z

itest/zero_value_anchor_test.go

+	// Send full amount of the new asset. This should sweep Alice's
+	// first tombstone and create a new one.
+	bobAddr2, err := secondTapd.NewAddr(ctxb, &taprpc.NewAddrRequest{
+		AssetId:      genInfo2.AssetId,
+		Amt:          assetAmount,
+		AssetVersion: rpcAssets2[0].Version,
+	})
+	require.NoError(t.t, err)
+
+	sendResp2, _ := sendAssetsToAddr(t, t.tapd, bobAddr2)
+
+	ConfirmAndAssertOutboundTransfer(
+		t.t, t.lndHarness.Miner().Client, t.tapd, sendResp2,
+		genInfo2.AssetId,
+		[]uint64{0, assetAmount}, 1, 2,
+	)
+	AssertNonInteractiveRecvComplete(t.t, secondTapd, 2)


Maybe we can inspect this transaction to ensure that the additional zero-value tombstone input is present.

ffranr · 2025-11-18T18:32:36Z

itest/zero_value_anchor_test.go

+	// Wait for the node to fully sync after restart.
+	time.Sleep(2 * time.Second)
+
+	// Verify that the zero-value UTXOs are still present after restart.
+	//nolint:lll
+	tombstoneUtxosAfterRestart, err := t.tapd.ListUtxos(ctxb, &taprpc.ListUtxosRequest{


Instead of this sleep you could possibly use ListUtxosRPC itest helper. Sleep can cause flakes.

ffranr · 2025-11-18T18:37:40Z

itest/assertions.go

-	require.NoError(t, err)
-
-	transferResp, err := sender.ListTransfers(
-		ctxb, &taprpc.ListTransfersRequest{},
-	)
-	require.NoError(t, err)
-
-	transferRespJSON, err := formatProtoJSON(transferResp)
-	require.NoError(t, err)
-	t.Logf("Got response from list transfers: %v", transferRespJSON)


Why remove these lines? That doesn't seem to fix under the commit subject. Same further above also.

ffranr · 2025-11-18T18:47:44Z

itest/test_list_on_test.go

 		test: testZeroValueAnchorSweep,
+		tapdOptions: []Option{
+			WithSweepOrphanUtxos(),
+		},
+	},


hmm I don't think I'm a fan of this change. I think it was clearer if t.tapd had the default options and we just create a new tapd instance if we need to specify configuration. I think we should remove the proofCourierType field also.

I don't think we should consult this list to see how the primary tapd node is configured. But I won't block on this change, we can revisit later.

darioAnongba self-assigned this Oct 1, 2025

levmi added this to Taproot-Assets Project Board Oct 2, 2025

github-project-automation bot moved this to 🆕 New in Taproot-Assets Project Board Oct 2, 2025

levmi moved this from 🆕 New to 🏗 In progress in Taproot-Assets Project Board Oct 2, 2025

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from f3a3bff to f55b6c5 Compare October 3, 2025 16:21

darioAnongba changed the base branch from main to 0-8-0-staging October 6, 2025 09:43

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 3 times, most recently from 3bd769d to 6fe11a5 Compare October 6, 2025 15:03

darioAnongba changed the base branch from 0-8-0-staging to main October 6, 2025 15:03

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 10 times, most recently from a8f3ce4 to afbfebf Compare October 9, 2025 13:04

darioAnongba marked this pull request as ready for review October 9, 2025 13:09

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 4 times, most recently from bf3e3ee to d812a8e Compare October 9, 2025 16:04

darioAnongba requested review from Roasbeef and ffranr October 9, 2025 16:25

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from d812a8e to 0ceaf76 Compare October 9, 2025 16:32

darioAnongba moved this from 🏗 In progress to 👀 In review in Taproot-Assets Project Board Oct 9, 2025

darioAnongba requested a review from ffranr October 28, 2025 17:43

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 4 times, most recently from ef93717 to d9383ba Compare October 28, 2025 18:58

GeorgeTsagk reviewed Oct 30, 2025

View reviewed changes

ffranr added this to the v0.8 milestone Oct 30, 2025

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from d9383ba to 773c448 Compare October 30, 2025 13:26

darioAnongba requested a review from GeorgeTsagk October 30, 2025 13:50

GeorgeTsagk approved these changes Oct 30, 2025

View reviewed changes

docs/release-notes/release-notes-0.8.0.md Show resolved Hide resolved

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from 773c448 to 4fb0135 Compare October 30, 2025 15:58

ffranr reviewed Nov 10, 2025

View reviewed changes

tapdb/assets_store.go Show resolved Hide resolved

darioAnongba force-pushed the feat/zero-value-utxo-selection branch 4 times, most recently from 1cf422a to 66faf3f Compare November 10, 2025 21:00

darioAnongba requested a review from ffranr November 10, 2025 21:31

darioAnongba force-pushed the 0-8-0-staging branch from 2b3ac4f to 035a840 Compare November 14, 2025 09:38

darioAnongba added 7 commits November 14, 2025 10:51

tapdb: add support for zero-value utxo sweeping

aef5d1f

This includes the addition of a "swept" field in the "managed_utxos" table with the corresponding migration and the "MarkManagedUTXOAsSwept" function.

tapfreighter: add call to FetchOrphanUTXOs to retrieve zero-value inputs

565abe4

tapsend: include BIP32 key derivation to allocation to preserve signi…

551d563

…ng family and index

taprpc+rpc: Apply garbage collection in AnchorVirtualPsbts

17e710a

itest: add test for zero-value utxo garbage collection

280d823

tapcfg: configure sweeping with wallet flag

b5a1f49

docs: add release notes

61a19e9

darioAnongba force-pushed the feat/zero-value-utxo-selection branch from 66faf3f to 61a19e9 Compare November 14, 2025 09:51

ffranr reviewed Nov 18, 2025

View reviewed changes

		err := readOutPoint(
		bytes.NewReader(u.Outpoint), 0, 0, &anchorPoint)

garbage collect zero-value UTXOs #1832

Are you sure you want to change the base?

garbage collect zero-value UTXOs #1832

Uh oh!

Conversation

darioAnongba commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 19360804740

Details

💛 - Coveralls

Uh oh!

darioAnongba commented Oct 28, 2025

Uh oh!

GeorgeTsagk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GeorgeTsagk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lightninglabs-deploy commented Nov 18, 2025

Uh oh!

ffranr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

darioAnongba commented Oct 1, 2025 •

edited

Loading

coveralls commented Oct 1, 2025 •

edited

Loading