Skip to content

(draft) refactor(esplora): clear remaining panic paths #2053

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+63 −8
Open

(draft) refactor(esplora): clear remaining panic paths #2053

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e35a862
fix(esplora): propagate thread join failures
reez Sep 29, 2025
7d67591
fix(esplora): error if keychain iteration yields no indices
reez Sep 29, 2025
481f92d
fix(esplora): replace .expect() with error handling in chain_update
reez Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions crates/esplora/src/async_ext.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ async fn chain_update<S: Sleeper>(

tip = tip
.extend(conflicts.into_iter().rev().map(|b| (b.height, b.hash)))
.expect("evicted are in order");
.map_err(|_| Box::new(esplora_client::Error::InvalidResponse))?;

for (anchor, _txid) in anchors {
let height = anchor.block_id.height;
Expand Down Expand Up @@ -348,6 +348,9 @@ where
.collect::<FuturesOrdered<_>>();

if handles.is_empty() {
if last_index.is_none() {
return Err(Box::new(esplora_client::Error::InvalidResponse));
}
break;
}

Expand All @@ -368,7 +371,8 @@ where
.extend(evicted.into_iter().map(|txid| (txid, start_time)));
}

let last_index = last_index.expect("Must be set since handles wasn't empty.");
let last_index =
last_index.ok_or_else(|| Box::new(esplora_client::Error::InvalidResponse))?;
Comment on lines +374 to +375
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One suggestion is to leave last_index as an Option and wrap stop_gap in Some(..) when doing the comparison.

But there's a much better way to handle this, which is to introduce an unused_count variable. Now when processing handles, if txs.is_empty() we increment the unused count, else update the last active index (and reset the unused count). Finally, break once the unused count reaches the gap limit. AFAICT there'd be no reason to keep track of the last_index scanned.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took your suggestion and worked it in; didn't want to push it to this branch yet if I had the wrong idea, but here is the commit reez@35538e2 on a branch that builds on this branch.

I felt like I had to keep two small guardrails from the old logic:

  • processed_any (still surface the “no handles ran” error)
  • gap_limit = stop_gap.max(1)(keeps old “stop_gap==0 still means one empty derivation” behavior)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's an error if handles turns up empty - we need a way to break from the loop once all of the keychain-spks are consumed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe stop_gap = stop_gap.max(parallel_requests), since we check at least 1 spk-index for every request in parallel_requests.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super helpful thanks mammal, I made the below changes to that my side branch esplora+mammal that I can eventually fold back into this esplora branch if I'm on the right track

I don't think it's an error if handles turns up empty - we need a way to break from the loop once all of the keychain-spks are consumed.

Good call. I made this change 36073d0 based on that

Maybe stop_gap = stop_gap.max(parallel_requests), since we check at least 1 spk-index for every request in parallel_requests.

Good catch, made another change 590edcf based on this

let gap_limit_reached = if let Some(i) = last_active_index {
last_index >= i.saturating_add(stop_gap as u32)
} else {
Expand Down Expand Up @@ -571,6 +575,15 @@ mod test {
}};
}

#[test]
fn ensure_last_index_none_returns_error() {
let last_index: Option<u32> = None;
let err = last_index
.ok_or_else(|| Box::new(esplora_client::Error::InvalidResponse))
.unwrap_err();
assert!(matches!(*err, esplora_client::Error::InvalidResponse));
}

// Test that `chain_update` fails due to wrong network.
#[tokio::test]
async fn test_chain_update_wrong_network_error() -> anyhow::Result<()> {
Expand Down
54 changes: 48 additions & 6 deletions crates/esplora/src/blocking_ext.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ fn chain_update(

tip = tip
.extend(conflicts.into_iter().rev().map(|b| (b.height, b.hash)))
.expect("evicted are in order");
.map_err(|_| Box::new(esplora_client::Error::InvalidResponse))?;

for (anchor, _) in anchors {
let height = anchor.block_id.height;
Expand Down Expand Up @@ -316,11 +316,17 @@ fn fetch_txs_with_keychain_spks<I: Iterator<Item = Indexed<SpkWithExpectedTxids>
.collect::<Vec<JoinHandle<Result<TxsOfSpkIndex, Error>>>>();

if handles.is_empty() {
if last_index.is_none() {
return Err(Box::new(esplora_client::Error::InvalidResponse));
}
break;
}

for handle in handles {
let (index, txs, evicted) = handle.join().expect("thread must not panic")?;
let handle_result = handle
.join()
.map_err(|_| Box::new(esplora_client::Error::InvalidResponse))?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because you're dropping the error it won't be very informative to propagate it upward. Instead this module can define its own Error type to handle errors that aren't covered by esplora_client::Error.

let (index, txs, evicted) = handle_result?;
last_index = Some(index);
if !txs.is_empty() {
last_active_index = Some(index);
Expand All @@ -337,7 +343,8 @@ fn fetch_txs_with_keychain_spks<I: Iterator<Item = Indexed<SpkWithExpectedTxids>
.extend(evicted.into_iter().map(|txid| (txid, start_time)));
}

let last_index = last_index.expect("Must be set since handles wasn't empty.");
let last_index =
last_index.ok_or_else(|| Box::new(esplora_client::Error::InvalidResponse))?;
let gap_limit_reached = if let Some(i) = last_active_index {
last_index >= i.saturating_add(stop_gap as u32)
} else {
Expand Down Expand Up @@ -417,7 +424,10 @@ fn fetch_txs_with_txids<I: IntoIterator<Item = Txid>>(
}

for handle in handles {
let (txid, tx_info) = handle.join().expect("thread must not panic")?;
let handle_result = handle
.join()
.map_err(|_| Box::new(esplora_client::Error::InvalidResponse))?;
let (txid, tx_info) = handle_result?;
if let Some(tx_info) = tx_info {
if inserted_txs.insert(txid) {
update.txs.push(tx_info.to_tx().into());
Expand Down Expand Up @@ -478,7 +488,10 @@ fn fetch_txs_with_outpoints<I: IntoIterator<Item = OutPoint>>(
}

for handle in handles {
if let Some(op_status) = handle.join().expect("thread must not panic")? {
let handle_result = handle
.join()
.map_err(|_| Box::new(esplora_client::Error::InvalidResponse))?;
if let Some(op_status) = handle_result? {
let spend_txid = match op_status.txid {
Some(txid) => txid,
None => continue,
Expand Down Expand Up @@ -511,7 +524,7 @@ fn fetch_txs_with_outpoints<I: IntoIterator<Item = OutPoint>>(
#[cfg(test)]
#[cfg_attr(coverage_nightly, coverage(off))]
mod test {
use crate::blocking_ext::{chain_update, fetch_latest_blocks};
use crate::blocking_ext::{chain_update, fetch_latest_blocks, Error};
use bdk_chain::bitcoin;
use bdk_chain::bitcoin::hashes::Hash;
use bdk_chain::bitcoin::Txid;
Expand All @@ -529,6 +542,35 @@ mod test {
}};
}

#[test]
fn thread_join_panic_maps_to_error() {
let handle = std::thread::spawn(|| -> Result<(), Error> {
panic!("expected panic for test coverage");
});

let res = (|| -> Result<(), Error> {
let handle_result = handle
.join()
.map_err(|_| Box::new(esplora_client::Error::InvalidResponse))?;
handle_result?;
Ok(())
})();

assert!(matches!(
*res.unwrap_err(),
esplora_client::Error::InvalidResponse
));
}

#[test]
fn ensure_last_index_none_returns_error() {
let last_index: Option<u32> = None;
let err = last_index
.ok_or_else(|| Box::new(esplora_client::Error::InvalidResponse))
.unwrap_err();
assert!(matches!(*err, esplora_client::Error::InvalidResponse));
}

macro_rules! local_chain {
[ $(($height:expr, $block_hash:expr)), * ] => {{
#[allow(unused_mut)]
Expand Down
Loading