We've introduced usage of tenacity in #366, but we're using normal retries. It'd be nicer to have exponential backoff.