Skip to content

Conversation

soonping-amzn
Copy link
Contributor

@soonping-amzn soonping-amzn commented Jun 8, 2023

What this PR does:
Adds context timeouts to calls to etcd from the KV store etcd implementation.

When called with a context without deadline, underlying calls to etcd from the etcd implementation of KV store can block forever when etcd is unavailable, emitting no logs or KV metrics indicating the problem.

For consistency and predictability, this PR adds timeouts to the context at the KV store level, rather than relying on clients of the KV store to specify deadlines at every call. This allows calls to timeout and emit metrics that can be alerted and acted on.

Which issue(s) this PR fixes:
N/A

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@alanprot
Copy link
Member

It make sense to me! Thanks

@soonping-amzn soonping-amzn force-pushed the 20230607-etcd-context-timeout branch from a169a48 to 0e25ee1 Compare June 14, 2023 00:58
@friedrichg friedrichg merged commit aeec731 into cortexproject:master Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants