Enable TTL on DynamoDB table creation #170
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds TTL enablement to the DynamoDB store setup process. Previously, TTL values were stored in items but the table-level TTL feature was never enabled, meaning expired items would never be automatically deleted by DynamoDB. Changes: - Added update_time_to_live call in _setup() method - Enabled TTL on the 'ttl' attribute after table creation - Added error handling for ValidationException (already enabled case) This addresses one of the critical issues identified in #168 for marking DynamoDB store as stable. Batch operations implementation is deferred to a future PR. Co-authored-by: William Easton <[email protected]>
Contributor
📝 WalkthroughWalkthroughAfter ensuring the DynamoDB table exists, the code now queries TTL status via DescribeTimeToLive and, if status is DISABLED, calls UpdateTimeToLive to enable TTL on the "ttl" attribute; ValidationException from the update is suppressed, other ClientError exceptions are re-raised. (46 words) Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Caller
participant Store as DynamoDBStore
participant AWS as DynamoDBAPI
Caller->>Store: _setup()
Store->>AWS: DescribeTable (ensure exists)
AWS-->>Store: Table exists
Store->>AWS: DescribeTimeToLive(tableName)
AWS-->>Store: TimeToLiveStatus: ENABLED/DISABLED/ENABLING/DISABLING
alt TTL DISABLED
Store->>AWS: UpdateTimeToLive(enable on "ttl")
AWS-->>Store: Success / ValidationException / ClientError
alt ValidationException
Note over Store: Suppress (TTL already set)
else Other ClientError
Note over Store: Re-raise error
end
else TTL not DISABLED
Note over Store: No update performed
end
Store-->>Caller: _setup() completes
Possibly related PRs
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
key-value/key-value-aio/src/key_value/aio/stores/dynamodb/store.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
- GitHub Check: test_quick (3.10, windows-latest, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, windows-latest, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, windows-2022, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, windows-2022, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, macos-14, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, macos-latest, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, macos-latest, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, ubuntu-latest, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, macos-14, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, ubuntu-22.04, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, ubuntu-latest, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, ubuntu-22.04, key-value/key-value-sync)
key-value/key-value-aio/src/key_value/aio/stores/dynamodb/store.py
Outdated
Show resolved
Hide resolved
Owner
Author
|
@claude can we check if the table has ttl on it before we try to update it? |
|
Claude finished @strawgate's task —— View job TTL Check Implementation Complete ✅
SummaryI've successfully updated the DynamoDB store implementation to check if TTL is enabled before attempting to enable it. This addresses your request for a more proactive approach. Changes MadeFile modified: The implementation now:
This approach is cleaner because:
Code snippet:# Enable TTL on the table if not already enabled
ttl_response = await self._connected_client.describe_time_to_live(
TableName=self._table_name
)
ttl_status = ttl_response.get("TimeToLiveDescription", {}).get("TimeToLiveStatus")
# Only enable TTL if it's currently disabled
if ttl_status == "DISABLED":
await self._connected_client.update_time_to_live(
TableName=self._table_name,
TimeToLiveSpecification={
"Enabled": True,
"AttributeName": "ttl",
},
)✅ All checks passed: linting, type checking --- | |
Instead of attempting to enable TTL and catching a ValidationException, we now proactively check the TTL status using describe_time_to_live and only call update_time_to_live if the status is DISABLED. This approach: - Avoids relying on exception handling for flow control - Prevents masking legitimate ValidationException errors - Makes the code more explicit and easier to understand Co-authored-by: William Easton <[email protected]>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
key-value/key-value-aio/src/key_value/aio/stores/dynamodb/store.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: test_quick (3.10, windows-2022, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, windows-2022, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, macos-14, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, windows-latest, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, windows-latest, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, windows-latest, key-value/key-value-shared)
- GitHub Check: test_quick (3.10, ubuntu-latest, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, macos-latest, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, macos-latest, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, macos-14, key-value/key-value-shared)
- GitHub Check: test_quick (3.10, ubuntu-22.04, key-value/key-value-sync)
- GitHub Check: test_quick (3.10, ubuntu-22.04, key-value/key-value-aio)
- GitHub Check: test_quick (3.10, ubuntu-latest, key-value/key-value-aio)
Comment on lines
+167
to
+182
| # Enable TTL on the table if not already enabled | ||
| ttl_response = await self._connected_client.describe_time_to_live( # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] | ||
| TableName=self._table_name | ||
| ) | ||
| ttl_status = ttl_response.get("TimeToLiveDescription", {}).get("TimeToLiveStatus") # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] | ||
|
|
||
| # Only enable TTL if it's currently disabled | ||
| if ttl_status == "DISABLED": | ||
| await self._connected_client.update_time_to_live( # pyright: ignore[reportUnknownMemberType] | ||
| TableName=self._table_name, | ||
| TimeToLiveSpecification={ | ||
| "Enabled": True, | ||
| "AttributeName": "ttl", | ||
| }, | ||
| ) | ||
|
|
Contributor
There was a problem hiding this comment.
Add error handling and handle transient TTL states.
The check-before-update approach addresses the previous concern about broad exception handling. However, the current implementation has several issues:
-
Missing error handling: The
update_time_to_livecall has no error handling. If this fails (due to permissions, rate limiting, transient errors, or concurrent updates), the exception will propagate and break the setup process. -
Unhandled transient states: DynamoDB TTL can be in
ENABLINGorDISABLINGstates during transitions. The code only checks forDISABLED, so:- If TTL is
ENABLING, the code skips the update (good) but doesn't verify it completes - If TTL is
DISABLING, the code skips the update, leaving TTL in an incomplete state - If
ttl_statusisNone(malformed response), the check silently fails
- If TTL is
-
Race condition: Between checking the status and calling update, another process could start enabling TTL, causing a
ValidationExceptionfrom the update call.
Apply this diff to add proper error handling and state management:
# Enable TTL on the table if not already enabled
ttl_response = await self._connected_client.describe_time_to_live( # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]
TableName=self._table_name
)
ttl_status = ttl_response.get("TimeToLiveDescription", {}).get("TimeToLiveStatus") # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]
- # Only enable TTL if it's currently disabled
- if ttl_status == "DISABLED":
- await self._connected_client.update_time_to_live( # pyright: ignore[reportUnknownMemberType]
- TableName=self._table_name,
- TimeToLiveSpecification={
- "Enabled": True,
- "AttributeName": "ttl",
- },
- )
+ # Only enable TTL if it's currently disabled or unknown
+ if ttl_status in ("DISABLED", None):
+ try:
+ await self._connected_client.update_time_to_live( # pyright: ignore[reportUnknownMemberType]
+ TableName=self._table_name,
+ TimeToLiveSpecification={
+ "Enabled": True,
+ "AttributeName": "ttl",
+ },
+ )
+ except self._connected_client.exceptions.ClientError as e: # pyright: ignore[reportUnknownMemberType]
+ error_code = e.response["Error"]["Code"] # pyright: ignore[reportUnknownMemberType]
+ # Suppress only if TTL is already enabled/enabling; re-raise other errors
+ if error_code == "ValidationException":
+ error_message = e.response["Error"]["Message"] # pyright: ignore[reportUnknownMemberType]
+ if "already enabled" not in error_message.lower() and "already being" not in error_message.lower(): # pyright: ignore[reportUnknownArgumentType]
+ raise
+ else:
+ raise
+ # If status is ENABLING, TTL will be enabled soon; if ENABLED, nothing to doNote: This reintroduces exception handling but only suppresses the specific "already enabled" case, addressing the concern from the previous review while maintaining robustness.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # Enable TTL on the table if not already enabled | |
| ttl_response = await self._connected_client.describe_time_to_live( # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] | |
| TableName=self._table_name | |
| ) | |
| ttl_status = ttl_response.get("TimeToLiveDescription", {}).get("TimeToLiveStatus") # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] | |
| # Only enable TTL if it's currently disabled | |
| if ttl_status == "DISABLED": | |
| await self._connected_client.update_time_to_live( # pyright: ignore[reportUnknownMemberType] | |
| TableName=self._table_name, | |
| TimeToLiveSpecification={ | |
| "Enabled": True, | |
| "AttributeName": "ttl", | |
| }, | |
| ) | |
| # Enable TTL on the table if not already enabled | |
| ttl_response = await self._connected_client.describe_time_to_live( # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] | |
| TableName=self._table_name | |
| ) | |
| ttl_status = ttl_response.get("TimeToLiveDescription", {}).get("TimeToLiveStatus") # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] | |
| # Only enable TTL if it's currently disabled or unknown | |
| if ttl_status in ("DISABLED", None): | |
| try: | |
| await self._connected_client.update_time_to_live( # pyright: ignore[reportUnknownMemberType] | |
| TableName=self._table_name, | |
| TimeToLiveSpecification={ | |
| "Enabled": True, | |
| "AttributeName": "ttl", | |
| }, | |
| ) | |
| except self._connected_client.exceptions.ClientError as e: # pyright: ignore[reportUnknownMemberType] | |
| error_code = e.response["Error"]["Code"] # pyright: ignore[reportUnknownMemberType] | |
| # Suppress only if TTL is already enabled/enabling; re-raise other errors | |
| if error_code == "ValidationException": | |
| error_message = e.response["Error"]["Message"] # pyright: ignore[reportUnknownMemberType] | |
| if "already enabled" not in error_message.lower() and "already being" not in error_message.lower(): # pyright: ignore[reportUnknownArgumentType] | |
| raise | |
| else: | |
| raise | |
| # If status is ENABLING, TTL will be enabled soon; if ENABLED, nothing to do |
🤖 Prompt for AI Agents
In key-value/key-value-aio/src/key_value/aio/stores/dynamodb/store.py around
lines 167 to 182, add robust error handling and transient-state management for
TTL: validate ttl_status for None and treat it as unknown, if ttl_status is
"ENABLING" wait/poll with a short backoff until it becomes "ENABLED" or a
timeout expires, if ttl_status is "DISABLING" wait/poll until it is "DISABLED"
or timeout before attempting change, wrap the update_time_to_live call in a
try/except to catch and suppress the specific ValidationException/error that
indicates TTL was enabled by a concurrent process while logging other exceptions
and re-raising or handling retries for transient errors (throttling/network)
with a small retry/backoff loop; ensure all errors are logged with details.
Owner
Author
|
@claude please review the PR feedback, there is a lock on setup from basestore so i think the feedback might be wrong |
|
Claude finished @strawgate's task —— View job Reviewing PR FeedbackTask Checklist:
AnalysisYou are correct - CodeRabbit's feedback about race conditions is not applicable to this implementation. Here's why: The Setup Lock MechanismThe async def setup(self) -> None:
if not self._setup_complete:
async with self._setup_lock: # 🔒 This lock prevents race conditions
if not self._setup_complete:
try:
await self._setup()
except Exception as e:
raise StoreSetupError(...) from e
self._setup_complete = TrueKey points:
Evaluating CodeRabbit's Concerns❌ Race Condition Concern (Invalid)CodeRabbit's second review stated:
This is incorrect because:
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR enables Time-To-Live (TTL) functionality on the DynamoDB table during store setup. Previously, TTL values were stored in items but the table-level TTL feature was never enabled, meaning expired items would never be automatically deleted.
Changes
update_time_to_livecall in_setup()methodRelated Issues
Addresses one of the critical issues identified in #168 for marking DynamoDB store as stable.
Batch operations implementation (Phases 2-3) and enumeration capabilities (Phase 4) are deferred to future PRs.
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes