Skip to content

Commit 69c62ad

Browse files
committed
chore(docs): added docs about expires_in_progress
1 parent 33fb5a4 commit 69c62ad

File tree

1 file changed

+53
-1
lines changed

1 file changed

+53
-1
lines changed

docs/utilities/idempotency.md

Lines changed: 53 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ times with the same parameters**. This makes idempotent operations safe to retry
2222
* Ensure Lambda handler returns the same result when called with the same payload
2323
* Select a subset of the event as the idempotency key using JMESPath expressions
2424
* Set a time window in which records with the same payload should be considered duplicates
25+
* Optionally expires in-progress executions after the Lambda handler timeout
2526

2627
## Getting started
2728

@@ -402,13 +403,14 @@ def call_external_service(data: dict, **kwargs):
402403

403404
This persistence layer is built-in, and you can either use an existing DynamoDB table or create a new one dedicated for idempotency state (recommended).
404405

405-
```python hl_lines="5-9" title="Customizing DynamoDBPersistenceLayer to suit your table structure"
406+
```python hl_lines="5-10" title="Customizing DynamoDBPersistenceLayer to suit your table structure"
406407
from aws_lambda_powertools.utilities.idempotency import DynamoDBPersistenceLayer
407408

408409
persistence_layer = DynamoDBPersistenceLayer(
409410
table_name="IdempotencyTable",
410411
key_attr="idempotency_key",
411412
expiry_attr="expires_at",
413+
in_progress_expiry_attr="in_progress_expires_at",
412414
status_attr="current_status",
413415
data_attr="result_data",
414416
validation_key_attr="validation_key",
@@ -422,6 +424,7 @@ Parameter | Required | Default | Description
422424
**table_name** | :heavy_check_mark: | | Table name to store state
423425
**key_attr** | | `id` | Partition key of the table. Hashed representation of the payload (unless **sort_key_attr** is specified)
424426
**expiry_attr** | | `expiration` | Unix timestamp of when record expires
427+
**in_progress_expiry_attr** | | `in_progress_expiration` | Unix timestamp of when record expires while in progress (in case of the invocation times out)
425428
**status_attr** | | `status` | Stores status of the lambda execution during and after invocation
426429
**data_attr** | | `data` | Stores results of successfully executed Lambda handlers
427430
**validation_key_attr** | | `validation` | Hashed representation of the parts of the event used for validation
@@ -440,6 +443,7 @@ Parameter | Default | Description
440443
**payload_validation_jmespath** | `""` | JMESPath expression to validate whether certain parameters have changed in the event while the event payload
441444
**raise_on_no_idempotency_key** | `False` | Raise exception if no idempotency key was found in the request
442445
**expires_after_seconds** | 3600 | The number of seconds to wait before a record is expired
446+
**expires_in_progress** | `False`| Enables expiry of invocations that time out during execution
443447
**use_local_cache** | `False` | Whether to locally cache idempotency results
444448
**local_cache_max_items** | 256 | Max number of items to store in local cache
445449
**hash_function** | `md5` | Function to use for calculating hashes, as provided by [hashlib](https://docs.python.org/3/library/hashlib.html) in the standard library.
@@ -853,6 +857,54 @@ class DynamoDBPersistenceLayer(BasePersistenceLayer):
853857

854858
For example, the `_put_record` method needs to raise an exception if a non-expired record already exists in the data store with a matching key.
855859

860+
### Expiring in-progress invocations
861+
862+
The default behavior when a Lambda invocation times out is for the event to stay locked until `expire_seconds` have passed. Powertools
863+
has no way of knowing if it's safe to retry the operation in this scenario, so we assume the safest approach: to not
864+
retry the operation.
865+
866+
However, certain types of invocation have less strict requirements, and can benefit from faster expiry of invocations. Ideally, we
867+
can make an in-progress invocation expire as soon as the [Lambda invocation expires](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/).
868+
869+
When using this option, powertools will calculate the remaining available time for the invocation, and save it on the idempotency record.
870+
This way, if a second invocation happens after this timestamp, and the record is stil marked `INPROGRESS`, we execute the invocation again
871+
as if it was already expired. This means that if an invocation expired during execution, it will be quickly executed again on the next try.
872+
873+
This setting introduces no change on the regular behavior where if an invocation succeeds, the results are cached for `expire_seconds` seconds.
874+
875+
???+ warning "Warning"
876+
Consider whenever you really want this behavior. Powertools can't make any garantee on which state your application was
877+
when it time outed. Ensure that your business logic can be retried at any stage.
878+
879+
???+ info "Info: Calculating the remaining available time"
880+
For now this only works with the `idempotent` decorator. At the moment we don't have access to the Lambda context when using
881+
the `idempotent_function` so enabling this option is a no-op in that scenario.
882+
883+
To activate this behaviour, enable the `expires_in_progress` option on the configuration:
884+
885+
=== "app.py"
886+
887+
```python hl_lines="7"
888+
from aws_lambda_powertools.utilities.idempotency import (
889+
DynamoDBPersistenceLayer, idempotent
890+
)
891+
892+
persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
893+
894+
@idempotent(persistence_store=persistence_layer, expires_in_progress=True)
895+
def handler(event, context):
896+
payment = create_subscription_payment(
897+
user=event['user'],
898+
product=event['product_id']
899+
)
900+
...
901+
return {
902+
"payment_id": payment.id,
903+
"message": "success",
904+
"statusCode": 200,
905+
}
906+
```
907+
856908
## Compatibility with other utilities
857909

858910
### Validation utility

0 commit comments

Comments
 (0)