Skip to content

chore: Sync dev to main #398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Aug 29, 2023
Merged

chore: Sync dev to main #398

merged 9 commits into from
Aug 29, 2023

Conversation

hjgraca
Copy link
Contributor

@hjgraca hjgraca commented Aug 29, 2023

Issue number: #396

Summary

Changes

Please provide a summary of what's being changed

In the current implementation of Idempotency we don't have the InProgressExpiration timestamp. So currently if a lambda invocation expired there would be no way to retry it, when using the Idempotent attribute in the lambda handler or Idempotent attribute on another method

This field is required to prevent against extended failed retries when a Lambda function times out, Powertools for AWS Lambda (.NET) calculates and includes the remaining invocation available time as part of the idempotency record.

If a second invocation happens after this timestamp, and the record is marked as INPROGRESS, we will execute the invocation again as if it was in the EXPIRED state (e.g, expire_seconds field elapsed).

This means that if an invocation expired during execution, it will be quickly executed again on the next retry.

User experience

Please share what the user experience looks like before and after this change

When decorating the handler with Idempotent attribute we will do it automatically by getting the RemainingTime value from the ILambdaContext that is passed to the handler.
RemainingTime is the remaining execution time till the function will be terminated. At the time you create the Lambda function you set maximum time limit, at which time AWS Lambda will terminate the function execution.
Information about the remaining time of function execution can be used to specify function behavior when nearing the timeout.

When using Idempotent attribute on another method to guard isolated parts of your code, you must use RegisterLambdaContext available in the Idempotency static class to benefit from this protection.

Here is an example on how you register the Lambda context in your handler:

public class Function
{
    public Function()
    {
        Idempotency.Configure(builder => builder.UseDynamoDb("idempotency_table"));
    }
    
    public Task<string> FunctionHandler(string input, ILambdaContext context)
    {
        Idempotency.RegisterLambdaContext(context);

        MyInternalMethod("hello", "world")
        return Task.FromResult(input.ToUpper());
    }

    [Idempotent]
    private string MyInternalMethod(string argOne, [IdempotencyKey] string argTwo) {
        return "something";
    }
}

Lambda request timeout diagram

sequenceDiagram
    participant Client
    participant Lambda
    participant Persistence Layer
    alt initial request
        Client->>Lambda: Invoke (event)
        Lambda->>Persistence Layer: Get or set idempotency_key=hash(payload)
        activate Persistence Layer
        Note over Lambda,Persistence Layer: Set record status to INPROGRESS. <br> Prevents concurrent invocations <br> with the same payload
        Lambda-->>Lambda: Call your function
        Note right of Lambda: Time out
        Lambda--xLambda: Time out error
        Lambda-->>Client: Return error response
        deactivate Persistence Layer
    else retry after Lambda timeout elapses
        Client->>Lambda: Invoke (event)
        Lambda->>Persistence Layer: Get or set idempotency_key=hash(payload)
        activate Persistence Layer
        Note over Lambda,Persistence Layer: Set record status to INPROGRESS. <br> Reset in_progress_expiry attribute
        Lambda-->>Lambda: Call your function
        Lambda->>Persistence Layer: Update record with result
        deactivate Persistence Layer
        Persistence Layer-->>Persistence Layer: Update record
        Lambda-->>Client: Response sent to client
    end
Loading

Checklist

Please leave checklist items unchecked if they do not apply to your change.

Is this a breaking change?

RFC issue number:

Checklist:

  • Migration process documented
  • Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@auto-assign auto-assign bot requested review from amirkaws and sliedig August 29, 2023 09:41
@boring-cyborg boring-cyborg bot added area/idempotency documentation Improvements or additions to documentation tests labels Aug 29, 2023
@pull-request-size pull-request-size bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Aug 29, 2023
@hjgraca hjgraca changed the title chore: Idempotency InProgressExpiration chore: Sync dev to main Aug 29, 2023
@github-actions github-actions bot added the internal Maintenance changes label Aug 29, 2023
@codecov-commenter
Copy link

Codecov Report

Patch coverage: 93.44% and project coverage change: +0.68% 🎉

Comparison is base (1378d75) 72.34% compared to head (84477e8) 73.02%.
Report is 18 commits behind head on main.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #398      +/-   ##
==========================================
+ Coverage   72.34%   73.02%   +0.68%     
==========================================
  Files          82       82              
  Lines        3594     3644      +50     
==========================================
+ Hits         2600     2661      +61     
+ Misses        994      983      -11     
Flag Coverage Δ
unittests 73.02% <93.44%> (+0.68%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...dempotency/Persistence/DynamoDBPersistenceStore.cs 93.30% <84.00%> (-1.31%) ⬇️
...c/AWS.Lambda.Powertools.Idempotency/Idempotency.cs 78.18% <100.00%> (+1.71%) ⬆️
...mbda.Powertools.Idempotency/IdempotentAttribute.cs 87.69% <100.00%> (+1.25%) ⬆️
...s.Idempotency/Internal/IdempotencyAspectHandler.cs 62.93% <100.00%> (+10.10%) ⬆️
...ls.Idempotency/Persistence/BasePersistenceStore.cs 98.78% <100.00%> (+0.05%) ⬆️
...a.Powertools.Idempotency/Persistence/DataRecord.cs 92.30% <100.00%> (+0.64%) ⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@hjgraca hjgraca merged commit 46c85ee into main Aug 29, 2023
@rubenfonseca rubenfonseca temporarily deployed to analytics August 29, 2023 10:01 — with GitHub Actions Inactive
@rubenfonseca rubenfonseca temporarily deployed to analytics August 29, 2023 11:01 — with GitHub Actions Inactive
@rubenfonseca rubenfonseca temporarily deployed to analytics August 29, 2023 12:01 — with GitHub Actions Inactive
@rubenfonseca rubenfonseca temporarily deployed to analytics August 29, 2023 13:02 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/idempotency documentation Improvements or additions to documentation internal Maintenance changes size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants