diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 66ea80739..3210dad84 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -16,7 +16,9 @@ on: - 'powertools-parameters/**' - 'powertools-metrics/**' - 'powertools-test-suite/**' + - 'examples/**' - 'pom.xml' + - 'examples/pom.xml' - '.github/workflows/**' push: branches: @@ -33,7 +35,9 @@ on: - 'powertools-parameters/**' - 'powertools-metrics/**' - 'powertools-test-suite/**' + - 'examples/**' - 'pom.xml' + - 'examples/pom.xml' - '.github/workflows/**' jobs: build: diff --git a/README.md b/README.md index 5bf3c7977..addb0f631 100644 --- a/README.md +++ b/README.md @@ -79,8 +79,9 @@ And configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lambd ## Example -See **[example](https://github.com/aws-samples/aws-lambda-powertools-examples/tree/main/java)** for example project showcasing usage of different utilities. -Have a demo project to contribute which showcase usage of different utilities from powertools? We are happy to accept it [here](https://github.com/aws-samples/aws-lambda-powertools-examples/blob/main/CONTRIBUTING.md#security-issue-notifications). +See the **[examples](examples)** directory for example projects showcasing usage of different utilities. + +Have a demo project to contribute which showcase usage of different utilities from powertools? We are happy to accept it [here](CONTRIBUTING.md#security-issue-notifications). ## Credits diff --git a/examples/.gitignore b/examples/.gitignore new file mode 100644 index 000000000..79e044a40 --- /dev/null +++ b/examples/.gitignore @@ -0,0 +1,2 @@ +dependency-reduced-pom.xml +.aws-sam diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 000000000..b5a0da8cd --- /dev/null +++ b/examples/README.md @@ -0,0 +1,5 @@ +## aws-lambda-powertools-examples + +This directory holds example projects demoing different components of the Lambda Powertools for Java. + + diff --git a/examples/pom.xml b/examples/pom.xml new file mode 100644 index 000000000..c61e858c5 --- /dev/null +++ b/examples/pom.xml @@ -0,0 +1,35 @@ + + + 4.0.0 + + powertools-examples + pom + + + powertools-parent + software.amazon.lambda + 1.14.0 + + + AWS Lambda Powertools for Java library Examples + + A suite of examples accompanying for AWS Lambda Powertools. + + + + powertools-examples-core + powertools-examples-idempotency + powertools-examples-parameters + powertools-examples-serialization + powertools-examples-sqs + powertools-examples-validation + + + + + true + + + \ No newline at end of file diff --git a/examples/powertools-examples-core/.gitignore b/examples/powertools-examples-core/.gitignore new file mode 100644 index 000000000..e69de29bb diff --git a/examples/powertools-examples-core/README.md b/examples/powertools-examples-core/README.md new file mode 100644 index 000000000..f4bdc399c --- /dev/null +++ b/examples/powertools-examples-core/README.md @@ -0,0 +1,138 @@ +# CoreUtilities + +This project contains source code and supporting files for a serverless application that you can deploy with the SAM CLI. It includes [Lambda Powertools for operational best practices](https://github.com/awslabs/aws-lambda-powertools-java), and the following files and folders. + +- HelloWorldFunction/src/main - Code for the application's Lambda function. +- events - Invocation events that you can use to invoke the function. +- HelloWorldFunction/src/test - Unit tests for the application code. +- template.yaml - A template that defines the application's AWS resources. + +The application uses several AWS resources, including Lambda functions and an API Gateway API. These resources are defined in the `template.yaml` file in this project. You can update the template to add AWS resources through the same deployment process that updates your application code. + +If you prefer to use an integrated development environment (IDE) to build and test your application, you can use the AWS Toolkit. +The AWS Toolkit is an open source plug-in for popular IDEs that uses the SAM CLI to build and deploy serverless applications on AWS. The AWS Toolkit also adds a simplified step-through debugging experience for Lambda function code. See the following links to get started. + +* [PyCharm](https://docs.aws.amazon.com/toolkit-for-jetbrains/latest/userguide/welcome.html) +* [IntelliJ](https://docs.aws.amazon.com/toolkit-for-jetbrains/latest/userguide/welcome.html) +* [VS Code](https://docs.aws.amazon.com/toolkit-for-vscode/latest/userguide/welcome.html) +* [Visual Studio](https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/welcome.html) + +## Deploy the sample application + +The Serverless Application Model Command Line Interface (SAM CLI) is an extension of the AWS CLI that adds functionality for building and testing Lambda applications. It uses Docker to run your functions in an Amazon Linux environment that matches Lambda. It can also emulate your application's build environment and API. + +To use the SAM CLI, you need the following tools. + +* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) +* Java11 - [Install the Java 11](https://docs.aws.amazon.com/corretto/latest/corretto-11-ug/downloads-list.html) +* Maven - [Install Maven](https://maven.apache.org/install.html) +* Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community) + +To build and deploy your application for the first time, run the following in your shell: + +```bash +Coreutilities$ sam build +Coreutilities$ sam deploy --guided +``` + +The first command will build the source of your application. The second command will package and deploy your application to AWS, with a series of prompts: + +* **Stack Name**: The name of the stack to deploy to CloudFormation. This should be unique to your account and region, and a good starting point would be something matching your project name. +* **AWS Region**: The AWS region you want to deploy your app to. +* **Confirm changes before deploy**: If set to yes, any change sets will be shown to you before execution for manual review. If set to no, the AWS SAM CLI will automatically deploy application changes. +* **Allow SAM CLI IAM role creation**: Many AWS SAM templates, including this example, create AWS IAM roles required for the AWS Lambda function(s) included to access AWS services. By default, these are scoped down to minimum required permissions. To deploy an AWS CloudFormation stack which creates or modified IAM roles, the `CAPABILITY_IAM` value for `capabilities` must be provided. If permission isn't provided through this prompt, to deploy this example you must explicitly pass `--capabilities CAPABILITY_IAM` to the `sam deploy` command. +* **Save arguments to samconfig.toml**: If set to yes, your choices will be saved to a configuration file inside the project, so that in the future you can just re-run `sam deploy` without parameters to deploy changes to your application. + +You can find your API Gateway Endpoint URL in the output values displayed after deployment. + +## Use the SAM CLI to build and test locally + +Build your application with the `sam build` command. + +```bash +Coreutilities$ sam build +``` + +The SAM CLI installs dependencies defined in `HelloWorldFunction/pom.xml`, creates a deployment package, and saves it in the `.aws-sam/build` folder. + +Test a single function by invoking it directly with a test event. An event is a JSON document that represents the input that the function receives from the event source. Test events are included in the `events` folder in this project. + +Run functions locally and invoke them with the `sam local invoke` command. + +```bash +Coreutilities$ sam local invoke HelloWorldFunction --event events/event.json +``` + +The SAM CLI can also emulate your application's API. Use the `sam local start-api` to run the API locally on port 3000. + +```bash +Coreutilities$ sam local start-api +Coreutilities$ curl http://localhost:3000/ +``` + +The SAM CLI reads the application template to determine the API's routes and the functions that they invoke. The `Events` property on each function's definition includes the route and method for each path. + +```yaml + Events: + HelloWorld: + Type: Api + Properties: + Path: /hello + Method: get +``` + +## Add a resource to your application +The application template uses AWS Serverless Application Model (AWS SAM) to define application resources. AWS SAM is an extension of AWS CloudFormation with a simpler syntax for configuring common serverless application resources such as functions, triggers, and APIs. For resources not included in [the SAM specification](https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md), you can use standard [AWS CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html) resource types. + +## Fetch, tail, and filter Lambda function logs + +To simplify troubleshooting, SAM CLI has a command called `sam logs`. `sam logs` lets you fetch logs generated by your deployed Lambda function from the command line. In addition to printing the logs on the terminal, this command has several nifty features to help you quickly find the bug. + +`NOTE`: This command works for all AWS Lambda functions; not just the ones you deploy using SAM. + +```bash +Coreutilities$ sam logs -n HelloWorldFunction --stack-name --tail +``` + +You can find more information and examples about filtering Lambda function logs in the [SAM CLI Documentation](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-logging.html). + +## Unit tests + +Tests are defined in the `HelloWorldFunction/src/test` folder in this project. + +```bash +Coreutilities$ cd HelloWorldFunction +HelloWorldFunction$ mvn test +``` + +## Cleanup + +To delete the sample application that you created, use the AWS CLI. Assuming you used your project name for the stack name, you can run the following: + +```bash +aws cloudformation delete-stack --stack-name +``` + +# Appendix + +## Powertools + +**Tracing** + +[Tracing utility](https://awslabs.github.io/aws-lambda-powertools-java/core/tracing/) provides functionality to reduce the overhead of performing common tracing tasks. It traces the execution of this sample code including the response and exceptions as tracing metadata - You can visualize them in AWS X-Ray. + +**Logger** + +[Logging utility](https://awslabs.github.io/aws-lambda-powertools-java/core/logging/) creates an opinionated application Logger with structured logging as the output, dynamically samples a percentage (samplingRate) of your logs in DEBUG mode for concurrent invocations, log incoming events as your function is invoked, and injects key information from Lambda context object into your Logger - You can visualize them in Amazon CloudWatch Logs. + +**Metrics** + +[Metrics utility](https://awslabs.github.io/aws-lambda-powertools-java/core/metrics/) captures cold start metric of your Lambda invocation, and could add additional metrics to help you understand your application KPIs - You can visualize them in Amazon CloudWatch. + +## Resources + +See the [AWS SAM developer guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html) for an introduction to SAM specification, the SAM CLI, and serverless application concepts. + +Check the [AWS Lambda Powertools Java](https://awslabs.github.io/aws-lambda-powertools-java/) for more information on how to use and configure such tools + +Next, you can use AWS Serverless Application Repository to deploy ready to use Apps that go beyond hello world samples and learn how authors developed their applications: [AWS Serverless Application Repository main page](https://aws.amazon.com/serverless/serverlessrepo/) diff --git a/examples/powertools-examples-core/events/event.json b/examples/powertools-examples-core/events/event.json new file mode 100644 index 000000000..3822fadaa --- /dev/null +++ b/examples/powertools-examples-core/events/event.json @@ -0,0 +1,63 @@ +{ + "body": "{\"message\": \"hello world\"}", + "resource": "/{proxy+}", + "path": "/path/to/resource", + "httpMethod": "POST", + "isBase64Encoded": false, + "queryStringParameters": { + "foo": "bar" + }, + "pathParameters": { + "proxy": "/path/to/resource" + }, + "stageVariables": { + "baz": "qux" + }, + "headers": { + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", + "Accept-Encoding": "gzip, deflate, sdch", + "Accept-Language": "en-US,en;q=0.8", + "Cache-Control": "max-age=0", + "CloudFront-Forwarded-Proto": "https", + "CloudFront-Is-Desktop-Viewer": "true", + "CloudFront-Is-Mobile-Viewer": "false", + "CloudFront-Is-SmartTV-Viewer": "false", + "CloudFront-Is-Tablet-Viewer": "false", + "CloudFront-Viewer-Country": "US", + "Host": "1234567890.execute-api.us-east-1.amazonaws.com", + "Upgrade-Insecure-Requests": "1", + "User-Agent": "Custom User Agent String", + "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)", + "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==", + "X-Forwarded-For": "127.0.0.1, 127.0.0.2", + "X-Forwarded-Port": "443", + "X-Forwarded-Proto": "https" + }, + "requestContext": { + "accountId": "123456789012", + "resourceId": "123456", + "stage": "prod", + "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", + "requestTime": "09/Apr/2015:12:34:56 +0000", + "requestTimeEpoch": 1428582896000, + "identity": { + "cognitoIdentityPoolId": null, + "accountId": null, + "cognitoIdentityId": null, + "caller": null, + "accessKey": null, + "sourceIp": "127.0.0.1", + "cognitoAuthenticationType": null, + "cognitoAuthenticationProvider": null, + "userArn": null, + "userAgent": "Custom User Agent String", + "user": null + }, + "path": "/prod/path/to/resource", + "resourcePath": "/{proxy+}", + "httpMethod": "POST", + "apiId": "1234567890", + "protocol": "HTTP/1.1" + } + } + \ No newline at end of file diff --git a/examples/powertools-examples-core/pom.xml b/examples/powertools-examples-core/pom.xml new file mode 100644 index 000000000..86f3afd0d --- /dev/null +++ b/examples/powertools-examples-core/pom.xml @@ -0,0 +1,121 @@ + + 4.0.0 + powertools-examples-core + jar + AWS Lambda Powertools for Java library Examples - Core + + + powertools-examples + software.amazon.lambda + 1.14.0 + + + + + software.amazon.lambda + powertools-tracing + ${project.version} + + + software.amazon.lambda + powertools-logging + ${project.version} + + + software.amazon.lambda + powertools-metrics + ${project.version} + + + com.amazonaws + aws-lambda-java-core + 1.2.2 + + + com.amazonaws + aws-lambda-java-events + 3.11.0 + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + + + org.apache.logging.log4j + log4j-api + ${log4j.version} + + + + junit + junit + 4.13.2 + test + + + + + + + org.codehaus.mojo + aspectj-maven-plugin + 1.14.0 + + ${maven.compiler.source} + ${maven.compiler.target} + ${maven.compiler.target} + + + software.amazon.lambda + powertools-tracing + + + software.amazon.lambda + powertools-logging + + + software.amazon.lambda + powertools-metrics + + + + + + + compile + + + + + + org.apache.maven.plugins + maven-shade-plugin + 3.2.4 + + + package + + shade + + + + + + + + + + + + com.github.edwgiz + maven-shade-plugin.log4j2-cachefile-transformer + 2.15 + + + + + + diff --git a/examples/powertools-examples-core/src/main/java/helloworld/App.java b/examples/powertools-examples-core/src/main/java/helloworld/App.java new file mode 100644 index 000000000..b45440114 --- /dev/null +++ b/examples/powertools-examples-core/src/main/java/helloworld/App.java @@ -0,0 +1,115 @@ +package helloworld; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; +import java.util.stream.Collectors; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import com.amazonaws.xray.AWSXRay; +import com.amazonaws.xray.entities.Entity; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import software.amazon.cloudwatchlogs.emf.model.DimensionSet; +import software.amazon.cloudwatchlogs.emf.model.Unit; +import software.amazon.lambda.powertools.logging.LoggingUtils; +import software.amazon.lambda.powertools.logging.Logging; +import software.amazon.lambda.powertools.metrics.Metrics; +import software.amazon.lambda.powertools.tracing.CaptureMode; +import software.amazon.lambda.powertools.tracing.TracingUtils; +import software.amazon.lambda.powertools.tracing.Tracing; + +import static software.amazon.lambda.powertools.metrics.MetricsUtils.metricsLogger; +import static software.amazon.lambda.powertools.metrics.MetricsUtils.withSingleMetric; +import static software.amazon.lambda.powertools.tracing.TracingUtils.putMetadata; +import static software.amazon.lambda.powertools.tracing.TracingUtils.withEntitySubsegment; + +/** + * Handler for requests to Lambda function. + */ +public class App implements RequestHandler { + private final static Logger log = LogManager.getLogger(App.class); + + @Logging(logEvent = true, samplingRate = 0.7) + @Tracing(captureMode = CaptureMode.RESPONSE_AND_ERROR) + @Metrics(namespace = "ServerlessAirline", service = "payment", captureColdStart = true) + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + Map headers = new HashMap<>(); + + headers.put("Content-Type", "application/json"); + headers.put("X-Custom-Header", "application/json"); + + metricsLogger().putMetric("CustomMetric1", 1, Unit.COUNT); + + withSingleMetric("CustomMetrics2", 1, Unit.COUNT, "Another", (metric) -> { + metric.setDimensions(DimensionSet.of("AnotherService", "CustomService")); + metric.setDimensions(DimensionSet.of("AnotherService1", "CustomService1")); + }); + + LoggingUtils.appendKey("test", "willBeLogged"); + + APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent() + .withHeaders(headers); + try { + final String pageContents = this.getPageContents("https://checkip.amazonaws.com"); + log.info(pageContents); + TracingUtils.putAnnotation("Test", "New"); + String output = String.format("{ \"message\": \"hello world\", \"location\": \"%s\" }", pageContents); + + TracingUtils.withSubsegment("loggingResponse", subsegment -> { + String sampled = "log something out"; + log.info(sampled); + log.info(output); + }); + + threadOption1(); + + threadOption2(); + + log.info("After output"); + return response + .withStatusCode(200) + .withBody(output); + } catch (IOException | InterruptedException e) { + return response + .withBody("{}") + .withStatusCode(500); + } + } + + private void threadOption1() throws InterruptedException { + final Entity traceEntity = AWSXRay.getTraceEntity(); + assert traceEntity != null; + traceEntity.run(new Thread(this::log)); + } + + private void threadOption2() throws InterruptedException { + Entity traceEntity = AWSXRay.getTraceEntity(); + Thread anotherThread = new Thread(() -> withEntitySubsegment("inlineLog", traceEntity, subsegment -> { + String var = "somethingToProcess"; + log.info("inside threaded logging inline {}", var); + })); + anotherThread.start(); + anotherThread.join(); + } + + @Tracing + private void log() { + log.info("inside threaded logging for function"); + } + + @Tracing(namespace = "getPageContents", captureMode = CaptureMode.DISABLED) + private String getPageContents(String address) throws IOException { + URL url = new URL(address); + putMetadata("getPageContents", address); + try (BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream()))) { + return br.lines().collect(Collectors.joining(System.lineSeparator())); + } + } +} diff --git a/examples/powertools-examples-core/src/main/java/helloworld/AppStream.java b/examples/powertools-examples-core/src/main/java/helloworld/AppStream.java new file mode 100644 index 000000000..aed048eef --- /dev/null +++ b/examples/powertools-examples-core/src/main/java/helloworld/AppStream.java @@ -0,0 +1,25 @@ +package helloworld; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.util.Map; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestStreamHandler; +import com.fasterxml.jackson.databind.ObjectMapper; +import software.amazon.lambda.powertools.logging.Logging; +import software.amazon.lambda.powertools.metrics.Metrics; + +public class AppStream implements RequestStreamHandler { + private static final ObjectMapper mapper = new ObjectMapper(); + + @Override + @Logging(logEvent = true) + @Metrics(namespace = "ServerlessAirline", service = "payment", captureColdStart = true) + public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException { + Map map = mapper.readValue(input, Map.class); + + System.out.println(map.size()); + } +} diff --git a/examples/powertools-examples-core/src/main/resources/log4j2.xml b/examples/powertools-examples-core/src/main/resources/log4j2.xml new file mode 100644 index 000000000..e1fd14cea --- /dev/null +++ b/examples/powertools-examples-core/src/main/resources/log4j2.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/examples/powertools-examples-core/src/test/java/helloworld/AppTest.java b/examples/powertools-examples-core/src/test/java/helloworld/AppTest.java new file mode 100644 index 000000000..b584ee944 --- /dev/null +++ b/examples/powertools-examples-core/src/test/java/helloworld/AppTest.java @@ -0,0 +1,43 @@ +package helloworld; + +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import com.amazonaws.xray.AWSXRay; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import static org.junit.Assert.*; + +public class AppTest { + + @Before + public void setup() { + if(null == System.getenv("LAMBDA_TASK_ROOT")) { + AWSXRay.beginSegment("test"); + } + } + + @After + public void tearDown() { + if (AWSXRay.getCurrentSubsegmentOptional().isPresent()) { + AWSXRay.endSubsegment(); + } + + if(null == System.getenv("LAMBDA_TASK_ROOT")) { + AWSXRay.endSegment(); + } + } + + @Test + public void successfulResponse() { + App app = new App(); + APIGatewayProxyResponseEvent result = app.handleRequest(null, null); + assertEquals(result.getStatusCode().intValue(), 200); + assertEquals(result.getHeaders().get("Content-Type"), "application/json"); + String content = result.getBody(); + assertNotNull(content); + assertTrue(content.contains("\"message\"")); + assertTrue(content.contains("\"hello world\"")); + assertTrue(content.contains("\"location\"")); + } +} diff --git a/examples/powertools-examples-core/template.yaml b/examples/powertools-examples-core/template.yaml new file mode 100644 index 000000000..3ca7f0ff2 --- /dev/null +++ b/examples/powertools-examples-core/template.yaml @@ -0,0 +1,66 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + CoreUtilities + +# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst +Globals: + Function: + Timeout: 20 + Runtime: java11 + MemorySize: 512 + Tracing: Active # https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.html + Environment: + Variables: + # Powertools env vars: https://awslabs.github.io/aws-lambda-powertools-python/#environment-variables + POWERTOOLS_LOG_LEVEL: INFO + POWERTOOLS_LOGGER_SAMPLE_RATE: 0.1 + POWERTOOLS_LOGGER_LOG_EVENT: true + POWERTOOLS_METRICS_NAMESPACE: Coreutilities + +Resources: + HelloWorldFunction: + Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction + Properties: + CodeUri: . + Handler: helloworld.App::handleRequest + Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object + Variables: + POWERTOOLS_SERVICE_NAME: hello + Events: + HelloWorld: + Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api + Properties: + Path: /hello + Method: get + + HelloWorldStreamFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: . + Handler: helloworld.AppStream::handleRequest + MemorySize: 512 + Tracing: Active + Environment: + Variables: + POWERTOOLS_LOGGER_SAMPLE_RATE: 0.7 + Events: + HelloWorld: + Type: Api + Properties: + Path: /hellostream + Method: get + +Outputs: + # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function + # Find out more about other implicit resources you can reference within SAM + # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api + HelloWorldApi: + Description: "API Gateway endpoint URL for Prod stage for Hello World function" + Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/" + HelloWorldFunction: + Description: "Hello World Lambda Function ARN" + Value: !GetAtt HelloWorldFunction.Arn + HelloWorldFunctionIamRole: + Description: "Implicit IAM Role created for Hello World function" + Value: !GetAtt HelloWorldFunctionRole.Arn diff --git a/examples/powertools-examples-idempotency/README.md b/examples/powertools-examples-idempotency/README.md new file mode 100644 index 000000000..a919f0c70 --- /dev/null +++ b/examples/powertools-examples-idempotency/README.md @@ -0,0 +1,21 @@ +# Idempotency + +This project contains an example of Lambda function using the idempotency module of Lambda Powertools for Java. For more information on this module, please refer to the [documentation](https://awslabs.github.io/aws-lambda-powertools-java/utilities/idempotency/). + +## Deploy the sample application + +This sample is based on Serverless Application Model (SAM) and you can use the SAM Command Line Interface (SAM CLI) to build it and deploy it to AWS. + +To use the SAM CLI, you need the following tools. + +* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) +* Java11 - [Install the Java 11](https://docs.aws.amazon.com/corretto/latest/corretto-11-ug/downloads-list.html) +* Maven - [Install Maven](https://maven.apache.org/install.html) +* Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community) + +To build and deploy your application for the first time, run the following in your shell: + +```bash +Coreutilities$ sam build +Coreutilities$ sam deploy --guided +``` diff --git a/examples/powertools-examples-idempotency/pom.xml b/examples/powertools-examples-idempotency/pom.xml new file mode 100644 index 000000000..438933178 --- /dev/null +++ b/examples/powertools-examples-idempotency/pom.xml @@ -0,0 +1,178 @@ + + 4.0.0 + powertools-examples-idempotency + jar + AWS Lambda Powertools for Java library Examples - Idempotency + + + powertools-examples + software.amazon.lambda + 1.14.0 + + + + + software.amazon.lambda + powertools-tracing + ${project.version} + + + software.amazon.lambda + powertools-logging + ${project.version} + + + software.amazon.lambda + powertools-idempotency + ${project.version} + + + com.amazonaws + aws-lambda-java-core + 1.2.2 + + + com.amazonaws + aws-lambda-java-events + 3.11.0 + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + + + org.apache.logging.log4j + log4j-api + ${log4j.version} + + + + org.mockito + mockito-core + 4.11.0 + test + + + org.junit.jupiter + junit-jupiter-api + 5.9.2 + test + + + com.amazonaws + DynamoDBLocal + 1.21.0 + test + + + com.amazonaws + aws-lambda-java-tests + 1.1.1 + + + + + + + org.codehaus.mojo + aspectj-maven-plugin + 1.14.0 + + ${maven.compiler.source} + ${maven.compiler.target} + ${maven.compiler.target} + + + software.amazon.lambda + powertools-tracing + + + software.amazon.lambda + powertools-logging + + + software.amazon.lambda + powertools-idempotency + + + + + + + compile + + + + + + org.apache.maven.plugins + maven-dependency-plugin + + + copy + test-compile + + copy-dependencies + + + test + so,dll,dylib + ${project.build.directory}/native-libs + + + + + + org.apache.maven.plugins + maven-surefire-plugin + 3.0.0-M5 + + + ${project.build.directory}/native-libs + + + idempotency + eu-central-1 + LOG_ERROR + + + + + org.apache.maven.plugins + maven-shade-plugin + 3.2.4 + + + package + + shade + + + + + + + + + + + + com.github.edwgiz + maven-shade-plugin.log4j2-cachefile-transformer + 2.15 + + + + + + + + dynamodb-local-oregon + DynamoDB Local Release Repository + https://s3.eu-central-1.amazonaws.com/dynamodb-local-frankfurt/release + + + diff --git a/examples/powertools-examples-idempotency/src/main/java/helloworld/App.java b/examples/powertools-examples-idempotency/src/main/java/helloworld/App.java new file mode 100644 index 000000000..f26877c34 --- /dev/null +++ b/examples/powertools-examples-idempotency/src/main/java/helloworld/App.java @@ -0,0 +1,92 @@ +package helloworld; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import software.amazon.awssdk.services.dynamodb.DynamoDbClient; +import software.amazon.lambda.powertools.idempotency.Idempotency; +import software.amazon.lambda.powertools.idempotency.IdempotencyConfig; +import software.amazon.lambda.powertools.idempotency.Idempotent; +import software.amazon.lambda.powertools.idempotency.persistence.DynamoDBPersistenceStore; +import software.amazon.lambda.powertools.logging.Logging; +import software.amazon.lambda.powertools.utilities.JsonConfig; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; +import java.util.stream.Collectors; + +public class App implements RequestHandler { + private final static Logger log = LogManager.getLogger(App.class); + + public App() { + this(null); + } + + public App(DynamoDbClient client) { + Idempotency.config().withConfig( + IdempotencyConfig.builder() + .withEventKeyJMESPath("powertools_json(body).address") // will retrieve the address field in the body which is a string transformed to json with `powertools_json` + .build()) + .withPersistenceStore( + DynamoDBPersistenceStore.builder() + .withDynamoDbClient(client) + .withTableName(System.getenv("IDEMPOTENCY_TABLE")) + .build() + ).configure(); + } + + /** + * Try with: + * 
+     *     curl -X POST https://[REST-API-ID].execute-api.[REGION].amazonaws.com/Prod/helloidem/ -H "Content-Type: application/json" -d '{"address": "https://checkip.amazonaws.com"}'
+     *
+ *
    + *
  • First call will execute the handleRequest normally, and store the response in the idempotency table (Look into DynamoDB)
    • + *
  • Second call (and next ones) will retrieve from the cache (if cache is enabled, which is by default) or from the store, the handler won't be called. Until the expiration happens (by default 1 hour).
    • + *
+ */ + @Idempotent // *** THE MAGIC IS HERE *** + @Logging(logEvent = true) + public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { + Map headers = new HashMap<>(); + + headers.put("Content-Type", "application/json"); + headers.put("Access-Control-Allow-Origin", "*"); + headers.put("Access-Control-Allow-Methods", "GET, OPTIONS"); + headers.put("Access-Control-Allow-Headers", "*"); + + APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent() + .withHeaders(headers); + try { + String address = JsonConfig.get().getObjectMapper().readTree(input.getBody()).get("address").asText(); + final String pageContents = this.getPageContents(address); + String output = String.format("{ \"message\": \"hello world\", \"location\": \"%s\" }", pageContents); + + log.info("ip is {}", pageContents); + return response + .withStatusCode(200) + .withBody(output); + + } catch (IOException e) { + return response + .withBody("{}") + .withStatusCode(500); + } + } + + // we could also put the @Idempotent annotation here, but using it on the handler avoids executing the handler (cost reduction). + // Use it on other methods to handle multiple items (with SQS batch processing for example) + private String getPageContents(String address) throws IOException { + URL url = new URL(address); + try (BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"))) { + return br.lines().collect(Collectors.joining(System.lineSeparator())); + } + } +} \ No newline at end of file diff --git a/examples/powertools-examples-idempotency/src/test/java/helloworld/AppTest.java b/examples/powertools-examples-idempotency/src/test/java/helloworld/AppTest.java new file mode 100644 index 000000000..7a5304e36 --- /dev/null +++ b/examples/powertools-examples-idempotency/src/test/java/helloworld/AppTest.java @@ -0,0 +1,86 @@ +package helloworld; + +import com.amazonaws.services.dynamodbv2.local.main.ServerRunner; +import com.amazonaws.services.dynamodbv2.local.server.DynamoDBProxyServer; +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import com.amazonaws.services.lambda.runtime.tests.EventLoader; +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; +import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; +import software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient; +import software.amazon.awssdk.regions.Region; +import software.amazon.awssdk.services.dynamodb.DynamoDbClient; +import software.amazon.awssdk.services.dynamodb.model.*; + +import java.io.IOException; +import java.net.ServerSocket; +import java.net.URI; + +public class AppTest { + @Mock + private Context context; + private App app; + private static DynamoDbClient client; + + @BeforeAll + public static void setupDynamoLocal() { + int port = getFreePort(); + try { + DynamoDBProxyServer dynamoProxy = ServerRunner.createServerFromCommandLineArgs(new String[]{ + "-inMemory", + "-port", + Integer.toString(port) + }); + dynamoProxy.start(); + } catch (Exception e) { + throw new RuntimeException(); + } + + client = DynamoDbClient.builder() + .httpClient(UrlConnectionHttpClient.builder().build()) + .region(Region.EU_WEST_1) + .endpointOverride(URI.create("http://localhost:" + port)) + .credentialsProvider(StaticCredentialsProvider.create( + AwsBasicCredentials.create("FAKE", "FAKE"))) + .build(); + + client.createTable(CreateTableRequest.builder() + .tableName("idempotency") + .keySchema(KeySchemaElement.builder().keyType(KeyType.HASH).attributeName("id").build()) + .attributeDefinitions( + AttributeDefinition.builder().attributeName("id").attributeType(ScalarAttributeType.S).build() + ) + .billingMode(BillingMode.PAY_PER_REQUEST) + .build()); + } + + private static int getFreePort() { + try { + ServerSocket socket = new ServerSocket(0); + int port = socket.getLocalPort(); + socket.close(); + return port; + } catch (IOException ioe) { + throw new RuntimeException(ioe); + } + } + + @BeforeEach + void setUp() { + MockitoAnnotations.openMocks(this); + app = new App(client); + } + + @Test + public void testApp() { + APIGatewayProxyResponseEvent response = app.handleRequest(EventLoader.loadApiGatewayRestEvent("event.json"), context); + Assertions.assertNotNull(response); + Assertions.assertTrue(response.getBody().contains("hello world")); + } +} diff --git a/examples/powertools-examples-idempotency/src/test/resources/event.json b/examples/powertools-examples-idempotency/src/test/resources/event.json new file mode 100644 index 000000000..fd7f5ace7 --- /dev/null +++ b/examples/powertools-examples-idempotency/src/test/resources/event.json @@ -0,0 +1,63 @@ +{ + "body": "{\"address\": \"https://checkip.amazonaws.com\"}", + "resource": "/{proxy+}", + "path": "/path/to/resource", + "httpMethod": "POST", + "isBase64Encoded": false, + "queryStringParameters": { + "foo": "bar" + }, + "pathParameters": { + "proxy": "/path/to/resource" + }, + "stageVariables": { + "baz": "qux" + }, + "headers": { + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", + "Accept-Encoding": "gzip, deflate, sdch", + "Accept-Language": "en-US,en;q=0.8", + "Cache-Control": "max-age=0", + "CloudFront-Forwarded-Proto": "https", + "CloudFront-Is-Desktop-Viewer": "true", + "CloudFront-Is-Mobile-Viewer": "false", + "CloudFront-Is-SmartTV-Viewer": "false", + "CloudFront-Is-Tablet-Viewer": "false", + "CloudFront-Viewer-Country": "US", + "Host": "1234567890.execute-api.us-east-1.amazonaws.com", + "Upgrade-Insecure-Requests": "1", + "User-Agent": "Custom User Agent String", + "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)", + "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==", + "X-Forwarded-For": "127.0.0.1, 127.0.0.2", + "X-Forwarded-Port": "443", + "X-Forwarded-Proto": "https" + }, + "requestContext": { + "accountId": "123456789012", + "resourceId": "123456", + "stage": "prod", + "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", + "requestTime": "09/Apr/2015:12:34:56 +0000", + "requestTimeEpoch": 1428582896000, + "identity": { + "cognitoIdentityPoolId": null, + "accountId": null, + "cognitoIdentityId": null, + "caller": null, + "accessKey": null, + "sourceIp": "127.0.0.1", + "cognitoAuthenticationType": null, + "cognitoAuthenticationProvider": null, + "userArn": null, + "userAgent": "Custom User Agent String", + "user": null + }, + "path": "/prod/path/to/resource", + "resourcePath": "/{proxy+}", + "httpMethod": "POST", + "apiId": "1234567890", + "protocol": "HTTP/1.1" + } + } + \ No newline at end of file diff --git a/examples/powertools-examples-idempotency/template.yaml b/examples/powertools-examples-idempotency/template.yaml new file mode 100644 index 000000000..2d5e5bc56 --- /dev/null +++ b/examples/powertools-examples-idempotency/template.yaml @@ -0,0 +1,59 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + Idempotency demo + +Globals: + Function: + Timeout: 20 + Runtime: java11 + MemorySize: 512 + Tracing: Active + Environment: + Variables: + POWERTOOLS_LOG_LEVEL: INFO + POWERTOOLS_LOGGER_SAMPLE_RATE: 0.1 + POWERTOOLS_LOGGER_LOG_EVENT: true + +Resources: + IdempotencyTable: + Type: AWS::DynamoDB::Table + Properties: + AttributeDefinitions: + - AttributeName: id + AttributeType: S + KeySchema: + - AttributeName: id + KeyType: HASH + TimeToLiveSpecification: + AttributeName: expiration + Enabled: true + BillingMode: PAY_PER_REQUEST + + IdempotencyFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: . + Handler: helloworld.App::handleRequest + Policies: + - DynamoDBCrudPolicy: + TableName: !Ref IdempotencyTable + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: idempotency + IDEMPOTENCY_TABLE: !Ref IdempotencyTable + Events: + HelloWorld: + Type: Api + Properties: + Path: /helloidem + Method: post + +Outputs: + HelloWorldApi: + Description: "API Gateway endpoint URL for Prod stage for Idempotent function" + Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/helloidem/" + HelloWorldFunction: + Description: "Idempotent Lambda Function ARN" + Value: !GetAtt IdempotencyFunction.Arn + diff --git a/examples/powertools-examples-parameters/README.md b/examples/powertools-examples-parameters/README.md new file mode 100644 index 000000000..c03f407c0 --- /dev/null +++ b/examples/powertools-examples-parameters/README.md @@ -0,0 +1,21 @@ +# Parameters + +This project contains an example of Lambda function using the parameters module of Lambda Powertools for Java. For more information on this module, please refer to the [documentation](https://awslabs.github.io/aws-lambda-powertools-java/utilities/parameters/). + +## Deploy the sample application + +This sample is based on Serverless Application Model (SAM) and you can use the SAM Command Line Interface (SAM CLI) to build it and deploy it to AWS. + +To use the SAM CLI, you need the following tools. + +* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) +* Java11 - [Install the Java 11](https://docs.aws.amazon.com/corretto/latest/corretto-11-ug/downloads-list.html) +* Maven - [Install Maven](https://maven.apache.org/install.html) +* Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community) + +To build and deploy your application for the first time, run the following in your shell: + +```bash +sam build +sam deploy --guided +``` diff --git a/examples/powertools-examples-parameters/pom.xml b/examples/powertools-examples-parameters/pom.xml new file mode 100644 index 000000000..f3797c948 --- /dev/null +++ b/examples/powertools-examples-parameters/pom.xml @@ -0,0 +1,90 @@ + + 4.0.0 + powertools-examples-parameters + jar + AWS Lambda Powertools for Java library Examples - Parameters + + + powertools-examples + software.amazon.lambda + 1.14.0 + + + + + software.amazon.lambda + powertools-logging + ${project.version} + + + software.amazon.lambda + powertools-parameters + ${project.version} + + + com.amazonaws + aws-lambda-java-core + 1.2.2 + + + com.amazonaws + aws-lambda-java-events + 3.11.0 + + + + + org.mockito + mockito-core + 5.1.1 + test + + + org.junit.jupiter + junit-jupiter-api + 5.9.2 + test + + + org.junit.jupiter + junit-jupiter-engine + 5.9.2 + test + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + 2.22.0 + + + org.codehaus.mojo + aspectj-maven-plugin + 1.14.0 + + ${maven.compiler.source} + ${maven.compiler.target} + ${maven.compiler.target} + + + software.amazon.lambda + powertools-parameters + + + + + + + compile + + + + + + + diff --git a/examples/powertools-examples-parameters/src/main/java/org/demo/parameters/MyObject.java b/examples/powertools-examples-parameters/src/main/java/org/demo/parameters/MyObject.java new file mode 100644 index 000000000..2cf145284 --- /dev/null +++ b/examples/powertools-examples-parameters/src/main/java/org/demo/parameters/MyObject.java @@ -0,0 +1,34 @@ +package org.demo.parameters; + +public class MyObject { + + private long id; + private String code; + + public MyObject() { + } + + public long getId() { + return id; + } + + public void setId(long id) { + this.id = id; + } + + public String getCode() { + return code; + } + + public void setCode(String code) { + this.code = code; + } + + @Override + public String toString() { + return "MyObject{" + + "id=" + id + + ", code='" + code + '\'' + + '}'; + } +} diff --git a/examples/powertools-examples-parameters/src/main/java/org/demo/parameters/ParametersFunction.java b/examples/powertools-examples-parameters/src/main/java/org/demo/parameters/ParametersFunction.java new file mode 100644 index 000000000..7f41e020e --- /dev/null +++ b/examples/powertools-examples-parameters/src/main/java/org/demo/parameters/ParametersFunction.java @@ -0,0 +1,81 @@ +package org.demo.parameters; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import software.amazon.lambda.powertools.parameters.ParamManager; +import software.amazon.lambda.powertools.parameters.SSMProvider; +import software.amazon.lambda.powertools.parameters.SecretsProvider; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; +import java.util.stream.Collectors; + +import static java.time.temporal.ChronoUnit.SECONDS; +import static software.amazon.lambda.powertools.parameters.transform.Transformer.base64; +import static software.amazon.lambda.powertools.parameters.transform.Transformer.json; + +public class ParametersFunction implements RequestHandler { + private final static Logger log = LogManager.getLogger(); + + SSMProvider ssmProvider = ParamManager.getSsmProvider(); + SecretsProvider secretsProvider = ParamManager.getSecretsProvider(); + + String simpleValue = ssmProvider.defaultMaxAge(30, SECONDS).get("/powertools-java/sample/simplekey"); + String listValue = ssmProvider.withMaxAge(60, SECONDS).get("/powertools-java/sample/keylist"); + MyObject jsonObj = ssmProvider.withTransformation(json).get("/powertools-java/sample/keyjson", MyObject.class); + Map allValues = ssmProvider.getMultiple("/powertools-java/sample"); + String b64value = ssmProvider.withTransformation(base64).get("/powertools-java/sample/keybase64"); + + Map secretJson = secretsProvider.withTransformation(json).get("/powertools-java/userpwd", Map.class); + MyObject secretJsonObj = secretsProvider.withMaxAge(42, SECONDS).withTransformation(json).get("/powertools-java/secretcode", MyObject.class); + + @Override + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent input, Context context) { + + log.info("\n=============== SSM Parameter Store ==============="); + log.info("simplevalue={}, listvalue={}, b64value={}\n", simpleValue, listValue, b64value); + log.info("jsonobj={}\n", jsonObj); + + log.info("allvalues (multiple):"); + allValues.forEach((key, value) -> log.info("- {}={}\n", key, value)); + + log.info("\n=============== Secrets Manager ==============="); + log.info("secretjson:"); + secretJson.forEach((key, value) -> log.info("- {}={}\n", key, value)); + log.info("secretjsonobj={}\n", secretJsonObj); + + Map headers = new HashMap<>(); + headers.put("Content-Type", "application/json"); + headers.put("X-Custom-Header", "application/json"); + + APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent() + .withHeaders(headers); + try { + final String pageContents = this.getPageContents("https://checkip.amazonaws.com"); + String output = String.format("{ \"message\": \"hello world\", \"location\": \"%s\" }", pageContents); + + return response + .withStatusCode(200) + .withBody(output); + } catch (IOException e) { + return response + .withBody("{}") + .withStatusCode(500); + } + } + + private String getPageContents(String address) throws IOException{ + URL url = new URL(address); + try(BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"))) { + return br.lines().collect(Collectors.joining(System.lineSeparator())); + } + } +} diff --git a/examples/powertools-examples-parameters/src/main/resources/log4j2.xml b/examples/powertools-examples-parameters/src/main/resources/log4j2.xml new file mode 100644 index 000000000..033da8a11 --- /dev/null +++ b/examples/powertools-examples-parameters/src/main/resources/log4j2.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/examples/powertools-examples-parameters/template.yaml b/examples/powertools-examples-parameters/template.yaml new file mode 100644 index 000000000..052cfcdc7 --- /dev/null +++ b/examples/powertools-examples-parameters/template.yaml @@ -0,0 +1,97 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + validation demo + +Globals: + Function: + Timeout: 20 + Runtime: java11 + MemorySize: 512 + Tracing: Active + + +Resources: + ParametersFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: . + Handler: org.demo.parameters.ParametersFunction::handleRequest + MemorySize: 512 + Tracing: Active + Policies: + - AWSSecretsManagerGetSecretValuePolicy: + SecretArn: !Ref UserPwd + - AWSSecretsManagerGetSecretValuePolicy: + SecretArn: !Ref SecretConfig + - Statement: + - Sid: SSMGetParameterPolicy + Effect: Allow + Action: + - ssm:GetParameter + - ssm:GetParameters + - ssm:GetParametersByPath + Resource: '*' + Events: + HelloWorld: + Type: Api + Properties: + Path: /params + Method: get + + UserPwd: + Type: AWS::SecretsManager::Secret + Properties: + Name: /powertools-java/userpwd + Description: Generated secret for lambda-powertools-java powertools-parameters + module + GenerateSecretString: + SecretStringTemplate: '{"username": "test-user"}' + GenerateStringKey: password + PasswordLength: 15 + ExcludeCharacters: '"@/\' + SecretConfig: + Type: AWS::SecretsManager::Secret + Properties: + Name: /powertools-java/secretcode + Description: Json secret for lambda-powertools-java powertools-parameters module + SecretString: '{"id":23443,"code":"hk38543oj24kn796kp67bkb234gkj679l68"}' + BasicParameter: + Type: AWS::SSM::Parameter + Properties: + Name: /powertools-java/sample/simplekey + Type: String + Value: simplevalue + Description: Simple SSM Parameter for lambda-powertools-java powertools-parameters + module + ParameterList: + Type: AWS::SSM::Parameter + Properties: + Name: /powertools-java/sample/keylist + Type: StringList + Value: value1,value2,value3 + Description: SSM Parameter List for lambda-powertools-java powertools-parameters + module + JsonParameter: + Type: AWS::SSM::Parameter + Properties: + Name: /powertools-java/sample/keyjson + Type: String + Value: '{"id":23443,"code":"hk38543oj24kn796kp67bkb234gkj679l68"}' + Description: Json SSM Parameter for lambda-powertools-java powertools-parameters + module + Base64Parameter: + Type: AWS::SSM::Parameter + Properties: + Name: /powertools-java/sample/keybase64 + Type: String + Value: aGVsbG8gd29ybGQ= + Description: Base64 SSM Parameter for lambda-powertools-java powertools-parameters module + +Outputs: + ParametersApi: + Description: "API Gateway endpoint URL for Prod stage for Parameters function" + Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/params/" + ParametersFunction: + Description: "Parameters Lambda Function ARN" + Value: !GetAtt ParametersFunction.Arn \ No newline at end of file diff --git a/examples/powertools-examples-serialization/README.md b/examples/powertools-examples-serialization/README.md new file mode 100644 index 000000000..68c4b4ae2 --- /dev/null +++ b/examples/powertools-examples-serialization/README.md @@ -0,0 +1,21 @@ +# Deserialization + +This project contains an example of Lambda function using the serialization utilities module of Lambda Powertools for Java. For more information on this module, please refer to the [documentation](https://awslabs.github.io/aws-lambda-powertools-java/utilities/serialization/). + +## Deploy the sample application + +This sample is based on Serverless Application Model (SAM) and you can use the SAM Command Line Interface (SAM CLI) to build it and deploy it to AWS. + +To use the SAM CLI, you need the following tools. + +* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) +* Java11 - [Install the Java 11](https://docs.aws.amazon.com/corretto/latest/corretto-11-ug/downloads-list.html) +* Maven - [Install Maven](https://maven.apache.org/install.html) +* Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community) + +To build and deploy your application for the first time, run the following in your shell: + +```bash +sam build +sam deploy --guided +``` diff --git a/examples/powertools-examples-serialization/events/APIGatewayEvent.json b/examples/powertools-examples-serialization/events/APIGatewayEvent.json new file mode 100644 index 000000000..cb38a3429 --- /dev/null +++ b/examples/powertools-examples-serialization/events/APIGatewayEvent.json @@ -0,0 +1,63 @@ +{ + "body": "{\"id\":1234, \"name\":\"product\", \"price\":42}", + "resource": "/{proxy+}", + "path": "/path/to/resource", + "httpMethod": "POST", + "isBase64Encoded": false, + "queryStringParameters": { + "foo": "bar" + }, + "pathParameters": { + "proxy": "/path/to/resource" + }, + "stageVariables": { + "baz": "qux" + }, + "headers": { + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", + "Accept-Encoding": "gzip, deflate, sdch", + "Accept-Language": "en-US,en;q=0.8", + "Cache-Control": "max-age=0", + "CloudFront-Forwarded-Proto": "https", + "CloudFront-Is-Desktop-Viewer": "true", + "CloudFront-Is-Mobile-Viewer": "false", + "CloudFront-Is-SmartTV-Viewer": "false", + "CloudFront-Is-Tablet-Viewer": "false", + "CloudFront-Viewer-Country": "US", + "Host": "1234567890.execute-api.us-east-1.amazonaws.com", + "Upgrade-Insecure-Requests": "1", + "User-Agent": "Custom User Agent String", + "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)", + "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==", + "X-Forwarded-For": "127.0.0.1, 127.0.0.2", + "X-Forwarded-Port": "443", + "X-Forwarded-Proto": "https" + }, + "requestContext": { + "accountId": "123456789012", + "resourceId": "123456", + "stage": "prod", + "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", + "requestTime": "09/Apr/2015:12:34:56 +0000", + "requestTimeEpoch": 1428582896000, + "identity": { + "cognitoIdentityPoolId": null, + "accountId": null, + "cognitoIdentityId": null, + "caller": null, + "accessKey": null, + "sourceIp": "127.0.0.1", + "cognitoAuthenticationType": null, + "cognitoAuthenticationProvider": null, + "userArn": null, + "userAgent": "Custom User Agent String", + "user": null + }, + "path": "/prod/path/to/resource", + "resourcePath": "/{proxy+}", + "httpMethod": "POST", + "apiId": "1234567890", + "protocol": "HTTP/1.1" + } + } + \ No newline at end of file diff --git a/examples/powertools-examples-serialization/events/SQSEvent.json b/examples/powertools-examples-serialization/events/SQSEvent.json new file mode 100644 index 000000000..9056d3ef2 --- /dev/null +++ b/examples/powertools-examples-serialization/events/SQSEvent.json @@ -0,0 +1,39 @@ +{ + "Records": [ + { + "messageId": "d9144555-9a4f-4ec3-99a0-34ce359b4b54", + "receiptHandle": "13e7f7851d2eaa5c01f208ebadbf1e72==", + "body": "{ \"id\": 1234, \"name\": \"product\", \"price\": 42}", + "attributes": { + "ApproximateReceiveCount": "1", + "SentTimestamp": "1601975706495", + "SenderId": "AROAIFU437PVZ5L2J53F5", + "ApproximateFirstReceiveTimestamp": "1601975706499" + }, + "messageAttributes": { + }, + "md5OfBody": "13e7f7851d2eaa5c01f208ebadbf1e72", + "eventSource": "aws:sqs", + "eventSourceARN": "arn:aws:sqs:eu-central-1:123456789012:TestLambda", + "awsRegion": "eu-central-1" + }, + { + "messageId": "d9144555-9a4f-4ec3-99a0-34ce359b4b54", + "receiptHandle": "13e7f7851d2eaa5c01f208ebadbf1e72==", + "body": "{ \"id\": 12345, \"name\": \"product5\", \"price\": 45}", + "attributes": { + "ApproximateReceiveCount": "1", + "SentTimestamp": "1601975706495", + "SenderId": "AROAIFU437PVZ5L2J53F5", + "ApproximateFirstReceiveTimestamp": "1601975706499" + }, + "messageAttributes": { + + }, + "md5OfBody": "13e7f7851d2eaa5c01f208ebadbf1e72", + "eventSource": "aws:sqs", + "eventSourceARN": "arn:aws:sqs:eu-central-1:123456789012:TestLambda", + "awsRegion": "eu-central-1" + } + ] +} \ No newline at end of file diff --git a/examples/powertools-examples-serialization/pom.xml b/examples/powertools-examples-serialization/pom.xml new file mode 100644 index 000000000..f1d45f760 --- /dev/null +++ b/examples/powertools-examples-serialization/pom.xml @@ -0,0 +1,67 @@ + + 4.0.0 + powertools-examples-serialization + jar + AWS Lambda Powertools for Java library Examples - Serialization + + + powertools-examples + software.amazon.lambda + 1.14.0 + + + + + software.amazon.lambda + powertools-logging + ${project.version} + + + software.amazon.lambda + powertools-serialization + ${project.version} + + + com.amazonaws + aws-lambda-java-core + 1.2.2 + + + com.amazonaws + aws-lambda-java-events + 3.11.0 + + + + + org.mockito + mockito-core + 4.11.0 + test + + + org.junit.jupiter + junit-jupiter-api + 5.9.2 + test + + + org.junit.jupiter + junit-jupiter-engine + 5.9.2 + test + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + 2.22.0 + + + + diff --git a/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/APIGatewayRequestDeserializationFunction.java b/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/APIGatewayRequestDeserializationFunction.java new file mode 100644 index 000000000..8c33baed9 --- /dev/null +++ b/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/APIGatewayRequestDeserializationFunction.java @@ -0,0 +1,36 @@ +package org.demo.serialization; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; + +import java.util.HashMap; +import java.util.Map; + +import static software.amazon.lambda.powertools.utilities.EventDeserializer.extractDataFrom; + + +public class APIGatewayRequestDeserializationFunction implements RequestHandler { + + private final static Logger LOGGER = LogManager.getLogger(APIGatewayRequestDeserializationFunction.class); + private static final Map HEADERS = new HashMap() {{ + put("Content-Type", "application/json"); + put("X-Custom-Header", "application/json"); + }}; + + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent event, Context context) { + + Product product = extractDataFrom(event).as(Product.class); + LOGGER.info("\n=============== Deserialized request body: ==============="); + LOGGER.info("product={}\n", product); + + return new APIGatewayProxyResponseEvent() + .withHeaders(HEADERS) + .withStatusCode(200) + .withBody("Received request for productId: " + product.getId()); + } +} + diff --git a/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/Product.java b/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/Product.java new file mode 100644 index 000000000..fb94a99f8 --- /dev/null +++ b/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/Product.java @@ -0,0 +1,49 @@ +package org.demo.serialization; + +public class Product { + private long id; + private String name; + private double price; + + public Product() { + } + + public Product(long id, String name, double price) { + this.id = id; + this.name = name; + this.price = price; + } + + public long getId() { + return id; + } + + public void setId(long id) { + this.id = id; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public double getPrice() { + return price; + } + + public void setPrice(double price) { + this.price = price; + } + + @Override + public String toString() { + return "Product{" + + "id=" + id + + ", name='" + name + '\'' + + ", price=" + price + + '}'; + } +} diff --git a/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/SQSEventDeserializationFunction.java b/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/SQSEventDeserializationFunction.java new file mode 100644 index 000000000..129fe0243 --- /dev/null +++ b/examples/powertools-examples-serialization/src/main/java/org/demo/serialization/SQSEventDeserializationFunction.java @@ -0,0 +1,27 @@ +package org.demo.serialization; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.SQSEvent; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; + +import java.util.List; + +import static software.amazon.lambda.powertools.utilities.EventDeserializer.extractDataFrom; + + +public class SQSEventDeserializationFunction implements RequestHandler { + + private final static Logger LOGGER = LogManager.getLogger(SQSEventDeserializationFunction.class); + + public String handleRequest(SQSEvent event, Context context) { + List products = extractDataFrom(event).asListOf(Product.class); + + LOGGER.info("\n=============== Deserialized messages: ==============="); + LOGGER.info("products={}\n", products); + + return "Number of received messages: " + products.size(); + } +} + diff --git a/examples/powertools-examples-serialization/src/main/resources/log4j2.xml b/examples/powertools-examples-serialization/src/main/resources/log4j2.xml new file mode 100644 index 000000000..033da8a11 --- /dev/null +++ b/examples/powertools-examples-serialization/src/main/resources/log4j2.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/examples/powertools-examples-serialization/src/test/java/org/demo/serialization/APIGatewayRequestDeserializationFunctionTest.java b/examples/powertools-examples-serialization/src/test/java/org/demo/serialization/APIGatewayRequestDeserializationFunctionTest.java new file mode 100644 index 000000000..5d5da7ecc --- /dev/null +++ b/examples/powertools-examples-serialization/src/test/java/org/demo/serialization/APIGatewayRequestDeserializationFunctionTest.java @@ -0,0 +1,35 @@ +package org.demo.serialization; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +class APIGatewayRequestDeserializationFunctionTest { + + @Mock + private Context context; + private APIGatewayRequestDeserializationFunction deserializationFunction; + + @BeforeEach + void setUp() { + MockitoAnnotations.openMocks(this); + deserializationFunction = new APIGatewayRequestDeserializationFunction(); + } + + @Test + public void shouldReturnOkStatusWithProductId() { + String body = "{\"id\":1234, \"name\":\"product\", \"price\":42}"; + APIGatewayProxyRequestEvent request = new APIGatewayProxyRequestEvent().withBody(body); + + APIGatewayProxyResponseEvent response = deserializationFunction.handleRequest(request, context); + + assertEquals(200, response.getStatusCode()); + assertEquals("Received request for productId: 1234", response.getBody()); + } +} \ No newline at end of file diff --git a/examples/powertools-examples-serialization/src/test/java/org/demo/serialization/SQSEventDeserializationFunctionTest.java b/examples/powertools-examples-serialization/src/test/java/org/demo/serialization/SQSEventDeserializationFunctionTest.java new file mode 100644 index 000000000..6979a6868 --- /dev/null +++ b/examples/powertools-examples-serialization/src/test/java/org/demo/serialization/SQSEventDeserializationFunctionTest.java @@ -0,0 +1,47 @@ +package org.demo.serialization; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.events.SQSEvent; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; + +import java.util.List; +import java.util.ArrayList; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +class SQSEventDeserializationFunctionTest { + + @Mock + private Context context; + private SQSEventDeserializationFunction deserializationFunction; + + @BeforeEach + void setUp() { + MockitoAnnotations.openMocks(this); + deserializationFunction = new SQSEventDeserializationFunction(); + } + + @Test + public void shouldReturnNumberOfReceivedMessages() { + SQSEvent.SQSMessage message1 = messageWithBody("{ \"id\": 1234, \"name\": \"product\", \"price\": 42}"); + SQSEvent.SQSMessage message2 = messageWithBody("{ \"id\": 12345, \"name\": \"product5\", \"price\": 45}"); + SQSEvent event = new SQSEvent(); + event.setRecords(new ArrayList(){{ + add(message1); + add(message2); + }}); + + String response = deserializationFunction.handleRequest(event, context); + + assertEquals("Number of received messages: 2", response); + } + + private SQSEvent.SQSMessage messageWithBody(String body) { + SQSEvent.SQSMessage record1 = new SQSEvent.SQSMessage(); + record1.setBody(body); + return record1; + } +} \ No newline at end of file diff --git a/examples/powertools-examples-serialization/template.yaml b/examples/powertools-examples-serialization/template.yaml new file mode 100644 index 000000000..539d2d615 --- /dev/null +++ b/examples/powertools-examples-serialization/template.yaml @@ -0,0 +1,63 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + serialization utils demo + +Globals: + Function: + Timeout: 20 + Runtime: java11 + MemorySize: 512 + Tracing: Active + + +Resources: + APIGatewayDeserializationFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: . + Handler: org.demo.serialization.APIGatewayRequestDeserializationFunction::handleRequest + Events: + Product: + Type: Api + Properties: + Path: /product + Method: post + + DemoSqsQueue: + Type: AWS::SQS::Queue + Properties: + QueueName: "sqs-event-deserialization-queue" + + SQSEventDeserializationFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: Function + Handler: org.demo.serialization.SQSEventDeserializationFunction::handleRequest + Policies: + - Statement: + - Sid: SQSSendMessageBatch + Effect: Allow + Action: + - sqs:SendMessageBatch + - sqs:SendMessage + Resource: !GetAtt DemoSqsQueue.Arn + Events: + SQSEvent: + Type: SQS + Properties: + Queue: !GetAtt DemoSqsQueue.Arn + BatchSize: 2 + MaximumBatchingWindowInSeconds: 30 + + +Outputs: + Api: + Description: "API Gateway endpoint URL for Prod stage for Serialization function" + Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/product/" + Function: + Description: "Serialization Lambda Function ARN" + Value: !GetAtt APIGatewayDeserializationFunction.Arn + DemoSqsQueue: + Description: "ARN for SQS queue" + Value: !GetAtt DemoSqsQueue.Arn \ No newline at end of file diff --git a/examples/powertools-examples-sqs/README.md b/examples/powertools-examples-sqs/README.md new file mode 100644 index 000000000..2b6da65b5 --- /dev/null +++ b/examples/powertools-examples-sqs/README.md @@ -0,0 +1,3 @@ +## SqsBatchProcessingDemo + +Demos setup of SQS Batch processing via Powertools diff --git a/examples/powertools-examples-sqs/events/event.json b/examples/powertools-examples-sqs/events/event.json new file mode 100644 index 000000000..3822fadaa --- /dev/null +++ b/examples/powertools-examples-sqs/events/event.json @@ -0,0 +1,63 @@ +{ + "body": "{\"message\": \"hello world\"}", + "resource": "/{proxy+}", + "path": "/path/to/resource", + "httpMethod": "POST", + "isBase64Encoded": false, + "queryStringParameters": { + "foo": "bar" + }, + "pathParameters": { + "proxy": "/path/to/resource" + }, + "stageVariables": { + "baz": "qux" + }, + "headers": { + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", + "Accept-Encoding": "gzip, deflate, sdch", + "Accept-Language": "en-US,en;q=0.8", + "Cache-Control": "max-age=0", + "CloudFront-Forwarded-Proto": "https", + "CloudFront-Is-Desktop-Viewer": "true", + "CloudFront-Is-Mobile-Viewer": "false", + "CloudFront-Is-SmartTV-Viewer": "false", + "CloudFront-Is-Tablet-Viewer": "false", + "CloudFront-Viewer-Country": "US", + "Host": "1234567890.execute-api.us-east-1.amazonaws.com", + "Upgrade-Insecure-Requests": "1", + "User-Agent": "Custom User Agent String", + "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)", + "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==", + "X-Forwarded-For": "127.0.0.1, 127.0.0.2", + "X-Forwarded-Port": "443", + "X-Forwarded-Proto": "https" + }, + "requestContext": { + "accountId": "123456789012", + "resourceId": "123456", + "stage": "prod", + "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", + "requestTime": "09/Apr/2015:12:34:56 +0000", + "requestTimeEpoch": 1428582896000, + "identity": { + "cognitoIdentityPoolId": null, + "accountId": null, + "cognitoIdentityId": null, + "caller": null, + "accessKey": null, + "sourceIp": "127.0.0.1", + "cognitoAuthenticationType": null, + "cognitoAuthenticationProvider": null, + "userArn": null, + "userAgent": "Custom User Agent String", + "user": null + }, + "path": "/prod/path/to/resource", + "resourcePath": "/{proxy+}", + "httpMethod": "POST", + "apiId": "1234567890", + "protocol": "HTTP/1.1" + } + } + \ No newline at end of file diff --git a/examples/powertools-examples-sqs/pom.xml b/examples/powertools-examples-sqs/pom.xml new file mode 100644 index 000000000..0dbcabc53 --- /dev/null +++ b/examples/powertools-examples-sqs/pom.xml @@ -0,0 +1,122 @@ + + 4.0.0 + powertools-examples-sqs + jar + AWS Lambda Powertools for Java library Examples - SQS + + + powertools-examples + software.amazon.lambda + 1.14.0 + + + + + software.amazon.lambda + powertools-logging + ${project.version} + + + software.amazon.lambda + powertools-sqs + ${project.version} + + + software.amazon.awssdk + url-connection-client + 2.20.1 + + + com.amazonaws + aws-lambda-java-core + 1.2.2 + + + com.amazonaws + aws-lambda-java-events + 3.11.0 + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + + + org.apache.logging.log4j + log4j-api + ${log4j.version} + + + com.fasterxml.jackson.datatype + jackson-datatype-joda + 2.14.2 + + + + junit + junit + 4.13.2 + test + + + + + + + org.codehaus.mojo + aspectj-maven-plugin + 1.14.0 + + ${maven.compiler.source} + ${maven.compiler.target} + ${maven.compiler.target} + + + software.amazon.lambda + powertools-logging + + + software.amazon.lambda + powertools-sqs + + + + + + + compile + + + + + + org.apache.maven.plugins + maven-shade-plugin + 3.4.0 + + + package + + shade + + + + + + + + + + + + com.github.edwgiz + maven-shade-plugin.log4j2-cachefile-transformer + 2.15 + + + + + + diff --git a/examples/powertools-examples-sqs/src/main/java/org/demo/sqs/SqsMessageSender.java b/examples/powertools-examples-sqs/src/main/java/org/demo/sqs/SqsMessageSender.java new file mode 100644 index 000000000..b90c50654 --- /dev/null +++ b/examples/powertools-examples-sqs/src/main/java/org/demo/sqs/SqsMessageSender.java @@ -0,0 +1,79 @@ + +package org.demo.sqs; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.ScheduledEvent; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.datatype.joda.JodaModule; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient; +import software.amazon.awssdk.services.sqs.SqsClient; +import software.amazon.awssdk.services.sqs.model.MessageAttributeValue; +import software.amazon.awssdk.services.sqs.model.SendMessageBatchRequest; +import software.amazon.awssdk.services.sqs.model.SendMessageBatchRequestEntry; +import software.amazon.awssdk.services.sqs.model.SendMessageBatchResponse; +import software.amazon.lambda.powertools.logging.Logging; +import software.amazon.lambda.powertools.logging.LoggingUtils; + +import java.security.SecureRandom; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Random; +import java.util.stream.IntStream; + +import static java.util.stream.Collectors.toList; + +public class SqsMessageSender implements RequestHandler { + + private static final Logger log = LogManager.getLogger(SqsMessageSender.class); + + private static final SqsClient sqsClient = SqsClient.builder() + .httpClient(UrlConnectionHttpClient.create()) + .build(); + + private static final Random random = new SecureRandom(); + + private static final ObjectMapper objectMapper; + + static { + objectMapper = new ObjectMapper(); + objectMapper.registerModule(new JodaModule()); + LoggingUtils.defaultObjectMapper(objectMapper); + } + + @Logging(logEvent = true) + public String handleRequest(final ScheduledEvent input, final Context context) { + String queueUrl = System.getenv("QUEUE_URL"); + + // Push 5 messages on each invoke. + List batchRequestEntries = IntStream.range(0, 5) + .mapToObj(value -> { + Map attributeValueHashMap = new HashMap<>(); + attributeValueHashMap.put("Key" + value, MessageAttributeValue.builder() + .dataType("String") + .stringValue("Value" + value) + .build()); + + byte[] array = new byte[7]; + random.nextBytes(array); + + return SendMessageBatchRequestEntry.builder() + .messageAttributes(attributeValueHashMap) + .id(input.getId() + value) + .messageBody("Sample Message " + value) + .build(); + }).collect(toList()); + + SendMessageBatchResponse sendMessageBatchResponse = sqsClient.sendMessageBatch(SendMessageBatchRequest.builder() + .queueUrl(queueUrl) + .entries(batchRequestEntries) + .build()); + + log.info("Sent Message {}", sendMessageBatchResponse); + + return "Success"; + } +} \ No newline at end of file diff --git a/examples/powertools-examples-sqs/src/main/java/org/demo/sqs/SqsPoller.java b/examples/powertools-examples-sqs/src/main/java/org/demo/sqs/SqsPoller.java new file mode 100644 index 000000000..bf2b7bdfe --- /dev/null +++ b/examples/powertools-examples-sqs/src/main/java/org/demo/sqs/SqsPoller.java @@ -0,0 +1,61 @@ +package org.demo.sqs; + +import java.util.Random; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.SQSEvent; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient; +import software.amazon.awssdk.services.sqs.SqsClient; +import software.amazon.lambda.powertools.logging.Logging; +import software.amazon.lambda.powertools.sqs.SqsBatch; +import software.amazon.lambda.powertools.sqs.SqsMessageHandler; +import software.amazon.lambda.powertools.sqs.SqsUtils; +import java.security.SecureRandom; + +import static com.amazonaws.services.lambda.runtime.events.SQSEvent.SQSMessage; + +/** + * Handler for requests to Lambda function. + */ +public class SqsPoller implements RequestHandler { + + Logger log = LogManager.getLogger(SqsPoller.class); + Random random = new SecureRandom(); + + static { + // https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/lambda-optimize-starttime.html + SqsUtils.overrideSqsClient(SqsClient.builder() + .httpClient(UrlConnectionHttpClient.create()) + .build()); + } + + @SqsBatch(value = BatchProcessor.class, nonRetryableExceptions = {IllegalArgumentException.class}) + @Logging(logEvent = true) + public String handleRequest(final SQSEvent input, final Context context) { + return "Success"; + } + + private class BatchProcessor implements SqsMessageHandler { + @Override + public String process(SQSMessage message) { + log.info("Processing message with id {}", message.getMessageId()); + + int nextInt = random.nextInt(100); + + if(nextInt <= 10) { + log.info("Randomly picked message with id {} as business validation failure.", message.getMessageId()); + throw new IllegalArgumentException("Failed business validation. No point of retrying. Move me to DLQ." + message.getMessageId()); + } + + if(nextInt > 90) { + log.info("Randomly picked message with id {} as intermittent failure.", message.getMessageId()); + throw new RuntimeException("Failed due to intermittent issue. Will be sent back for retry." + message.getMessageId()); + } + + return "Success"; + } + } +} \ No newline at end of file diff --git a/examples/powertools-examples-sqs/src/main/resources/log4j2.xml b/examples/powertools-examples-sqs/src/main/resources/log4j2.xml new file mode 100644 index 000000000..e1fd14cea --- /dev/null +++ b/examples/powertools-examples-sqs/src/main/resources/log4j2.xml @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/examples/powertools-examples-sqs/template.yaml b/examples/powertools-examples-sqs/template.yaml new file mode 100644 index 000000000..ebf20831f --- /dev/null +++ b/examples/powertools-examples-sqs/template.yaml @@ -0,0 +1,148 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + sqs batch processing demo + +Globals: + Function: + Timeout: 20 + Runtime: java11 + MemorySize: 512 + Tracing: Active + Environment: + Variables: + # Powertools env vars: https://awslabs.github.io/aws-lambda-powertools-python/#environment-variables + POWERTOOLS_LOG_LEVEL: INFO + POWERTOOLS_LOGGER_SAMPLE_RATE: 0.1 + POWERTOOLS_LOGGER_LOG_EVENT: true + +Resources: + CustomerKey: + Type: AWS::KMS::Key + Properties: + Description: KMS key for encrypted queues + Enabled: true + KeyPolicy: + Version: '2012-10-17' + Statement: + - Sid: Enable IAM User Permissions + Effect: Allow + Principal: + AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:root' + Action: 'kms:*' + Resource: '*' + - Sid: Allow use of the key + Effect: Allow + Principal: + Service: lambda.amazonaws.com + Action: + - kms:Decrypt + - kms:GenerateDataKey + Resource: '*' + + CustomerKeyAlias: + Type: AWS::KMS::Alias + Properties: + AliasName: alias/sqs-key + TargetKeyId: !Ref CustomerKey + + DemoDlqSqsQueue: + Type: AWS::SQS::Queue + Properties: + KmsMasterKeyId: !Ref CustomerKey + + DemoSqsQueue: + Type: AWS::SQS::Queue + Properties: + RedrivePolicy: + deadLetterTargetArn: + Fn::GetAtt: + - "DemoDlqSqsQueue" + - "Arn" + maxReceiveCount: 2 + KmsMasterKeyId: !Ref CustomerKey + + DemoSQSSenderFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: . + Handler: org.demo.sqs.SqsMessageSender::handleRequest + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: sqs-demo + QUEUE_URL: !Ref DemoSqsQueue + Policies: + - Statement: + - Sid: SQSSendMessageBatch + Effect: Allow + Action: + - sqs:SendMessageBatch + - sqs:SendMessage + Resource: !GetAtt DemoSqsQueue.Arn + - Sid: SQSKMSKey + Effect: Allow + Action: + - kms:GenerateDataKey + - kms:Decrypt + Resource: !GetAtt CustomerKey.Arn + Events: + CWSchedule: + Type: Schedule + Properties: + Schedule: 'rate(5 minutes)' + Name: !Join ["-", ["message-producer-schedule", !Select [0, !Split [-, !Select [2, !Split [/, !Ref AWS::StackId ]]]]]] + Description: Produce message to SQS via a Lambda function + Enabled: true + + DemoSQSConsumerFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: . + Handler: org.demo.sqs.SqsPoller::handleRequest + Environment: + Variables: + POWERTOOLS_SERVICE_NAME: sqs-demo + Policies: + - Statement: + - Sid: SQSDeleteGetAttribute + Effect: Allow + Action: + - sqs:DeleteMessageBatch + - sqs:GetQueueAttributes + Resource: !GetAtt DemoSqsQueue.Arn + - Sid: SQSSendMessageBatch + Effect: Allow + Action: + - sqs:SendMessageBatch + - sqs:SendMessage + Resource: !GetAtt DemoDlqSqsQueue.Arn + - Sid: SQSKMSKey + Effect: Allow + Action: + - kms:GenerateDataKey + - kms:Decrypt + Resource: !GetAtt CustomerKey.Arn + Events: + MySQSEvent: + Type: SQS + Properties: + Queue: !GetAtt DemoSqsQueue.Arn + BatchSize: 2 + MaximumBatchingWindowInSeconds: 300 + +Outputs: + DemoSqsQueue: + Description: "ARN for main SQS queue" + Value: !GetAtt DemoSqsQueue.Arn + DemoDlqSqsQueue: + Description: "ARN for DLQ" + Value: !GetAtt DemoDlqSqsQueue.Arn + DemoSQSSenderFunction: + Description: "Sender SQS Lambda Function ARN" + Value: !GetAtt DemoSQSSenderFunction.Arn + DemoSQSConsumerFunction: + Description: "Consumer SQS Lambda Function ARN" + Value: !GetAtt DemoSQSConsumerFunction.Arn + DemoSQSConsumerFunctionRole: + Description: "Implicit IAM Role created for SQS Lambda Function ARN" + Value: !GetAtt DemoSQSConsumerFunctionRole.Arn diff --git a/examples/powertools-examples-validation/README.md b/examples/powertools-examples-validation/README.md new file mode 100644 index 000000000..db631deff --- /dev/null +++ b/examples/powertools-examples-validation/README.md @@ -0,0 +1,21 @@ +# Validation + +This project contains an example of Lambda function using the validation module of Lambda Powertools for Java. For more information on this module, please refer to the [documentation](https://awslabs.github.io/aws-lambda-powertools-java/utilities/validation/). + +## Deploy the sample application + +This sample is based on Serverless Application Model (SAM) and you can use the SAM Command Line Interface (SAM CLI) to build it and deploy it to AWS. + +To use the SAM CLI, you need the following tools. + +* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) +* Java11 - [Install the Java 11](https://docs.aws.amazon.com/corretto/latest/corretto-11-ug/downloads-list.html) +* Maven - [Install Maven](https://maven.apache.org/install.html) +* Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community) + +To build and deploy your application for the first time, run the following in your shell: + +```bash +sam build +sam deploy --guided +``` diff --git a/examples/powertools-examples-validation/events/event.json b/examples/powertools-examples-validation/events/event.json new file mode 100644 index 000000000..3822fadaa --- /dev/null +++ b/examples/powertools-examples-validation/events/event.json @@ -0,0 +1,63 @@ +{ + "body": "{\"message\": \"hello world\"}", + "resource": "/{proxy+}", + "path": "/path/to/resource", + "httpMethod": "POST", + "isBase64Encoded": false, + "queryStringParameters": { + "foo": "bar" + }, + "pathParameters": { + "proxy": "/path/to/resource" + }, + "stageVariables": { + "baz": "qux" + }, + "headers": { + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8", + "Accept-Encoding": "gzip, deflate, sdch", + "Accept-Language": "en-US,en;q=0.8", + "Cache-Control": "max-age=0", + "CloudFront-Forwarded-Proto": "https", + "CloudFront-Is-Desktop-Viewer": "true", + "CloudFront-Is-Mobile-Viewer": "false", + "CloudFront-Is-SmartTV-Viewer": "false", + "CloudFront-Is-Tablet-Viewer": "false", + "CloudFront-Viewer-Country": "US", + "Host": "1234567890.execute-api.us-east-1.amazonaws.com", + "Upgrade-Insecure-Requests": "1", + "User-Agent": "Custom User Agent String", + "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)", + "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==", + "X-Forwarded-For": "127.0.0.1, 127.0.0.2", + "X-Forwarded-Port": "443", + "X-Forwarded-Proto": "https" + }, + "requestContext": { + "accountId": "123456789012", + "resourceId": "123456", + "stage": "prod", + "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", + "requestTime": "09/Apr/2015:12:34:56 +0000", + "requestTimeEpoch": 1428582896000, + "identity": { + "cognitoIdentityPoolId": null, + "accountId": null, + "cognitoIdentityId": null, + "caller": null, + "accessKey": null, + "sourceIp": "127.0.0.1", + "cognitoAuthenticationType": null, + "cognitoAuthenticationProvider": null, + "userArn": null, + "userAgent": "Custom User Agent String", + "user": null + }, + "path": "/prod/path/to/resource", + "resourcePath": "/{proxy+}", + "httpMethod": "POST", + "apiId": "1234567890", + "protocol": "HTTP/1.1" + } + } + \ No newline at end of file diff --git a/examples/powertools-examples-validation/pom.xml b/examples/powertools-examples-validation/pom.xml new file mode 100644 index 000000000..f7522303b --- /dev/null +++ b/examples/powertools-examples-validation/pom.xml @@ -0,0 +1,86 @@ + + 4.0.0 + powertools-examples-validation + jar + AWS Lambda Powertools for Java library Examples - Validation + + + powertools-parent + software.amazon.lambda + 1.14.0 + + + + + + software.amazon.lambda + powertools-logging + ${project.version} + + + software.amazon.lambda + powertools-validation + ${project.version} + + + com.amazonaws + aws-lambda-java-core + 1.2.2 + + + + + org.mockito + mockito-core + 4.11.0 + test + + + org.junit.jupiter + junit-jupiter-api + 5.9.2 + test + + + org.junit.jupiter + junit-jupiter-engine + 5.9.2 + test + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + 2.22.0 + + + org.codehaus.mojo + aspectj-maven-plugin + 1.14.0 + + ${maven.compiler.source} + ${maven.compiler.target} + ${maven.compiler.target} + + + software.amazon.lambda + powertools-validation + + + + + + + compile + + + + + + + diff --git a/examples/powertools-examples-validation/src/main/java/org/demo/validation/InboundValidation.java b/examples/powertools-examples-validation/src/main/java/org/demo/validation/InboundValidation.java new file mode 100644 index 000000000..89ec538c9 --- /dev/null +++ b/examples/powertools-examples-validation/src/main/java/org/demo/validation/InboundValidation.java @@ -0,0 +1,51 @@ +package org.demo.validation; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.RequestHandler; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import software.amazon.lambda.powertools.validation.Validation; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStreamReader; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * Request handler for Lambda function which demonstrates validation of request message. + */ +public class InboundValidation implements RequestHandler { + + @Validation(inboundSchema = "classpath:/schema.json") + public APIGatewayProxyResponseEvent handleRequest(APIGatewayProxyRequestEvent apiGatewayProxyRequestEvent, Context context) { + Map headers = new HashMap<>(); + + headers.put("Content-Type", "application/json"); + headers.put("X-Custom-Header", "application/json"); + + APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent() + .withHeaders(headers); + try { + final String pageContents = this.getPageContents("https://checkip.amazonaws.com"); + String output = String.format("{ \"message\": \"hello world\", \"location\": \"%s\" }", pageContents); + + return response + .withStatusCode(200) + .withBody(output); + } catch (IOException e) { + return response + .withBody("{}") + .withStatusCode(500); + } + } + + private String getPageContents(String address) throws IOException { + URL url = new URL(address); + try (BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"))) { + return br.lines().collect(Collectors.joining(System.lineSeparator())); + } + } +} diff --git a/examples/powertools-examples-validation/src/main/resources/schema.json b/examples/powertools-examples-validation/src/main/resources/schema.json new file mode 100644 index 000000000..f38272f2d --- /dev/null +++ b/examples/powertools-examples-validation/src/main/resources/schema.json @@ -0,0 +1,55 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema", + "$id": "http://example.com/product.json", + "type": "object", + "title": "Product schema", + "description": "JSON schema to validate Products", + "default": {}, + "examples": [ + { + "id": 43242, + "name": "FooBar XY", + "price": 258 + } + ], + "required": [ + "id", + "name", + "price" + ], + "properties": { + "id": { + "$id": "#/properties/id", + "type": "integer", + "title": "Id of the product", + "description": "Unique identifier of the product", + "default": 0, + "examples": [ + 43242 + ] + }, + "name": { + "$id": "#/properties/name", + "type": "string", + "title": "Name of the product", + "description": "Explicit name of the product", + "minLength": 5, + "default": "", + "examples": [ + "FooBar XY" + ] + }, + "price": { + "$id": "#/properties/price", + "type": "number", + "title": "Price of the product", + "description": "Positive price of the product", + "default": 0, + "exclusiveMinimum": 0, + "examples": [ + 258.99 + ] + } + }, + "additionalProperties": true +} \ No newline at end of file diff --git a/examples/powertools-examples-validation/src/test/java/org/demo/validation/InboundValidationTest.java b/examples/powertools-examples-validation/src/test/java/org/demo/validation/InboundValidationTest.java new file mode 100644 index 000000000..af47d3d87 --- /dev/null +++ b/examples/powertools-examples-validation/src/test/java/org/demo/validation/InboundValidationTest.java @@ -0,0 +1,51 @@ +package org.demo.validation; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent; +import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.mockito.Mock; +import org.mockito.MockitoAnnotations; +import software.amazon.lambda.powertools.validation.ValidationException; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +public class InboundValidationTest { + + @Mock + private Context context; + private InboundValidation inboundValidation; + + @BeforeEach + void setUp() { + MockitoAnnotations.openMocks(this); + inboundValidation = new InboundValidation(); + } + + @Test + public void shouldReturnOkStatusWhenInputIsValid() { + String body = "{\n" + + " \"id\": 43242,\n" + + " \"name\": \"FooBar XY\",\n" + + " \"price\": 258\n" + + " }"; + APIGatewayProxyRequestEvent request = new APIGatewayProxyRequestEvent().withBody(body); + + APIGatewayProxyResponseEvent response = inboundValidation.handleRequest(request, context); + + assertEquals(200, response.getStatusCode()); + } + + @Test + public void shouldThrowExceptionWhenRequestInInvalid() { + String bodyWithMissedId = "{\n" + + " \"name\": \"FooBar XY\",\n" + + " \"price\": 258\n" + + " }"; + APIGatewayProxyRequestEvent request = new APIGatewayProxyRequestEvent().withBody(bodyWithMissedId); + + assertThrows(ValidationException.class, () -> inboundValidation.handleRequest(request, context)); + } +} \ No newline at end of file diff --git a/examples/powertools-examples-validation/template.yaml b/examples/powertools-examples-validation/template.yaml new file mode 100644 index 000000000..f18a9fb00 --- /dev/null +++ b/examples/powertools-examples-validation/template.yaml @@ -0,0 +1,33 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + validation demo + +Globals: + Function: + Timeout: 20 + Runtime: java11 + MemorySize: 512 + Tracing: Active + + +Resources: + ValidationFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: . + Handler: org.demo.validation.InboundValidation::handleRequest + Events: + HelloWorld: + Type: Api + Properties: + Path: /hello + Method: post + +Outputs: + Api: + Description: "API Gateway endpoint URL for Prod stage for Validation function" + Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/" + Function: + Description: "Validation Lambda Function ARN" + Value: !GetAtt ValidationFunction.Arn \ No newline at end of file diff --git a/examples/spotbugs-exclude.xml b/examples/spotbugs-exclude.xml new file mode 100644 index 000000000..e33e65478 --- /dev/null +++ b/examples/spotbugs-exclude.xml @@ -0,0 +1,20 @@ + + + + + + + + + + + + \ No newline at end of file diff --git a/pom.xml b/pom.xml index 5ca158654..121b3459e 100644 --- a/pom.xml +++ b/pom.xml @@ -37,7 +37,8 @@ powertools-validation powertools-test-suite powertools-cloudformation - powertools-idempotency + powertools-idempotency + examples @@ -450,7 +451,7 @@ build-with-spotbugs - true + false diff --git a/spotbugs-exclude.xml b/spotbugs-exclude.xml index fe4194bfd..5a5e3bed8 100644 --- a/spotbugs-exclude.xml +++ b/spotbugs-exclude.xml @@ -152,4 +152,13 @@ + + + + + + + + + \ No newline at end of file