Skip to content

Commit 67549f6

Browse files
dreamorosisvozza
andauthored
feat(parser): support Standard Schema and upgrade to Zod v4 (#4164)
Co-authored-by: Stefano Vozza <[email protected]>
1 parent 191c846 commit 67549f6

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+1473
-1194
lines changed

docs/features/parser.md

Lines changed: 47 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -1,63 +1,57 @@
11
---
2-
title: Parser (Zod)
2+
title: Parser (Standard Schema)
33
descrition: Utility
44
---
55

66
<!-- markdownlint-disable MD043 --->
77

8-
This utility provides data validation and parsing using [Zod](https://zod.dev){target="_blank"}, a TypeScript-first schema declaration and validation library.
8+
This utility provides data validation and parsing for [Standard Schema](https://github.com/standard-schema/standard-schema){target="_blank"}, together with a collection of built-in [Zod](https://zod.dev){target="_blank"} schemas and envelopes to parse and unwrap popular AWS event sources payloads.
99

1010
## Key features
1111

12-
* Define data schema as Zod schema, then parse, validate and extract only what you want
13-
* Built-in envelopes to unwrap and validate popular AWS event sources payloads
14-
* Extend and customize envelopes to fit your needs
15-
* Safe parsing option to avoid throwing errors and custom error handling
16-
* Available for Middy.js middleware and TypeScript method decorators
12+
* Accept a [Standard Schema](https://github.com/standard-schema/standard-schema) and parse incoming payloads
13+
* Built-in Zod schemas and envelopes to unwrap and validate popular AWS event sources payloads
14+
* Extend and customize built-in Zod schemas to fit your needs
15+
* Safe parsing option to avoid throwing errors and allow custom error handling
16+
* Available as Middy.js middleware and TypeScript class method decorator
1717

1818
## Getting started
1919

2020
```bash
21-
npm install @aws-lambda-powertools/parser zod@~3
21+
npm install @aws-lambda-powertools/parser zod
2222
```
2323

24-
!!! warning "Zod version"
25-
The package is compatible with Zod v3 only.<br/>
26-
We're considering Zod v4 support and we'd love to hear your feedback. Please [leave a comment here](https://github.com/aws-powertools/powertools-lambda-typescript/issues/3951) to let us know your thoughts.
27-
28-
## Define schema
29-
30-
You can define your schema using Zod:
31-
32-
```typescript title="schema.ts"
33-
--8<-- "examples/snippets/parser/schema.ts"
34-
```
35-
36-
This is a schema for `Order` object using Zod.
37-
You can create complex schemas by using nested objects, arrays, unions, and other types, see [Zod documentation](https://zod.dev) for more details.
38-
3924
## Parse events
4025

4126
You can parse inbound events using `parser` decorator, Middy.js middleware, or [manually](#manual-parsing) using built-in envelopes and schemas.
42-
Both are also able to parse either an object or JSON string as an input.
4327

44-
???+ warning
45-
The decorator and middleware will replace the event object with the parsed schema if successful.
46-
Be cautious when using multiple decorators that expect event to have a specific structure, the order of evaluation for decorators is from bottom to top.
28+
When using the decorator or middleware, you can specify a schema to parse the event, this can be a [built-in Zod schema](#built-in-schemas) or a custom schema you defined. Custom schemas can be defined using Zod or any other [Standard Schema compatible library](https://standardschema.dev/#what-schema-libraries-implement-the-spec){target="_blank"}.
4729

48-
=== "Middy middleware"
30+
=== "Middy.js middleware with Zod schema"
4931
```typescript hl_lines="22"
5032
--8<-- "examples/snippets/parser/middy.ts"
5133
```
5234

35+
=== "Middy.js middleware with Valibot schema"
36+
```typescript hl_lines="30"
37+
--8<-- "examples/snippets/parser/middyValibot.ts"
38+
```
39+
5340
=== "Decorator"
41+
!!! warning
42+
The decorator and middleware will replace the event object with the parsed schema if successful.
43+
Be cautious when using multiple decorators that expect an event to have a specific structure, the order of evaluation for decorators is from the inner to the outermost decorator.
44+
5445
```typescript hl_lines="25"
5546
--8<-- "examples/snippets/parser/decorator.ts"
5647
```
5748

5849
## Built-in schemas
5950

60-
**Parser** comes with the following built-in schemas:
51+
**Parser** comes with the following built-in Zod schemas:
52+
53+
!!! note "Looking for other libraries?"
54+
The built-in schemas are defined using Zod, if you would like us to support other libraries like [valibot](https://valibot.dev){target="_blank"} please [open an issue](https://github.com/aws-powertools/powertools-lambda-typescript/issues/new?template=feature_request.yml){target="_blank"} and we will consider it based on the community's feedback.
6155

6256
| Model name | Description |
6357
| -------------------------------------------- | ------------------------------------------------------------------------------------- |
@@ -198,7 +192,7 @@ This can become difficult quite quickly. Parser simplifies the development throu
198192
Envelopes can be used via envelope parameter available in middy and decorator.
199193
Here's an example of parsing a custom schema in an event coming from EventBridge, where all you want is what's inside the detail key.
200194

201-
=== "Middy middleware"
195+
=== "Middy.js middleware"
202196
```typescript hl_lines="23"
203197
--8<-- "examples/snippets/parser/envelopeMiddy.ts"
204198
```
@@ -221,24 +215,27 @@ We have also complex envelopes that parse the payload from a string, decode base
221215

222216
### Built-in envelopes
223217

224-
Parser comes with the following built-in envelopes:
218+
Parser comes with the following built-in Zod envelopes:
219+
220+
!!! note "Looking for other libraries?"
221+
The built-in schemas are defined using Zod, if you would like us to support other libraries like [valibot](https://valibot.dev){target="_blank"} please [open an issue](https://github.com/aws-powertools/powertools-lambda-typescript/issues/new?template=feature_request.yml){target="_blank"} and we will consider it based on the community's feedback.
225222

226223
| Envelope name | Behaviour |
227224
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
228-
| **apiGatewayEnvelope** | 1. Parses data using `APIGatewayProxyEventSchema`. <br/> 2. Parses `body` key using your schema and returns it. |
229-
| **apiGatewayV2Envelope** | 1. Parses data using `APIGatewayProxyEventV2Schema`. <br/> 2. Parses `body` key using your schema and returns it. |
230-
| **cloudWatchEnvelope** | 1. Parses data using `CloudwatchLogsSchema` which will base64 decode and decompress it. <br/> 2. Parses records in `message` key using your schema and return them in a list. |
231-
| **dynamoDBStreamEnvelope** | 1. Parses data using `DynamoDBStreamSchema`. <br/> 2. Parses records in `NewImage` and `OldImage` keys using your schema. <br/> 3. Returns a list with a dictionary containing `NewImage` and `OldImage` keys |
232-
| **eventBridgeEnvelope** | 1. Parses data using `EventBridgeSchema`. <br/> 2. Parses `detail` key using your schema and returns it. |
233-
| **kafkaEnvelope** | 1. Parses data using `KafkaRecordSchema`. <br/> 2. Parses `value` key using your schema and returns it. |
234-
| **kinesisEnvelope** | 1. Parses data using `KinesisDataStreamSchema` which will base64 decode it. <br/> 2. Parses records in `Records` key using your schema and returns them in a list. |
235-
| **kinesisFirehoseEnvelope** | 1. Parses data using `KinesisFirehoseSchema` which will base64 decode it. <br/> 2. Parses records in `Records` key using your schema and returns them in a list. |
236-
| **lambdaFunctionUrlEnvelope** | 1. Parses data using `LambdaFunctionUrlSchema`. <br/> 2. Parses `body` key using your schema and returns it. |
237-
| **snsEnvelope** | 1. Parses data using `SnsSchema`. <br/> 2. Parses records in `body` key using your schema and return them in a list. |
238-
| **snsSqsEnvelope** | 1. Parses data using `SqsSchema`. <br/> 2. Parses SNS records in `body` key using `SnsNotificationSchema`. <br/> 3. Parses data in `Message` key using your schema and return them in a list. |
239-
| **sqsEnvelope** | 1. Parses data using `SqsSchema`. <br/> 2. Parses records in `body` key using your schema and return them in a list. |
240-
| **vpcLatticeEnvelope** | 1. Parses data using `VpcLatticeSchema`. <br/> 2. Parses `value` key using your schema and returns it. |
241-
| **vpcLatticeV2Envelope** | 1. Parses data using `VpcLatticeSchema`. <br/> 2. Parses `value` key using your schema and returns it. |
225+
| **ApiGatewayEnvelope** | 1. Parses data using `APIGatewayProxyEventSchema`. <br/> 2. Parses `body` key using your schema and returns it. |
226+
| **ApiGatewayV2Envelope** | 1. Parses data using `APIGatewayProxyEventV2Schema`. <br/> 2. Parses `body` key using your schema and returns it. |
227+
| **CloudWatchEnvelope** | 1. Parses data using `CloudwatchLogsSchema` which will base64 decode and decompress it. <br/> 2. Parses records in `message` key using your schema and return them in a list. |
228+
| **DynamoDBStreamEnvelope** | 1. Parses data using `DynamoDBStreamSchema`. <br/> 2. Parses records in `NewImage` and `OldImage` keys using your schema. <br/> 3. Returns a list with a dictionary containing `NewImage` and `OldImage` keys |
229+
| **EventBridgeEnvelope** | 1. Parses data using `EventBridgeSchema`. <br/> 2. Parses `detail` key using your schema and returns it. |
230+
| **KafkaEnvelope** | 1. Parses data using `KafkaRecordSchema`. <br/> 2. Parses `value` key using your schema and returns it. |
231+
| **KinesisEnvelope** | 1. Parses data using `KinesisDataStreamSchema` which will base64 decode it. <br/> 2. Parses records in `Records` key using your schema and returns them in a list. |
232+
| **KinesisFirehoseEnvelope** | 1. Parses data using `KinesisFirehoseSchema` which will base64 decode it. <br/> 2. Parses records in `Records` key using your schema and returns them in a list. |
233+
| **LambdaFunctionUrlEnvelope** | 1. Parses data using `LambdaFunctionUrlSchema`. <br/> 2. Parses `body` key using your schema and returns it. |
234+
| **SnsEnvelope** | 1. Parses data using `SnsSchema`. <br/> 2. Parses records in `body` key using your schema and return them in a list. |
235+
| **SnsSqsEnvelope** | 1. Parses data using `SqsSchema`. <br/> 2. Parses SNS records in `body` key using `SnsNotificationSchema`. <br/> 3. Parses data in `Message` key using your schema and return them in a list. |
236+
| **SnsEnvelope** | 1. Parses data using `SqsSchema`. <br/> 2. Parses records in `body` key using your schema and return them in a list. |
237+
| **VpcLatticeEnvelope** | 1. Parses data using `VpcLatticeSchema`. <br/> 2. Parses `value` key using your schema and returns it. |
238+
| **VpcLatticeV2Envelope** | 1. Parses data using `VpcLatticeSchema`. <br/> 2. Parses `value` key using your schema and returns it. |
242239

243240
## Safe parsing
244241

@@ -248,7 +245,7 @@ The handler `event` object will be replaced with `ParsedResult<Input?, Oputput?>
248245
The `ParsedResult` object will have `success`, `data`, or `error` and `originalEvent` fields, depending on the outcome.
249246
If the parsing is successful, the `data` field will contain the parsed event, otherwise you can access the `error` field and the `originalEvent` to handle the error and recover the original event.
250247

251-
=== "Middy middleware"
248+
=== "Middy.js middleware"
252249
```typescript hl_lines="23 28 32-33"
253250
--8<-- "examples/snippets/parser/safeParseMiddy.ts"
254251
```
@@ -320,10 +317,11 @@ Use `z.infer` to extract the type of the schema, so you can use types during dev
320317

321318
### Compatibility with `@types/aws-lambda`
322319

323-
The package `@types/aws-lambda` is a popular project that contains type definitions for many AWS service event invocations.
324-
Powertools parser utility also bring AWS Lambda event types based on the built-in schema definitions.
320+
The package `@types/aws-lambda` is a popular project that contains type definitions for many AWS service event invocations, support for these types is provided on a best effort basis.
321+
322+
We recommend using the types provided by the Parser utility under `@aws-powertools/parser/types` when using the built-in schemas and envelopes, as they are inferred directly from the Zod schemas and are more accurate.
325323

326-
We recommend to use the types provided by the parser utility. If you encounter any issues or have any feedback, please [submit an issue](https://github.com/aws-powertools/powertools-lambda-typescript/issues/new/choose).
324+
If you encounter any type compatibility issues with `@types/aws-lambda`, please [submit an issue](https://github.com/aws-powertools/powertools-lambda-typescript/issues/new/choose).
327325

328326
## Testing your code
329327

examples/snippets/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@
4343
"@valkey/valkey-glide": "^2.0.1",
4444
"aws-sdk": "^2.1692.0",
4545
"aws-sdk-client-mock": "^4.1.0",
46-
"zod": "^3.25.76"
46+
"zod": "^4.0.5"
4747
},
4848
"dependencies": {
4949
"arktype": "^2.1.20",

examples/snippets/parser/extendAlbSchema.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ const customSchema = z.object({
88
});
99

1010
const extendedSchema = AlbSchema.extend({
11-
body: JSONStringified(customSchema),
11+
body: JSONStringified(customSchema), // (1)!
1212
});
1313

1414
type _ExtendedAlbEvent = z.infer<typeof extendedSchema>;

examples/snippets/parser/extendSqsSchema.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ const customSchema = z.object({
1313
const extendedSchema = SqsSchema.extend({
1414
Records: z.array(
1515
SqsRecordSchema.extend({
16-
body: JSONStringified(customSchema), // (1)!
16+
body: JSONStringified(customSchema),
1717
})
1818
),
1919
});

examples/snippets/parser/middy.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ const orderSchema = z.object({
1111
items: z.array(
1212
z.object({
1313
id: z.number().positive(),
14-
quantity: z.number(),
14+
quantity: z.number().positive(),
1515
description: z.string(),
1616
})
1717
),
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
import { Logger } from '@aws-lambda-powertools/logger';
2+
import { parser } from '@aws-lambda-powertools/parser/middleware';
3+
import middy from '@middy/core';
4+
import {
5+
array,
6+
number,
7+
object,
8+
optional,
9+
pipe,
10+
string,
11+
toMinValue,
12+
} from 'valibot';
13+
14+
const logger = new Logger();
15+
16+
const orderSchema = object({
17+
id: pipe(number(), toMinValue(0)),
18+
description: string(),
19+
items: array(
20+
object({
21+
id: pipe(number(), toMinValue(0)),
22+
quantity: pipe(number(), toMinValue(1)),
23+
description: string(),
24+
})
25+
),
26+
optionalField: optional(string()),
27+
});
28+
29+
export const handler = middy()
30+
.use(parser({ schema: orderSchema }))
31+
.handler(async (event): Promise<void> => {
32+
for (const item of event.items) {
33+
// item is parsed as OrderItem
34+
logger.info('Processing item', { item });
35+
}
36+
});

package-lock.json

Lines changed: 26 additions & 15 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/kafka/package.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@
5353
"peerDependencies": {
5454
"arktype": ">=2.0.0",
5555
"valibot": ">=1.0.0",
56-
"zod": ">=3.24.0"
56+
"zod": "^3.25.0 || ^4.0.0"
5757
},
5858
"peerDependenciesMeta": {
5959
"zod": {
@@ -117,6 +117,6 @@
117117
"devDependencies": {
118118
"avro-js": "^1.12.0",
119119
"protobufjs": "^7.5.3",
120-
"zod": "^3.25.76"
120+
"zod": "^4.0.5"
121121
}
122122
}

0 commit comments

Comments
 (0)