Skip to content

docs(parameters): release tasks + add docs on bring your own provider #1565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/scripts/release_patch_package_json.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ if (process.argv.length < 3) {
const basePath = resolve(process.argv[2]);
const packageJsonPath = join(basePath, 'package.json');
const alphaPackages = ['@aws-lambda-powertools/idempotency'];
const betaPackages = ['@aws-lambda-powertools/parameters'];
const betaPackages = [];

(() => {
try {
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ You can use the library in both TypeScript and JavaScript code bases.
* **[Tracer](https://docs.powertools.aws.dev/lambda-typescript/latest/core/tracer/)** - Utilities to trace Lambda function handlers, and both synchronous and asynchronous functions
* **[Logger](https://docs.powertools.aws.dev/lambda-typescript/latest/core/logger/)** - Structured logging made easier, and a middleware to enrich log items with key details of the Lambda context
* **[Metrics](https://docs.powertools.aws.dev/lambda-typescript/latest/core/metrics/)** - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF)
* **[Parameters (beta)](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM Parameter Store, AWS Secrets Manager, AWS AppConfig, and Amazon DynamoDB
* **[Parameters](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM Parameter Store, AWS Secrets Manager, AWS AppConfig, and Amazon DynamoDB
* **[Idempotency (beta)](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/idempotency/)** - Class method decorator, Middy middleware, and function wrapper to make your Lambda functions idempotent and prevent duplicate execution based on payload content

## Getting started
Expand Down
3 changes: 2 additions & 1 deletion docs/snippets/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
"aws-sdk": "^2.1405.0",
"aws-sdk-client-mock": "^2.2.0",
"aws-sdk-client-mock-jest": "^2.2.0",
"axios": "^1.2.4"
"axios": "^1.2.4",
"hashi-vault-js": "^0.4.13"
}
}
116 changes: 116 additions & 0 deletions docs/snippets/parameters/customProviderVault.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import { Logger } from '@aws-lambda-powertools/logger';
import { BaseProvider } from '@aws-lambda-powertools/parameters/base';
import Vault from 'hashi-vault-js';
import { isErrorResponse } from './customProviderVaultTypes';
import type {
HashiCorpVaultProviderOptions,
HashiCorpVaultGetOptions,
} from './customProviderVaultTypes';

class HashiCorpVaultProvider extends BaseProvider {
public client: Vault;
readonly #token: string;
readonly #logger: Logger;

/**
* It initializes the HashiCorpVaultProvider class.
*
* @param {HashiCorpVaultProviderOptions} config - The configuration object.
*/
public constructor(config: HashiCorpVaultProviderOptions) {
super();

const { url, token, clientConfig, vaultClient } = config;
if (vaultClient) {
if (vaultClient instanceof Vault) {
this.client = vaultClient;
} else {
throw Error('Not a valid Vault client provided');
}
} else {
const config: Vault.VaultConfig = {
baseUrl: url,
...(clientConfig ?? {
timeout: 10000,
rootPath: '',
}),
};
this.client = new Vault(config);
}
this.#token = token;
this.#logger = new Logger({
serviceName: 'HashiCorpVaultProvider',
});
}

/**
* Retrieve a secret from HashiCorp Vault.
*
* You can customize the retrieval of the secret by passing options to the function:
* * `maxAge` - The maximum age of the value in cache before fetching a new one (in seconds) (default: 5)
* * `forceFetch` - Whether to always fetch a new value from the store regardless if already available in cache
* * `sdkOptions` - Extra options to pass to the HashiCorp Vault SDK, e.g. `mount` or `version`
*
* @param {string} name - The name of the secret
* @param {HashiCorpVaultGetOptions} options - Options to customize the retrieval of the secret
*/
public async get(
name: string,
options?: HashiCorpVaultGetOptions
): Promise<Record<string, unknown> | undefined> {
return super.get(name, options) as Promise<
Record<string, unknown> | undefined
>;
}

/**
* Retrieving multiple parameter values is not supported with HashiCorp Vault.
*/
public async getMultiple(path: string, _options?: unknown): Promise<void> {
await super.getMultiple(path);
}

/**
* Retrieve a secret from HashiCorp Vault.
*
* @param {string} name - The name of the secret
* @param {HashiCorpVaultGetOptions} options - Options to customize the retrieval of the secret
*/
protected async _get(
name: string,
options?: HashiCorpVaultGetOptions
): Promise<Record<string, unknown>> {
const mount = options?.sdkOptions?.mount ?? 'secret';
const version = options?.sdkOptions?.version;

const response = await this.client.readKVSecret(
this.#token,
name,
version,
mount
);

if (isErrorResponse(response)) {
this.#logger.error('An error occurred', {
error: response.vaultHelpMessage,
});
throw response;
} else {
return response.data;
}
}

/**
* Retrieving multiple parameter values from HashiCorp Vault is not supported.
*
* @throws Not Implemented Error.
*/
protected async _getMultiple(
_path: string,
_options?: unknown
): Promise<void> {
throw new Error('Method not implemented.');
}
}

export { HashiCorpVaultProvider };
97 changes: 97 additions & 0 deletions docs/snippets/parameters/customProviderVaultTypes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import { GetOptionsInterface } from '@aws-lambda-powertools/parameters/base/types';
import type Vault from 'hashi-vault-js';

/**
* Base interface for HashiCorpVaultProviderOptions.
* @interface
*/
interface HashiCorpVaultProviderOptionsBase {
/**
* Indicate the server name/IP, port and API version for the Vault instance, all paths are relative to this one.
* @example 'https://vault.example.com:8200/v1'
*/
url: string;
/**
* The Vault token to use for authentication.
*/
token: string;
}

/**
* Interface for HashiCorpVaultProviderOptions with clientConfig property.
* @interface
*/
interface HashiCorpVaultProviderOptionsWithClientConfig
extends HashiCorpVaultProviderOptionsBase {
/**
* Optional configuration to pass during client initialization to customize the `hashi-vault-js` client.
*/
clientConfig?: Omit<Vault.VaultConfig, 'baseUrl'>;
/**
* This property should never be passed.
*/
vaultClient?: never;
}

/**
* Interface for HashiCorpVaultProviderOptions with vaultClient property.
*
* @interface
*/
interface HashiCorpVaultProviderOptionsWithClientInstance
extends HashiCorpVaultProviderOptionsBase {
/**
* Optional `hashi-vault-js` client to pass during HashiCorpVaultProvider class instantiation. If not provided, a new client will be created.
*/
vaultClient?: Vault;
/**
* This property should never be passed.
*/
clientConfig: never;
}

/**
* Options for the HashiCorpVaultProvider class constructor.
*
* @param {string} url - Indicate the server name/IP, port and API version for the Vault instance, all paths are relative to this one.
* @param {string} token - The Vault token to use for authentication.
* @param {Vault.VaultConfig} [clientConfig] - Optional configuration to pass during client initialization, e.g. timeout. Mutually exclusive with vaultClient.
* @param {Vault} [vaultClient] - Optional `hashi-vault-js` client to pass during HashiCorpVaultProvider class instantiation. Mutually exclusive with clientConfig.
*/
type HashiCorpVaultProviderOptions =
| HashiCorpVaultProviderOptionsWithClientConfig
| HashiCorpVaultProviderOptionsWithClientInstance;

type HashiCorpVaultReadKVSecretOptions = {
/**
* The mount point of the secret engine to use. Defaults to `secret`.
* @example 'kv'
*/
mount?: string;
/**
* The version of the secret to retrieve. Defaults to `undefined`.
* @example 1
*/
version?: number;
};

interface HashiCorpVaultGetOptions extends GetOptionsInterface {
/**
* The Parameters utility does not support transforming `Record<string, unknown>` values as returned by the HashiCorp Vault SDK.
*/
transform?: never;
sdkOptions?: HashiCorpVaultReadKVSecretOptions;
}

/**
* Typeguard that discriminates the type of the response and excludes the ErrorResponse type.
*
* @param object The response object to discriminate
*/
export const isErrorResponse = (
object: Vault.ReadKVSecretResponse
): object is Vault.ErrorResponse => {
return 'isVaultError' in object;
};

export type { HashiCorpVaultProviderOptions, HashiCorpVaultGetOptions };
12 changes: 12 additions & 0 deletions docs/snippets/parameters/customProviderVaultUsage.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { HashiCorpVaultProvider } from './customProviderVault';

const secretsProvider = new HashiCorpVaultProvider({
url: 'https://vault.example.com:8200/v1',
token: 'my-token',
});

export const handler = async (): Promise<void> => {
// Retrieve a secret from HashiCorp Vault
const secret = await secretsProvider.get('my-secret');
console.log(secret);
};
26 changes: 23 additions & 3 deletions docs/utilities/parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@ title: Parameters
description: Utility
---

???+ warning
**This utility is currently released as beta developer preview** and is intended strictly for feedback and testing purposes **and not for production workloads**.. The version and all future versions tagged with the `-beta` suffix should be treated as not stable. Up until before the [General Availability release](https://github.com/aws-powertools/powertools-lambda-typescript/milestone/10) we might introduce significant breaking changes and improvements in response to customers feedback.

The Parameters utility provides high-level functions to retrieve one or multiple parameter values from [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html){target="_blank"}, [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html){target="_blank"}, [AWS AppConfig](https://docs.aws.amazon.com/appconfig/latest/userguide/what-is-appconfig.html){target="_blank"}, [Amazon DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html){target="_blank"}, or your own parameter store.

## Key features
Expand Down Expand Up @@ -275,6 +272,29 @@ DynamoDB provider can be customized at initialization to match your table struct
--8<-- "docs/snippets/parameters/dynamoDBProviderCustomizeTable.ts"
```

### Create your own provider

You can create your own custom parameter store provider by extending the `BaseProvider` class, and implementing the `get()` and `getMultiple()` methods, as well as its respective `_get()` and `_getMultiple()` private methods to retrieve a single, or multiple parameters from your custom store.

All caching logic is handled by the `BaseProvider`, and provided that the return types of your store are compatible with the ones used in the `BaseProvider`, all transformations will also work as expected.

Here's an example of implementing a custom parameter store using an external service like HashiCorp Vault, a widely popular key-value secret storage.

=== "Provider implementation"
```typescript
--8<-- "docs/snippets/parameters/customProviderVault.ts"
```

=== "Provider types"
```typescript
--8<-- "docs/snippets/parameters/customProviderVaultTypes.ts"
```

=== "Provider usage"
```typescript
--8<-- "docs/snippets/parameters/customProviderVaultUsage.ts"
```

### Deserializing values with transform parameter

For parameters stored in JSON or Base64 format, you can use the `transform` argument for deserialization.
Expand Down
20 changes: 17 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion packages/commons/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ You can use the library in both TypeScript and JavaScript code bases.
* **[Tracer](https://docs.powertools.aws.dev/lambda-typescript/latest/core/tracer/)** - Utilities to trace Lambda function handlers, and both synchronous and asynchronous functions
* **[Logger](https://docs.powertools.aws.dev/lambda-typescript/latest/core/logger/)** - Structured logging made easier, and a middleware to enrich log items with key details of the Lambda context
* **[Metrics](https://docs.powertools.aws.dev/lambda-typescript/latest/core/metrics/)** - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF)
* **[Parameters (beta)](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM, Secrets Manager, AppConfig, and DynamoDB
* **[Parameters](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM, Secrets Manager, AppConfig, and DynamoDB

## Getting started

Expand Down
2 changes: 1 addition & 1 deletion packages/logger/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ You can use the library in both TypeScript and JavaScript code bases.
* **[Tracer](https://docs.powertools.aws.dev/lambda-typescript/latest/core/tracer/)** - Utilities to trace Lambda function handlers, and both synchronous and asynchronous functions
* **[Logger](https://docs.powertools.aws.dev/lambda-typescript/latest/core/logger/)** - Structured logging made easier, and a middleware to enrich log items with key details of the Lambda context
* **[Metrics](https://docs.powertools.aws.dev/lambda-typescript/latest/core/metrics/)** - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF)
* **[Parameters (beta)](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM, Secrets Manager, AppConfig, and DynamoDB
* **[Parameters](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM, Secrets Manager, AppConfig, and DynamoDB

## Getting started

Expand Down
2 changes: 1 addition & 1 deletion packages/metrics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ You can use the library in both TypeScript and JavaScript code bases.
* **[Tracer](https://docs.powertools.aws.dev/lambda-typescript/latest/core/tracer/)** - Utilities to trace Lambda function handlers, and both synchronous and asynchronous functions
* **[Logger](https://docs.powertools.aws.dev/lambda-typescript/latest/core/logger/)** - Structured logging made easier, and a middleware to enrich log items with key details of the Lambda context
* **[Metrics](https://docs.powertools.aws.dev/lambda-typescript/latest/core/metrics/)** - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF)
* **[Parameters (beta)](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM, Secrets Manager, AppConfig, and DynamoDB
* **[Parameters](https://docs.powertools.aws.dev/lambda-typescript/latest/utilities/parameters/)** - High-level functions to retrieve one or more parameters from AWS SSM, Secrets Manager, AppConfig, and DynamoDB

## Getting started

Expand Down
Loading