-
Notifications
You must be signed in to change notification settings - Fork 1.8k
feat(NODE-4696): add FaaS env information to client metadata #3626
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
durran
merged 33 commits into
main
from
NODE-4696-faas-metadata-handshake-5x-spec-updates
Apr 12, 2023
Merged
Changes from 16 commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
c623356
FAAS metadata commit
baileympearson 1b86b2d
fix lint
baileympearson ed44549
misc fixes
baileympearson da62ec4
remove truncated client metadata
baileympearson a98a423
chore: everything but truncation
baileympearson ef58d3f
chore: misc updates
baileympearson 51ae18b
misc changes
baileympearson 6853019
use `Int32`
baileympearson c81552a
Merge branch 'main' into NODE-4696-faas-metadata-handshake-5x
nbbeeken cc1bf22
refactor(NODE-4696): use an additive approach toward metadata limit
nbbeeken 21dc9e3
test: env omission
nbbeeken b8108a7
fix
nbbeeken 4be609f
test: make sure there's a failure with too large a document
nbbeeken 70da1d6
fix: tests
nbbeeken 871d208
fix: lb mode error
nbbeeken 6bccf1e
test: os type omission, env omission, merge faas logic
nbbeeken 01b9140
inheritdoc unsupported
nbbeeken 757cc0b
perf: only serialize new element for fits check
nbbeeken 8f28668
fix: vercel having aws env vars
nbbeeken beba14a
fix: gcp bool
nbbeeken d82591d
style: early return = no else
nbbeeken 85a4551
chore: add comment about vercel
nbbeeken 52000e2
Merge branch 'main' into NODE-4696-faas-metadata-handshake-5x-spec-up…
nbbeeken d20c333
fix: update error message
nbbeeken 69c01d7
fix: rm vercel_url
nbbeeken 5cc2efb
titles
nbbeeken 6c82746
undo cmap change
nbbeeken 83f5e6c
titles2
nbbeeken 6c4d67d
lint
nbbeeken 2c9b4c4
moved throwing tests
nbbeeken f7b1ea3
fix before/after inside loop
nbbeeken 92c0a18
Merge branch 'main' into NODE-4696-faas-metadata-handshake-5x-spec-up…
nbbeeken ec0372f
fix: test locations
nbbeeken File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,227 @@ | ||
import * as os from 'os'; | ||
import * as process from 'process'; | ||
|
||
import { BSON, Int32 } from '../../bson'; | ||
import { MongoInvalidArgumentError } from '../../error'; | ||
import type { MongoOptions } from '../../mongo_client'; | ||
|
||
// eslint-disable-next-line @typescript-eslint/no-var-requires | ||
const NODE_DRIVER_VERSION = require('../../../package.json').version; | ||
|
||
/** | ||
* @public | ||
* @see https://github.com/mongodb/specifications/blob/master/source/mongodb-handshake/handshake.rst#hello-command | ||
*/ | ||
export interface ClientMetadata { | ||
driver: { | ||
name: string; | ||
version: string; | ||
}; | ||
os: { | ||
type: string; | ||
name?: NodeJS.Platform; | ||
architecture?: string; | ||
version?: string; | ||
}; | ||
platform: string; | ||
application?: { | ||
name: string; | ||
}; | ||
/** FaaS environment information */ | ||
env?: { | ||
name: 'aws.lambda' | 'gcp.func' | 'azure.func' | 'vercel'; | ||
timeout_sec?: Int32; | ||
memory_mb?: Int32; | ||
region?: string; | ||
url?: string; | ||
}; | ||
} | ||
|
||
/** @public */ | ||
export interface ClientMetadataOptions { | ||
driverInfo?: { | ||
name?: string; | ||
version?: string; | ||
platform?: string; | ||
}; | ||
appName?: string; | ||
} | ||
|
||
/** @internal */ | ||
export class LimitedSizeDocument { | ||
private document = new Map(); | ||
constructor(private maxSize: number) {} | ||
|
||
private getBsonByteLength() { | ||
return BSON.serialize(this.document).byteLength; | ||
nbbeeken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
/** Only adds key/value if the bsonByteLength is less than MAX_SIZE */ | ||
public ifFitsSets(key: string, value: Record<string, any> | string): boolean { | ||
durran marked this conversation as resolved.
Show resolved
Hide resolved
|
||
this.document.set(key, value); | ||
|
||
if (this.getBsonByteLength() > this.maxSize) { | ||
this.document.delete(key); | ||
return false; | ||
} | ||
|
||
return true; | ||
} | ||
|
||
toObject(): ClientMetadata { | ||
return BSON.deserialize(BSON.serialize(this.document), { | ||
promoteLongs: false, | ||
promoteBuffers: false, | ||
promoteValues: false, | ||
useBigInt64: false | ||
}) as ClientMetadata; | ||
} | ||
} | ||
|
||
type MakeClientMetadataOptions = Pick<MongoOptions, 'appName' | 'driverInfo'>; | ||
/** | ||
* From the specs: | ||
* Implementors SHOULD cumulatively update fields in the following order until the document is under the size limit: | ||
* 1. Omit fields from `env` except `env.name`. | ||
* 2. Omit fields from `os` except `os.type`. | ||
* 3. Omit the `env` document entirely. | ||
* 4. Truncate `platform`. -- special we do not truncate this field | ||
*/ | ||
export function makeClientMetadata(options: MakeClientMetadataOptions): ClientMetadata { | ||
const metadataDocument = new LimitedSizeDocument(512); | ||
|
||
const { appName = '' } = options; | ||
// Add app name first, it must be sent | ||
if (appName.length > 0) { | ||
const name = | ||
Buffer.byteLength(appName, 'utf8') <= 128 | ||
? options.appName | ||
: Buffer.from(appName, 'utf8').subarray(0, 128).toString('utf8'); | ||
metadataDocument.ifFitsSets('application', { name }); | ||
} | ||
|
||
const { name = '', version = '', platform = '' } = options.driverInfo; | ||
|
||
const driverInfo = { | ||
name: name.length > 0 ? `nodejs|${name}` : 'nodejs', | ||
version: version.length > 0 ? `${NODE_DRIVER_VERSION}|${version}` : NODE_DRIVER_VERSION | ||
}; | ||
|
||
if (!metadataDocument.ifFitsSets('driver', driverInfo)) { | ||
throw new MongoInvalidArgumentError('driverInfo name and version exceed limit of 512 bytes'); | ||
} | ||
|
||
const platformInfo = | ||
platform.length > 0 | ||
? `Node.js ${process.version}, ${os.endianness()}|${platform}` | ||
: `Node.js ${process.version}, ${os.endianness()}`; | ||
|
||
if (!metadataDocument.ifFitsSets('platform', platformInfo)) { | ||
throw new MongoInvalidArgumentError('driverInfo platform exceeds the limit of 512 bytes'); | ||
} | ||
|
||
// Note: order matters, os.type is last so it will be removed last if we're at maxSize | ||
const osInfo = new Map() | ||
.set('name', process.platform) | ||
.set('architecture', process.arch) | ||
.set('version', os.release()) | ||
.set('type', os.type()); | ||
|
||
if (!metadataDocument.ifFitsSets('os', osInfo)) { | ||
for (const key of osInfo.keys()) { | ||
osInfo.delete(key); | ||
if (osInfo.size === 0) break; | ||
if (metadataDocument.ifFitsSets('os', osInfo)) break; | ||
} | ||
nbbeeken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
const faasEnv = getFAASEnv(); | ||
if (faasEnv != null) { | ||
if (!metadataDocument.ifFitsSets('env', faasEnv)) { | ||
for (const key of faasEnv.keys()) { | ||
faasEnv.delete(key); | ||
if (faasEnv.size === 0) break; | ||
if (metadataDocument.ifFitsSets('env', faasEnv)) break; | ||
} | ||
} | ||
} | ||
|
||
return metadataDocument.toObject(); | ||
} | ||
|
||
/** | ||
* Collects FaaS metadata. | ||
* - `name` MUST be the last key in the Map returned. | ||
*/ | ||
export function getFAASEnv(): Map<string, string | Int32> | null { | ||
const { | ||
AWS_EXECUTION_ENV = '', | ||
AWS_LAMBDA_RUNTIME_API = '', | ||
FUNCTIONS_WORKER_RUNTIME = '', | ||
K_SERVICE = '', | ||
FUNCTION_NAME = '', | ||
VERCEL = '', | ||
AWS_LAMBDA_FUNCTION_MEMORY_SIZE = '', | ||
AWS_REGION = '', | ||
FUNCTION_MEMORY_MB = '', | ||
FUNCTION_REGION = '', | ||
FUNCTION_TIMEOUT_SEC = '', | ||
VERCEL_URL = '', | ||
VERCEL_REGION = '' | ||
} = process.env; | ||
nbbeeken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
const isAWSFaaS = AWS_EXECUTION_ENV.length > 0 || AWS_LAMBDA_RUNTIME_API.length > 0; | ||
const isAzureFaaS = FUNCTIONS_WORKER_RUNTIME.length > 0; | ||
const isGCPFaaS = K_SERVICE.length > 0 || FUNCTION_NAME.length > 0; | ||
const isVercelFaaS = VERCEL.length > 0; | ||
|
||
// Note: order matters, name must always be the last key | ||
nbbeeken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
const faasEnv = new Map(); | ||
|
||
if (isAWSFaaS && !(isAzureFaaS || isGCPFaaS || isVercelFaaS)) { | ||
durran marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if (AWS_REGION.length > 0) { | ||
faasEnv.set('region', AWS_REGION); | ||
} | ||
|
||
if ( | ||
AWS_LAMBDA_FUNCTION_MEMORY_SIZE.length > 0 && | ||
Number.isInteger(+AWS_LAMBDA_FUNCTION_MEMORY_SIZE) | ||
) { | ||
faasEnv.set('memory_mb', new Int32(AWS_LAMBDA_FUNCTION_MEMORY_SIZE)); | ||
} | ||
|
||
faasEnv.set('name', 'aws.lambda'); | ||
return faasEnv; | ||
} else if (isAzureFaaS && !(isAWSFaaS || isGCPFaaS || isVercelFaaS)) { | ||
faasEnv.set('name', 'azure.func'); | ||
return faasEnv; | ||
} else if (isGCPFaaS && !(isAWSFaaS || isAzureFaaS || isVercelFaaS)) { | ||
if (FUNCTION_REGION.length > 0) { | ||
faasEnv.set('region', FUNCTION_REGION); | ||
} | ||
|
||
if (FUNCTION_MEMORY_MB.length > 0 && Number.isInteger(+FUNCTION_MEMORY_MB)) { | ||
faasEnv.set('memory_mb', new Int32(FUNCTION_MEMORY_MB)); | ||
} | ||
|
||
if (FUNCTION_TIMEOUT_SEC.length > 0 && Number.isInteger(+FUNCTION_TIMEOUT_SEC)) { | ||
faasEnv.set('timeout_sec', new Int32(FUNCTION_TIMEOUT_SEC)); | ||
} | ||
|
||
faasEnv.set('name', 'gcp.func'); | ||
return faasEnv; | ||
} else if (isVercelFaaS && !(isAWSFaaS || isAzureFaaS || isGCPFaaS)) { | ||
durran marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if (VERCEL_URL.length > 0) { | ||
faasEnv.set('url', VERCEL_URL); | ||
} | ||
|
||
if (VERCEL_REGION.length > 0) { | ||
faasEnv.set('region', VERCEL_REGION); | ||
} | ||
|
||
faasEnv.set('name', 'vercel'); | ||
return faasEnv; | ||
} else { | ||
return null; | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.