ES Module support

[edeesis]

Expected Behavior

According to this issue, Lambda has updated the latest version of the amazon/aws-lambda-nodejs image to support the top level await by adding the following code:

const loaded =
    (pjHasModule && _tryAwaitImport(lambdaStylePath, ".js")) ||
    _tryAwaitImport(lambdaStylePath, ".mjs") ||
    _tryRequireFile(lambdaStylePath, ".cjs");

Since this repo now has a vendored version of UserFunction.js, this code should be updated to work appropriately.

Actual Behavior

An error is raised on start up when attempting to load a ES Module file. See stacktrace below.

Steps to Reproduce the Problem

1. Use a ES Module file as your lambda handler (.mjs file should work)
2. Declare that as the DD_LAMBDA_HANDLER environment variable

Specifications

• Datadog Lambda Layer version: 4.65.0+
  • Since 4.64.0 has no vendored version
• Node version: 14.x

Stacktrace

{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module 'lambda'\nRequire stack:\n- /opt/nodejs/node_modules/datadog-lambda-js/runtime/user-function.js\n- /opt/nodejs/node_modules/datadog-lambda-js/runtime/index.js\n- /opt/nodejs/node_modules/datadog-lambda-js/handler.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
  "stack": [
      "Runtime.ImportModuleError: Error: Cannot find module 'lambda'",
      "Require stack:",
      "- /opt/nodejs/node_modules/datadog-lambda-js/runtime/user-function.js",
      "- /opt/nodejs/node_modules/datadog-lambda-js/runtime/index.js",
      "- /opt/nodejs/node_modules/datadog-lambda-js/handler.js",
      "- /var/runtime/UserFunction.js",
      "- /var/runtime/index.js",
      "    at ImportModuleError.ExtendedError [as constructor] (/opt/nodejs/node_modules/datadog-lambda-js/runtime/errors.js:113:28)",
      "    at new ImportModuleError (/opt/nodejs/node_modules/datadog-lambda-js/runtime/errors.js:123:42)",
      "    at _loadUserApp (/opt/nodejs/node_modules/datadog-lambda-js/runtime/user-function.js:113:19)",
      "    at load (/opt/nodejs/node_modules/datadog-lambda-js/runtime/user-function.js:148:19)",
      "    at Object.<anonymous> (/opt/nodejs/node_modules/datadog-lambda-js/handler.js:65:59)",
      "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
      "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
      "    at Module.load (internal/modules/cjs/loader.js:950:32)",
      "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
      "    at Module.require (internal/modules/cjs/loader.js:974:19)"
  ]
}

[DarcyRaynerDD]

Hi @edeesis. Thanks for the report. We've scheduled work to support this with our js client. Unfortunately it's not quite as simple as updating the UserFunction.js, since the new code depends on a native library called deasync to support top level await. This means we must publish separate layers for ARM64 and x86 architectures, and update some of our other tooling as well. We will update this issue with our progress.

[andcea]

I enabled ESM on my existing (and working) NodejsFunction using the new config bundling: { format: lambdaNodeJs.OutputFormat.ESM } and I'm getting the following error:

{
    "errorType": "Runtime.HandlerNotFound",
    "errorMessage": "index.post is undefined or not exported",
    "stack": [
        "Runtime.HandlerNotFound: index.post is undefined or not exported",
        "    at HandlerNotFound.ExtendedError [as constructor] (/opt/nodejs/node_modules/datadog-lambda-js/runtime/errors.js:113:28)",
        "    at new HandlerNotFound (/opt/nodejs/node_modules/datadog-lambda-js/runtime/errors.js:131:42)",
        "    at load (/opt/nodejs/node_modules/datadog-lambda-js/runtime/user-function.js:151:15)",
        "    at Object.<anonymous> (/opt/nodejs/node_modules/datadog-lambda-js/handler.js:65:59)",
        "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
        "    at Module.load (internal/modules/cjs/loader.js:950:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
        "    at Module.require (internal/modules/cjs/loader.js:974:19)",
        "    at require (internal/modules/cjs/helpers.js:93:18)"
    ]
}

I assume it's related to this issue and lack of support?

[astuyve]

ESM support is released in Layer version 78 and version 6.78 of this library!

[edeesis]

Thanks @astuyve! Fantastic news, and happy for the relatively quick turn around given the complexity of the change required.

[paco-sparta]

Just a bit more work to get this working :D #295

[beeequeue]

We tried to add the DD layer to our ESM built lambda but found that if we do the lambda cannot import dd-trace anymore. It is already marked as external and works with CJS but not with ESM. 😞