ESM modules support

[DarcyRaynerDD]

What does this PR do?

Adds support for ESM modules and top level async await support with the lambda layer.

JS files with a '.mjs' extension, or with a package.json containing 'type':'module' field, will be imported as ESM modules..

ESM modules look like this:

import {makePromise} from 'my-lib.js';

let prom = makePromise();
await prom(); // Top level await, this module won't finish importing until this is complete.

export async function handler() {
   return {
      status: 200,
      message: "Hello"
   }
}

VS CommonJS

const {makePromise} = require('my-lib');
exports.handler = async function {
   return {
      status: 200,
      message: "Hello"
   }
};

Motivation

This is a supported feature of Lambda, that we are catching up to.

Testing Guidelines

I've added an integration test for the .mjs use case. II have tested manually for ARM for now.

Additional Notes

We upgraded the handler wrapper script to use a dynamic imports for mjs modules, (node will throw an error if you try to use require).

Dynamic imports look like this:

const my_lib_req = require('my_lib_commonjs'); // Returns the exported object
const my_lib_prom = import('my_lib.mjs'); // Returns a promise
const my_lib_req = require('my_lib.mjs"); // Throws an error

We need a way to depromisify the handler library. Usually this is done be turning the calling function into an async function, and awaiting the result. However, this will be reflected all the way up the call hierachy, until we get to our root function, where we still have a promise.

The solution is to use the library deasync. This is a native library, (written in C), that needs to be compiled to use each CPU Architecture, and hooks more directly into the Node runtime. Unfortunately, deasync doesn't come with a prebuilt version of the library for linux-arm64, so we added a script to our build process that will create this. We also strip out other pre-built versions of deasync, since they would add > 6mb to the layer size.

Types of Changes

• Bug fix
• New feature
• Breaking change
• Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

• This PR's description is comprehensive
• This PR contains breaking changes that are documented in the description
• This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
• This PR impacts documentation, and it has been updated (or a ticket has been logged)
• This PR's changes are covered by the automated tests
• This PR collects user input/sensitive content into Datadog
• This PR passes the integration tests (ask a Datadog member to run the tests)

[codecov-commenter]

Codecov Report

Merging #282 (324208a) into main (748b1a3) will decrease coverage by 1.74%.
The diff coverage is n/a.

❗ Current head 324208a differs from pull request most recent head 97073e3. Consider uploading reports for the commit 97073e3 to get more accurate results

@@            Coverage Diff             @@
##             main     #282      +/-   ##
==========================================
- Coverage   82.43%   80.68%   -1.75%     
==========================================
  Files          37       37              
  Lines        1662     1698      +36     
  Branches      364      378      +14     
==========================================
  Hits         1370     1370              
- Misses        240      276      +36     
  Partials       52       52              

Impacted Files	Coverage Δ
src/runtime/user-function.ts	0.00% <ø> (ø)

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[IvanTopolcic]

I couldn't find anything immediately wrong with it. I'd say as long as it's tested well LGTM 👍

[DarcyRaynerDD]

Closing this, we are going to use another approach that doesn't involve using deasync