ESModule support failure due to `require`-ing our client code #295

paco-sparta · 2022-07-20T13:05:33Z

Expected Behavior

Pointing the CMD to handler.handle.mjs should make the lambda compatible with ESModules.

Actual Behavior

When launching our handler with package.json specifying "modules" and pointing our Docker file to "handler.handle.mjs", we see a require error at runtime.

2022-07-20T12:09:01.317Z	undefined	ERROR	Uncaught Exception 	
{
    "errorType": "Error",
    "errorMessage": "require() of ES Module /var/task/app.js from /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js not supported.\nInstead change the require of app.js in /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js to a dynamic import() which is available in all CommonJS modules.",
    "code": "ERR_REQUIRE_ESM",
    "stack": [
        "Error [ERR_REQUIRE_ESM]: require() of ES Module /var/task/app.js from /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js not supported.",
        "Instead change the require of app.js in /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js to a dynamic import() which is available in all CommonJS modules.",
        "    at Module.Hook.Module.require (/var/task/node_modules/dd-trace/packages/dd-trace/src/ritm.js:72:33)",
        "    at _tryRequireSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:248:12)",
        "    at _loadUserAppSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:286:16)",
        "    at loadSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:374:19)",
        "    at Object.<anonymous> (/var/task/node_modules/datadog-lambda-js/dist/handler.js:35:27)",
        "    at _tryRequireFile (file:///var/runtime/index.mjs:857:37)",
        "    at _tryRequire (file:///var/runtime/index.mjs:907:25)",
        "    at _loadUserApp (file:///var/runtime/index.mjs:933:22)",
        "    at Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:964:27)",
        "    at start (file:///var/runtime/index.mjs:1124:42)",
        "    at file:///var/runtime/index.mjs:1130:7",
        "    at async Promise.all (index 0)"
    ]
}

Where app.js is our client code, transpiled to ES2020 using tsc with esmoduleinterop and isolatedModules enabled. Node is configured to use experimental node resolution for imports so file extensions aren't needed.

Steps to Reproduce the Problem

Create a new Docker-based nodejs project with datadog. Use TSC to compile with configurations above.
Point Docker to CMD ["node_modules/datadog-lambda-js/dist/handler.handler.mjs"]
Deploy with serverless
See error above

Specifications

Datadog NPM version: 6.81.X
Node version: 16.X

Stacktrace

2022-07-20T12:09:01.317Z	undefined	ERROR	Uncaught Exception 	
{
  "errorType": "Error",
  "errorMessage": "require() of ES Module /var/task/app.js from /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js not supported.\nInstead change the require of app.js in /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js to a dynamic import() which is available in all CommonJS modules.",
  "code": "ERR_REQUIRE_ESM",
  "stack": [
      "Error [ERR_REQUIRE_ESM]: require() of ES Module /var/task/app.js from /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js not supported.",
      "Instead change the require of app.js in /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js to a dynamic import() which is available in all CommonJS modules.",
      "    at Module.Hook.Module.require (/var/task/node_modules/dd-trace/packages/dd-trace/src/ritm.js:72:33)",
      "    at _tryRequireSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:248:12)",
      "    at _loadUserAppSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:286:16)",
      "    at loadSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:374:19)",
      "    at Object.<anonymous> (/var/task/node_modules/datadog-lambda-js/dist/handler.js:35:27)",
      "    at _tryRequireFile (file:///var/runtime/index.mjs:857:37)",
      "    at _tryRequire (file:///var/runtime/index.mjs:907:25)",
      "    at _loadUserApp (file:///var/runtime/index.mjs:933:22)",
      "    at Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:964:27)",
      "    at start (file:///var/runtime/index.mjs:1124:42)",
      "    at file:///var/runtime/index.mjs:1130:7",
      "    at async Promise.all (index 0)"
  ]
}

The text was updated successfully, but these errors were encountered:

paco-sparta · 2022-07-20T14:49:01Z

After digging a bit more, it seems like it's not finding package.json, and fails this branch:

datadog-lambda-js/src/runtime/user-function.ts

Lines 130 to 137 in 826c36c

    
           // If package.json type != module, .js files are loaded via require. 
        
           const pjHasModule = _hasPackageJsonTypeModule(lambdaStylePath); 
        
           if (!pjHasModule) { 
        
             const loaded = _tryRequireFile(lambdaStylePath, ".js"); 
        
             if (loaded) { 
        
               return loaded; 
        
             } 
        
           }

DarcyRaynerDD · 2022-07-20T15:40:27Z

Hi Paco, is there a package.json next to your app.js file, that specifies it's a ES module?
We try to replicate the logic AWS describes here with how we load modules, and a .js file will be interpreted as a common js file unless there is a package.json file next to it declaring the module type: https://aws.amazon.com/blogs/compute/using-node-js-es-modules-and-top-level-await-in-aws-lambda/

paco-sparta · 2022-07-21T09:15:00Z

I have added a handful of logs to a modified node_modulesto follow the logic:

function _hasFolderPackageJsonTypeModule(folder) {
   ...
    var pj = path_1.default.join(folder, "/package.json");
    var pppp = fs_1.default.existsSync(pj);
    console.log(JSON.stringify({ path: pj, exists: pppp }));

Yields {"path":"/var/task/package.json","exists":true}, and a few lines later in

var pkg = JSON.parse(fs_1.default.readFileSync(pj, "utf-8"));
console.log(JSON.stringify({ pkg, hasModule: pkg.type === "module" }));

I see {"pkg":{...}, "hasModule":true}.

This means that

function _tryRequireSync(appRoot, moduleRoot, module) {
   ...

    // If package.json type != module, .js files are loaded via require.
    var pjHasModule = _hasPackageJsonTypeModule(lambdaStylePath);
    console.log(JSON.stringify({pjHasModule}));
    if (!pjHasModule) {

Returns {"pjHasModule":true} and the logic falls through to require at the end of the method _tryRequireSync, which causes the exception due to incompatibility:

{
    "errorType": "Error",
    "errorMessage": "require() of ES Module /var/task/app.js from /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js not supported.\nInstead change the require of app.js in /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js to a dynamic import() which is available in all CommonJS modules.",
    "trace": [
        "Error [ERR_REQUIRE_ESM]: require() of ES Module /var/task/app.js from /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js not supported.",
        "Instead change the require of app.js in /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js to a dynamic import() which is available in all CommonJS modules.",
        "    at Module.Hook.Module.require (/var/task/node_modules/dd-trace/packages/dd-trace/src/ritm.js:72:33)",
        "    at _tryRequireSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:252:12)",
        "    at _loadUserAppSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:290:16)",
        "    at loadSync (/var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js:378:19)",
        "    at Object.<anonymous> (/var/task/node_modules/datadog-lambda-js/dist/handler.js:35:27)",
        "    at _tryRequireFile (file:///var/runtime/index.mjs:857:37)",
        "    at _tryRequire (file:///var/runtime/index.mjs:907:25)",
        "    at _loadUserApp (file:///var/runtime/index.mjs:933:22)",
        "    at Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:964:27)",
        "    at start (file:///var/runtime/index.mjs:1124:42)",
        "    at file:///var/runtime/index.mjs:1130:7",
        "    at async Promise.all (index 0)"
    ]
}

paco-sparta · 2022-07-21T09:16:26Z

Let me follow why _tryRequire falls through this path.

EDIT: The lambda environment is picking up the js file with require-style imports loadSync instead of the mjs with load. This is even if I specify the mjs file in the command. WTF.

paco-sparta · 2022-07-21T09:38:07Z

I have copied the contents of .mjs into .js and another WTF

{
    "errorType": "Runtime.UserCodeSyntaxError",
    "errorMessage": "SyntaxError: Cannot use import statement outside a module",
    "trace": [
        "Runtime.UserCodeSyntaxError: SyntaxError: Cannot use import statement outside a module",
        "    at _loadUserApp (file:///var/runtime/index.mjs:936:17)",
        "    at async Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:964:21)",
        "    at async start (file:///var/runtime/index.mjs:1124:23)",
        "    at async file:///var/runtime/index.mjs:1130:1"
    ]
}

The lambda runtime is not picking up that we're working with modules. What.

paco-sparta · 2022-07-21T10:29:12Z

I have moved the whole repo to mjs and mts and still no result. I removed the node experimental resolver and still nothing.

My Dockerfile is lean

FROM public.ecr.aws/lambda/nodejs:16

COPY --from=public.ecr.aws/datadog/lambda-extension:latest /opt/extensions/ /opt/extensions

# Configure by --buildArgs
ENV DD_LAMBDA_HANDLER "app.handler"
ENV DD_SITE "datadoghq.eu"
ENV DD_API_KEY "XXXX"

COPY package.json ./
COPY package-lock.json ./

COPY lib ./ # contains a copy of node_modules

CMD ["node_modules/datadog-lambda-js/dist/handler.handler"]

And the serverless one too

provider:
  name: aws
  architecture: x86_64
  stage: ${opt:stage, 'dev'}
  region: ${opt:region, 'eu-west-1'}
  apiGateway:
    minimumCompressionSize: 1024
    shouldStartNameWithService: true
  environment:
    AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1'
    NODE_OPTIONS: '--enable-source-maps --stack-trace-limit=1000'
  ecr:
    images:
      appimage:
        path: ./
        platform: linux/amd64

I know it's all there.

[12:32:49] ƛ ls -1
app.mjs
app.mjs.map
node_modules/
package-lock.json
package.json
tracer.mjs
tracer.mjs.map

[12:32:51] ƛ bat package.json 
   1   │ {
   2   │   "name": "XXX",
   3   │   "version": "1.0.0",
   4   │   "description": "Serverless aws-nodejs-typescript template",
   5   │   "author": "YYY",
   6   │   "license": "SEE LICENSE IN ../LICENSE",
   7   │   "main": "app.mjs",
   8   │   "type": "module",

I'm at my wits' end to force it to load the handler. The exception is already telling me that Lambda is using the mjs runtime file:///var/runtime/index.mjs:1130:1

Why why why

DarcyRaynerDD · 2022-07-22T19:30:57Z

What happens if you change

CMD ["node_modules/datadog-lambda-js/dist/handler.handler"]

to

CMD ["node_modules/datadog-lambda-js/dist/handler.mjs.handler"]

The reason we have two files, is there is a cjs entry point for the node 12 runtime, (which doesn't support .mjs). If you use one of our lambda layers, we make sure node_modules/datadog-lambda-js/dist/handler.handler uses the right version for your runtime, but in the npm module we don't know which runtime you are using, so we leave both in. If this change does work, I will update our documentation to be cleared for Dockerfile users.

If that doesn't work, I think a productive next step would be to send us a zip with a minimum reproduction.

paco-sparta · 2022-07-25T09:27:15Z

Your suggestion was helpful but didn't work :(

Here's the minimum repro I was able to put together: lambda-failure-example.zip

npm i && sls deploy --aws-profile <prof> && sls invoke -f hello --aws-profile <prof> --log --verbose

bgeese-szdm · 2022-07-26T08:30:10Z

We have the same or a very similar issue. We client code in a container image with modules and no bundling or processing of the JavaScript at all. package.json specifies the type "module" and we have a simply app.js file that exports a handler. That worked fine until we integrated the datadog handler.

The Dockerfile is straightforward:

FROM public.ecr.aws/lambda/nodejs:14

ENV NODE_ENV=production
ENV DD_LAMBDA_HANDLER="app.handler"
ENV DD_TRACE_ENABLED="true"
ENV DD_FLUSH_TO_LOG="true"

# Install Chrome to get all of the dependencies installed
# ...

COPY .npmrc ${LAMBDA_TASK_ROOT}/
COPY package*.json ${LAMBDA_TASK_ROOT}/
COPY src ${LAMBDA_TASK_ROOT}

RUN npm install --production

RUN rm .npmrc

CMD ["node_modules/datadog-lambda-js/dist/handler.handler"]

When we change the CMD back to CMD ["app.handler"], the lambda works, but of course, without the integration we wanted.

Changing to CMD to CMD ["node_modules/datadog-lambda-js/dist/handler.mjs.handler"], as you suggested, does not help:

    "require() of ES modules is not supported.",
    "require() of /var/task/app.js from /var/task/node_modules/datadog-lambda-js/dist/runtime/user-function.js is an ES module file as it is a .js file whose nearest parent package.json contains \"type\": \"module\" which defines all .js files in that package scope as ES modules.",
    "Instead rename app.js to end in .cjs, change the requiring code to use import(), or remove \"type\": \"module\" from /var/task/package.json.",
    "",

DarcyRaynerDD · 2022-07-26T20:27:32Z

@paco-sparta Thanks for the reproduction, it helps a lot. We will work on a fix for this soon.

kimi-p · 2022-07-27T21:31:37Z

@paco-sparta @bgeese-szdm
We found the issue is due to node_modules/datadog-lambda-js/dist/handler.js file exists (for node12 backward compatibility, we cannot just remove it) and the existence of the file will make _tryRequire() try to import commonJS first.

The temporary workaround is to remove the file for now. We will think about the Lambda layer and npm use cases, and come up with a solution for the long term. Thank you both for letting us know the issue!

RUN rm node_modules/datadog-lambda-js/dist/handler.js
CMD ["node_modules/datadog-lambda-js/dist/handler.handler"]

DataDog/datadog-lambda-js#295 (comment)

kimi-p · 2022-07-28T20:17:17Z

Closing this one now and we will remove this handler.js file later this year after AWS deprecates their support of Node 12. Thanks again for flagging!

pakoito · 2022-07-28T23:54:03Z

Thank you both for the help!

EDIT: same guy, personal account :D

paco-sparta changed the title ~~ESModule support failure due to require~~ ESModule support failure due to require-ing our client code Jul 20, 2022

paco-sparta mentioned this issue Jul 20, 2022

ES Module support #266

Closed

kimi-p added a commit to DataDog/documentation that referenced this issue Jul 28, 2022

Update ESModule Instrumentation for Container Image

0b793ec

DataDog/datadog-lambda-js#295 (comment)

kimi-p mentioned this issue Jul 28, 2022

Update ESModule Instrumentation for Container Image DataDog/documentation#14813

Merged

4 tasks

kimi-p closed this as completed Jul 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ESModule support failure due to `require`-ing our client code #295

ESModule support failure due to `require`-ing our client code #295

paco-sparta commented Jul 20, 2022 •

edited

Loading

paco-sparta commented Jul 20, 2022

Uh oh!

DarcyRaynerDD commented Jul 20, 2022

Uh oh!

paco-sparta commented Jul 21, 2022 •

edited

Loading

Uh oh!

paco-sparta commented Jul 21, 2022 •

edited

Loading

Uh oh!

paco-sparta commented Jul 21, 2022 •

edited

Loading

Uh oh!

paco-sparta commented Jul 21, 2022 •

edited

Loading

Uh oh!

DarcyRaynerDD commented Jul 22, 2022

Uh oh!

paco-sparta commented Jul 25, 2022 •

edited

Loading

Uh oh!

bgeese-szdm commented Jul 26, 2022 •

edited

Loading

Uh oh!

DarcyRaynerDD commented Jul 26, 2022

Uh oh!

kimi-p commented Jul 27, 2022 •

edited

Loading

Uh oh!

kimi-p commented Jul 28, 2022

Uh oh!

pakoito commented Jul 28, 2022 •

edited

Loading

Uh oh!

ESModule support failure due to require-ing our client code #295

ESModule support failure due to require-ing our client code #295

Comments

paco-sparta commented Jul 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

Specifications

Stacktrace

paco-sparta commented Jul 20, 2022

Uh oh!

DarcyRaynerDD commented Jul 20, 2022

Uh oh!

paco-sparta commented Jul 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paco-sparta commented Jul 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paco-sparta commented Jul 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paco-sparta commented Jul 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DarcyRaynerDD commented Jul 22, 2022

Uh oh!

paco-sparta commented Jul 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bgeese-szdm commented Jul 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DarcyRaynerDD commented Jul 26, 2022

Uh oh!

kimi-p commented Jul 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kimi-p commented Jul 28, 2022

Uh oh!

pakoito commented Jul 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ESModule support failure due to `require`-ing our client code #295

ESModule support failure due to `require`-ing our client code #295

paco-sparta commented Jul 20, 2022 •

edited

Loading

paco-sparta commented Jul 21, 2022 •

edited

Loading

paco-sparta commented Jul 21, 2022 •

edited

Loading

paco-sparta commented Jul 21, 2022 •

edited

Loading

paco-sparta commented Jul 21, 2022 •

edited

Loading

paco-sparta commented Jul 25, 2022 •

edited

Loading

bgeese-szdm commented Jul 26, 2022 •

edited

Loading

kimi-p commented Jul 27, 2022 •

edited

Loading

pakoito commented Jul 28, 2022 •

edited

Loading