Error using v8 lambda layer (243)

[JoaquinFernandez]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/aws-serverless

SDK Version

8.2.1

Framework Version

No response

Link to Sentry event

No response

SDK Setup

No response

Steps to Reproduce

Load the lambda layer 243 and initialize as documented -r @sentry/aws-serverless/awslambda-auto

Expected Result

Sentry is imported and initialized as expected

Actual Result

Error: Expected Module._resolveFilename to be a function (was: undefined) - aborting!
--
Please report this error as an issue related to Node.js v16.20.2 at https://github.com/elastic/require-in-the-middle/issues
/opt/nodejs/node_modules/@sentry/aws-serverless/build/npm/cjs/index-BL-xGqAo.js:2
var t=require("perf_hooks"),n=require("path"),e=require("util"),r=require("module"),o=require("fs"),i=require("os"),s=require("tty"),c=require("url"),u=require("process"),a=require("child_process"),l=require("events"),d=require("worker_threads"),f=require("inspector"),h=require("http"),p=require("node:http"),S=require("node:https"),E=require("stream"),T=require("zlib"),m=require("net"),_=require("tls"),b=require("async_hooks"),v="undefined"!=typeof document?document.currentScript:null;function R(t){var n=Object.create(null);return t&&Object.keys(t).forEach((function(e){if("default"!==e){var r=Object.getOwnPropertyDescriptor(t,e);Object.defineProperty(n,e,r.get?r:{enumerable:!0,get:function(){return t[e]}})}})),n.default=t,Object.freeze(n)}function y(t
INIT_REPORT Init Duration: 255.94 ms	Phase: init	Status: error	Error Type: Runtime.ExitError
Error: Expected Module._resolveFilename to be a function (was: undefined) - aborting!
Please report this error as an issue related to Node.js v16.20.2 at https://github.com/elastic/require-in-the-middle/issues
/opt/nodejs/node_modules/@sentry/aws-serverless/build/npm/cjs/index-BL-xGqAo.js:2

[Lms24]

Hey @JoaquinFernandez thanks for reporting (again)! We'll look into this tomorrow. Sorry for the prolonged trouble

[Lms24]

Quick update: We tracked this down to a build issue with the lambda layer specifically. We're currently investigating how to fix it.

As a workaround (while definitely not optimal), you can set up the SDK via an NPM package instead of the layer. I just tested this locally and on AWS
and this seems to work. Again, we recognize this is far from ideal for current layer users. We're on the fix right now.

[Lms24]

Update before the weekend:

We're still not sure what exactly is broken; lest how to fix that but we narrowed it down a bit. I'm gonna document my findings here for some more context and I'll pick this up on Tuesday again (Monday is a holiday here 🙃).

Workarounds

We fully recognize this is not ideal but there are two workarounds:

Using v7 of the Lambda layer

We just re-published the most recent v7 version of our AWS lambda layer. It got a new name (which is why the number starts at 1 again) but apart from that it can be used exactly like the layer up to version 235.

1. Choose the ARN for your region:

us-west-1:
        arn:aws:lambda:us-west-1:943013980633:layer:SentryNodeServerlessSDKv7:1
eu-central-1:
        arn:aws:lambda:eu-central-1:943013980633:layer:SentryNodeServerlessSDKv7:1
us-west-2:
        arn:aws:lambda:us-west-2:943013980633:layer:SentryNodeServerlessSDKv7:1
ap-northeast-2:
        arn:aws:lambda:ap-northeast-2:943013980633:layer:SentryNodeServerlessSDKv7:1
us-east-2:
        arn:aws:lambda:us-east-2:943013980633:layer:SentryNodeServerlessSDKv7:1
eu-west-1:
        arn:aws:lambda:eu-west-1:943013980633:layer:SentryNodeServerlessSDKv7:1
ap-northeast-1:
        arn:aws:lambda:ap-northeast-1:943013980633:layer:SentryNodeServerlessSDKv7:1
ca-central-1:
        arn:aws:lambda:ca-central-1:943013980633:layer:SentryNodeServerlessSDKv7:1
sa-east-1:
        arn:aws:lambda:sa-east-1:943013980633:layer:SentryNodeServerlessSDKv7:1
eu-west-2:
        arn:aws:lambda:eu-west-2:943013980633:layer:SentryNodeServerlessSDKv7:1
us-east-1:
        arn:aws:lambda:us-east-1:943013980633:layer:SentryNodeServerlessSDKv7:1
eu-west-3:
        arn:aws:lambda:eu-west-3:943013980633:layer:SentryNodeServerlessSDKv7:1
eu-south-1:
        arn:aws:lambda:eu-south-1:943013980633:layer:SentryNodeServerlessSDKv7:1
eu-north-1:
        arn:aws:lambda:eu-north-1:943013980633:layer:SentryNodeServerlessSDKv7:1
me-south-1:
        arn:aws:lambda:me-south-1:943013980633:layer:SentryNodeServerlessSDKv7:1
ap-northeast-3:
        arn:aws:lambda:ap-northeast-3:943013980633:layer:SentryNodeServerlessSDKv7:1
af-south-1:
        arn:aws:lambda:af-south-1:943013980633:layer:SentryNodeServerlessSDKv7:1
ap-south-1:
        arn:aws:lambda:ap-south-1:943013980633:layer:SentryNodeServerlessSDKv7:1
ap-east-1:
        arn:aws:lambda:ap-east-1:943013980633:layer:SentryNodeServerlessSDKv7:1
ap-southeast-2:
        arn:aws:lambda:ap-southeast-2:943013980633:layer:SentryNodeServerlessSDKv7:1
ap-southeast-1:
        arn:aws:lambda:ap-southeast-1:943013980633:layer:SentryNodeServerlessSDKv7:1

2. Make sure your NODE_OPTIONS env variable is set up for v7

NODE_OPTIONS="-r @sentry/serverless/dist/awslambda-auto"

Using v8 via the @sentry/aws-serverless NPM package

You can also use the @sentry/aws-serverless SDK directly as an NPM package/dependency and call Sentry.init in your lambda code. Follow the docs for this approach. Please make sure the require call and Sentry.init are the first two lines of code in your lambda function.

Findings

Here's what we know so far:

• The lambda layer artifact bundles the entire SDK code (including all dependencies of @sentry/aws-serverless) into one index.js file (which is called from awslambda-auto.js when automatically initializing the SDK via the -r node option.
• Initializing the SDK via -r argument OR by requiring @sentry/aws-serverless and calling Sentry.ini in the lambda code both throw the same error (the one posted in the issue description)
• This is also reproducible in a small test repo that I made to simulate a simple lambda with the layer: https://github.com/Lms24/gh-sentry-javascript-12089-repro
• The error only occurs if any of these three integrations are initialized:
  • Sentry.awsLambdaIntegration(),
  • Sentry.awsIntegration(),
  • Sentry.httpIntegration(),
• Which makes me think that the root cause lies somewhere in the Otel Http instrumentation that doesn't play well with how we bundle everything together for the layer. But so far I don't know yet how to fix this.

I know I must sound like a broken record by now but sorry for the prolonged troubles with the Lambda layer. We hope to find a fix soon.

[Lms24]

Hey @JoaquinFernandez some updates on this:

I opened two PRs today

• fix(aws-serverless): Avoid minifying Module._resolveFilename in Lambda layer bundle #12232 fixes an issue with require-in-the-middle which was caused by a bundling errors on our end
• fix(aws): Ensure lambda layer uses default export from ImportInTheMiddle #12233 temporarily fixes an issue with import-in-the-middle. We technically don't need this library in the AWS bundle but can't get rid of it easily as it's a dependency of our OpenTelemetry instrumentation. However, it nevertheless threw an error that crashed the lambda function.

Once this is released (probably today or tomorrow), the lambda layer should be usable again 🤞