Problems with `@sentry/astro` on Netlify

[mydea]

Raised by @whitep4nth3r , there seem to be some issues with an astro app deployed to netfliy.

Repro: https://github.com/whitep4nth3r/p4nth3rworld

When running locally, stuff seems to work as far as I can tell. But when deployed to production (on Netlify), you can see that often no server transaction is sent: https://the-claw.sentry.io/performance/trace/ffcc30d02ff374907a3bef54c998f338/ (only a pageload).

It is sent sometimes but not always, leading to broken traces etc.

While looking more into this, when enabling debug logs, we found some things:

This log:

WARN   Sentry Logger [warn]: Failed to register ESM hook Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/var/task/node_modules/@opentelemetry/instrumentation/hook.mjs' imported from /var/task/node_modules/@sentry/node/esm/sdk/initOtel.js
    at finalizeResolution (node:internal/modules/esm/resolve:269:11)
    at moduleResolve (node:internal/modules/esm/resolve:937:10)
    at moduleResolveWithNodePath (node:internal/modules/esm/resolve:1161:14)
    at defaultResolve (node:internal/modules/esm/resolve:1204:79)
    at nextResolve (node:internal/modules/esm/hooks:866:28)
    at Hooks.resolve (node:internal/modules/esm/hooks:304:30)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:345:35)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:227:38)
    at ModuleLoader.import (node:internal/modules/esm/loader:315:34)
    at Hooks.register (node:internal/modules/esm/hooks:165:53) {
  code: 'ERR_MODULE_NOT_FOUND',
  url: 'file:///var/task/node_modules/@opentelemetry/instrumentation/hook.mjs'
}

These logs:

something seems to be off there 🤔

[whitep4nth3r]

If anyone needs any more info, I'm here!

[s1gr1d]

I could reproduce it with my own example (repo: https://github.com/s1gr1d/example.astro-sentry-netlify).

Some things I saw in the function log on Netlify:

• Sentry Logger [log]: Could not load NodeFetch instrumentation. on Netlify (does not happen locally)
• I don't see all events in Sentry (but I am getting them in the function logs). I just got the success spans in Sentry
  • [Tracing] Finishing "http.server" root span "GET /" with ID 4743f742c010d44a
  • [Tracing] Finishing "http.server" root span "GET /" with ID 44621bba8f17b5e9
  • [Tracing] Finishing "http.server" root span "GET /test-param-success/[param]" with ID e6ce337a1e36aec9
  • [Tracing] Finishing "http.server" root span "GET /test-param-error/[param]" with ID b95ae735524c9139
  • the span for /test-success/ is missing -> this one is a static page with no fetch call
• ESM error on server start

ESM hook error

a7364239 WARN   Sentry Logger [warn]: Failed to register ESM hook Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/var/task/node_modules/@opentelemetry/instrumentation/hook.mjs' imported from /var/task/node_modules/@sentry/node/esm/sdk/initOtel.js
    at new NodeError (node:internal/errors:405:5)
    at finalizeResolution (node:internal/modules/esm/resolve:332:11)
    at moduleResolve (node:internal/modules/esm/resolve:1029:10)
    at moduleResolveWithNodePath (node:internal/modules/esm/resolve:876:12)
    at defaultResolve (node:internal/modules/esm/resolve:1242:79)
    at nextResolve (node:internal/modules/esm/hooks:864:28)
    at Hooks.resolve (node:internal/modules/esm/hooks:302:30)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:366:35)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:250:38)
    at ModuleLoader.import (node:internal/modules/esm/loader:336:34) {
  url: 'file:///var/task/node_modules/@opentelemetry/instrumentation/hook.mjs',
  code: 'ERR_MODULE_NOT_FOUND'
}

Complete Logs

Jun 24, 11:23:13 AM: INIT_START Runtime Version: nodejs:18.v30	Runtime Version ARN: arn:aws:lambda:us-east-2::runtime:f89c264158db39a1cfcbb5f9b3741413df1cfce4d550c9a475a67d923e19e2f4
Jun 24, 11:23:15 AM: a7364239 WARN   Sentry Logger [warn]: Failed to register ESM hook Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/var/task/node_modules/@opentelemetry/instrumentation/hook.mjs' imported from /var/task/node_modules/@sentry/node/esm/sdk/initOtel.js
    at new NodeError (node:internal/errors:405:5)
    at finalizeResolution (node:internal/modules/esm/resolve:332:11)
    at moduleResolve (node:internal/modules/esm/resolve:1029:10)
    at moduleResolveWithNodePath (node:internal/modules/esm/resolve:876:12)
    at defaultResolve (node:internal/modules/esm/resolve:1242:79)
    at nextResolve (node:internal/modules/esm/hooks:864:28)
    at Hooks.resolve (node:internal/modules/esm/hooks:302:30)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:366:35)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:250:38)
    at ModuleLoader.import (node:internal/modules/esm/loader:336:34) {
  url: 'file:///var/task/node_modules/@opentelemetry/instrumentation/hook.mjs',
  code: 'ERR_MODULE_NOT_FOUND'
}
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Initializing Sentry: process: 17, thread: main.
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: InboundFilters
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: FunctionToString
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: LinkedErrors
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: RequestData
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Console
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Http
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: NodeFetch
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: OnUncaughtException
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: OnUnhandledRejection
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: ContextLines
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: LocalVariables
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Context
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Express
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Fastify
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Graphql
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Mongo
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Mongoose
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Mysql
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Mysql2
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Redis
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Postgres
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Nest
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Hapi
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Koa
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Integration installed: Connect
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Running in ESM mode.
Jun 24, 11:23:15 AM: a7364239 DEBUG  Sentry Logger [debug]: @opentelemetry/api: Registered a global for diag v1.9.0.
Jun 24, 11:23:15 AM: a7364239 DEBUG  Sentry Logger [debug]: @opentelemetry/api: Registered a global for trace v1.9.0.
Jun 24, 11:23:15 AM: a7364239 DEBUG  Sentry Logger [debug]: @opentelemetry/api: Registered a global for context v1.9.0.
Jun 24, 11:23:15 AM: a7364239 DEBUG  Sentry Logger [debug]: @opentelemetry/api: Registered a global for propagation v1.9.0.
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: [Tracing] Starting sampled root span
  op: < unknown op >
  name: GET /
  ID: 4743f742c010d44a
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: [Tracing] Finishing "http.server" root span "GET /" with ID 4743f742c010d44a
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: Could not load NodeFetch instrumentation.
Jun 24, 11:23:15 AM: a7364239 INFO   Sentry Logger [log]: SpanExporter exported 1 spans, 0 unsent spans remaining
Jun 24, 11:23:15 AM: a7364239 Duration: 675.74 ms	Memory Usage: 176 MB	Init Duration: 1475.44 ms	
Jun 24, 11:28:07 AM: 4d234179 INFO   Sentry Logger [log]: [Tracing] Starting sampled root span
  op: < unknown op >
  name: GET /
  ID: 44621bba8f17b5e9
Jun 24, 11:28:07 AM: 4d234179 INFO   Sentry Logger [log]: [Tracing] Finishing "http.server" root span "GET /" with ID 44621bba8f17b5e9
Jun 24, 11:28:07 AM: 4d234179 INFO   Sentry Logger [log]: SpanExporter exported 1 spans, 0 unsent spans remaining
Jun 24, 11:28:07 AM: 4d234179 Duration: 188.16 ms	Memory Usage: 178 MB	
Jun 24, 11:28:33 AM: INIT_START Runtime Version: nodejs:18.v30	Runtime Version ARN: arn:aws:lambda:us-east-2::runtime:f89c264158db39a1cfcbb5f9b3741413df1cfce4d550c9a475a67d923e19e2f4
Jun 24, 11:28:35 AM: c1a73ec1 INFO   Sentry Logger [log]: [Tracing] Finishing "http.server" root span "GET /test-param-success/[param]" with ID e6ce337a1e36aec9
Jun 24, 11:28:35 AM: c1a73ec1 INFO   Sentry Logger [log]: SpanExporter exported 1 spans, 0 unsent spans remaining
Jun 24, 11:28:35 AM: c1a73ec1 Duration: 2072.18 ms	Memory Usage: 178 MB	
Jun 24, 11:28:55 AM: 52973d5e INFO   Sentry Logger [log]: [Tracing] Starting sampled root span
  op: < unknown op >
  name: GET /test-param-error/[param]
  ID: b95ae735524c9139
Jun 24, 11:28:55 AM: fc013641 Duration: 109.92 ms	Memory Usage: 160 MB	
Jun 24, 11:28:55 AM: 52973d5e INFO   Sentry Logger [log]: [Tracing] Finishing "http.server" root span "GET /test-param-error/[param]" with ID b95ae735524c9139
Jun 24, 11:28:55 AM: 52973d5e INFO   Sentry Logger [log]: SpanExporter exported 1 spans, 0 unsent spans remaining
Jun 24, 11:28:55 AM: 52973d5e Duration: 202.21 ms	Memory Usage: 178 MB	
Jun 24, 11:29:34 AM: INIT_START Runtime Version: nodejs:18.v30	Runtime Version ARN: arn:aws:lambda:us-east-2::runtime:f89c264158db39a1cfcbb5f9b3741413df1cfce4d550c9a475a67d923e19e2f4

[timfish]

Without trying the code out yet myself, the ESM hook error suggests the app is bundled in some way or the full node_modules isn't getting deployed.

It's trying to load /var/task/node_modules/@opentelemetry/instrumentation/hook.mjs but it can't be found.

[timfish]

I still need to test this locally to confirm but I think this issue is caused by the Netlify tooling not including all the required files in production builds. Netlify uses @netlify/zip-it-and-ship-it to collect all the files required for a production deploy, which in turn uses @vercel/nft to resolve all the required source files.

My best guess so far is that this does not pick up that the otel hook should be included in the production build.