Skip to content

Type-safe Embind port, compatible with the current API #7553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

GulgDev
Copy link
Contributor

@GulgDev GulgDev commented Apr 25, 2025

This PR moves the JS wrapper to C++ by utilizing Embind. It aims at automating the process of binding as much as possible. Ideally, I want to get rid of any wrappers and automatically generate compatible bindings by wrapping the necessary methods in lambdas. If this succeeds, it may be possible in future to adapt API to make it more natural (including breaking changes).

@GulgDev GulgDev changed the title [PoC] Use Embind on the C wrapper Type-safe Embind port, compatible with the current API Apr 26, 2025
@GulgDev
Copy link
Contributor Author

GulgDev commented Apr 27, 2025

@kripken For some reason I get Maximum call stack size exceeded when importing the module after it has been built. Do you know why is that happening? Setting --stack-size to 2000 solves the problem.

@kripken
Copy link
Member

kripken commented Apr 28, 2025

For some reason I get Maximum call stack size exceeded when importing the module

Hmm, strange. What is the full stack trace when you get that error? That could help understand if this is during the VM's compilation of the code, or in the JS itself as it executes during startup.

cc @brendandahl - are there known issues with embind there?

@GulgDev
Copy link
Contributor Author

GulgDev commented Apr 28, 2025

@kripken It seems like Node.JS fails to load the script:

node:internal/modules/cjs/loader:1427
  const result = compileFunctionForCJSLoader(content, filename, false /* is_sea_main */, shouldDetectModule);
                 ^

RangeError: Maximum call stack size exceeded
    at wrapSafe (node:internal/modules/cjs/loader:1427:18)
    at Module._compile (node:internal/modules/cjs/loader:1449:20)
    at Module._extensions..js (node:internal/modules/cjs/loader:1588:10)
    at Module.load (node:internal/modules/cjs/loader:1282:32)
    at Module._load (node:internal/modules/cjs/loader:1098:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:215:24)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:158:5)
    at node:internal/main/run_main_module:30:49

The call stack is different when type is set to module:

node:internal/modules/esm/utils:337
  const wrap = new ModuleWrap(url, undefined, source, 0, 0, hostDefinedOption);
               ^

RangeError: Maximum call stack size exceeded
    at compileSourceTextModule (node:internal/modules/esm/utils:337:16)
    at ModuleLoader.moduleStrategy (node:internal/modules/esm/translators:164:18)
    at callTranslator (node:internal/modules/esm/loader:439:14)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:445:30)
    at async ModuleJob._link (node:internal/modules/esm/module_job:106:19)

@GulgDev
Copy link
Contributor Author

GulgDev commented Apr 28, 2025

It seems like the error is coming not from the count of stack frames, but instead from the size of stack. Increasing --stack-size also fixes the problem. The resulting module is about 16 Mb. Is the problem coming from the size of the bundle?

@kripken
Copy link
Member

kripken commented Apr 28, 2025

Looks like Node is failing to load and compile the code, yeah. How big is the code? Maybe it's just the size or the amount of nesting.

If this is a debug build, perhaps try an optimized one? Linking with -O1 might help, or even -O2 --profiling (optimized, but profiling prevents full JS minification, so the JS should be somewhat readable).

@GulgDev
Copy link
Contributor Author

GulgDev commented Apr 28, 2025

I've built the project in debug mode, and got 3 million lines. Is this ok? Is the size of bundle coming from embind?

@brendandahl
Copy link
Collaborator

Binaryen is a pretty big project with lots of templated c++, so it seems reasonable the debug output is going to be large.

@GulgDev
Copy link
Contributor Author

GulgDev commented Apr 29, 2025

It's just strange to me that the crashes appeared when using Embind.

I've found the root of the problem. I removed the expression wrappers and now it loads just fine. But what do we do now? Expression wrappers are crucial.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants