Huge WASM binary size with -sMAIN_MODULE=1 or 2

[skullernet]

Version of emscripten/emsdk:

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 4.0.3 (a9651ff57165f5710bb09a5fe52590fd6ddb72df)
clang version 21.0.0git (https:/github.com/llvm/llvm-project 6dc41a639334b913e762f65410fcd14a722b137f)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /home/skuller/src/emsdk/upstream/bin

Sample program:

#include <stdio.h>

char test[1024*1024*50];

int main(void)
{
    puts(test);
    return 0;
}

Compiler command line:

emcc -o test.wasm -O3 -sMAIN_MODULE=<X> -sTOTAL_MEMORY=200MB test.c

Problem description:

When compiling above program with -sMAIN_MODULE=0 resulting test.wasm binary size is 2 KB, but with -sMAIN_MODULE=1 it increases to 52 MB. Maybe I'm missing something, but is such dramatic code size increase normal/expected? I can compress the output, sure (it compresses well since it's mostly zeros), but isn't Emscripten all about minimizing code size for use on the web?

This behavior stays the same with -sMAIN_MODULE=2. wasm-objdump -h shows there's a huge Data section consisting mostly of zeros:

Data start=0x00002077 end=0x03202730 (size=0x032006b9) count: 1

My apologies if this is a known/duplicate issue but I couldn't find any similar bug report.

[kripken]

Yes, this is normal. MAIN_MODULE=1 links in all of libc and other system libraries, which allows loading dynamic libraries that use any part of libc etc. later. That has a big price in terms of code size. On a native linux system libc is just in the system libraries anyhow, but for wasm we bundle that code.

MAIN_MODULE=2 is exactly meant to fix this (with some limitations).

[skullernet]

Yes, I understand that. But this problem is reproducible with MAIN_MODULE=2 too. I don't think it is related to statically linking libc and other system libraries.

The core issue here is that the large 50 MiB statically allocated array in the example program above gets included in Data section and is seemingly stored in WASM file. This doesn't seem logical to me. I'd expect it to be never actually stored on disk, because it's not initialized.

[kripken]

Ah, sorry, I missed that array of 50MB static data.

Relocatable binaries are not able to ignore zeroes at the end of data segments. The segment is not at a fixed place, but at one determined by the __memory_base location, determined at runtime. Those zeroes might be needed, depending on that location.

Two things might help here:

1. @sbc100 was thinking about making main modules non-relocatable.
2. We could optimize huge segments of zeroes in wasm-opt, replacing them with a memory.fill of 0.

[sbc100]

Yes, we can/should probably replace .bss segments with a memory.fill instruction. I actually thought that wasm-ld already did this. Checking now.

[sbc100]

I have a fix up on the llvm side: llvm/llvm-project#127735

[oatgnauh]

I have a fix up on the llvm side: llvm/llvm-project#127735

great ! I wonder if I can try this commit. I am using emsdk 3.1.54,can I replace the clang version with the latest code without changing the emsdk version?

[sbc100]

Once the change land in the llvm side it will make it into and emsdk tot build within a few hours. You can then try it out with emsdk install tot. You will need to upgrade to the latest emsdk version though, is there some reason you cannot do this? Using a new clang with and emsdk its not likely to work well.