Ensure the WASM output is compatible with wasmtime + WASI

[certik]

We should ensure that the subset of WASM that we generate in the ASR->WASM is compatible with wasmtime + WASI, so that one can use wasmtime and other tools like (wasm2c) out of the box.

[certik]

The goal here would be to do:

$ lfortran --backend=wasm examples/expr2.f90 -o expr2.wasm
$ wasmtime expr2.wasm
Error: failed to run main module `expr2.wasm`

Caused by:
    0: failed to instantiate "expr2.wasm"
    1: unknown import: `js::print_i32` has not been defined

We should standardize the WASM output that we generate, and it seems the change is minor, in this example the WAT looks like:

$ lfortran --show-wat examples/expr2.f90              
(module
    (type (;0;) (func (param i32) (result)))
    (type (;1;) (func (param i64) (result)))
    (type (;2;) (func (param f32) (result)))
    (type (;3;) (func (param f64) (result)))
    (type (;4;) (func (param i32 i32) (result)))
    (type (;5;) (func (param) (result)))
    (type (;6;) (func (param i32) (result)))
    (type (;7;) (func (param) (result)))
    (import "js" "print_i32" (func (;0;) (type 0)))
    (import "js" "print_i64" (func (;1;) (type 1)))
    (import "js" "print_f32" (func (;2;) (type 2)))
    (import "js" "print_f64" (func (;3;) (type 3)))
    (import "js" "print_str" (func (;4;) (type 4)))
    (import "js" "flush_buf" (func (;5;) (type 5)))
    (import "js" "set_exit_code" (func (;6;) (type 6)))
    (import "js" "memory" (memory (;0;) 100 100))
    (func $7 (type 7) (param) (result)
        (local i32)
        i32.const 25
        local.set 0
        local.get 0
        call 0
        call 5
        i32.const 0
        call 6
        return
    )
    (export "_lcompilers_main" (func $7))
)

And we just need to change the imports:

    (import "js" "print_i32" (func (;0;) (type 0)))
    (import "js" "print_i64" (func (;1;) (type 1)))
    (import "js" "print_f32" (func (;2;) (type 2)))
    (import "js" "print_f64" (func (;3;) (type 3)))
    (import "js" "print_str" (func (;4;) (type 4)))
    (import "js" "flush_buf" (func (;5;) (type 5)))
    (import "js" "set_exit_code" (func (;6;) (type 6)))
    (import "js" "memory" (memory (;0;) 100 100))

To be compatible with WASI/wasmtime. Details to be figured out.

Then in JavaScript we change the API of our implementation to be also compatible with WASI for the subset that we generate, and so the generated wasm will continue working at https://dev.lfortran.org/.

[certik]

For printing, we can see how Clang does it using this tutorial: https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-tutorial.md

[certik]

As an example, instead of sys_exit_code, use the proc_exit import:

https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md#-proc_exitrval-exitcode

[ubaidsk]

For printing, we can see how Clang does it using this tutorial:

Would it be fine if we use emscripten in place of clang for compiling to wasm? It seems emscripten uses clang internally (https://stackoverflow.com/a/64691937). Emscripten compiled wasm runs with wasmtime. For clang, I am currently facing the following error:

(base) compile_to_wasm$ clang --target=wasm32 main.cpp -o clang_main.wasm
wasm-ld-14: error: cannot open crt1.o: No such file or directory
wasm-ld-14: error: unable to find library -lc
wasm-ld-14: error: unable to find library -lgcc
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I guess there might be a fix for the above. I am concerned if the fix might break/affect other development environment/tools like lpython/lfortran environment or emscripten tool.

[certik]

Yes, I think emscripten is fine. As long as it runs in wasmtime. The goal is to just figure out how they do it, what API they use.

[ubaidsk]

Yes, I think emscripten is fine. As long as it runs in wasmtime. The goal is to just figure out how they do it, what API they use.

Got it.

[ubaidsk]

It seems WASI does not have support for printing integers/floats, instead it depends on source language's library (https://bytecodealliance.zulipchat.com/#narrow/stream/217126-wasmtime/topic/import.20memory/near/325736801). Please, could someone possibly share what we should do? Shall we implement a function/class/entity in the frontend language that could print formatted integers/floats? Or Shall we implement something similar in the lines of printing integers/floats in the wasm_x64 backend?

[certik]

Here is how to experiment with WASI. First install https://github.com/WebAssembly/wasi-sdk, you can go to releases and unpack a tarball. Let's create a simple program:

#include <stdio.h>

int main() {
    int x;
    x = (2+3)*5;
    printf("%d\n", x);
    return 0;
}

Compile to WASM + WASI:

$ /path/to/wasi-sdk-19.0/bin/clang expr2.c -o expr2.wasm
$ file expr2.wasm 
expr2.wasm: WebAssembly (wasm) binary module version 0x1 (MVP)
$ ll -h expr2.wasm
-rwxr-xr-x  1 ondrej  staff    18K Feb  3 20:07 expr2.wasm

You can run it with wasmtime or wasmer as follows:

$ wasmtime expr2.wasm 
25
$ wasmer expr2.wasm
25

You can compile to a binary using:

$ wasmer create-exe expr2.wasm -o expr2              
Compiler: cranelift
Target: aarch64-apple-darwin
Format: Symbols
Using path `/Users/ondrej/.wasmer/lib/libwasmer.a` as libwasmer path.
✔ Native executable compiled successfully to `expr2`.
$ file expr2
expr2: Mach-O 64-bit executable arm64
$ ll -h expr2
-rwxr-xr-x  1 ondrej  staff    20M Feb  3 20:09 expr2

And you can run the binary:

$ ./expr2
25

The expr2.wasm is relatively small (18K) but the expr2 binary is quite big (20M).

You can then disassemble using:

$ /path/to/wabt/build/wasm2wat expr2.wasm -o expr2.wat
$ head -n 20 expr2.wat 
(module
  (type (;0;) (func (param i32 i32 i32) (result i32)))
  (type (;1;) (func (param i32 i64 i32) (result i64)))
  (type (;2;) (func (param i32) (result i32)))
  (type (;3;) (func (param i32 i32) (result i32)))
  (type (;4;) (func (param i32 i64 i32 i32) (result i32)))
  (type (;5;) (func (param i32 i32 i32 i32) (result i32)))
  (type (;6;) (func (param i32)))
  (type (;7;) (func))
  (type (;8;) (func (result i32)))
  (type (;9;) (func (param i32 i32 i32 i32 i32) (result i32)))
  (type (;10;) (func (param i32 i32 i32)))
  (type (;11;) (func (param i32 i32 i32 i32 i32)))
  (type (;12;) (func (param f64 i32) (result f64)))
  (import "wasi_snapshot_preview1" "fd_close" (func $__imported_wasi_snapshot_preview1_fd_close (type 2)))
  (import "wasi_snapshot_preview1" "fd_fdstat_get" (func $__imported_wasi_snapshot_preview1_fd_fdstat_get (type 3)))
  (import "wasi_snapshot_preview1" "fd_seek" (func $__imported_wasi_snapshot_preview1_fd_seek (type 4)))
  (import "wasi_snapshot_preview1" "fd_write" (func $__imported_wasi_snapshot_preview1_fd_write (type 5)))
  (import "wasi_snapshot_preview1" "proc_exit" (func $__imported_wasi_snapshot_preview1_proc_exit (type 6)))
  (func $__wasm_call_ctors (type 7))

So the WASM binary is using exactly 5 imports. The WASM that we generate thus should use exactly these imports and things should work.

Regarding how to print floats: we have to implement it one way or the other.

Shall we implement a function/class/entity in the frontend language that could print formatted integers/floats?

We can, although it would not be used by the frontend (which would continue using the current approach with existing ASR nodes), only by the WASM backend.

Or Shall we implement something similar in the lines of printing integers/floats in the wasm_x64 backend?

Yes, the backend has to implement it. Here are some ideas how to do it:

• It can implement it by using the frontend language, compile using LPython to WASM and the WASM backend would use it
• It can be implemented in C, compiled to WASM using wasi-sdk and used by the WASM backend
• It can be implemented directly in the backend by generating the proper WASM instructions like the wasm_x64 backend does.
• We write an ASR->ASR transformation that transforms the Print node into an actual runtime implementation: we know the types at compile time, so Print([str, int, str]) would transform into a sequence of only strings Print([str, convert_int_to_str(int), str]), where convert_int_to_str is an ASR implementation of the string conversion.

The wasm_x64 backend will thus not print any integers directly anymore, it just implements fd_write (using the API above) that just prints strings, so things becomes simpler for wasm_x64.

Another way to look at it: currently we implement the int to str conversion in the wasm_x64 backend in assembly. The first step is to "lift" it up into the WASM backend to implement in WASM, and the wasm_x64 only implements printing of strings. The next step is to lift it up from the WASM backend into ASR, so WASM backend only prints strings, and ASR implements the conversion. The frontend typically generates Print(int), so it would probably be an ASR pass that does it. That way backends that don't want to implement can just call the pass.

The mechanism to do that is similar to the IntrinsicFunction mechanism that I am working on in: lfortran/lfortran#1045. Until then, just implement the int to str in Python, compile to WASM using your backend, and then you can "semi-manually" call it from the backend, or just implement in WASM by hand, it's probably not that hard, if you did it already in assembly before. That is, take this:

lpython/src/libasr/codegen/x86_assembler.cpp

Line 139 in acd593e

void emit_print_int(X86Assembler &a, const std::string &name)

, and implement it in the WASM backend using WASM instructions.

[certik]

Here is another related issue: I realized today that wasmtime cannot create a standalone binary executable, it does compile WASM to native code, but it relies on various hooks into the wasmtime runtime: bytecodealliance/wasmtime#4563 (comment). Wasmer can create an executable and it works for the simplest examples, as shown above, but the executable is large (bloated). Also the generated binary doesn't seem to have the capability to read files: wasmerio/wasmer#3574, which is a problem. To read files it seems one must run it via wasmtime or wasmer.

The goal of our wasm_x64 backend should be to create a very small lean x64 standalone binary, and if allowed, the binary should be able to read files.

[certik]

Here is an example how to get lean WASM output using Clang: apparently we cannot use the libc library which is bloated (18K):

int main() {
    int x;
    x = (2+3)*5;
    return x;
}

this gives:

$ /path/to/wasi-sdk-19.0/bin/clang -Os expr2.c -o expr2.wasm
$ /path/to/wabt/build/wasm2wat expr2.wasm -o expr2.wat
$ ll expr2.wasm
-rwxr-xr-x  1 ondrej  staff  507 Feb  3 20:57 expr2.wasm
$ wasmtime expr2.wasm 
$ echo $?
25

The expr2.wat is just 507 bytes:

(module
  (type (;0;) (func (param i32)))
  (type (;1;) (func))
  (type (;2;) (func (result i32)))
  (import "wasi_snapshot_preview1" "proc_exit" (func $__imported_wasi_snapshot_preview1_proc_exit (type 0)))
  (func $__wasm_call_ctors (type 1))
  (func $_start (type 1)
    (local i32)
    block  ;; label = @1
      block  ;; label = @2
        i32.const 0
        i32.load offset=1024
        br_if 0 (;@2;)
        i32.const 0
        i32.const 1
        i32.store offset=1024
        call $__wasm_call_ctors
        call $__original_main
        local.set 0
        call $__wasm_call_dtors
        local.get 0
        br_if 1 (;@1;)
        return
      end
      unreachable
      unreachable
    end
    local.get 0
    call $__wasi_proc_exit
    unreachable)
  (func $__original_main (type 2) (result i32)
    i32.const 25)
  (func $__wasi_proc_exit (type 0) (param i32)
    local.get 0
    call $__imported_wasi_snapshot_preview1_proc_exit
    unreachable)
  (func $dummy (type 1))
  (func $__wasm_call_dtors (type 1)
    call $dummy
    call $dummy)
  (table (;0;) 1 1 funcref)
  (memory (;0;) 2)
  (global $__stack_pointer (mut i32) (i32.const 66576))
  (export "memory" (memory 0))
  (export "_start" (func $_start)))

Unfortunately the wasmer created binary does not work:

$ wasmer create-exe expr2.wasm -o expr2
Compiler: cranelift
Target: aarch64-apple-darwin
Format: Symbols
Using path `~/.wasmer/lib/libwasmer.a` as libwasmer path.
✔ Native executable compiled successfully to `expr2`.
$ ll -h expr2
-rwxr-xr-x  1 ondrej  staff    20M Feb  3 20:59 expr2
$ ./expr2
Trap is not NULL: TODO:
$ echo $?
255

The 507 bytes of WASM still generates a 20M binary, I think they are linking the wasmer runtime in it:

$ ll -h ~/.wasmer/lib/libwasmer.a
-rw-r--r--  1 ondrej  staff    52M Jan 26 08:44 ~/.wasmer/lib/libwasmer.a

which makes it bloated. Our backend must generate truly native executable, no runtime like this.

I think part of the reason the wasmer generated binary is so large is because they assume the webassembly code cannot be trusted, so they implement WASM in a "safe" way, handling WASM traps correctly, etc. While we assume the code can be trusted and we care more about the size of the binary and speed of compilation, so this should allow us to create small binaries.

[certik]

@Shaikh-Ubaid given that neither wasmtime nor wasmer can create binaries that robustly work yet (and if they work, they are still about 1,000x larger than they should be), I would not worry about creating binaries using them, but let's use wasmtime and optionally wasmer to test that the WASM that LPython generates runs via their runtime (wasmtime expr2.wasm). And then let's make sure our wasm_x64 can create a small lean binary, that runs and behaves just like wasmtime expr2.wasm.