Jupyterlite integration

[davy39]

Hello,
I'm trying to compile and install this kernel to built it with Jupyterlite.
Finally, I think the build is good, but there is an error when I try to integrate it inside Jupyterlite :

$ jupyter lite build --XeusAddon.prefix=/home/davy/miniforge3/envs/xeus-cpp
....
/home/davy/miniforge3/envs/xeus-cpp/lib/python3.12/site-packages/jupyterlite_xeus/add_on.py:56: UserWarning: kernel binaries not found for xcpp
  warnings.warn(f"kernel binaries not found for {path.name}")

Any idea on why this is not detecting xcpp bin which have been build and install ?

Any chance we can get it out of the box soon ?

[vgvassilev]

Looks like we are not installing the xcpp binary at the expected place. Do you know where it is expected to be?

[davy39]

Thanks for your answer.
I had a look at jupyterlite-xeus addon sourcefile and it looks like it's looking for xcpp.js and xcpp.wasm files which are not build.

        kernel_binary_js = Path(kernel_binary + ".js")
        kernel_binary_wasm = Path(kernel_binary + ".wasm")

        if kernel_binary_js.exists() and kernel_binary_wasm.exists():
            return kernel_binary_js, kernel_binary_wasm
        else:
            warnings.warn(f"kernel binaries not found for {path.name}")

Sorry I'm very new to this... Is there a way to get a working cpp kernel in jupyterlite yet ?

[vgvassilev]

Ha, okay, we do not have such files. I do not know whole a lot about wasm but let me ping the experts ;)

@JohanMabille, @anutosh491, @mcbarton.

[davy39]

I read more carefully Jupyterlite xeus doc and setup emsdk to build the js kernel.

I got the following error, due to upgrade from xeus 3->4 :

[ 72%] Building CXX object CMakeFiles/xeus-cpp-static.dir/src/xinterpreter_wasm.cpp.o
/home/davy/Bureau/xeus-cpp-0.4.0/src/xinput.cpp:36:69: error: too few arguments to function call, expected 3, have 2
   36 |                       value = xeus::blocking_input_request("", false);
      |                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~          ^

When I revert to xeus=3.2.0 and xeus-lite=1.0.3, I got plenty errors of this kind :

error: undefined symbol: LLVMInitializeWebAssemblyAsmPrinter (referenced by root reference (e.g. compiled C/C++ code))
warning: To disable errors for undefined symbols use `-sERROR_ON_UNDEFINED_SYMBOLS=0`
warning: _LLVMInitializeWebAssemblyAsmPrinter may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: LLVMInitializeWebAssemblyTarget (referenced by root reference (e.g. compiled C/C++ code))
warning: _LLVMInitializeWebAssemblyTarget may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
error: undefined symbol: LLVMInitializeWebAssemblyTargetInfo (referenced by root reference (e.g. compiled C/C++ code))
warning: _LLVMInitializeWebAssemblyTargetInfo may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library

I guess it's not functional right now. It's far from my skills, so I'll have to wait before playing with c++ on Jupyterlite ...

I already can play with python, js, ocaml and sql :)

Thnaks for your work !

[JohanMabille]

Hi,

Thanks for tackling this! Not sure what the issue with JupyterLite integration is, but the code base shoud not be upgraded to xeus 4 yet; we need to upgrade other dependencies before.

[davy39]

Thanks. I cannot build it even with xeus 3, but as I said, I'm very new to this.

Is that working for you ? Have you got a procedure to make it work, or at least a working build of jupyterlite for me to play with ?

[davy39]

Hi,
Thanks for guiding me to CI : indeed Jupyterlite actually builds with C++ kernels. I got it with this artifact.
However when I try to run this jupyterlite build locally, I can choose C++ kernels, but they don't work : I'm not even able to declare variables.
Am I missing something ?

[vgvassilev]

If you are referring to being able to run with wasm inside a web browser we know that's still broken and we have a plan how to fix it. That requires some changes to llvm.

[davy39]

Ok, thanks for your answer. I'll wait for you to work this out then.

[anutosh491]

Hello @davy39

I was trying out the jupyterlite integration. Indeed JupyterLite builds the c++ kernel and keeps it in the _output directory. I am facing some errors running the build locally though. I am doing it correctly ? (or did you encounter any similar error)

(xeus-lite-host) anutosh491@spbhat68:~/QuantStack/xeus-cpp/build$ jupyter lite serve --port=8888
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:         jupyterlite-app-0.3.0.tgz 13MB
    output:          /home/anutosh491/QuantStack/xeus-cpp/build/_output
    lite dir:        /home/anutosh491/QuantStack/xeus-cpp/build
    apps:            
    sourcemaps:      True
    unused packages: True
archive:archive
contents:contents
icons:icons
lite:jupyter-lite.json
mimetypes:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteServeApp] No archive (yet): build-jupyterlite.tgz
.  status:contents:contents
    contents: 0 files
.  status:icons:icons
    favicon files: 7 files
.  status:lite:jupyter-lite.json
[LiteServeApp]     jupyter-lite.(json|ipynb): 0 files
.  status:mimetypes:jupyter-lite.json
    filetypes:         26 
.  status:serve:contents
    url: http://127.0.0.1:8888/
    server: tornado
    headers:
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 2 files
static:output_dir
-- pre_init:static:output_dir
static:unpack
-- init:static:unpack
federated_extensions:copy:ext:jupyterlab_pygments
federated_extensions:copy:ext:@jupyterlite/xeus
-- pre_build:federated_extensions:copy:ext:jupyterlab_pygments
-- pre_build:federated_extensions:copy:ext:@jupyterlite/xeus
icons:copy
translation:copy
.  build:icons:copy
.  build:translation:copy
federated_extensions:patch
federated_extensions:settings
Channels:
 - conda-forge
 - https://repo.mamba.pm/emscripten-forge
 - nodefaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
    current version: 24.1.2
    latest version: 24.5.0

Please update conda by running

    $ conda update -n base -c conda-forge conda



## Package Plan ##

  environment location: /tmp/tmp9rowkcua/env/envs/xeus-env



Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate /tmp/tmp9rowkcua/env/envs/xeus-env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Channels:
 - conda-forge
 - https://repo.mamba.pm/emscripten-forge
 - nodefaults
Platform: emscripten-wasm32
Collecting package metadata (repodata.json): done
Solving environment: / warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
- warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
warning  libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
failed

LibMambaUnsatisfiableError: Encountered problems while solving:
  - nothing provides __win needed by ipython-8.4.0-pyh08f2357_1

Could not solve for environment specs
The following package could not be installed
└─ xeus-python is not installable because it requires
   └─ ipython but there are no viable options
      ├─ ipython [8.10.0|8.11.0|...|8.9.0] would require
      │  └─ __win, which is missing on the system;
      ├─ ipython [8.10.0|8.11.0|...|8.9.0] would require
      │  └─ __linux, which is missing on the system;
      ├─ ipython [8.10.0|8.11.0|...|8.9.0] would require
      │  └─ __osx, which is missing on the system;
      ├─ ipython [8.18.1|8.19.0|...|8.24.0] would require
      │  └─ __unix, which is missing on the system;
      └─ ipython [8.14.0|8.20.0|...|8.24.0] conflicts with any installable versions previously reported.

[LiteServeApp] ERROR | [lite] [post_build] [jupyterlite-xeus] [ERR] Command '['/home/anutosh491/conda_root/condabin/conda', 'install', '--yes', '--prefix', PosixPath('/tmp/tmp9rowkcua/env/envs/xeus-env'), '-c', 'conda-forge', '-c', 'https://repo.mamba.pm/emscripten-forge', 'xeus-python']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/doit_cmd.py", line 294, in run
    return command.parse_execute(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/cmd_base.py", line 150, in parse_execute
    return self.execute(params, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/cmd_base.py", line 570, in execute
    return self._execute(**exec_params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/cmd_run.py", line 265, in _execute
    return runner.run_all(self.control.task_dispatcher())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/runner.py", line 254, in run_all
    self.run_tasks(task_dispatcher)
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/runner.py", line 213, in run_tasks
    node = task_dispatcher.generator.send(node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/control.py", line 629, in _dispatcher_generator
    next_step = node.step()
                ^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/control.py", line 336, in step
    return next(self.generator)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/control.py", line 345, in _func
    for value in decorated(*args, **kwargs):
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/control.py", line 473, in _add_task
    new_tasks = generate_tasks(to_load, task_gen, ref.__doc__)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/loader.py", line 390, in generate_tasks
    for task_dict, x_doc in flat_generator(gen_result, gen_doc):
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/doit/loader.py", line 27, in flat_generator
    for item in gen:
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_core/manager.py", line 138, in _delayed_gather
    yield from _gather()
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_core/manager.py", line 131, in _gather
    raise error
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_core/manager.py", line 123, in _gather
    for task in getattr(addon, attr)(self):
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_xeus/add_on.py", line 116, in post_build
    self.create_prefix()
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_xeus/add_on.py", line 142, in create_prefix
    create_conda_env_from_specs(
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_xeus/create_conda_env.py", line 69, in create_conda_env_from_specs
    _create_conda_env_from_specs_impl(
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_xeus/create_conda_env.py", line 129, in _create_conda_env_from_specs_impl
    return _create_env_with_config(CONDA_COMMAND, prefix_path, specs, channels_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/site-packages/jupyterlite_xeus/create_conda_env.py", line 144, in _create_env_with_config
    subprocess_run(
  File "/home/anutosh491/y/envs/xeus-lite-host/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/anutosh491/conda_root/condabin/conda', 'install', '--yes', '--prefix', PosixPath('/tmp/tmp9rowkcua/env/envs/xeus-env'), '-c', 'conda-forge', '-c', 'https://repo.mamba.pm/emscripten-forge', 'xeus-python']' returned non-zero exit status 1.

The port 8888 is even blocked but I don't see the jupyter lite frontend.