Install on Pyodide/JupyterLite

[deeplook]

It would be really cool to run leafmap on Pyodide and hence JupyterLite using micropip.

I gave it a try and found basically two issues with non-existing Python 3 source wheels (needed by micropip) for pycrs and pyshp on PyPI for which I've created these tickets (but there might be wheels for them hosted somewhere else): karimbahgat/PyCRS#56 and GeospatialPython/pyshp#217. And here-map-widget-for-jupyter can be installed, too, but still has some issue with the JupyterLab extension, as described on heremaps/here-map-widget-for-jupyter#25. Folium can be installed in a somewhat complicated manner, see the notebook pyolite folium.ipynb on JupyterLite.

Not sure if anything else is needed... ;)

[giswqs]

@deeplook Thanks a lot for looking into this. It would be super awesome to have leafmap run on JupyterLite. Are the source wheels required by micropip the same as the one hosted on PyPI? I just built the pyshp wheel and made it available at https://spatial.utk.edu/download/pyshp-2.1.3-py3-none-any.whl. However, it seems micropip failed to fetch the wheel. I also tried uploading the wheel to JupyterLite and installed it, but it failed as well. Any advice? @jtpio @martinRenou

await micropip.install('https://spatial.utk.edu/download/pyshp-2.1.3-py3-none-any.whl')

await micropip.install('pyshp-2.1.3-py3-none-any.whl')

[deeplook]

I'm pretty new to this, too. The geojson package which is not preinstalled in JupyterLite can be installed with either of these lines (and the wheel was created in August 2019, see https://pypi.org/project/geojson/#files ;):

await micropip.install("https://files.pythonhosted.org/packages/e4/8d/9e28e9af95739e6d2d2f8d4bef0b3432da40b7c3588fbad4298c1be09e48/geojson-2.5.0-py2.py3-none-any.whl")

await micropip.install("geojson")

There is also this description for creating a Pyodide package, but I need to digest more of it...

And then, I read somewhere in the Pyodide issues on Github (I believe) that micropip will load the wheels using HTTP range requests, which maybe not all servers do support...

[jtpio]

Yes it would be interesting to double check installing pyshp from another server.

The content type seems to be differing:

$ curl -i https://spatial.utk.edu/download/pyshp-2.1.3-py3-none-any.whl
HTTP/1.1 200 OK
Date: Wed, 23 Jun 2021 20:53:41 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips
Vary: User-Agent,Accept-Encoding
Last-Modified: Wed, 23 Jun 2021 17:05:46 GMT
ETag: "9190-5c571eaedc79d"
Accept-Ranges: bytes
Content-Length: 37264
Content-Type: application/x-troff-man

and

$ curl -i https://files.pythonhosted.org/packages/e4/8d/9e28e9af95739e6d2d2f8d4bef0b3432da40b7c3588fbad4298c1be09e48/geojson-2.5.0-py2.py3-none-any.whl
HTTP/2 200
last-modified: Wed, 26 Feb 2020 17:59:43 GMT
etag: "183b6f5ccb61818543f29c62a47eff16"
content-type: binary/octet-stream
server: UploadServer
cache-control: max-age=365000000, immutable, public
accept-ranges: bytes
date: Wed, 23 Jun 2021 20:55:48 GMT
age: 1251652
x-served-by: cache-sea4481-SEA, cache-hhn4082-HHN
x-cache: HIT, HIT
x-cache-hits: 1, 1
x-timer: S1624481749.825108,VS0,VE1
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-frame-options: deny
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
x-robots-header: noindex
access-control-allow-methods: GET
access-control-allow-origin: *
content-length: 14839

[deeplook]

There is also this pyodide/pyodide#1653 (comment) in pyodide/pyodide#1653 linking such installation issues with coors and specific browser versions.

[giswqs]

@deeplook This is great to know. Should we wait for JupyterLite to incorporate the latest pyodide?

[jtpio]

@giswqs @deeplook this PR updates to the latest (dev) version of Pyodide: https://github.com/jtpio/jupyterlite/pull/178

Would you be able to test on the RTD preview? https://jupyterlite--178.org.readthedocs.build/en/178/_static/lab/index.html

[deeplook]

@jtpio Sorry, but I seem to wait forever to import micropip there.

[jtpio]

Ah, probably something that changed in pyodide?

[RandomFractals]

well, I got it to work with a custom leaflet map renderer in vscode: https://github.com/RandomFractals/vscode-leaflet#pyolite-notebook-example

only using Pyolite to load geoJSON tho :(

[Niko-La]

anyone get leafmap to install via pywheel or micro on jupyterlite. this would be great to skip the binder builds. I believe lot of new features are coming in pyodide that should make it easier.

[deeplook]

@giswqs Meanwhile it is possible to run geopandas, shapely, fiona and gdal inside pyodide, which seems like providing a big step further in running more of leafmap inside the browser! See pyodide/pyodide#1569 and pyodide/pyodide#3213 (with a test using pyscript in the comments of the latter).

[giswqs]

@deeplook This is awesome! I will look into it this weekend. PR is also welcome!