-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Closed
Labels
bugConfirmed bugsConfirmed bugs
Description
🐛 Bug
Following the latest doc to install mlc_chat.rest
from nightly build, the script will crash when started with a model (doing --help
works fine)
To Reproduce
Steps to reproduce the behavior:
- create a new conda environment with python 3.11
- follow latest doc to install nightly pip wheels
- run the mlc_chat.rest on a small model
python -m mlc_chat.rest --model mlc-chat-RedPajama-INCITE-Chat-3B-v1-q4f16_1
INFO: Started server process [18484]
INFO: Waiting for application startup.
ERROR: Traceback (most recent call last):
File "C:\Users\allmy\miniconda3\envs\restapi\Lib\site-packages\starlette\routing.py", line 677, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "C:\Users\allmy\miniconda3\envs\restapi\Lib\contextlib.py", line 204, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\allmy\miniconda3\envs\restapi\Lib\site-packages\mlc_chat\rest.py", line 115, in lifespan
chat_mod = ChatModule(
^^^^^^^^^^^
TypeError: ChatModule.__init__() got an unexpected keyword argument 'lib_path'
ERROR: Application startup failed. Exiting.
Expected behavior
REST api to work as it had before recent code changes.
Environment
- Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA): Vulkan
- Operating system (e.g. Ubuntu/Windows/MacOS/...): Windows 11
- Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...) Nvidia 2050
- How you installed MLC-LLM (
conda
, source): pip wheel - How you installed TVM-Unity (
pip
, source): pip wheel - Python version (e.g. 3.10): 3.11
- GPU driver version (if applicable):
- CUDA/cuDNN version (if applicable):
- TVM Unity Hash Tag (
python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))"
, applicable if you compile models): - Any other relevant information:
Additional context
Metadata
Metadata
Assignees
Labels
bugConfirmed bugsConfirmed bugs