Skip to content

[Bug] REST API broken from recent code updates #1116

@Sing-Li

Description

@Sing-Li

🐛 Bug

Following the latest doc to install mlc_chat.rest from nightly build, the script will crash when started with a model (doing --help works fine)

To Reproduce

Steps to reproduce the behavior:

  1. create a new conda environment with python 3.11
  2. follow latest doc to install nightly pip wheels
  3. run the mlc_chat.rest on a small model
python -m mlc_chat.rest --model  mlc-chat-RedPajama-INCITE-Chat-3B-v1-q4f16_1
INFO:     Started server process [18484]
INFO:     Waiting for application startup.
ERROR:    Traceback (most recent call last):
  File "C:\Users\allmy\miniconda3\envs\restapi\Lib\site-packages\starlette\routing.py", line 677, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "C:\Users\allmy\miniconda3\envs\restapi\Lib\contextlib.py", line 204, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\allmy\miniconda3\envs\restapi\Lib\site-packages\mlc_chat\rest.py", line 115, in lifespan
    chat_mod = ChatModule(
               ^^^^^^^^^^^
TypeError: ChatModule.__init__() got an unexpected keyword argument 'lib_path'

ERROR:    Application startup failed. Exiting.

Expected behavior

REST api to work as it had before recent code changes.

Environment

  • Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA): Vulkan
  • Operating system (e.g. Ubuntu/Windows/MacOS/...): Windows 11
  • Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...) Nvidia 2050
  • How you installed MLC-LLM (conda, source): pip wheel
  • How you installed TVM-Unity (pip, source): pip wheel
  • Python version (e.g. 3.10): 3.11
  • GPU driver version (if applicable):
  • CUDA/cuDNN version (if applicable):
  • TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):
  • Any other relevant information:

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugConfirmed bugs

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions