[Bug] android app crashes

[randomshinichi]

🐛 Bug

Android app crashed while printing out a response to my question.

To Reproduce

Steps to reproduce the behavior:

1. Install android app on Galaxy S22 Ultra
2. Ask a question
3. Answer interrupted by crash

MLCChat failed

Stack trace:
org.apache.tvm.Base$TVMError: InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure, attempted to allocate 4202496 at offset 0 in region that is 4194304bytes
Stack trace:
  File "/Users/houbohan/tvm/src/runtime/relax_vm/memory_manager.cc", line 115

	at org.apache.tvm.Base.checkCall(Base.java:173)
	at org.apache.tvm.Function.invoke(Function.java:130)
	at ai.mlc.mlcllm.ChatModule.decode(ChatModule.java:71)
	at ai.mlc.mlcchat.AppViewModel$ChatState$requestGenerate$1$2.invoke(AppViewModel.kt:654)
	at ai.mlc.mlcchat.AppViewModel$ChatState$requestGenerate$1$2.invoke(AppViewModel.kt:653)
	at ai.mlc.mlcchat.AppViewModel$ChatState.callBackend(AppViewModel.kt:537)
	at ai.mlc.mlcchat.AppViewModel$ChatState.requestGenerate$lambda$4(AppViewModel.kt:653)
	at ai.mlc.mlcchat.AppViewModel$ChatState.$r8$lambda$lluIrcsPALEW5nCb2tohZYadhTY(Unknown Source:0)
	at ai.mlc.mlcchat.AppViewModel$ChatState$$ExternalSyntheticLambda2.run(Unknown Source:6)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:463)
	at java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:637)
	at java.lang.Thread.run(Thread.java:1012)


Error message:
InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure, attempted to allocate 4202496 at offset 0 in region that is 4194304bytes
Stack trace:
  File "/Users/houbohan/tvm/src/runtime/relax_vm/memory_manager.cc", line 115

Expected behavior

Environment

• Android
• Device: Galaxy S22 Ultra
  

Additional context

[Sing-Li]

Interesting. This is the same error we get on Pixel 6 and 7 when trying to start the Llama2 model (even though we have enough memory to run it).

InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure, attempted to allocate 4202496 at offset 0 in region that is 4194304bytes
Stack trace:
File "/Users/houbohan/tvm/src/runtime/relax_vm/memory_manager.cc", line 115

@spectrometerHBH (sorry the error message has your name on it 🙂 ) the new APK crashes even Qualcomm devices after a while. Perhaps this is related to the _0 vs _1 format switch 🤔

[spectrometerHBH]

@Sing-Li Thanks for the input. I'll take a look.

[spectrometerHBH]

Hi, we just updated the Android apk. https://github.com/mlc-ai/binary-mlc-llm-libs/blob/main/mlc-chat.apk
Hopefully, it fixed the OOM crash issue. It's caused by some bug introduced when we were developing multi-GPU support.