Gemma3 only responding with <unused32>

[MrMilitaryMech]

Which version of LM Studio?
LM Studio 0.3.13

Which operating system?
Windows

What is the bug?
When sending Gemma3 a single image, it will either fail to process the prompt or simply respond with <unused32> repeated infinitely. The response was manually stopped bc it was using up resources

Screenshots
Screenshot of app:

Image prompt:

The chat seen in the screenshot was a new chat made in the hopes that it would fix the issue. There was a previous chat with 3 images.

Logs
There were no developer logs present.

To Reproduce
Steps to reproduce the behavior:

1. Load up the model gemma-3-12b-it
2. Make a new chat
3. Within the chat, prompt it with 3+ images
4. All further responses, even in different chats, will either fail to generate or will return <unused32> repeatedly
5. After this, the model will have to be reloaded to fix the issue

[pteropid]

I can confirm that I am seeing the same issue, although for me it appears to occur randomly. Sometimes the error occurs immediately, sometimes after after a dozen replies.

I'm running LM Studio 0.3.13 (Build 1) on a 9950X system with 192GB RAM and an RTX 4090, llama.cpp runtimes are v1.20.1

I've trialled Gemma 3 12B with Q6_K quant (lmstudio-community) and Q5_K_L quant (bartowski) with a 24,576 token context window. I've also trialled Gemma 3 27B with iQ4_XS quant (bartowski) with an 8,192 token context window. The behaviour is identical across all three models.

I've varied temp between 0.1-1.0, top_k 40 and 64, top_p 0.90 and 0.95, min_p 0.0 and 0.05, repeat penalty 1.0 and 1.1, to no avail.

Sorry I can't be of more help.

[caith-h]

same for me. happens on multiple linux machines. If the error is caused once, the chance of it happening again is increased, almost guaranteed for every subsequent request.
longer context causes it to show up much quicker, usually instantly at 4k context system prompts - if images are included.

happens on a RTX 3090 (24gb vram) with gemma-3-27b Q4_K_M loaded as well as on a machine with multiple rtx 6000 adas (96gb pooled vram) with 27b q8 loaded

happens regardless of context size the model is loaded with (8k, 20k, 40k)

happens regardless of flash attention being enabled or not

llama.cpp is on v1.20.1
LM Studio is on 0.3.13

[Zuzakiin]

Happens for gemma-3-12b Q4_K_M model too.

directly in LM studio

even for chats with only one input picture, and some questions about it, then talking about other things 10+ messages after

LM Studio is on 0.3.13
AMDRyzen5 3600x cpu, 3090 24gb video card

Still, before it errors out, it is awesome.

[nyukers]

I have unused32 in chat with different Gemma3 models every 5 minutes.

[Mozer]

Same error (spam with tag unused32) when using llama-gemma3-cli.exe from llama.cpp package after several messages with images.
So I think it's the problem with llama.cpp. Can anyone open a bug report there?

llama-gemma3-cli.exe -c 8192 -ngl 99 --main-gpu 0 --split-mode layer --tensor-split 0.7,0.3 -fa -m gemma-3-27b-it-Q6_K.gguf --mmproj mmproj-model-f16.gguf

[monksofzero]

Same issue here: lots of spam resulting from prompt. First prompt gets expected results; next prompt results in spam of repeating . System: LM Studio 0.3.13, Gemma-3-27b-it Q4_K_M, Windows 11, dual GPU (3060 and 4060ti). When I also use the smaller Gemma-3-4b-it Q4_K_M model it results in same (first prompt works, subsequent prompts return .

[caith-h]

Vllm implementation doesn't cause this. It's an inference bug, on the llama.cpp side. Will probably be fixed in a few days

…

On Fri, 14 Mar 2025, 17:03 monksofzero, ***@***.***> wrote: Is this an LM Studio issue or a model issue? Same issue here: lots of spam resulting from prompt. First prompt gets expected results; next prompt results in spam of repeating . System: LM Studio 0.3.13, Gemma-3-27b-it Q4_K_M, Windows 11, dual GPU (3060 and 4060ti). When I also use the smaller Gemma-3-4b-it Q4_K_M model it results in same (first prompt works, subsequent prompts return . — Reply to this email directly, view it on GitHub <#520 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4NQIHJGKUPSIMGFNBPIOL2UL4WLAVCNFSM6AAAAABY7CDOGKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRVGEZDEOJWHE> . You are receiving this because you commented.Message ID: ***@***.***> [image: monksofzero]*monksofzero* left a comment (lmstudio-ai/lmstudio-bug-tracker#520) <#520 (comment)> Is this an LM Studio issue or a model issue? Same issue here: lots of spam resulting from prompt. First prompt gets expected results; next prompt results in spam of repeating . System: LM Studio 0.3.13, Gemma-3-27b-it Q4_K_M, Windows 11, dual GPU (3060 and 4060ti). When I also use the smaller Gemma-3-4b-it Q4_K_M model it results in same (first prompt works, subsequent prompts return . — Reply to this email directly, view it on GitHub <#520 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4NQIHJGKUPSIMGFNBPIOL2UL4WLAVCNFSM6AAAAABY7CDOGKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRVGEZDEOJWHE> . You are receiving this because you commented.Message ID: ***@***.***>

[Mozer]

Recent koboldcpp_cu12.exe doesn't have this problem (tested for 30m, lots of images).

ollama has similar problem (it just crashes) after some time.

[GIN] 2025/03/14 - 20:24:02 | 200 | 6.0961414s | 127.0.0.1 | POST "/api/chat"
C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\im2col.cu:72: GGML_ASSERT(src1->type == GGML_TYPE_F32) failed
[GIN] 2025/03/14 - 20:24:10 | 200 | 1.1510863s | 127.0.0.1 | POST "/api/chat"
time=2025-03-14T20:24:10.253+03:00 level=ERROR source=server.go:449 msg="llama runner terminated" error="exit status 0xc0000409"
time=2025-03-14T20:29:15.237+03:00 level=WARN source=sched.go:647 msg="gpu VRAM usage didn't recover within timeout" seconds=5.0550955 model=sha256-377655e65351a68cddfbd69b7c8dc60c1890466254628c3e494661a52c2c5ada

[Mozer]

I think ggml doesn't support F32 operations for GGML_OP_IM2COL. SYCL GGML code has a comment: TODO: add support for the new F32 operations.
CUDA code asserts if src1->type == GGML_TYPE_F32:

https://github.com/ggml-org/llama.cpp/blob/9f2250ba722738ec0e6ab684636268a79160c854/ggml/src/ggml-cuda/im2col.cu#L72C5-L72C46

[wgabrys88]

I can confirm that I am seeing the same issue, although for me it appears to occur randomly. Sometimes the error occurs immediately, sometimes after after a dozen replies.

I'm running LM Studio 0.3.13 (Build 1) on a 9950X system with 192GB RAM and an RTX 4090, llama.cpp runtimes are v1.20.1

I've trialled Gemma 3 12B with Q6_K quant (lmstudio-community) and Q5_K_L quant (bartowski) with a 24,576 token context window. I've also trialled Gemma 3 27B with iQ4_XS quant (bartowski) with an 8,192 token context window. The behaviour is identical across all three models.

I've varied temp between 0.1-1.0, top_k 40 and 64, top_p 0.90 and 0.95, min_p 0.0 and 0.05, repeat penalty 1.0 and 1.1, to no avail.

Sorry I can't be of more help.

randomly for me also

[Lunaman06]

Ok - have the same problem. 4B Q8 on MacOS, V0.3.13 (Build2)
To me it is not exactly repeatable.
If I start a new chat and first have some text only prompts and add than a picture, I get and than subsequently always.
If I unload and reload the same model, start the conversation with the same picture, I get the a proper response.
Any subsequent use of images is than working to an extend before it breaks again.

[Zengrath]

Same issue here. it is fairly repeatable but it required many inputs in a same session before it happens, usually around 6-8 inputs it happens majority of time. When the spam occurs, reloading model and regenerating only works once. Also interestingly even if you start a completely new chat without reloading the model it continues this issue. Temporary fix is to completely clear or start new chat, then reload model, at this point it'll work for the next 6-8 entries again before it breaks.
This is all in LM Studio 0.3.13d (build 1) and on a rtx 4090 and 64gb ram and windows 11

[HoneyBerries]

Same, I am getting annoyed. UHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH

[nevoin]

i tried set old version llama, but it is not support gemma 3 (
is this only CUDA problem? Vulkan too?

[mattjcly]

Thank you all for the detailed reports. We are investigating to fix ASAP and will keep you all posted on this issue.