[model] Add support for Mistral models

[orionpapadakis]

Summary

This PR integrates the Mistral LLM into the GPULlama3 repository.

To support this integration, several architectural changes and refactorings were made to promote component abstractions and a modular, extensible design.

---

Key Changes

1. Model Abstraction

• The original Llama class has been refactored into a Model interface (under a new model package).
• Llama-specific functionality is moved to model.llama.Llama.
• A new implementation for Mistral is introduced in model.mistral.Mistral.

2. Tokenizer Abstraction

• Introduced a Tokenizer interface under the tokenizer.impl package.

• Implemented two tokenizers:

  • LlamaTokenizer for GPT-2-style BPE.
  • MistralTokenizer for TikToken-style BPE.

• The Vocabulary class has been relocated to the tokenizer package.

3. ChatFormat Abstraction

• ChatFormat functionality has been refactored into an abstract form to support model-specific formatting and enable future extensions.

4. Inference Refactoring

• Inference logic is decoupled from the model classes.

• Introduced a dedicated inference package with:

  • InferenceEngine: Entry point for token generation (generateToken, generateTokenGPU methods).
  • InferenceCore: Contains reusable core operations (e.g., rmsnorm, forward, etc.).

[CLAassistant]

All committers have signed the CLA.

[Copilot]

Pull Request Overview

This PR integrates the Mistral LLM into the GPULlama3 codebase by introducing model and tokenizer abstractions, extending chat formatting logic, and refactoring inference loading.

• Refactor Llama into a generic Model interface and add Mistral implementation
• Introduce Tokenizer interface with LlamaTokenizer and MistralTokenizer
• Abstract ChatFormat for both Llama and Mistral and enhance ModelLoader to detect and load each

Reviewed Changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
model/llama/LlamaConfiguration.java	New record implementing Configuration for Llama
model/llama/Llama.java	Updated to implement generic Model interface
model/format/ChatFormat.java	Interface and factory to select Llama/Mistral format
model/format/MistralChatFormat.java	New chat formatter for Mistral-specific tokens
model/format/LlamaChatFormat.java	Refined Llama chat formatting under ChatFormat
loader/weights/ModelLoader.java	Detects GGUF metadata and loads the appropriate model
Model.java	Unified inference entry points under the Model interface

Comments suppressed due to low confidence (2)

src/main/java/com/example/model/format/LlamaChatFormat.java:60

• This loop refers to LlamaChatFormat.Message, but Message is defined in the ChatFormat interface. Change to for (ChatFormat.Message message : dialog) to match the interface type.

for (LlamaChatFormat.Message message : dialog) {

src/main/java/com/example/Model.java:103

• List<Integer> does not have getLast() or removeLast() methods. Use responseTokens.get(responseTokens.size() - 1) and responseTokens.remove(responseTokens.size() - 1) instead.

if (!responseTokens.isEmpty() && stopTokens.contains(responseTokens.getLast())) {

[mikepapadim]

Is there any other way to detect the model here instead of checking sizes?

[orionpapadakis]

indeed checking the vocab size seems odd, but, it should be noted that it's the third check in line and just acts as a fallback in case the model name (1st check) and the tokenizer metadata (2nd check) are not enough. imho, we can keep only the model name check.

[mikepapadim]

Fixes #18

[mikepapadim]

One more thing! Can you update the README with the link and instruction to download the mistral model ->https://huggingface.co/beehive-lab

[orionpapadakis]

One more thing! Can you update the README with the link and instruction to download the mistral model ->https://huggingface.co/beehive-lab

done

[mikepapadim]

@orionpapadakis thanks LGTM, let me test it and add the changes for the normalization layers, then we can merge.