Add arcee model

[alyosha-swamy]

[New Model] Support Arcee (Arcee Foundational Models)

1. Purpose (Why this PR?)

Add inference support for Arcee Foundational Model (AFM) so that users can serve it with vLLM in both Python and API-server workflows. AFM uses a unique ReLU² activation in its MLP layers, differentiating it from standard Llama-based models.

2. Model details

Field	Value / Reference
Source repo / HF id	huggingface.co/arcee-ai/AFM-4.5B-Base
Architecture	Llama-style decoder-only transformer with ReLU² MLP activation
Context length	64k tokens
Hidden size / #layers	4096 / 32
License	CC BY-NC 4.0
Special quirks	Uses ReLU² (squared ReLU) activation instead of SiLU in MLP layers

3. Implementation overview

• Added ArceeForCausalLM class in vllm/model_executor/models/arcee.py with custom ArceeMLP using ReLU² activation
• Registered model in _TEXT_GENERATION_MODELS in vllm/model_executor/models/registry.py
• Updated docs/models/supported_models.md with Arcee entry in text generation table
• Reused LlamaAttention from existing Llama implementation for attention layers
• Implemented proper LoRA and Pipeline Parallelism support

4. Performance / sanity check

$ python -m vllm.entrypoints.openai.api_server --model arcee-ai/AFM-4.5B-Base --trust-remote-code
$ curl http://localhost:8000/v1/completions -H "Content-Type: application/json" -d '{
    "model": "arcee-ai/AFM-4.5B-Base",
    "prompt": "The future of artificial intelligence is",
    "max_tokens": 50
}'

Expected: Coherent completion about life's meaning

Observed: " a question that has been asked throughout the history of mankind. The search for an answer to this question has inspired countless works of art, literature, and philosophy. Whether we consider the existentialist ideas of Albert Camus or the religious perspectives of spiritual leaders"

5. Test plan ✔️

Test	Command	Expected
Unit	pytest tests/models/test_arcee.py	All tests pass
Model Loading	python -c "from vllm import LLM; llm = LLM('arcee-ai/AFM-4.5B-Base')"	Model loads without errors
Integration	vllm serve arcee-ai/AFM-4.5B-Base --trust-remote-code	Server starts, responds to requests
Generation	curl localhost:8000/v1/completions	200 OK + valid completions

6. Documentation

• Added row to docs/models/supported_models.md under Text Generation models
• Model listed as ArceeForCausalLM with example model arcee-ai/AFM-4.5B-Base
• Marked as supporting LoRA (✅), Pipeline Parallel (✅), and V1 (✅)

Checklist

• I ran pre-commit run --all-files (ruff formatting)
• All CI tests pass locally (pytest -q)
• The PR description follows vLLM's "Essential Elements" template
• No breaking changes for existing model classes

Notes for reviewers

The key architectural difference from standard Llama models is the MLP activation function. Arcee uses ReLU² (squared ReLU) instead of SiLU:

• ArceeMLP implements: x = torch.pow(torch.relu(x), 2)
• No gating mechanism (no gate_proj), only up_proj and down_proj
• All other components (attention, layer norm, etc.) reuse existing Llama implementations

The model has been tested with an internal HF repo during development, but the official model is arcee-ai/AFM-4.5B-Base.

Test result

seq	Prompt	vLLM Output
0	"The meaning of life is"	" a question that has been asked throughout the history of mankind. The search for an answer to this question has inspired countless works of art, literature, and philosophy. Whether we consider the existentialist ideas of Albert Camus or the religious perspectives of spiritual leaders"
1	"Climate change is primarily caused by"	" human activity, specifically the emission of greenhouse gases such as carbon dioxide (CO2) and methane (CH4). It leads to changes in average temperatures and weather patterns, impacting both nature and human society."
2	"Machine learning algorithms work by"	" training a predictive model using labeled training data: the model detects patterns in the training data and learns from it. That model is then tested using a test set, which it must predict to achieve a good accuracy rate."

All outputs are coherent and contextually appropriate.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[alyosha-swamy]

Generated Outputs:

Prompt: 'Hello, my name is'
Output: ' Helen and I am from Shanghai. I’m 15 years old. I’m'

Prompt: 'The president of the United States is'
Output: ' the head of state and head of government of the United States. He or she'

Prompt: 'The capital of France is'
Output: ' Paris. Paris is also the largest city in France. The city of Paris is'

Prompt: 'The future of AI is'
Output: ' being decided now,, and you can make a difference.\nFebruary 21,'

cc @jeejeelee

[gemini-code-assist]

Code Review

The code changes introduce the Arcee model. Refactor the weight loading mechanism to align better with vLLM's standard practices, and correct the test registry entry for the Arcee model.

[gemini-code-assist]

This load_weights method can be improved to fully leverage AutoWeightsLoader and simplify the overall implementation. The custom load_weights method in ArceeModel (lines 275-368) duplicates logic from AutoWeightsLoader and is less maintainable. By adding skip_prefixes and skip_substrs to the AutoWeightsLoader initialization here, you can handle the Arcee-specific weight skipping. This allows for the complete removal of the ArceeModel.load_weights method, resulting in cleaner code that follows vLLM's standard practices.

from vllm.model_executor.models.utils import AutoWeightsLoader
        loader = AutoWeightsLoader(
            self,
            skip_prefixes=(['lm_head.']
                           if self.config.tie_word_embeddings else None),
            skip_substrs=["gate_proj"])
        return loader.load_weights(weights)

[jeejeelee]

overall LGTM, please fix the pre-commit failure

[hmellor]

@alyosha-swamy please stop making new PRs for this model.

It's making discussion about the PR unnecessarily hard to follow:

• Add Arcee (AFM) model support to vLLM #21263
• Add Arcee (AFM) model support to vLLM #21264
• Add Arcee (AFM) model support to vLLM #21265
• Add Arcee (AFM) model support to vLLM #21267
• Arcee model support #21293

[hmellor]

In #21267 it was pointed out that

If the architecture is the same as Llama with the activation function changed it probably will work with --model-impl transformers.

Meaning that it's not necessary to make any changes to vLLM.

---

Repeatedly making new PRs will hide this from reviewers.

[alyosha-swamy]

In #21267 it was pointed out that

If the architecture is the same as Llama with the activation function changed it probably will work with --model-impl transformers.
Meaning that it's not necessary to make any changes to vLLM.

Repeatedly making new PRs will hide this from reviewers.

Understood, however to enable native vLLM support for better performance, it would be better if we could add this

[hmellor]

Have you tested the performance with --model-impl transformers? (you'll probably also need --trust-remote-code because your model is not in the Transformers library yet)

The performance should be close to or the same as a natively supported model.

[alyosha-swamy]

Model outputs are nearly identical; however, the model loads in less than half the time when using it directly versus with the model impl HF flag.

[alyosha-swamy]

AFM is in this release https://github.com/huggingface/transformers/releases/tag/v4.53.0

[hmellor]

Model outputs are nearly identical; however, the model loads in less than half the time when using it directly versus with the model impl HF flag.

How is the performance of the model once loaded with --model-impl transformers?

Thank you for the feedback about model loading speed, this is very useful to know. Some effort could be put into improving this, which would affect all models loaded with --model-impl transformers.

AFM is in this release https://github.com/huggingface/transformers/releases/tag/v4.53.0

Oh I see, thank you for the additional context. When you said that the checkpoint hadn't been released, I assumed the implementation hadn't been released on Transformers either. In that case you wouldn't need --trust-remote-code to use --model-impl transformers.

[alyosha-swamy]

How is the performance of the model once loaded with --model-impl transformers?

I have verified the logprobs for these and can confirm they are matching with the native impl.

Since all the other fully supported models still need to be included in registry.py as well as have their own file, we would like to have this PR implemented.

[hmellor]

Since all the other fully supported models still need to be included in registry.py as well as have their own file, we would like to have this PR implemented.

This isn't actually necessary. Models can be officially supported using the Transformers backend with an entry like:

    "ArceeForCausalLM": ("transformers", "TransformersForCausalLM"),

If it's necessary to add this model explicitly now, I won't block it. But in future we'd prefer not to maintain copies of models from Transformers.

[alyosha-swamy]

I'd prefer to merge it for now, understood for future reference