Candle-binding: Supporting more embedding models

[rootfs]

Is your feature request related to a problem? Please describe.
Current candle binding supports bert embedding models that usually have limited sequence length (512).
While the Embedding Gemma supports 2K and Qwen3 0.6B embedding models supports up to 32K

Describe the solution you'd like
We should add more embedding models in candle-binding

Describe alternatives you've considered

Additional context

[Xunzhuo]

This is cool, a plugin system for embedding models is important to match different users request for the latency and accuracy

[alvarobartt]

Hey @rootfs, nice to meet you! I'm a maintainer at Hugging Face working on Text Embeddings Inference among other things. Just leaving here a couple of candle-based references that might be useful:

• EmbeddingGemma (note it also requires a couple of Dense layers after the Pooling)
• Qwen3-Embedding

Happy to help if needed 🤗

[rootfs]

@alvarobartt Thank you for the pointer! This is great news, candle is really helpful in supporting new models. If you can lend hands in reviewing, tutoring, or helping getting these embedding models to the candle-binding library, that'll be highly appreciated!

At the moment, we are using bert embeddings here. I have not personally used the gemma or qwen3 embedding models yet.

[rootfs]

cc @OneZero-Y

[zhengkezhou1]

It seems we currently don't have a test for sequence length limiting. Should we add an E2E test for this first?

[rootfs]

@zhengkezhou1 that's a good idea!

[OneZero-Y]

Hi @rootfs,

Sorry, I've been quite busy lately.

After studying the current codebase and researching best practices, I'd like to propose a two-phase approach .

🏗️ Current vs. Proposed Architecture

Current Structure:

candle-binding/src/
├── lib.rs
├── bert_official.rs
├── modernbert.rs
└── unified_classifier.rs

Proposed Architecture:

candle-binding/src/
├── lib.rs                    # Main entry point
├── ffi/                      # C FFI Interface Layer
├── core/                     # Pure Business Logic
├── model_architectures/      # Dual-Path Model Layer
│   ├── traditional/          # Traditional Fine-Tuning Path
│   │   ├── bert.rs           # Enhanced BERT implementation
│   │   └── modernbert.rs     # ModernBERT implementation
│   ├── lora/                 # LoRA High-Performance Path
│   │   ├── bert_lora.rs      # BERT+LoRA implementation
│   │   └── parallel_lora.rs  # Parallel processing
│   └── future/               # Future Model Extensions
│       ├── gemma.rs          # Ready for Gemma integration
│       └── qwen.rs           # Ready for Qwen integration
├── classifiers/              # Classification Systems
│   ├── unified.rs            # Intelligent routing
│   ├── traditional/          # Traditional classifiers
│   └── lora/                 # LoRA classifiers
└── utils/                    # Essential utilities

🚀 Key Advantages of Proposed Architecture

1. Performance Improvements

• Dual-Path Processing: Traditional path for accuracy, LoRA path for speed
• Parallel Multi-Task: Intent + PII + Security classification run concurrently
• Intelligent Routing: Automatic selection of optimal processing path

2. Future Model Integration Made Simple

The modular design makes adding new models straightforward:

Adding Gemma (Example):

// Step 1: Add model architecture
// model_architectures/future/gemma.rs
impl ModelBackbone for GemmaModel {
    fn load(config: &GemmaConfig) -> Result<Self> { /* Implementation */ }
}

// Step 2: Update routing to detect Gemma models
// Automatic integration with existing infrastructure

Adding Qwen follows the same pattern - just add qwen.rs and update the routing logic.

3. Maintainability Benefits

• Clear Separation: Each model type has its own module
• Code Reuse: Shared utilities and interfaces
• Easy Testing: Modular structure enables focused testing
• Backward Compatibility: Existing functionality preserved

🔧 Implementation Approach

Phase 1: Architecture Refactor

1. Restructure existing code into modular architecture
2. Implement dual-path routing system
3. Maintain 100% backward compatibility
4. Add comprehensive testing

Phase 2: New Model Integration

1. Add Gemma support using the new framework
2. Add Qwen support following the same pattern
3. Extend LoRA training scripts for new models

💡 Why This Approach?

1. Future-Proof: Easy to add Gemma, Qwen, or any new model
2. Low Risk: Maintains backward compatibility while improving structure
3. Scalable: Clean architecture supports project growth

🤔 Questions for Discussion

1. Does this approach align with your vision for the project's architecture?
2. Would you like to proceed with Phase 1 (architecture refactor) as a foundation for future model additions?
3. Are there specific requirements for Gemma/Qwen integration that should influence the design?

Thanks for considering this proposal!

[rootfs]

@OneZero-Y yes, this is a much cleaner and scalable idea. If you can help drive the phase 1 refactoring, that'll be great! Thanks for considering this.

[OneZero-Y]

@rootfs Okay, I'll open a pr for phase 1