BlackOps Solver Development Roadmap

[blackopsrepl]

Hello everyone,

I'm excited to share the official development roadmap for the BlackOps Solver project. This post outlines our strategy to deliver high-performance constraint solving for Python, building upon the solid foundation of Timefold Solver.

Core Objective

To provide the Python and Rust ecosystems with a first-class, high-performance Constraint Programming and Optimization solver, offering a seamless experience and unlocking new possibilities for the ML and data science communities.

Core Challenge: The Python-Java Bridge

The primary performance bottleneck in JVM-based Python software is the interoperability layer. Our strategy is a forward-looking, ground-up rewrite to eliminate this bottleneck entirely by migrating the entire interoperability layer to native Rust via PyO3 and JNI, aiming for near-native performance.

---

Roadmap Phases

Phase 1: Stabilize the Legacy Bridge (Present - Q4 2025)

• Objective: Provide a stable and maintained continuation to the Timefold Solver 1.24.0 Python API for current users.
• Key Deliverables:
  • blackops-solver-legacy (Released): A debranded, maintained fork of the original solver. Available on PyPI: pip install blackops-legacy. This serves as a reliable, baseline reference.
  • Quickstart Benchmarks: The blackops-quickstarts repository serves as our central hub for benchmarking and validating performance across different versions and implementations.
• Status: Complete. This phase provides continuity for existing projects.

Phase 2: Alpha Release of Native Rust Solver (Q4 2025 - Q1 2026)

• Objective: Deliver a fully functional alpha version of the solver with a native Rust backend, eliminating the traditional Python-Java bridge and its overhead.
• Key Deliverables:
  • blackops-solver Repository: A from-the-ground-up rewrite using Rust (via PyO3) and JNI for direct, efficient communication with the Timefold JVM core.
  • Full API Compatibility: 100% compatibility with the discontinued Timefold Python API.
  • Comprehensive Lambda Support: A hybrid approach ensuring full compatibility:
    • Simple Lambdas: Direct translation to Java bytecode for native performance.
    • Complex Lambdas: Automatic fallback to efficient proxy classes for full Python compatibility, maintaining feature parity.
  • Target Performance: Aiming for within 0.5% overhead compared to pure Java, a massive reduction from the current interpreter overhead.
• Status: In Active Development. Core infrastructure is implemented and functional.

Phase 3: Production-Ready Rust Solver & Ecosystem Expansion (H1 2026 - Q3 2026)

• Objective: Solidify the Rust solver, achieve production-ready stability, and begin exploring integrations with the wider data ecosystem.
• Key Deliverables:
  • Production Release: A stable, well-tested blackops PyPI package (pip install blackops).
  • Comprehensive Documentation: Dedicated documentation covering installation, migration from Timefold, and advanced usage.
  • Advanced Quickstarts: Examples featuring predictive and ML-integrated scheduling (e.g., using linear regression to forecast resource demand).
  • Native Rust API: Exposing a first-class Rust API for the solver, enabling its use in pure Rust applications.
  • PyO3 API:: Python interface with near-zero overhead
  • HuggingFace Ecosystem:: Publishing Dockerized quickstarts and models on HuggingFace to engage the ML community.

---

How You Can Contribute

This project thrives on community input. Here’s how you can help:

• Test & Benchmark: Try the blackops-legacy package and report issues. Run benchmarks from the quickstarts repo.
• Join the Discussion: Share your thoughts on this roadmap below! What features are most critical for your use case?
• Spread the Word: Star the GitHub repositories and share this project with anyone who might be interested in high-performance optimization in Python.

We believe this focused strategy provides a clear path to a fundamentally faster and more robust solver for the Python and Rust communities. Thank you for your support.

Best regards,

Vittorio Distefano
Principal Maintainer

---

Repositories:

• BlackOps AI Organization
• blackops-solver-legacy
• blackops-quickstarts

Related Links:

• Timefold Solver

[pchtsp]

Ciao Vittorio, I'm interested in contributing with testing if /when needed. It would be great to see a rust implementation of timefold that can be used from python. I'm very familiar with python, not so much (yet?) with rust.

[blackopsrepl]

Ciao Franco, thank you for reaching out!

The upcoming Rust solver aims to provide a Python API that is 100% compatibile with the current one, therefore the alpha release will occur only when capable of running most existing quickstarts.

If you want to participate to earlier testing phases for the core solver, which would indeed be welcome, I could arrange an early access repo and start pushing what works incrementally - but this would be slightly more about internals than quickstart material. Let me know!

On another note, have you checked out the new "fast" quickstarts? They use version 1.24, but are based on a different data model and have shown some performance increase with the generated demo datasets (see benchmarks), but have not been tested at a larger scale, yet!

[pchtsp]

do you prefer for me to run and benchmark the quickstarts? or do you prefer if I benchmark the solver with my own set of problems?
Can I install the solver as a pypi package from python (i.e., pip install git+ssh://[email protected]/BlackOpsAI/blackops-solver-legacy.git)?

I don't have that much experience with the quickstarts. But I can prepare a set of known problems (graph coloring, tsp, vrp, assignment, facility location) where I have other solvers implemented and example problems and benchmark.

Finally, I have bigger, more "complex"/ real life problems, that I can also benchmark: https://github.com/pchtsp/ihtc2024/tree/master/python/ihtc2024/solver/timefold

[blackopsrepl]

You can install by running pip install blackops-legacy and changing your imports accordingly.

As for benchmarks, the current priority is observing performance for the new data model, as used in the 'fast' quickstarts (https://github.com/BlackOpsAI/blackops-quickstarts/tree/main/fast) and in the current, relevant benchmarks (https://github.com/BlackOpsAI/blackops-quickstarts/blob/main/benchmarks/README.md), but with larger scale datasets.
This data model is based on dataclasses during solving + pydantic at the API boundary, with the aim of avoiding runtime validation at each solver move. Therefore, this is more about what data model we suggest to users than about the core solver itself i.e. no need for comparative benchmarks right now!

[pchtsp]

I understand the data model change, which involves avoiding the use of pydantic during solving.
I've been checking the benchmarking report and scripts.
I see that you're calling the solver through the rest api in the benchmarking script. Why not just fill the domain dataclasses and call the SolverManager directly from the benchmarking script? I don't like having to start a webserver just to run the solver. And taking the server out of the picture should help with better, simpler reproducible results (and, maybe running benchmarks in parallel in the future). I can propose an alternative code though a PR.

In any case, if I understand correctly you need me to:

1. corroborate the results you get by running the benchmarking script to test the existing quickstarts + existing demo datasets with the new data model proposal and compare them with the previous one.
2. find larger datasets for the three problems in quick-starts (employee-scheduling, meeting-scheduling and vehicle-routing) to run the benchmark script. Of these, I'm sure we can get easy well-known datasets, especially for the VRP: https://neo.lcc.uma.es/vrp/vrp-instances/capacitated-vrp-instances/ and http://vrp.atd-lab.inf.puc-rio.br/media/com_vrp/instances/Vrp-Set-X.tgz. We can even use the https://github.com/PyVRP/VRPLIB package to import the datasets into the correct data format.

Let me know if I understood correctly or you meant other types of help.

cheers,

Franco