Add performance statistics display to mlc_llm serve command

[Copilot]

Summary

Adds real-time performance monitoring to the mlc_llm serve command by displaying request completion statistics in the terminal. When a request finishes processing, users will now see performance metrics including token counts and throughput rates.

Changes

• Modified cpp/serve/engine_actions/action_commons.cc:
  • Added #include <cstdio> header for printf functionality
  • Added console output in ProcessFinishedRequestStateEntries function
  • Displays prefill/decode token counts and TPS (tokens per second) calculations
  • Added safety checks to handle edge cases (zero tokens, zero timing)

Example Output

[Request Completed] Prefill: 50 tokens, Decode: 100 tokens, Prefill TPS: 1200.5, Decode TPS: 25.3

Implementation Details

• Minimal impact: Only 18 lines of code added using existing metrics infrastructure
• Safe: Includes proper validation to prevent division by zero and invalid data
• Efficient: Uses existing RequestMetrics data with no additional computation overhead
• Non-blocking: Uses printf with fflush for immediate console output
• Style compliant: Code follows project formatting standards via clang-format

Benefits

• Real-time monitoring: Users can track model performance without external tools
• Performance debugging: Helps identify bottlenecks in prefill vs decode phases
• Production monitoring: Enables live performance assessment during serving

The feature only displays statistics when meaningful data is available (both prefill and decode tokens > 0, with valid timing), ensuring clean output without noise from incomplete requests.

Testing

• Code compiles successfully with existing build system
• Follows project code style guidelines (clang-format compliant)
• Uses existing metrics infrastructure with no breaking changes

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[alohachen]

thanks