review: PR #1436

[Pouyanpi]

Cache:

• Added create_normalized_cache_key() utility for consistent cache key generation with whitespace normalization
• Renamed capacity_per_model → maxsize in ModelCacheConfig for clarity
• Implemented _create_model_cache() and _initialize_model_caches() in LLMRails for dynamic cache creation
• Registered caches as model_caches dict in runtime action parameters

Content Safety Integration:

• Modified content_safety_check_input() to use centralized cache registry (model_caches)
• Cache entries now store both result and LLM stats for accurate metrics restoration
• Added _restore_llm_stats_from_cache() and _extract_llm_stats_for_cache() helpers
• Cache hits restore duration, token counts, and set from_cache=True flag

Observability:

• Added from_cache field to LLMCallInfo for cache hit tracking
• Enhanced span extraction to include from_cache in custom attributes
• Enabled tracing in content safety example config

Testing:

• Added test_cache_utils.py with comprehensive normalization tests
• Added test_content_safety_cache.py with cache storage/retrieval tests
• Added test_integration_cache.py with end-to-end integration tests
• Added test_llmrails.py tests for cache initialization and config validation
• Updated existing cache tests to use new maxsize parameter

Test Plan:

run https://gist.github.com/Pouyanpi/3461b5310a17da17097280b5efc9cebe

TODO:

• push the restructuring commit that moves cache to llm package once review is done.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 97.19626% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
nemoguardrails/cache/utils.py	95.52%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[hazai]

Approved with some comments