Skip to content

Conversation

Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Oct 2, 2025

Stack Info

This PR is part of a stack:

  1. refactor!: drop reasoning trace extraction logic #1427
  2. feat: add reasoning trace extraction from llm calls #1431
  3. feat: emit BotThinking events with reasoning traces #1432
  4. feat(logging): improve event logging and add bot thinking display #1434 (this PR)

⚠️ Depends on:

#1427#1431#1432

Description

Enhanced event logging in the runtime and callbacks to better display bot thinking events with reasoning traces. Moved event logging to occur after event generation for proper ordering, added reasoning content extraction from LLM completions, and improved console output formatting for different event types.

Changes

  • nemoguardrails/colang/v1_0/runtime/runtime.py: Moved event logging to occur after events are generated to ensure proper logging order
  • nemoguardrails/logging/callbacks.py: Extract and include reasoning_content from ChatGeneration messages in completion logs
  • nemoguardrails/logging/verbose.py: Add special formatting for BotThinking events and improve action event display
  • tests/test_runtime_event_logging.py: Add comprehensive tests for event logging behavior including BotThinking, BotMessage, ContextUpdate events and ordering

@Pouyanpi Pouyanpi self-assigned this Oct 2, 2025
@Pouyanpi Pouyanpi added this to the v0.18.0 milestone Oct 2, 2025
@codecov-commenter
Copy link

codecov-commenter commented Oct 2, 2025

Codecov Report

❌ Patch coverage is 86.66667% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
nemoguardrails/logging/callbacks.py 88.88% 1 Missing ⚠️
nemoguardrails/logging/verbose.py 50.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Collaborator

@tgasser-nv tgasser-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few small comments.

BEFORE MERGING: Could you run a local integration test with all the reasoning-models we claim to support? Copy-and-paste the output into the description as a test-plan. We need to check this new approach works with production models

@Pouyanpi Pouyanpi force-pushed the feat/bot-thinking-events branch from 8ae50ef to 98c8911 Compare October 13, 2025 08:13
Base automatically changed from feat/bot-thinking-events to develop October 13, 2025 09:04
@Pouyanpi Pouyanpi force-pushed the feat/runtime-event-logging branch from 341297f to ead6ff7 Compare October 13, 2025 09:08
@Pouyanpi Pouyanpi force-pushed the feat/runtime-event-logging branch from ead6ff7 to bc47bea Compare October 13, 2025 09:08
@Pouyanpi Pouyanpi merged commit d660d66 into develop Oct 13, 2025
7 checks passed
@Pouyanpi Pouyanpi deleted the feat/runtime-event-logging branch October 13, 2025 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants