Skip to content

Conversation

@ixchio
Copy link

@ixchio ixchio commented Dec 6, 2025

Add built-in health-aware middleware for LangChain v1 agents that monitors tool/model health and provides automatic retry, fallback, and telemetry capabilities.

Fixes #34193

Features

  • HealthStats: Rolling window statistics tracking per-tool/model (error rate, P95 latency)
  • HealthPolicy: Configurable degradation thresholds
  • HealthMiddleware: Auto retry with exponential backoff, model fallback, tool disabling
  • Telemetry: Pluggable emitter with create_langsmith_emitter() helper
  • Persistence: Optional store integration for cross-run stats via store param
  • Provider tool preservation: Keep provider dict tools active even when degraded

Usage

from langchain.agents.middleware import HealthMiddleware, HealthPolicy

health = HealthMiddleware(
    policy=HealthPolicy(max_error_rate=0.2, consecutive_failures=3),
    fallback_model="openai:gpt-4o-mini",
    disable_tools=True,
)

agent = create_agent(model="openai:gpt-4o", tools=[...], middleware=[health])

Testing
35 unit tests covering HealthStats, HealthPolicy, HealthMiddleware
All 534 middleware tests pass

@github-actions github-actions bot added langchain `langchain` package issues & PRs feature For PRs that implement a new feature; NOT A FEATURE REQUEST labels Dec 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature For PRs that implement a new feature; NOT A FEATURE REQUEST langchain `langchain` package issues & PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add built-in health-aware middleware for tools/models with auto retry/fallback

1 participant