Add comprehensive documentation for both tag-based routing and the new conditional routing feature. Fixes #2199.

[eschabell]

Add comprehensive documentation for both tag-based routing and the new conditional routing feature introduced in Fluent Bit 4.0. Fixes #2199.

Summary by CodeRabbit

• Documentation
  • Reworked routing docs into two clear approaches: Tag-based routing and a full Conditional routing chapter.
  • Expanded Tag-based routing with wildcard and regex matching and concrete configuration examples.
  • Added comprehensive Conditional routing syntax: per-record routing, named routes, operators, signal types, defaults and evaluation semantics.
  • Numerous practical examples and guidance on when to use or combine each routing approach.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Reworks pipeline/router.md to separate tag-based routing and add a full Conditional routing chapter: introduces Tag and Match concepts (wildcards/regex), a new routes block with per-record routing, condition syntax/operators/signals, many YAML/classic examples, and guidance for choosing approaches.

Changes

Cohort / File(s)	Summary
Router documentation expansion pipeline/router.md	Reorganized routing narrative into two mechanisms: Tag-based routing (Tag & Match, wildcard/regex examples) and Conditional routing (new routes block, per-record routing toggle per_record_routing, route parameters, condition operators and signals). Added YAML/classic config examples, regex/multi-condition samples, operator tables, and a "Choosing a routing approach" guidance section.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Source as Data source
  participant Router as Router docs
  participant TagMatch as Tag matcher
  participant CondEval as Condition evaluator
  participant Output as Output plugins

  rect rgb(250,250,240)
    note left of Router `#f7f3c6`: Tag-based routing (chunk-level)
    Source->>Router: emit chunk (tag + records)
    Router->>TagMatch: match by tag / wildcard / regex
    alt tag matched
      TagMatch->>Output: route whole chunk to outputs
    else no tag match
      TagMatch-->>Router: pass to conditional routing phase
    end
  end

  rect rgb(240,250,250)
    note left of Router `#e6f7f7`: Conditional routing (per-record)
    Router->>CondEval: evaluate `routes` block per record
    loop per record
      CondEval->>CondEval: evaluate condition(s) & operators
      alt condition true
        CondEval->>Output: send record to specified outputs (may include multiple)
      else condition false
        CondEval-->>Router: try next route or default route
      end
    end
  end

  Note right of Output: Outputs may receive whole chunks (tag-based) or individual records (conditional)

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

• Verify accuracy and consistency of YAML/classic examples with implementation/spec.
• Check condition operator table and supported signal types (logs, metrics, traces, any).
• Review examples using regex, multi-condition routes, and per_record_routing semantics for correctness and edge cases.

Poem

🐇 I hop through tags and patterns bright,
Per-record whispers guide the flight.
Chunks roll left, and rules send right,
I nibble docs till routes are light. ✨

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding comprehensive documentation for tag-based and conditional routing features.
Linked Issues check	✅ Passed	The PR directly addresses issue #2199 by expanding the Router documentation to include both tag-based routing details and the new conditional routing feature.
Out of Scope Changes check	✅ Passed	All changes are scoped to expanding router documentation with tag-based and conditional routing content, directly aligned with issue #2199.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4655e73 and 75945b3.

📒 Files selected for processing (1)

• pipeline/router.md (5 hunks)

🔇 Additional comments (14)

pipeline/router.md (14)

175-177: Version requirement correctly updated to Fluent Bit 4.2.

The hint now correctly specifies that conditional routing requires Fluent Bit version 4.2 and greater, matching the actual feature availability.

---

7-11: Clear and accurate introduction to dual routing mechanisms.

The opening clearly distinguishes between tag-based routing (traditional, chunk-level) and conditional routing (newer, per-record), setting appropriate context for the rest of the document.

---

26-84: Well-structured Tag-based routing section with clear examples.

The section effectively introduces Tags and Match concepts, provides concrete YAML/classic config examples, and demonstrates wildcards and regex matching. The progression from basic to advanced matching patterns is pedagogically sound.

---

189-227: Configuration syntax clearly presented with proper YAML structure.

The syntax block shows the routes structure within inputs, condition/rule definitions, and output mappings. The use of placeholder syntax {and|or}, {field_name}, etc. effectively communicates the template nature without being overly verbose.

---

251-267: Comparison operators table is comprehensive and accurate.

All standard comparison operators are documented (eq, neq, gt, lt, gte, lte, regex, not_regex, in, not_in) with clear descriptions. Coverage is sufficient for the conditional routing feature.

---

268-278: Signal types table clearly documents routing scope options.

The four signal types (logs, metrics, traces, any) are defined concisely, with the any option providing a catchall for users who want unified signal routing.

---

281-351: Route logs by severity—concrete and realistic example.

Demonstrates error/fatal routing to one destination, info logs to another, and a default catch-all. The use of or and and operators, field accessors ($level), and multiple output aliases clearly illustrates practical log segmentation.

---

352-396: Route by service name—demonstrates in operator for multi-value matching.

Shows how to route logs from a list of critical services to a dedicated output using the in operator, with a sensible default for all other services. This pattern is common in real deployments.

---

398-454: Multi-condition routing example combines multiple field checks effectively.

Demonstrates chaining conditions (level=error AND environment=production AND response_time>5000ms) to identify high-priority alerts, routing to both PagerDuty and Elasticsearch simultaneously. The example is realistic and actionable.

---

456-503: Regex-based routing shows field pattern matching with case-insensitive flag.

The regex patterns for security event detection ((?i)(unauthorized|forbidden|...) and ^security\\.) are well-chosen and demonstrate regex syntax clearly. The or operator allows flexible matching.

---

505-574: Metrics routing example demonstrates metric name pattern matching.

Routes CPU and memory metrics to separate Prometheus backends based on $metric.name regex patterns, with a general backend for others. The pattern hierarchy (cpu → cpu backend, memory → memory backend, other → general) is clear and maintainable.

---

576-645: Traces routing example demonstrates context parameter usage.

Uses context: otel_resource_attributes to access OpenTelemetry resource fields, routes critical services separately, and demonstrates span-status-code and latency-based routing. Shows advanced feature usage well.

---

647-754: Multi-signal-type example demonstrates unified observability collection.

Routes logs, metrics, and traces with separate rule blocks within a single OpenTelemetry input. Shows error-severity logs, production metrics, and production traces to dedicated backends, with appropriate defaults. This example is comprehensive and realistic for OTEL-based deployments.

---

756-769: Choosing a routing approach table uses neutral terminology and provides clear guidance.

The comparison table correctly uses "Routing without content inspection" (not "Simple"), and the guidance to combine both approaches is sound. Helps users select the right mechanism for their use case.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3002555 and d49ee2f.

📒 Files selected for processing (1)

• pipeline/router.md (5 hunks)

🧰 Additional context used

🪛 GitHub Check: runner / vale

pipeline/router.md

[warning] 516-516:
[vale] reported by reviewdog 🐶
[FluentBit.Simplicity] Avoid words like "Simple" that imply ease of use, because the user may find this action difficult.

Raw Output:
{"message": "[FluentBit.Simplicity] Avoid words like "Simple" that imply ease of use, because the user may find this action difficult.", "location": {"path": "pipeline/router.md", "range": {"start": {"line": 516, "column": 3}}}, "severity": "WARNING"}

🪛 LanguageTool

pipeline/router.md

[style] ~248-~248: To form a complete sentence, be sure to include a subject or ‘there’.
Context: ...value | The value to compare against. Can be a single value or an array for in ...

(MISSING_IT_THERE)

🔇 Additional comments (4)

pipeline/router.md (4)

7-10: Strong introduction clarifying routing mechanisms.

The opening clearly distinguishes tag-based routing (chunk-level) from conditional routing (record-level), setting appropriate expectations for both approaches.

---

26-170: Well-organized tag-based routing documentation.

The section provides clear progression from basic matching through wildcards to regex patterns. Examples are consistent, complete, and effectively demonstrate each concept.

---

171-279: Comprehensive conditional routing parameter documentation.

The section effectively introduces the feature, explains evaluation mechanics, provides clear configuration syntax, and documents all parameters, operators, and signal types. The version requirement notice is helpful.

---

280-503: Well-designed, progressively complex examples.

The four examples effectively demonstrate conditional routing from simple field matching through regex patterns. Each includes realistic output configurations and clear explanations of routing behavior.

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d49ee2f and 103e83c.

📒 Files selected for processing (1)

• pipeline/router.md (5 hunks)

🧰 Additional context used

🪛 LanguageTool

pipeline/router.md

[grammar] ~248-~248: Ensure spelling is correct
Context: ...parison operator. | | value | This is thee value to compare against. It can be a s...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🔇 Additional comments (5)

pipeline/router.md (5)

7-11: Excellent documentation structure for dual routing mechanisms.

The introduction clearly and concisely explains the two routing approaches available in Fluent Bit. The distinction between tag-based (traditional, chunk-level) and conditional (per-record, content-based) routing helps users understand which to use. The bullet points are well-written and accessible.

---

26-169: Comprehensive tag-based routing documentation with clear progression.

The section effectively covers tag-based routing fundamentals through three increasingly advanced subsections: basic matching, wildcards, and regular expressions. Each subsection includes both YAML and Conf configuration examples, making the documentation accessible to users with different configuration preferences. The explanatory text is concise and links to the mermaid diagram appropriately.

---

171-278: Well-structured conditional routing core section with comprehensive parameter documentation.

The record accessor syntax documentation correctly references the feature for accessing fields inside logs, and the link to the record accessor reference page is helpful. The parameter tables (route configuration, condition rules, comparison operators, signal types) are thorough and well-organized. The version constraint correctly specifies that conditional routing is available in Fluent Bit version 4.0 and greater.

---

279-503: Excellent practical examples demonstrating progressive complexity and real-world scenarios.

The four examples (severity routing, service name routing, multiple conditions, regex matching) effectively demonstrate different use cases and progressively increase in complexity. Each example uses complete, executable YAML configurations with actual output aliases and plugin names. The explanatory text after each example clearly articulates the behavior and intent.

---

505-518: Helpful comparison table for routing approach selection.

The table effectively guides users in choosing between tag-based and conditional routing by presenting practical use cases. The guidance about combining both approaches in line 518 is valuable for users building complex pipelines.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

pipeline/router.md (1)

268-277: Consider demonstrating signal type routing with examples.

The Signal types section (lines 268-277) lists four distinct signal types (logs, metrics, traces, any) that can be routed separately. However, all five examples in the document only demonstrate logs: routing.

While the structure is intuitive (users would likely infer that metrics: and traces: work similarly), adding even a brief example or clarifying note about how to route metrics or traces would improve completeness. For instance, a note like "The same route structure applies to metrics: and traces: signal types" would set clearer expectations.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 103e83c and 4655e73.

📒 Files selected for processing (1)

• pipeline/router.md (5 hunks)

[eschabell]

Passed AI review (whew!), merging with good updates to the routing docs.