Skip to content

Conversation

@marcsanmi
Copy link
Contributor

@marcsanmi marcsanmi commented Nov 13, 2025

This PR implements backend support for exemplars in timeseries queries.

Changes

Protobuf API

  • Add Exemplar message with profile_id, timestamp, value, and labels fields
  • Extend Point message with exemplars field to embed exemplar metadata in timeseries responses
  • Add ExemplarType enum to control exemplar inclusion (NONE, INDIVIDUAL)

Query Pipeline

  • Thread profileID through v2 query backend via ProfileEntry.ID
  • Read profile UUID from Parquet ID column during timeseries queries
  • Use withFetchProfileIDs(true) option to enable profile ID fetching only when exemplars are requested
  • Maintain v1 compatibility by passing empty profileID in legacy query paths

Exemplar Management

  • ExemplarBuilder: New dedicated component for per-series exemplar tracking
    • Deduplicates exemplars by fingerprint (label set hash)
    • Merges exemplars with same (profileID, timestamp) by intersecting labels
    • Handles dynamic labels (e.g., pod changes during migration)
  • Early label filtering: Filter out groupBy labels at ingestion time using efficient WithoutLabels() method
  • Three-level label intersection:
    a. ExemplarBuilder: Within single block
    b. TimeSeriesMerger: Across blocks at same timestamp
    c. RangeSeries: Across timestamps within time bucket

Merging & Aggregation

  • Single-block path: Exemplars attached to points with matching timestamps
  • Multi-block path: TimeSeriesMerger merges exemplars across blocks, preserving highest values
  • sumTimeSeriesAggregator and avgTimeSeriesAggregator preserve and merge exemplars during re-aggregation
  • Top-N selection per point (default: 1, configurable via maxExemplarsPerPoint)

Performance Optimizations

  • Efficient sorted-list attachment algorithm (O(E+P) instead of O(E*P))
  • Conditional ID column fetching only when exemplars requested
  • In-place label filtering using Subtract() method
  • Early filtering eliminates redundant aggregator-level filtering

Implementation Flow

  1. v2 single-block: queryTimeSeries → TimeSeriesBuilder.Add() → filter labels → ExemplarBuilder.Add() → BuildWithExemplars()
  2. v2 multi-block: TimeSeriesBuilder → TimeSeriesMerger.MergeTimeSeries() → RangeSeries() re-aggregates with exemplars
  3. v1 paths: Empty profileID for backward compatibility (no exemplars)

@marcsanmi marcsanmi changed the title feat: add exemplar support to SelectSeries for individual profile retrieval feat: add exemplar support to timeseries responses for individual profile retrieval Nov 13, 2025
@marcsanmi marcsanmi marked this pull request as ready for review November 14, 2025 08:22
@marcsanmi marcsanmi requested a review from simonswine November 17, 2025 12:08
Copy link
Contributor

@simonswine simonswine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just focused on the query-backend for now

Base automatically changed from api/individual-profile-retrieval to main November 18, 2025 08:18
@marcsanmi marcsanmi requested a review from a team as a code owner November 18, 2025 08:18
simonswine added a commit that referenced this pull request Nov 18, 2025
@marcsanmi marcsanmi requested a review from simonswine November 19, 2025 14:32
simonswine added a commit that referenced this pull request Nov 20, 2025
* chore: Make profileEntryIterator more configurable

* Implement profileIDSelector and address review comments

* Disable ID retrieval until it is used in #4615
@marcsanmi marcsanmi requested a review from simonswine November 21, 2025 12:14
Copy link
Contributor

@simonswine simonswine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Great work, thanks for going through all my comments

@marcsanmi marcsanmi merged commit 9e07b6b into main Nov 21, 2025
21 checks passed
@marcsanmi marcsanmi deleted the marcsanmi/add-timeseries-exemplars branch November 21, 2025 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants