Skip to content

Full table scan in ClickHouse due to HyperDX query missing date filter #1036

@m0nikasingh

Description

@m0nikasingh

I'm running HyperDX with an external ClickHouse and MongoDB setup. Our ClickHouse instance has the "force_index_by_date" setting enabled, which appears to conflict with some of the queries HyperDX runs.

For example, the following query does not include a time filter, which leads to a full table scan:

SELECT DISTINCT lowCardinalityKeys(arrayJoin(LogAttributes.keys)) AS key 
FROM otel.otel_logs 
LIMIT _CAST(1000, 'Int32') 
FORMAT JSON

Is there a recommended way to address this without disabling force_index_by_date? We'd like to avoid full scans, especially given the volume of data we're working with.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions