[HDX-1828] Dashboards: Not all data is shown in charts when the legend count is greater than 20 #899
Description
Firstly, I love HyperDX!
We have a chart in one of our dashboards, however, that chart fails to show all data. We observed that this happens when the legend has more than 20 items (maybe in other situations too)
| Attempting to plot all data | Plotting more specific (i.e. subset) data to find missing data |
 |
 |
 |
 |
| Observe that none of the counts here have a value 64 | Issue: Upon adding a condition to explicitly filter the id, I start seeing datapoints which are missing in the left plot |
Could it be because of different granularity in the 2 plots?
No, the granularity is same, and I also checked [-2days,+2days] around that time; nowhere is the id corresponding to value 64 visible.
Copilot fix:
// in file /workspaces/hyperdx/packages/app/src/hooks/useMetadata.tsx
export function useGetKeyValues(
{
chartConfigs,
keys,
limit,
disableRowLimit,
}: {
chartConfigs: ChartConfigWithDateRange | ChartConfigWithDateRange[];
keys: string[];
limit?: number;
disableRowLimit?: boolean;
},
options?: Omit<UseQueryOptions<any, Error>, 'queryKey'>,
) {
const metadata = getMetadata();
const chartConfigsArr = toArray(chartConfigs);
// Helper to chunk an array, instead of only taking first 20
function chunkArray<T>(arr: T[], size: number): T[][] {
const res: T[][] = [];
for (let i = 0; i < arr.length; i += size) {
res.push(arr.slice(i, i + size));
}
return res;
}
return useQuery<{ key: string; value: string[] }[]>({
queryKey: [
'useMetadata.useGetKeyValues',
...chartConfigsArr.map(cc => ({ ...cc })),
...keys,
disableRowLimit,
],
queryFn: async () => {
// Batch keys in groups of 20
const keyChunks = chunkArray(keys, 20);
const results = await Promise.all(
chartConfigsArr.flatMap(chartConfig =>
keyChunks.map(chunk =>
metadata.getKeyValues({
chartConfig,
keys: chunk,
limit,
disableRowLimit,
}),
),
),
);
// Flatten all results
return results.flatMap((v: any) => v);
},
staleTime: 1000 * 60 * 5, // Cache every 5 min
enabled: !!keys.length,
placeholderData: keepPreviousData,
...options,
});
}