Skip to content

fix(core): Fix and add missing cache attributes in Vercel AI #17982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 21, 2025
Merged

fix(core): Fix and add missing cache attributes in Vercel AI #17982

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a6bcceb
fix(core): Fix and add cache attributes in Vercel AI
RulaKhaled Oct 21, 2025
b72f770
nice catch cursor
RulaKhaled Oct 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions packages/core/src/utils/ai/gen-ai-attributes.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,16 @@ export const GEN_AI_RESPONSE_STREAMING_ATTRIBUTE = 'gen_ai.response.streaming';
*/
export const GEN_AI_RESPONSE_TOOL_CALLS_ATTRIBUTE = 'gen_ai.response.tool_calls';

/**
* The number of cache write input tokens used
*/
export const GEN_AI_USAGE_INPUT_TOKENS_CACHE_WRITE_ATTRIBUTE = 'gen_ai.usage.input_tokens.cache_write';

/**
* The number of cached input tokens that were used
*/
export const GEN_AI_USAGE_INPUT_TOKENS_CACHED_ATTRIBUTE = 'gen_ai.usage.input_tokens.cached';

// =============================================================================
// OPENAI-SPECIFIC ATTRIBUTES
// =============================================================================
Expand Down
33 changes: 19 additions & 14 deletions packages/core/src/utils/vercel-ai/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ import type { Client } from '../../client';
import { SEMANTIC_ATTRIBUTE_SENTRY_OP, SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN } from '../../semanticAttributes';
import type { Event } from '../../types-hoist/event';
import type { Span, SpanAttributes, SpanAttributeValue, SpanJSON, SpanOrigin } from '../../types-hoist/span';
import {
GEN_AI_USAGE_INPUT_TOKENS_CACHE_WRITE_ATTRIBUTE,
GEN_AI_USAGE_INPUT_TOKENS_CACHED_ATTRIBUTE,
} from '../ai/gen-ai-attributes';
import { spanToJSON } from '../spanUtils';
import { toolCallSpanMap } from './constants';
import type { TokenSummary } from './types';
Expand All @@ -23,6 +27,7 @@ import {
AI_TOOL_CALL_ID_ATTRIBUTE,
AI_TOOL_CALL_NAME_ATTRIBUTE,
AI_TOOL_CALL_RESULT_ATTRIBUTE,
AI_USAGE_CACHED_INPUT_TOKENS_ATTRIBUTE,
AI_USAGE_COMPLETION_TOKENS_ATTRIBUTE,
AI_USAGE_PROMPT_TOKENS_ATTRIBUTE,
GEN_AI_RESPONSE_MODEL_ATTRIBUTE,
Expand Down Expand Up @@ -107,6 +112,7 @@ function processEndedVercelAiSpan(span: SpanJSON): void {

renameAttributeKey(attributes, AI_USAGE_COMPLETION_TOKENS_ATTRIBUTE, GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE);
renameAttributeKey(attributes, AI_USAGE_PROMPT_TOKENS_ATTRIBUTE, GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE);
renameAttributeKey(attributes, AI_USAGE_CACHED_INPUT_TOKENS_ATTRIBUTE, GEN_AI_USAGE_INPUT_TOKENS_CACHED_ATTRIBUTE);

if (
typeof attributes[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE] === 'number' &&
Expand Down Expand Up @@ -287,7 +293,7 @@ function addProviderMetadataToAttributes(attributes: SpanAttributes): void {
if (providerMetadataObject.openai) {
setAttributeIfDefined(
attributes,
'gen_ai.usage.input_tokens.cached',
GEN_AI_USAGE_INPUT_TOKENS_CACHED_ATTRIBUTE,
providerMetadataObject.openai.cachedPromptTokens,
);
setAttributeIfDefined(
Expand All @@ -309,35 +315,34 @@ function addProviderMetadataToAttributes(attributes: SpanAttributes): void {
}

if (providerMetadataObject.anthropic) {
setAttributeIfDefined(
attributes,
'gen_ai.usage.input_tokens.cached',
providerMetadataObject.anthropic.cacheReadInputTokens,
);
setAttributeIfDefined(
attributes,
'gen_ai.usage.input_tokens.cache_write',
providerMetadataObject.anthropic.cacheCreationInputTokens,
);
const cachedInputTokens =
providerMetadataObject.anthropic.usage?.cache_read_input_tokens ??
providerMetadataObject.anthropic.cacheCreationInputTokens;
setAttributeIfDefined(attributes, GEN_AI_USAGE_INPUT_TOKENS_CACHED_ATTRIBUTE, cachedInputTokens);
Copy link

@cursor cursor bot Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Cache Token Usage Reporting Error

The Anthropic cachedInputTokens attribute incorrectly falls back to cacheCreationInputTokens. This uses tokens for cache writes instead of tokens read from the cache, leading to inaccurate gen_ai.usage.input_tokens.cached values.

Fix in Cursor Fix in Web


const cacheWriteInputTokens =
providerMetadataObject.anthropic.usage?.cache_creation_input_tokens ??
providerMetadataObject.anthropic.cacheCreationInputTokens;
setAttributeIfDefined(attributes, GEN_AI_USAGE_INPUT_TOKENS_CACHE_WRITE_ATTRIBUTE, cacheWriteInputTokens);
}

if (providerMetadataObject.bedrock?.usage) {
setAttributeIfDefined(
attributes,
'gen_ai.usage.input_tokens.cached',
GEN_AI_USAGE_INPUT_TOKENS_CACHED_ATTRIBUTE,
providerMetadataObject.bedrock.usage.cacheReadInputTokens,
);
setAttributeIfDefined(
attributes,
'gen_ai.usage.input_tokens.cache_write',
GEN_AI_USAGE_INPUT_TOKENS_CACHE_WRITE_ATTRIBUTE,
providerMetadataObject.bedrock.usage.cacheWriteInputTokens,
);
}

if (providerMetadataObject.deepseek) {
setAttributeIfDefined(
attributes,
'gen_ai.usage.input_tokens.cached',
GEN_AI_USAGE_INPUT_TOKENS_CACHED_ATTRIBUTE,
providerMetadataObject.deepseek.promptCacheHitTokens,
);
setAttributeIfDefined(
Expand Down
23 changes: 23 additions & 0 deletions packages/core/src/utils/vercel-ai/vercel-ai-attributes.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,14 @@ export const AI_RESPONSE_PROVIDER_METADATA_ATTRIBUTE = 'ai.response.providerMeta
*/
export const AI_SETTINGS_MAX_RETRIES_ATTRIBUTE = 'ai.settings.maxRetries';

/**
* Basic LLM span information
* Multiple spans
*
* The number of cached input tokens that were used
* @see https://ai-sdk.dev/docs/ai-sdk-core/telemetry#basic-llm-span-information
*/
export const AI_USAGE_CACHED_INPUT_TOKENS_ATTRIBUTE = 'ai.usage.cachedInputTokens';
Copy link
Member

@JPeer264 JPeer264 Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: I couldn't find the ai.usage.cachedInputTokens in the given docs. Not sure if that is wanted

Copy link
Member Author

@RulaKhaled RulaKhaled Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it exists in their otel attributes, unfortunately not everything is documented.
Screenshot 2025-10-21 at 3 16 04 PM

/**
* Basic LLM span information
* Multiple spans
Expand Down Expand Up @@ -863,6 +871,21 @@ interface AnthropicProviderMetadata {
* @see https://ai-sdk.dev/providers/ai-sdk-providers/anthropic#cache-control
*/
cacheReadInputTokens?: number;

/**
* Usage metrics for the Anthropic model.
*/
usage?: {
input_tokens: number;
cache_creation_input_tokens?: number;
cache_read_input_tokens?: number;
cache_creation?: {
ephemeral_5m_input_tokens?: number;
ephemeral_1h_input_tokens?: number;
};
output_tokens?: number;
service_tier?: string;
};
}

/**
Expand Down
Loading