Skip to content

[DOCS] document how to obtain cache stats when caching system prompt #118

@panditkbgh

Description

@panditkbgh

Documentation Section

https://strandsagents.com/latest/user-guide/concepts/model-providers/amazon-bedrock/#system-prompt-caching

Current State

We should document on https://strandsagents.com/latest/user-guide/concepts/model-providers/amazon-bedrock/#system-prompt-caching how to get the cache stats. Currently the page does not have those details.

Suggested Improvement

CacheReadInputTokens and CacheWriteInputTokens in https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html have this info. It is available in the event event type in Strands async iterators and callback handlers, for example: https://github.com/strands-agents/sdk-python/blob/1421aadd73c7a497503e43864649c9016dadea69/tests-integ/test_bedrock_cache_point.py#L21-L27

Documentations improvements: getting info on them should be documented in https://strandsagents.com/latest/user-guide/concepts/model-providers/amazon-bedrock/#system-prompt-caching

Another possible improvement is to return this info as a part of AgentResult.metrics and OpenTelemetry traces. That way, one does not need to write async handlers to get at this info.

Rationale

better documentation and ease of use

Additional Context

No response

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions