Skip to content

Commit c5613fb

Browse files
ankursharmascopybara-github
authored andcommitted
feat: Add Rubrics to EvalCase datamodel
For advanced eval use cases, we do expect agent developers to have rubrics that are specific to an Eval Case and in some cases even specific to a single invocation/turn in the eval case conversation. A separate PR will be created to consume this data model changes in ADK Eval. PiperOrigin-RevId: 805588808
1 parent 0bc2ee6 commit c5613fb

File tree

1 file changed

+13
-1
lines changed

1 file changed

+13
-1
lines changed

src/google/adk/evaluation/eval_case.py

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@
2323
from pydantic import ConfigDict
2424
from pydantic import Field
2525

26+
from .eval_rubrics import Rubric
27+
2628

2729
class EvalBaseModel(BaseModel):
2830
model_config = ConfigDict(
@@ -55,7 +57,7 @@ class IntermediateData(EvalBaseModel):
5557
class Invocation(EvalBaseModel):
5658
"""Represents a single invocation."""
5759

58-
invocation_id: str = ''
60+
invocation_id: str = ""
5961
"""Unique identifier for the invocation."""
6062

6163
user_content: genai_types.Content
@@ -74,6 +76,11 @@ class Invocation(EvalBaseModel):
7476
creation_timestamp: float = 0.0
7577
"""Timestamp for the current invocation, primarily intended for debugging purposes."""
7678

79+
rubrics: Optional[list[Rubric]] = Field(
80+
default=None,
81+
)
82+
"""A list of rubrics that are applicable to only this invocation."""
83+
7784

7885
class SessionInput(EvalBaseModel):
7986
"""Values that help initialize a Session."""
@@ -105,3 +112,8 @@ class EvalCase(EvalBaseModel):
105112

106113
creation_timestamp: float = 0.0
107114
"""The time at which this eval case was created."""
115+
116+
rubrics: Optional[list[Rubric]] = Field(
117+
default=None,
118+
)
119+
"""A list of rubrics that are applicable to all the invocations in the conversation of this eval case."""

0 commit comments

Comments
 (0)