Skip to content

Conversation

salma-elshafey
Copy link
Contributor

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@salma-elshafey salma-elshafey requested a review from a team as a code owner October 5, 2025 09:52
@Copilot Copilot AI review requested due to automatic review settings October 5, 2025 09:52
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR renames the Tool Call Accuracy Evaluator to Tool Call Quality Evaluator to better reflect its purpose. The change involves updating class names, file names, constants, and all references throughout the codebase while maintaining backward compatibility through an alias.

  • Renamed ToolCallAccuracyEvaluator to ToolCallQualityEvaluator across all files
  • Updated associated constants, messages, and identifiers from "accuracy" to "quality"
  • Maintained backward compatibility by keeping the old class name as an alias

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test_tool_call_quality_evaluator.py Updated all test class names and references to use the new ToolCallQualityEvaluator
test_agent_evaluators.py Updated test method names and evaluator references to use new quality naming
evaluation_samples_evaluate_fdp.py Updated sample code to demonstrate new ToolCallQualityEvaluator usage
evaluation_samples_evaluate.py Updated sample code to demonstrate new ToolCallQualityEvaluator usage
tool_call_quality.ipynb Updated notebook title and content to reflect new quality evaluator naming
_exceptions.py Added new TOOL_CALL_QUALITY_EVALUATOR error target enum value
tool_call_quality.prompty Updated prompty file name and description from accuracy to quality
_tool_call_quality.py Renamed class and all internal references from accuracy to quality terminology
init.py Updated imports to use new ToolCallQualityEvaluator class name
_eval_mapping.py Added mapping for new evaluator while maintaining backward compatibility
init.py Added new evaluator to exports and created backward compatibility alias

@github-actions github-actions bot added Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Evaluation Issues related to the client library for Azure AI Evaluation labels Oct 5, 2025
Copy link

github-actions bot commented Oct 5, 2025

Thank you for your contribution @salma-elshafey! We will review the pull request and get back to you soon.

Copy link

github-actions bot commented Oct 5, 2025

API Change Check

APIView identified API level changes in this PR and created the following API reviews

azure-ai-evaluation

@luigiw luigiw merged commit 772ee5a into Azure:main Oct 6, 2025
20 checks passed
salma-elshafey added a commit to salma-elshafey/azure-sdk-for-python that referenced this pull request Oct 8, 2025
w-javed pushed a commit that referenced this pull request Oct 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Evaluation Issues related to the client library for Azure AI Evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants