Support customizing Swarm handoff trigger beyond model-based function call

[ekzhu]

Confirmation

• I confirm that I am a maintainer and so can use this template. If I am not, I understand this issue will be closed and I will be asked to use a different template.

Issue body

Currently handoff in AssistantAgent is through model-driven function call. The handoff is converted into a tool which the model can choose to call. We would like to be able to trigger handoff with a user-defined condition, which can take the input of the current model context or the latest model inference result.

This allows more robust handoff, especially when structured output is used.

[EItanya]

I agree with this plan for sure, this would also help build towards a DAG type system. Did you have a design in mind? Maybe a Handoff component?

The one issue I see is that it will be hard to maintain the current behavior AND make it pluggable. The current behavior is incredibly specific, and potentially interferes with the standard runtime of the agent. The Handoff feels like it should really happen AFTER the agent has finished its core loop. I think it could be possible if we split the current Handoff behavior into a separate LLM call specifically for this purpose. Maybe something like the following:

class HandofTool(BaseTool):
    ...

class LLMHandoff(BaseHandoff):
    def __init__(targets: list[str], model_client: ChatCompletionClient):
        ...

    def targets(output: TextMessage | ToolCallSummaryResult, cancellation_token: CancellationToken):
        # convert potential targets to HandoffTools
        handoff_tools = self._convert_to_handoff_tool(self._potential_targets)
        # Use a pre-built system prompt + a user message with the input
        result = self._model_client.create(self._system_message + UserMessage(output.content), handoff_tools)
        # Check if any of the "tools/handoffs" were called

class TextMentionHandoff(BaseHandoff):
    def __init__(targets: list[str], text: str)
        ...
    def targets(output: TextMessage | ToolCallSummaryResult, cancellation_token: CancellationToken):
        # Check the output for this specific text

[ekzhu]

I like the idea that handoff can use different implementations. We can extend the current Handoff to include different handoff conditions, such as ToolHandoff, ModelHandoff, and TextMentionHandoff, as you listed earlier. All handoff conditions subclass from the HandoffCondition base class, similar to TerminationCondition.

So a Handoff can be created like this:

from autogen_agentchat.base import Handoff
from autogen_agentchat.conditions import ToolHandoff
handoff = Handoff(target="agent2", condition=ToolHandoff(...)) # Same as the current behavior.

On whether to use a separate model call for handoff, I think we should keep the current approach as a default approach. Though, I do see it as a separate option we can support. E.g., ReflectionToolHandoff can be applied after the first model inference call.