b14a03d01f
Applies the same async-tool message routing introduced for AWSNovaSonicLLMService
and OpenAIRealtimeLLMService to additional realtime LLM services where the
flag's intent ("keep talking while the tool runs") is achievable:
- GrokRealtimeLLMService (xAI Realtime — also benefits the deprecated Grok
alias since it re-exports the xAI module)
- AzureRealtimeLLMService picks up the fix transitively by inheriting from
OpenAIRealtimeLLMService — no code change needed.
GrokRealtimeLLMService's _process_completed_function_calls now matches
the canonical pattern: skip LLMSpecificMessage, detect async-tool messages
via parse_message and route them — started skipped silently, intermediate
logged as an error and surfaced via push_error, final delivered through
the same channel as a synchronous result.
UltravoxRealtimeLLMService instead gets a one-time warning when async-tool
messages appear in the context. The Ultravox API freezes the conversation
during tool execution
(https://docs.ultravox.ai/tools/async-tools#custom-tool-timeouts), so the
flag's "keep talking while the tool runs" intent isn't achievable there —
applying the same code pattern would mislead users into expecting a UX
Ultravox can't deliver. Surfacing a clear warning is the right behavior
until Ultravox grows true async tool support.
Adds async-tool example files for Grok and Azure modeled on the existing
Nova Sonic / OpenAI Realtime ones (10s simulated network delay, weather
tool registered with cancel_on_interruption=False).
Two services remain excluded:
- GeminiLiveLLMService — the async-tool path needs deeper investigation.
- InworldRealtimeLLMService — appears to have a pre-existing problem
with even simple synchronous tool calling on its Realtime API (the
request reaches the server fine, but response generation fails with a
generic server_error).
1.1 KiB
1.1 KiB
- Extended the
cancel_on_interruption=Falseregression fix toGrokRealtimeLLMServiceandAzureRealtimeLLMService(the latter picks up the fix transitively by inheriting fromOpenAIRealtimeLLMService). Same shape as the original fix forAWSNovaSonicLLMServiceandOpenAIRealtimeLLMService: each service now detects async-tool messages in the LLM context and routes the final result to its formal tool-result channel. Streamed intermediate results (FunctionCallResultProperties(is_final=False)) are not supported on these realtime services.UltravoxRealtimeLLMServicenow logs a one-time warning when async-tool messages appear in the context, since Ultravox freezes the conversation during tool execution and so the "keep talking while the tool runs" intent ofcancel_on_interruption=Falseis structurally not achievable there.GeminiLiveLLMServiceandInworldRealtimeLLMServiceare excluded for now: Gemini Live's async-tool path needs deeper investigation, and Inworld appears to have a pre-existing problem with even simple tool calling on its Realtime API.