58333b2705
Same async-tool routing approach as #4441: detect async-tool messages in the LLM context, deliver the final result via the formal tool-result channel. Caveat: as of this writing, Inworld Realtime doesn't appear to handle the resulting delayed tool result reliably, so the routing is best-effort and the service emits a one-time warning when async-tool messages are seen. Streamed intermediate results remain unsupported. Also adds function calling to the realtime-inworld.py example, and softens the Inworld mention in the #4447 changelog now that the exclusion is being closed.
2 lines
1.1 KiB
Markdown
2 lines
1.1 KiB
Markdown
- Extended the `cancel_on_interruption=False` regression fix to `GrokRealtimeLLMService`, `AzureRealtimeLLMService`, and `UltravoxRealtimeLLMService`. Grok and Azure use the same approach as in #4441 (each service detects async-tool messages in the LLM context and routes the final result to its formal tool-result channel; Azure inherits transitively from `OpenAIRealtimeLLMService`). Ultravox needed a different approach because its API freezes the conversation between `client_tool_invocation` and the matching `client_tool_result` — for async-registered functions it now ships a placeholder `client_tool_result` immediately when the function is invoked (to unfreeze the conversation), then injects the real result as user-side text once the tool finishes. Streamed intermediate results (`FunctionCallResultProperties(is_final=False)`) are still not supported on any of these realtime services. `GeminiLiveLLMService` and `InworldRealtimeLLMService` are excluded for now: Gemini Live's async-tool path needs deeper investigation, and Inworld tool calling needs to be sorted out first.
|