7d3726a74b
Add support to AWSNovaSonicLLMService for the new "async tool call" mechanism activated by `cancel_on_interruption=False`, which includes: - delivering results asynchronously - delivering result streams - cancelling running async tools Note that the introduction of the new mechanism had actually caused a regression in AWS Nova Sonic, which previously supported `cancel_on_interruption=False` with the old mechanism (simply avoiding discarding tool calls on interruptions). Support for the other major realtime services (`GeminiLiveLLMService`, `OpenAIRealtimeLLMService`) will follow in a separate PR — Gemini Live in particular needs more work before it can support long-running tool calls reliably.
290 B
290 B
- Fixed a regression in
AWSNovaSonicLLMServicewherecancel_on_interruption=False(which previously worked under the old async-tool-call mechanism, by simply avoiding discarding tool calls on interruptions) stopped working after the introduction of the new "async tool call" mechanism.