wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: support new async-tool mechanism in AWS Nova Sonic

Browse Source 
Add support to AWSNovaSonicLLMService for the new "async tool call"
mechanism activated by `cancel_on_interruption=False`, which includes:

- delivering results asynchronously
- delivering result streams
- cancelling running async tools

Note that the introduction of the new mechanism had actually caused a
regression in AWS Nova Sonic, which previously supported
`cancel_on_interruption=False` with the old mechanism (simply avoiding
discarding tool calls on interruptions).

Support for the other major realtime services (`GeminiLiveLLMService`,
`OpenAIRealtimeLLMService`) will follow in a separate PR — Gemini Live
in particular needs more work before it can support long-running tool
calls reliably.
This commit is contained in:
Paul Kompfner
2026-05-01 15:08:06 -04:00
parent a745e8d318
commit 7d3726a74b
9 changed files with 964 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
changelog/4425.added.md Normal file
View File

@@ -0,0 +1 @@
- Added support to `AWSNovaSonicLLMService` for the new "async tool call" mechanism activated by `cancel_on_interruption=False`, which includes delivering results asynchronously, delivering result streams, and cancelling running async tools. Support for the other major realtime services (`GeminiLiveLLMService`, `OpenAIRealtimeLLMService`) will be added in a follow-up PR.

1
changelog/4425.fixed.md Normal file
View File

@@ -0,0 +1 @@
- Fixed a regression in `AWSNovaSonicLLMService` where `cancel_on_interruption=False` (which previously worked under the old async-tool-call mechanism, by simply avoiding discarding tool calls on interruptions) stopped working after the introduction of the new "async tool call" mechanism.