Compare commits

...

207 Commits

Author SHA1 Message Date
James Hush
21b13d85ea Update 2025-06-30 14:32:19 +08:00
James Hush
6088848209 Add comment 2025-06-30 12:29:58 +08:00
James Hush
f5d2dbe977 demo: mute while transfering 2025-06-30 12:27:19 +08:00
Mark Backman
0ecfa827e6 Improve docstrings for services and processors (#2087) 2025-06-28 13:39:45 -04:00
Aleix Conchillo Flaqué
e1b0db75eb Merge pull request #2086 from pipecat-ai/aleix/watchdog-coroutine-helper
add watchdog coroutine helper
2025-06-27 11:10:10 -07:00
Aleix Conchillo Flaqué
b0c773189f AWSNovaSonicLLMService: fix error with watchdog_coroutine() 2025-06-27 11:09:40 -07:00
Aleix Conchillo Flaqué
3064326834 utils.asyncio: added watchdog_coroutine() 2025-06-27 11:09:40 -07:00
Mark Backman
c67e50fe34 Merge pull request #2084 from pipecat-ai/mb/update-evals-nova-sonic
Add 40-aws-nova-sonic to release evals list
2025-06-27 09:47:59 -04:00
Mark Backman
9d45e3eca1 Merge pull request #2079 from pipecat-ai/mb/fix-42-incorrect-import
fix: example 42 incorrect import
2025-06-27 09:47:47 -04:00
Mark Backman
43a24d15f6 Add 40-aws-nova-sonic to release evals list 2025-06-27 08:34:39 -04:00
Mark Backman
2cf31884d0 fix: example 42 incorrect import 2025-06-26 21:52:14 -04:00
Aleix Conchillo Flaqué
19354c6f2d Merge pull request #2078 from pipecat-ai/aleix/hotfix-0.0.73
just a quick hotfix for 0.0.73
2025-06-26 17:31:40 -07:00
Aleix Conchillo Flaqué
0b2079ad41 update CHANGELOG for 0.0.73 2025-06-26 17:02:12 -07:00
Aleix Conchillo Flaqué
5f18c3af70 OpenAIRealtimeLLMContext: fix circular dependency 2025-06-26 17:01:45 -07:00
Aleix Conchillo Flaqué
0a40285d43 update FrameProcessor.watchdog_timers_enabled references 2025-06-26 16:26:12 -07:00
Vanessa Pyne
5b1c328541 Merge pull request #2075 from pipecat-ai/vp-mcp-lint
mcp_service: lint
2025-06-26 15:25:39 -05:00
vipyne
37929533af mcp_service: lint 2025-06-26 15:00:20 -05:00
Vanessa Pyne
3b92113680 Merge pull request #2030 from yousifa/mcp-streaming-http
MCPClient streamable_http transport support
2025-06-26 14:57:31 -05:00
Yousif
46b52cb9bb Merge branch 'main' into mcp-streaming-http 2025-06-26 12:30:43 -07:00
Mark Backman
f0bcc9d9ba Add MCPClient docstrings. Removed google specific cleanup, changed example to openai 2025-06-26 12:29:45 -07:00
Yousif Astarabadi
1cac028bfe example using http transport for mcp client 2025-06-26 12:16:35 -07:00
Yousif Astarabadi
4956886819 updated error message with StreamableHttpParameters 2025-06-26 12:16:28 -07:00
Yousif Astarabadi
c720cfc7c7 updated streamablehttp to use StreamableHttpParameters type 2025-06-26 12:16:26 -07:00
Yousif Astarabadi
8fcef5628f added streamablehttp support, bumped mcp version, added additional headers and streamable_http params to MCPClient 2025-06-26 12:16:19 -07:00
Aleix Conchillo Flaqué
c4a72802f0 Merge pull request #2074 from pipecat-ai/aleix/pipecat-0.0.72
update CHANGELOG for 0.0.72
2025-06-26 12:10:14 -07:00
Aleix Conchillo Flaqué
917394803c update CHANGELOG for 0.0.72 2025-06-26 11:42:52 -07:00
Mark Backman
01040ddcdd Merge pull request #2071 from pipecat-ai/mb/services-docstrings-update
Add/update docstrings to LLM services
2025-06-26 14:42:32 -04:00
Aleix Conchillo Flaqué
7947497f7e Merge pull request #2073 from a6kme/patch-1
Start HeartBeat when all processors have processed StartFrame
2025-06-26 11:34:46 -07:00
Aleix Conchillo Flaqué
539ca5856f Merge pull request #2072 from pipecat-ai/aleix/utils-watchdog-cleanup
utils(asyncio): simplify watchdog helpers
2025-06-26 11:29:21 -07:00
Abhishek
89c801f82c Start HeartBeat when all processors have processed StartFrame
Some of the processors like STTService and TTSService don't push StartFrame ahead in the pipeline, unless they have connected with their service providers. This delays StartFrame in downstream processors. 

If we receive HeartBeat frame before StartFrame, we will get AttributeError `'Processor' object has no attribute '_FrameProcessor__input_queue'`. 

Idea is to start HeartBeats after StartFrame has been processed by all the Processors in the pipeline.
2025-06-26 23:28:37 +05:30
Aleix Conchillo Flaqué
3de4f22d34 utils(asyncio): simplify watchdog helpers 2025-06-26 09:40:42 -07:00
Mark Backman
0e4d2be98c Update AzureRealtimeBetaLLMService docstrings 2025-06-26 12:12:00 -04:00
Mark Backman
d8ce108ccd Update OpenAIRealtimeBetaLLMService docstrings 2025-06-26 12:06:47 -04:00
Mark Backman
d123cd4b2b Update GeminiMultimodalLiveLLMService docstrings 2025-06-26 11:47:30 -04:00
Aleix Conchillo Flaqué
4d34aa7cd6 Merge pull request #2069 from pipecat-ai/aleix/utils-asyncio-package
move things to new utils.asyncio package
2025-06-26 08:26:47 -07:00
Aleix Conchillo Flaqué
b860e94582 move things to new utils.asyncio package 2025-06-26 08:24:25 -07:00
Aleix Conchillo Flaqué
9d653e3788 Merge pull request #2068 from pipecat-ai/aleix/task-manager-dont-warn-reset-watchdog
TaskManager: don't warn on reset_watchdog()
2025-06-26 08:23:51 -07:00
Mark Backman
9e518cf2ba Update AWSNovaSonicLLMService docstrings 2025-06-26 11:21:18 -04:00
Mark Backman
2856372ad6 Update TogetherLLMService docstrings 2025-06-26 11:01:35 -04:00
Mark Backman
efbf574613 Update SambaNovaLLMService docstrings 2025-06-26 11:00:40 -04:00
Mark Backman
c018eb2f0e Update QwenLLMService docstrings 2025-06-26 10:57:42 -04:00
Mark Backman
d7bfe54b7c Update PerplexityLLMService docstrings 2025-06-26 10:56:48 -04:00
Mark Backman
137282b7a9 Update OpenRouterLLMService docstrings 2025-06-26 10:53:42 -04:00
Mark Backman
769f8c8f34 Update OpenPipeLLMService docstrings 2025-06-26 10:53:05 -04:00
Mark Backman
8b8a37ae7c Update OLLamaLLMService docstrings 2025-06-26 10:48:19 -04:00
Mark Backman
56e2b006f5 Update NimLLMService docstrings 2025-06-26 10:47:26 -04:00
Mark Backman
79cca05e43 Update GroqLLMService docstrings 2025-06-26 10:46:07 -04:00
Mark Backman
166c8e8e82 Update GrokLLMService docstrings 2025-06-26 10:39:46 -04:00
Mark Backman
9b64d2c325 Update GoogleLLMService docstrings 2025-06-26 10:37:22 -04:00
Mark Backman
03e3e9fae9 Update FireworksLLMService docstrings 2025-06-26 10:28:35 -04:00
Mark Backman
65234ae41a Update DeepSeekLLMService docstrings 2025-06-26 10:27:36 -04:00
Mark Backman
3828df8cf9 Update CerebrasLLMService docstrings 2025-06-26 10:26:42 -04:00
Mark Backman
9cbe85bf99 Update AzureLLMService docstrings 2025-06-26 10:25:17 -04:00
Mark Backman
7bf805b829 Update AWSBedrock docstrings 2025-06-26 10:23:40 -04:00
Mark Backman
990ee436e1 Add Anthropic docstrings 2025-06-26 07:42:22 -04:00
Mark Backman
1cd42066a6 Merge pull request #2067 from pipecat-ai/mb/update-docstrings-for-ref-docs
Update base service class docstrings for better docs auto-generation
2025-06-26 07:07:59 -04:00
Filipi da Silva Fuchter
ba43558049 Merge pull request #2066 from pipecat-ai/filipi/sentry_freeze_test
Enabling watchdog and sentry into the freeze-test
2025-06-26 08:01:51 -03:00
Mark Backman
951c8d34da Add special case handling for STT, TTS, LLM 2025-06-26 00:15:09 -04:00
Mark Backman
ac61139243 Add OpenAI LLM docstrings 2025-06-26 00:06:57 -04:00
Mark Backman
5b8f1fe3e3 Add Cartesia TTS docstrings 2025-06-25 23:50:55 -04:00
Mark Backman
0aa197e4a4 Add docstrings to DeepgramSTTService 2025-06-25 23:36:04 -04:00
Mark Backman
f04e058c96 Programmatically set the copyright date in docs 2025-06-25 23:29:37 -04:00
Mark Backman
6ef2ae12b7 Mock mcp imports 2025-06-25 23:29:37 -04:00
Mark Backman
fe6bbdaefe Skip dataclass attributes to remove duplicate entries 2025-06-25 23:29:37 -04:00
Mark Backman
cc66fddca9 Modify docs auto-gen rules to remove duplicate parameters listing 2025-06-25 23:29:37 -04:00
Mark Backman
04b70ddf13 Add MCPClient docstrings 2025-06-25 22:38:11 -04:00
Mark Backman
bb3bb8d9c6 Improve WebsocketService docstrings 2025-06-25 22:38:11 -04:00
Mark Backman
f80f62c7d1 Add VisionService docstrings 2025-06-25 22:38:11 -04:00
Mark Backman
2007ae4317 Add ImageGenService docstrings 2025-06-25 22:38:11 -04:00
Mark Backman
a1e5a1eff4 Add AIService docstrings 2025-06-25 22:38:11 -04:00
Mark Backman
691999b402 Add AIServices docstring 2025-06-25 22:38:11 -04:00
Mark Backman
33f3a4cea1 Add TTSService docstrings 2025-06-25 22:38:11 -04:00
Mark Backman
ab1d2dbe6a Add STTService docstrings 2025-06-25 22:27:07 -04:00
Mark Backman
f622b281d0 Make call_start_function a private function in llm_service 2025-06-25 22:23:13 -04:00
Mark Backman
fb12bf9b4c Update LLMService docstrings 2025-06-25 22:23:13 -04:00
Aleix Conchillo Flaqué
27af50087e TaskManager: don't warn on reset_watchdog() 2025-06-25 17:29:45 -07:00
Filipi Fuchter
03502bed52 Enabling watchdog and sentry into the freeze-test 2025-06-25 20:53:30 -03:00
Aleix Conchillo Flaqué
27c7e2d150 Merge pull request #2063 from pipecat-ai/aleix/watchdog-timers-remove-start-watchdog
no need to call start_watchdog() only reset_watchdog()
2025-06-25 16:47:44 -07:00
Aleix Conchillo Flaqué
e81d387971 TaskManager: rely on add_done_callback() 2025-06-25 16:44:20 -07:00
Aleix Conchillo Flaqué
ef1ade3a71 allow enabling watchdog timers per frame processor or task 2025-06-25 16:36:19 -07:00
Aleix Conchillo Flaqué
4f032f5b96 update keepalive times depending on watchdog timers 2025-06-25 15:55:16 -07:00
Aleix Conchillo Flaqué
72cb967780 update CHANGELOG with watchdog timers updates 2025-06-25 15:55:16 -07:00
Aleix Conchillo Flaqué
357934a644 watchdog timers are disabled by default use enable_watchdog_timers 2025-06-25 15:55:16 -07:00
Aleix Conchillo Flaqué
327973657f TaskManager: remove wathcdog timer when main task is done 2025-06-25 11:26:21 -07:00
Aleix Conchillo Flaqué
d2730e6741 GooglSTTService: cleanup request queues 2025-06-25 11:12:32 -07:00
Aleix Conchillo Flaqué
eb5ecab104 no need to call start_watchdog() only reset_watchdog() 2025-06-25 11:12:32 -07:00
Mark Backman
202055a9b8 Merge pull request #2065 from pipecat-ai/mb/fix-configdict-openai-realtime
fix: add missing ConfigDict import in openai_realtime_beta/events
2025-06-25 11:40:35 -04:00
Mark Backman
7034a9e3fd fix: add missing ConfigDict import in openai_realtime_beta/events 2025-06-25 11:32:29 -04:00
Filipi da Silva Fuchter
8f7ed12262 Merge pull request #2061 from pipecat-ai/not_force_bot_speaking
Not forcing the bot resume speaking in case we receive no transcription.
2025-06-24 20:57:46 -03:00
Aleix Conchillo Flaqué
96b5320ef9 Merge pull request #2055 from pipecat-ai/aleix/fix-sentry-async
SentryMetrics: send metrics to sentry asynchronously
2025-06-24 16:32:01 -07:00
Filipi Fuchter
d5cd742237 Not forcing the bot resume speaking in case we receive no transcription. 2025-06-24 20:12:49 -03:00
Aleix Conchillo Flaqué
1f1da8942d SentryMetrics: send metrics to sentry asynchronously 2025-06-24 15:56:08 -07:00
Mark Backman
7953e1e9d9 Merge pull request #2054 from pipecat-ai/mb/telnyx-catch-hangup-error
fix: Telnyx, catch error when user has hung up the call first
2025-06-24 18:04:19 -04:00
Mark Backman
d6f7ecc0a3 fix: Telnyx, catch error when user has hung up the call first 2025-06-24 17:28:00 -04:00
Mark Backman
3eed316049 Merge pull request #2020 from snova-jorgep/snova-jorgep/sambanova-integration
Add Sambanova LLM and STT integration
2025-06-24 17:04:24 -04:00
Jorge Piedrahita Ortiz
851cf079c3 Merge branch 'main' into snova-jorgep/sambanova-integration 2025-06-24 16:00:28 -05:00
jhpiedrahitao
dfb0da32a9 fmt 2025-06-24 15:59:40 -05:00
Aleix Conchillo Flaqué
f450da57e5 Merge pull request #2056 from pipecat-ai/khk/fix-22d
Update google libraries used in google audio-in examples
2025-06-24 13:47:59 -07:00
Aleix Conchillo Flaqué
2ec6b6c995 Merge pull request #2060 from pipecat-ai/aleix/watchdog-timeout-secs
FrameProcessor: use watchdog_timeout_secs
2025-06-24 13:36:39 -07:00
Aleix Conchillo Flaqué
53b769a8ec FrameProcessor: use watchdog_timeout_secs 2025-06-24 13:33:47 -07:00
Filipi da Silva Fuchter
4f9adc173a Merge pull request #2004 from pipecat-ai/filipi/pipeline_freeze
Pipeline freeze improvements
2025-06-24 17:20:38 -03:00
Filipi Fuchter
dc4a58877e Fixing merge conflict. 2025-06-24 17:12:40 -03:00
Filipi Fuchter
a6243a6fe7 Merge branch 'main' into filipi/pipeline_freeze
# Conflicts:
#	CHANGELOG.md
#	src/pipecat/pipeline/task.py
#	src/pipecat/processors/frame_processor.py
#	src/pipecat/transports/base_input.py
2025-06-24 17:11:21 -03:00
Aleix Conchillo Flaqué
cf5f1b541a Merge pull request #2049 from pipecat-ai/aleix/introduce-watchdog-timers
introduce watchdog timers
2025-06-24 13:00:57 -07:00
Filipi Fuchter
70e6c48233 Mentioning the fixes in the changelog. 2025-06-24 16:56:46 -03:00
Filipi Fuchter
51f7d14d0a Merge branch 'main' into filipi/pipeline_freeze 2025-06-24 16:44:07 -03:00
Filipi Fuchter
4853d5d1fc Handling the case where user stopped speaking but no new aggregation received. 2025-06-24 16:42:10 -03:00
Aleix Conchillo Flaqué
076a8938f0 add start_watchdog/reset_watchdog to tasks 2025-06-24 11:56:20 -07:00
Aleix Conchillo Flaqué
5a3457ba33 introduce task watchdog timers 2025-06-24 11:56:20 -07:00
Aleix Conchillo Flaqué
2fc224384d Merge pull request #2059 from pipecat-ai/aleix/heartbeatframe-control-frames
HeartbeatFrames are now control frames
2025-06-24 11:55:18 -07:00
Aleix Conchillo Flaqué
a4e6ea5a3f HeartbeatFrames are now control frames 2025-06-24 11:27:39 -07:00
Vanessa Pyne
d3c211f293 Merge pull request #2058 from pipecat-ai/vp-mcp-sse-up
follow up to #1887 - proper MCP SSE support
2025-06-24 13:06:01 -05:00
vipyne
20047c369e mcp: update examples to use SseServerParameter 2025-06-24 12:58:39 -05:00
vipyne
dd1ff237a8 lint mcp_service 2025-06-24 12:58:33 -05:00
Vanessa Pyne
39d80d0b0e Merge pull request #1887 from ezun-kim/feat/mcp-sse-params
Fix SSE server connection handling for MCP client
2025-06-24 12:58:05 -05:00
Kwindla Hultman Kramer
7a48316534 update google libraries used in google audio-in examples 2025-06-24 09:52:04 -07:00
Filipi da Silva Fuchter
031a93ac46 Merge pull request #2053 from pipecat-ai/sentry_dsn_environment_variable
Creating an environment variable for sentry dsn.
2025-06-24 12:10:20 -03:00
Mark Backman
ea6cc1aa95 Merge pull request #2052 from pipecat-ai/mb/11labs-keepalive
Send context_id when available in ElevenLabsTTSService keepalive message
2025-06-24 11:07:07 -04:00
Filipi Fuchter
365260ec44 Creating an environment variable for sentry dsn. 2025-06-24 11:57:14 -03:00
Mark Backman
2eb244c80a Send context_id when available in ElevenLabsTTSService keepalive message 2025-06-24 10:52:49 -04:00
Mark Backman
aee3011d61 Merge pull request #2037 from pipecat-ai/mb/11labs-close-context
Fix: Correctly close the context for ElevenLabsTTSService
2025-06-24 07:44:22 -04:00
Aleix Conchillo Flaqué
40496e7b0f Merge pull request #2034 from pipecat-ai/khk/pause-frames
small fix for processor pause/resume frames
2025-06-23 17:08:41 -07:00
Kwindla Hultman Kramer
6b24f89fa7 small fix for processor pause/resume frames 2025-06-23 16:44:32 -07:00
Filipi Fuchter
2097800042 Allowing to clear the turn analyser 2025-06-23 18:50:37 -03:00
Filipi Fuchter
6739318e68 Forcing user stopped speaking due to timeout to receive audio frame! 2025-06-23 18:50:02 -03:00
Filipi Fuchter
d0bd563d42 Logging the BaseException inside the cancel_task. 2025-06-23 18:48:44 -03:00
Filipi Fuchter
74280829fc Fixed an issue with the FastAPIWebsocketClient to disconnect in case the websocket is already closed. 2025-06-23 18:48:03 -03:00
Filipi Fuchter
3fde8880f2 Fixed a couple of places inside the FrameProcessor where we should not raise the exceptions. 2025-06-23 18:47:54 -03:00
Filipi Fuchter
98d39e0d38 Logging the last 10 frames received in case idle timeout is detected. 2025-06-23 18:47:17 -03:00
Filipi Fuchter
c9cebb5ffe Created an example for testing the bot and try to create freezing conditions. 2025-06-23 18:46:58 -03:00
Mark Backman
f52ac6e99c Merge pull request #1998 from pipecat-ai/mb/fix-38-smart-turn-fal 2025-06-23 17:15:29 -04:00
Mark Backman
787a6b1c6a Merge pull request #2038 from pipecat-ai/mb/openai-realtime-model-update
Update OpenAIRealtimeBetaLLMService model to gpt-4o-realtime-preview-…
2025-06-23 16:30:31 -04:00
Mark Backman
d00a91074e Update OpenAIRealtimeBetaLLMService model to gpt-4o-realtime-preview-2025-06-03 2025-06-23 16:26:42 -04:00
Mark Backman
4e11497a38 Merge pull request #2048 from thibaudbrg/patch-1
Fix missing video_in_enabled in vision bot.py for Moondream template
2025-06-23 16:11:50 -04:00
Tibo
0443d5202a Fix missing video_in_enabled in vision bot.py for Moondream template
The parameter video_in_enabled=True was missing in DailyParams, which prevented image capture 
from working. Without this parameter, UserImageRequestFrame would be sent but no actual image data would be captured from participants.

This fix enables the "Let me take a look" functionality to work as 
intended by allowing the transport to capture video frames for vision processing with Moondream.
2025-06-23 21:17:41 +02:00
Mark Backman
633c25cb13 Merge pull request #2039 from pipecat-ai/mb/remove-lang-validation
OpenAIRealtimeBetaLLMService accepts language for all InputAudioTrans…
2025-06-23 14:41:09 -04:00
jhpiedrahitao
d07f45132f update changelog 2025-06-23 12:54:00 -05:00
jhpiedrahitao
a51280afa6 add 13 and 14 type foundational examples for sambanova iontegration 2025-06-23 12:53:32 -05:00
Jorge Piedrahita Ortiz
be14eb2460 Merge branch 'pipecat-ai:main' into snova-jorgep/sambanova-integration 2025-06-23 12:23:00 -05:00
jhpiedrahitao
e26dbffcbe update sambanova init imports 2025-06-23 12:22:08 -05:00
Mark Backman
59992fd24a Merge pull request #2044 from pipecat-ai/mb/daily-rest-docstring
Add missing arg docstring in DailyRESTHelper
2025-06-23 11:24:44 -04:00
Mark Backman
455362ccaf Merge pull request #2022 from pipecat-ai/mb/turn-tracking-end-cancel-frame
TurnTrackingObserver ends turn upon seeing EndFrame, CancelFrame
2025-06-23 11:24:27 -04:00
Mark Backman
16c0e2460b TurnTrackingObserver ends turn upon seeing EndFrame, CancelFrame 2025-06-23 11:08:51 -04:00
Mark Backman
92246f7125 Add missing arg docstring in DailyRESTHelper 2025-06-22 13:44:59 -04:00
Mark Backman
7737335ec9 OpenAIRealtimeBetaLLMService accepts language for all InputAudioTranscription models 2025-06-21 10:08:46 -04:00
Mark Backman
5cc9b7e0d1 Fix: Correctly close the context for ElevenLabsTTSService 2025-06-20 15:47:03 -04:00
Mark Backman
8c6a441064 Merge pull request #2035 from smokyabdulrahman/feat/aws-polly-lexicon-names-support
Support AWS Polly Lexicon Names parameter
2025-06-20 10:03:27 -04:00
Alrahma
fddc058ce2 add CHANGELOG entry 2025-06-20 14:15:24 +01:00
Alrahma
89750086c5 Support AWS Polly Lexicon Names parameter
Documentation reference
[AWS Managing
Lexicons](https://docs.aws.amazon.com/polly/latest/dg/managing-lexicons.html)
2025-06-20 09:47:46 +01:00
Aleix Conchillo Flaqué
e69406c7e2 Merge pull request #2032 from pipecat-ai/aleix/aws-nova-sonic-function-calls
AWSNovaSonicLLMService: fix function calling
2025-06-19 14:42:47 -07:00
Aleix Conchillo Flaqué
878ae42d84 AWSNovaSonicLLMService: fix function calling 2025-06-19 14:26:34 -07:00
Aleix Conchillo Flaqué
d34ebfc126 Merge pull request #2027 from pipecat-ai/aleix/task-on-idle-timeout-repeated
PipelineTask: fix repeated on_idle_timeout
2025-06-19 14:13:10 -07:00
Aleix Conchillo Flaqué
028f7b2d65 PipelineTask: fix repeated on_idle_timeout 2025-06-19 09:14:10 -07:00
Mark Backman
0aa3ec50f2 Merge pull request #2023 from pipecat-ai/mb/allow-interruptions-true
allow_interruptions=True
2025-06-19 10:24:53 -04:00
Mark Backman
9146def21b Update examples to use default allow_interruptions, fixes to align examples 2025-06-19 10:07:32 -04:00
Aleix Conchillo Flaqué
ebb23a5a8c Merge pull request #2024 from pipecat-ai/aleix/audio-buffer-processor-sync-issues
AudioBufferProcessor: treat all streams as intermittent
2025-06-18 18:26:38 -07:00
Aleix Conchillo Flaqué
b118082984 AudioBufferProcessor: treat all streams as intermittent
This fixes an issue with STTMuteFilter that prevents user audio to be pushed
downstream.
2025-06-18 18:23:31 -07:00
Mark Backman
b5c0ac5f25 allow_interruptions=True 2025-06-18 20:33:40 -04:00
Filipi da Silva Fuchter
dc78e874af Merge pull request #2021 from pipecat-ai/gladia_stt_improvements_changelog
Adding the GladiaSTTService improvements in the changelog.
2025-06-18 18:25:36 -03:00
Filipi Fuchter
c30bde0a2b Adding the GladiaSTTService improvements in the changelog. 2025-06-18 16:19:58 -03:00
Filipi da Silva Fuchter
171597fbe9 Merge pull request #1952 from jqueguiner/feat/gladia-auto-reconnect
feat: Enhance GladiaSTTService with reconnection and audio buffer management features
2025-06-18 16:14:58 -03:00
jhpiedrahitao
fae2d272d5 fmt 2025-06-18 10:53:06 -05:00
jhpiedrahitao
03a067d3e6 add sambanova llm and stt 2025-06-18 10:50:42 -05:00
Mark Backman
f5d028f3b3 Merge pull request #2017 from pipecat-ai/mb/fix-11labs-voice-settings
fix: ElevenLabsTTSService voice settings not being sent
2025-06-18 09:56:46 -04:00
Mark Backman
e5b7dbba90 fix: ElevenLabsTTSService voice settings not being sent 2025-06-18 09:49:17 -04:00
Filipi da Silva Fuchter
7ffba1e0b3 Merge pull request #1950 from pipecat-ai/filipi/tavus_custom_tracks
Sending audio to Tavus using custom tracks
2025-06-18 07:57:19 -03:00
Filipi Fuchter
72cdbf0b78 Mentioning the Tavus improvements in the changelog. 2025-06-18 07:46:04 -03:00
Filipi Fuchter
8b4a86f629 Ignoring the audio level when creating the custom tracks. 2025-06-18 07:45:54 -03:00
Filipi Fuchter
fa15e64fc9 Test script that mimics the behavior expected to be supported by Tavus. 2025-06-18 07:45:38 -03:00
Filipi Fuchter
564f064c71 Refactoring TavusVideoService to send audio using WebRTC audio tracks instead of app-messages. 2025-06-18 07:44:51 -03:00
Filipi Fuchter
4062c7afa0 Refactoring TavusTransport to send audio using WebRTC audio tracks instead of app-messages. 2025-06-18 07:44:38 -03:00
Jean-Louis Queguiner
8071c4ba1c Merge branch 'main' into feat/gladia-auto-reconnect 2025-06-18 08:57:21 +02:00
jqueguiner
3d0ffbc832 🐛 (stt.py): handle websocket connection closure gracefully and log warnings
♻️ (stt.py): refactor reconnection logic into a separate method for clarity
 (stt.py): implement exponential backoff for reconnection attempts to improve reliability
2025-06-18 08:52:43 +02:00
Filipi da Silva Fuchter
1cac94bf97 Merge pull request #1925 from pipecat-ai/filipi/websocket_transport_example_twilio
Websocket client web app to test Twilio.
2025-06-17 16:24:18 -03:00
Mark Backman
c94c51d44f Fix: 38-smart-turn-fal 2025-06-17 15:10:52 -04:00
Mark Backman
96958933af Merge pull request #2016 from pipecat-ai/aleix/example-params-allow-async-objects
examples: create transport params async
2025-06-17 15:08:37 -04:00
Filipi Fuchter
2300c2632e Refactoring how we are organizing the twilio chatbot examples and improving the readmes 2025-06-17 16:08:35 -03:00
Filipi Fuchter
cbd0529674 Merge branch 'main' into filipi/websocket_transport_example_twilio 2025-06-17 15:54:31 -03:00
Filipi da Silva Fuchter
5614e35ac4 Merge pull request #2015 from pipecat-ai/bumping_pipecat_required_versions
Bumping pipecat-ai-krisp required version
2025-06-17 15:42:20 -03:00
Aleix Conchillo Flaqué
c11172caba examples: create transport params async 2025-06-17 11:37:42 -07:00
Filipi Fuchter
11b6e409bb Bumping pipecat-ai-krisp required version 2025-06-17 15:22:31 -03:00
Aleix Conchillo Flaqué
3dca95aa3c Merge pull request #2014 from pipecat-ai/aleix/daily-python-0.19.3
update daily-python to 0.19.3
2025-06-17 10:10:23 -07:00
Aleix Conchillo Flaqué
7ddc706434 update daily-python to 0.19.3 2025-06-17 09:30:28 -07:00
Aleix Conchillo Flaqué
20eebb08e9 update CHANGELOG with AWSTranscribeSTTService Polish support 2025-06-16 10:34:56 -07:00
Aleix Conchillo Flaqué
4abf41b85a Merge pull request #2011 from wuodar/wuodar/polish-lang-aws-transcribe
Support polish language in Amazon Transcribe
2025-06-16 10:33:55 -07:00
Aleix Conchillo Flaqué
e426f7ee7c Merge pull request #2012 from pipecat-ai/aleix/frame-pause-resume-frames
FrameProcessor: handle new FrameProcessorPauseFrame/FrameProcessorResumeFrame
2025-06-16 10:32:38 -07:00
Aleix Conchillo Flaqué
14dc6a7984 FrameProcessor: handle new FrameProcessorPauseFrame/FrameProcessorResumeFrame 2025-06-16 10:31:33 -07:00
Mark Backman
e0a24a3f07 Merge pull request #2006 from pipecat-ai/mb/expose-function-calls-in-progress
Expose has_function_calls_in_progress property
2025-06-16 12:49:07 -04:00
Mark Backman
d1bee22d73 Expose has_function_calls_in_progress property 2025-06-16 12:45:16 -04:00
Jon Taylor
d73f7908f2 Merge pull request #2008 from pipecat-ai/khk/groq-audio
fix groq wav file header parsing
2025-06-16 14:09:09 +01:00
Aleix Conchillo Flaqué
a4ea0d2b82 dev-requirements: update pyright 1.1.400 and ruff 0.11.13 2025-06-15 21:05:03 -07:00
Kacper Włodarczyk
e2c15169b8 feat: support polish language in Amazon Transcribe 2025-06-15 21:44:06 +02:00
Kwindla Hultman Kramer
fe16ed3c73 added changelog entry 2025-06-15 10:49:40 -07:00
Filipi Fuchter
80ce097f90 Using relative URL for the websocket. 2025-06-15 10:49:25 -03:00
Filipi Fuchter
eceaf8a46b Making the path to the web client relative 2025-06-14 21:07:15 -03:00
Kwindla Hultman Kramer
1e3fa4a9c7 fix groq wav file header parsing 2025-06-14 17:41:44 -04:00
Filipi da Silva Fuchter
dc640a7591 Merge pull request #2001 from pipecat-ai/filipi/google_stt_reconnection_issue
Fixed an issue with `GoogleSTTService` where it was constantly reconnecting
2025-06-13 08:29:18 -03:00
Filipi Fuchter
1f072d182c Merge branch 'main' into filipi/google_stt_reconnection_issue
# Conflicts:
#	CHANGELOG.md
2025-06-13 08:26:00 -03:00
Mark Backman
1d64e04ed5 Merge pull request #2002 from pipecat-ai/mb/google-fix-ttfb
Fix: GoogleLLMService TTFB
2025-06-12 12:10:01 -04:00
Mark Backman
22f4f0b79e Update 14e example name 2025-06-12 11:45:59 -04:00
Mark Backman
69c63293fb fix: GoogleLLMService TTFB value 2025-06-12 11:43:27 -04:00
Filipi Fuchter
c1db13ceeb Fixed an issue with GoogleSTTService where it was constantly reconnecting before starting to receive audio from the user. 2025-06-12 12:07:33 -03:00
Filipi Fuchter
70eadee0aa Bumping the @pipecat-ai/websocket-transport dependency. 2025-06-11 18:30:16 -03:00
Filipi Fuchter
0073a868d4 Websocket client web app to test Twilio. 2025-06-10 11:34:02 -03:00
jqueguiner
25ff8ef37b (config.py): add new configuration options for lip-sync optimization, context adaptation, and additional context to enhance translation accuracy
♻️ (stt.py): increase default max buffer size from 5MB to 20MB to accommodate larger audio data
♻️ (stt.py): simplify audio sending logic by removing chunking and sending the entire buffered audio at once for improved performance
2025-06-05 16:51:29 -07:00
jqueguiner
02cc6f3d56 Enhance GladiaSTTService with reconnection and audio buffer management features
- Added parameters for maximum reconnection attempts, reconnection delay, and maximum audio buffer size.
- Implemented automatic reconnection logic with exponential backoff.
- Introduced audio buffer management to handle audio data efficiently, including trimming excess data.
- Updated connection handling to ensure proper cleanup and management of WebSocket connections.
- Enhanced audio sending logic to support buffered audio transmission after reconnections.
2025-06-03 03:16:57 -07:00
ezun-kim
3da711ba8b Fix SSE server connection handling for MCP client
### Summary
This PR improves the MCP (Model Context Protocol) client's SSE (Server-Sent Events) server connection handling by replacing the generic string parameter with a proper `SseServerParameters` class.

### Changes
- **Breaking Change**: Changed `server_params` type from `Union[StdioServerParameters, str]` to `Union[StdioServerParameters, SseServerParameters]`
- Added import for `SseServerParameters` from `mcp.client.session_group`
- Updated SSE client connection to use structured parameters instead of a simple URL string
- Fixed error message to correctly reflect the expected parameter types
- Improved logging by changing info-level log to debug-level for consistency

### Details

#### Before
The SSE client connection only accepted a URL string:
```python
async with self._client(self._server_params) as (read, write):
```

#### After
Now properly unpacks SSE server parameters:
```python
async with self._client(
    url=self._server_params.url,
    headers=self._server_params.headers,
    timeout=self._server_params.timeout,
    sse_read_timeout=self._server_params.sse_read_timeout
) as (read, write):
```

### Benefits
- **Type Safety**: Stronger type checking with dedicated `SseServerParameters` class
- **Extended Configuration**: Support for custom headers (authentication), timeouts, and SSE-specific settings
- **Better Error Messages**: Clear type error messages when incorrect parameters are provided
- **Improved Debugging**: Debug logging of SSE server parameters for troubleshooting

### Migration Guide
Users need to update their SSE server initialization:
```python
# Before
client = MCPClient("https://example.com/sse")

# After
from mcp.client.session_group import SseServerParameters
client = MCPClient(SseServerParameters(
    url="https://example.com/sse",
    headers={"Authorization": "Bearer token"},
    timeout=30,
    sse_read_timeout=60
))
```

### Testing
- [ ] Tested with StdioServerParameters (unchanged behavior)
- [ ] Tested with SseServerParameters with various configurations
- [ ] Verified error handling for invalid parameter types

---

This is a necessary change to support production-ready SSE connections with proper authentication and timeout handling.
2025-05-24 22:35:57 +09:00
333 changed files with 16708 additions and 2155 deletions

View File

@@ -5,6 +5,154 @@ All notable changes to **Pipecat** will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- Added `watchdog_coroutine()`. This is a watchdog helper for couroutines. So,
if you have a coroutine that is waiting for a result and that takes a long
time, you will need to wrap it with `watchdog_coroutine()` so the watchdog
timers are reset regularly.
### Fixed
- Fixed a `AWSNovaSonicLLMService` issue introduced in 0.0.72.
## [0.0.73] - 2025-06-26
### Fixed
- Fixed an issue introduced in 0.0.72 that would cause `ElevenLabsTTSService`,
`GladiaSTTService`, `NeuphonicTTSService` and `OpenAIRealtimeBetaLLMService`
to throw an error.
## [0.0.72] - 2025-06-26
### Added
- Added logging and improved error handling to help diagnose and prevent potential
Pipeline freezes.
- Added `WatchdogQueue`, `WatchdogPriorityQueue`, `WatchdogEvent` and
`WatchdogAsyncIterator`. These helper utilities reset watchdog timers
appropriately before they expire. When watchdog timers are disabled, the
utilities behave as standard counterparts without side effects.
- Introduce task watchdog timers. Watchdog timers are used to detect if a
Pipecat task is taking longer than expected (by default 5 seconds). Watchdog
timers are disabled by default and can be enabled globally by passing
`enable_watchdog_timers` argument to `PipelineTask` constructor. It is
possible to change the default watchdog timer timeout by using the
`watchdog_timeout` argument. You can also log how long it takes to reset the
watchdog timers which is done with the `enable_watchdog_logging`. You can
control all these settings per each frame processor or even per task. That is,
you can set `enable_watchdog_timers`, `enable_watchdog_logging` and
`watchdog_timeout` when creating any frame processor through their constructor
arguments or when you create a task with `FrameProcessor.create_task()`. Note
that watchdog timers only work with Pipecat tasks and will not work if you use
`asycio.create_task()` or similar.
- Added `lexicon_names` parameter to `AWSPollyTTSService.InputParams`.
- Added reconnection logic and audio buffer management to `GladiaSTTService`.
- The `TurnTrackingObserver` now ends a turn upon observing an `EndFrame` or
`CancelFrame`.
- Added Polish support to `AWSTranscribeSTTService`.
- Added new frames `FrameProcessorPauseFrame` and `FrameProcessorResumeFrame`
which allow pausing and resuming frame processing for a given frame
processor. These are control frames, so they are ordered. Pausing frame
processor will keep old frames in the internal queues until resume takes
place. Frames being pushed while a frame processor is paused will be pushed to
the queues. When frame processing is resumed all queued frames will be
processed in order. Also added `FrameProcessorPauseUrgentFrame` and
`FrameProcessorResumeUrgentFrame` which are system frames and therefore they
have high priority.
- Added a property called `has_function_calls_in_progress` in
`LLMAssistantContextAggregator` that exposes whether a function call is in
progress.
- Added `SambaNovaLLMService` which provides llm api integration with an
OpenAI-compatible interface.
- Added `SambaNovaTTSService` which provides speech-to-text functionality using
SambaNovas's (whisper) API.
- Add fundational examples for function calling and transcription
`14s-function-calling-sambanova.py`, `13g-sambanova-transcription.py`
### Changed
- `HeartbeatFrame`s are now control frames. This will make it easier to detect
pipeline freezes. Previously, heartbeat frames were system frames which meant
they were not get queued with other frames, making it difficult to detect
pipeline stalls.
- Updated `OpenAIRealtimeBetaLLMService` to accept `language` in the
`InputAudioTranscription` class for all models.
- Updated the default model for `OpenAIRealtimeBetaLLMService` to
`gpt-4o-realtime-preview-2025-06-03`.
- The `PipelineParams` arg `allow_interruptions` now defaults to `True`.
- `TavusTransport` and `TavusVideoService` now send audio to Tavus using WebRTC
audio tracks instead of `app-messages` over WebSocket. This should improve the
overall audio quality.
- Upgraded `daily-python` to 0.19.3.
### Fixed
- Fixed an issue that would cause heartbeat frames to be sent before processors
were started.
- Fixed an event loop blocking issue when using `SentryMetrics`.
- Fixed an issue in `FastAPIWebsocketClient` to ensure proper disconnection
when the websocket is already closed.
- Fixed an issue where the `UserStoppedSpeakingFrame` was not received if the
transport was not receiving new audio frames.
- Fixed an edge case where if the user interrupted the bot but no new aggregation
was received, the bot would not resume speaking.
- Fixed an issue with `TelnyxFrameSerializer` where it would throw an exception
when the user hung up the call.
- Fixed an issue with `ElevenLabsTTSService` where the context was not being
closed.
- Fixed function calling in `AWSNovaSonicLLMService`.
- Fixed an issue that would cause multiple `PipelineTask.on_idle_timeout`
events to be triggered repeatedly.
- Fixed an issue that was causing user and bot speech to not be synchronized
during recordings.
- Fixed an issue where voice settings weren't applied to ElevenLabsTTSService.
- Fixed an issue with `GroqTTSService` where it was not properly parsing the
WAV file header.
- Fixed an issue with `GoogleSTTService` where it was constantly reconnecting
before starting to receive audio from the user.
- Fixed an issue where `GoogleLLMService`'s TTFB value was incorrect.
### Deprecated
- `AudioBufferProcessor` parameter `user_continuos_stream` is deprecated.
### Other
- Rename `14e-function-calling-gemini.py` to `14e-function-calling-google.py`.
## [0.0.71] - 2025-06-10
### Added

View File

@@ -41,36 +41,107 @@ We use Ruff for code linting and formatting. Please ensure your code passes all
We follow Google-style docstrings with these specific conventions:
- Class docstrings should fully document all parameters used in `__init__`
- We don't require separate docstrings for `__init__` methods when parameters are documented in the class docstring
- Property methods should have docstrings explaining their purpose and return value
**Regular Classes:**
Example of correctly documented class:
- Class docstring describes the class purpose and key functionality
- `__init__` method has its own docstring with complete `Args:` section documenting all parameters
- All public methods must have docstrings with `Args:` and `Returns:` sections as appropriate
**Dataclasses:**
- Class docstring describes the purpose and documents all fields in a `Parameters:` section
- No `__init__` docstring (auto-generated)
**Properties:**
- Must have docstrings with `Returns:` section
**Abstract Methods:**
- Must have docstrings explaining what subclasses should implement
**`__init__.py` Files:**
- **Skip docstrings** for pure import/re-export modules
- **Add brief docstrings** for top-level packages or those with initialization logic
**Enums:**
- Class docstring describes the enumeration purpose
- Use `Parameters:` section to document each enum value and its meaning
- No `__init__` docstring (Enums don't have custom constructors)
#### Examples:
```python
class MyClass:
"""Class description.
# Regular class
class MyService(BaseService):
"""Description of what the service does.
Additional details about the class.
Args:
param1: Description of first parameter.
param2: Description of second parameter.
Provides detailed explanation of the service's functionality,
key features, and usage patterns.
"""
def __init__(self, param1, param2):
# No docstring required here as parameters are documented above
self.param1 = param1
self.param2 = param2
def __init__(self, param1: str, param2: bool = True, **kwargs):
"""Initialize the service.
Args:
param1: Description of param1.
param2: Description of param2. Defaults to True.
**kwargs: Additional arguments passed to parent.
"""
super().__init__(**kwargs)
@property
def some_property(self) -> str:
"""Get the formatted property value.
def sample_rate(self) -> int:
"""Get the current sample rate.
Returns:
A string representation of the property.
The sample rate in Hz.
"""
return f"Property: {self.param1}"
return self._sample_rate
async def process_data(self, data: str) -> bool:
"""Process the provided data.
Args:
data: The data to process.
Returns:
True if processing succeeded.
"""
pass
# Dataclass
@dataclass
class ConfigParams:
"""Configuration parameters for the service.
Parameters:
host: The host address.
port: The port number. Defaults to 8080.
timeout: Connection timeout in seconds.
"""
host: str
port: int = 8080
timeout: float = 30.0
# Enum class
class Status(Enum):
"""Status codes for processing operations.
Parameters:
PENDING: Operation is queued but not started.
RUNNING: Operation is currently in progress.
COMPLETED: Operation finished successfully.
FAILED: Operation encountered an error.
"""
PENDING = "pending"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
```
# Contributor Covenant Code of Conduct

View File

@@ -53,8 +53,8 @@ You can connect to Pipecat from any platform using our official SDKs:
| Category | Services |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Cartesia](https://docs.pipecat.ai/server/services/stt/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Parakeet (NVIDIA)](https://docs.pipecat.ai/server/services/stt/parakeet), [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
| LLMs | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [AWS](https://docs.pipecat.ai/server/services/llm/aws), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Qwen](https://docs.pipecat.ai/server/services/llm/qwen), [Together AI](https://docs.pipecat.ai/server/services/llm/together) |
| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Cartesia](https://docs.pipecat.ai/server/services/stt/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Parakeet (NVIDIA)](https://docs.pipecat.ai/server/services/stt/parakeet), [SambaNova (Whisper)](https://docs.pipecat.ai/server/services/stt/sambanova) [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
| LLMs | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [AWS](https://docs.pipecat.ai/server/services/llm/aws), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Qwen](https://docs.pipecat.ai/server/services/llm/qwen), [SambaNova](https://docs.pipecat.ai/server/services/llm/sambanova) [Together AI](https://docs.pipecat.ai/server/services/llm/together) |
| Text-to-Speech | [AWS](https://docs.pipecat.ai/server/services/tts/aws), [Azure](https://docs.pipecat.ai/server/services/tts/azure), [Cartesia](https://docs.pipecat.ai/server/services/tts/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/tts/deepgram), [ElevenLabs](https://docs.pipecat.ai/server/services/tts/elevenlabs), [FastPitch (NVIDIA)](https://docs.pipecat.ai/server/services/tts/fastpitch), [Fish](https://docs.pipecat.ai/server/services/tts/fish), [Google](https://docs.pipecat.ai/server/services/tts/google), [LMNT](https://docs.pipecat.ai/server/services/tts/lmnt), [MiniMax](https://docs.pipecat.ai/server/services/tts/minimax), [Neuphonic](https://docs.pipecat.ai/server/services/tts/neuphonic), [OpenAI](https://docs.pipecat.ai/server/services/tts/openai), [Piper](https://docs.pipecat.ai/server/services/tts/piper), [PlayHT](https://docs.pipecat.ai/server/services/tts/playht), [Rime](https://docs.pipecat.ai/server/services/tts/rime), [Sarvam](https://docs.pipecat.ai/server/services/tts/sarvam), [XTTS](https://docs.pipecat.ai/server/services/tts/xtts) |
| Speech-to-Speech | [AWS Nova Sonic](https://docs.pipecat.ai/server/services/s2s/aws), [Gemini Multimodal Live](https://docs.pipecat.ai/server/services/s2s/gemini), [OpenAI Realtime](https://docs.pipecat.ai/server/services/s2s/openai) |
| Transport | [Daily (WebRTC)](https://docs.pipecat.ai/server/services/transport/daily), [FastAPI Websocket](https://docs.pipecat.ai/server/services/transport/fastapi-websocket), [SmallWebRTCTransport](https://docs.pipecat.ai/server/services/transport/small-webrtc), [WebSocket Server](https://docs.pipecat.ai/server/services/transport/websocket-server), Local |

View File

@@ -3,11 +3,11 @@ coverage~=7.6.12
grpcio-tools~=1.67.1
pip-tools~=7.4.1
pre-commit~=4.0.1
pyright~=1.1.397
pyright~=1.1.400
pytest~=8.3.4
pytest-asyncio~=0.25.3
pytest-aiohttp==1.1.0
ruff~=0.11.1
ruff~=0.11.13
setuptools~=70.0.0
setuptools_scm~=8.1.0
python-dotenv~=1.0.1

View File

@@ -1,5 +1,6 @@
import logging
import sys
from datetime import datetime
from pathlib import Path
# Configure logging
@@ -13,7 +14,8 @@ sys.path.insert(0, str(project_root / "src"))
# Project information
project = "pipecat-ai"
copyright = "2024, Daily"
current_year = datetime.now().year
copyright = f"2024-{current_year}, Daily" if current_year > 2024 else "2024, Daily"
author = "Daily"
# General configuration
@@ -26,16 +28,14 @@ extensions = [
# Napoleon settings
napoleon_google_docstring = True
napoleon_numpy_docstring = False
napoleon_include_init_with_doc = True
# AutoDoc settings
autodoc_default_options = {
"members": True,
"member-order": "bysource",
"special-members": "__init__",
"undoc-members": True,
"exclude-members": "__weakref__",
"exclude-members": "__weakref__,model_config",
"no-index": True,
"show-inheritance": True,
}
@@ -145,12 +145,34 @@ autodoc_mock_imports = [
"transformers.AutoFeatureExtractor",
# Also add specific classes that are imported
"AutoFeatureExtractor",
# Sentry dependencies
"sentry_sdk",
# AWS Nova Sonic dependencies
"aws_sdk_bedrock_runtime",
"aws_sdk_bedrock_runtime.client",
"aws_sdk_bedrock_runtime.config",
"aws_sdk_bedrock_runtime.models",
"smithy_aws_core",
"smithy_aws_core.credentials_resolvers",
"smithy_aws_core.credentials_resolvers.static",
"smithy_aws_core.identity",
"smithy_core",
"smithy_core.aio",
"smithy_core.aio.eventstream",
# MCP dependencies (you may already have these)
"mcp",
"mcp.client",
"mcp.client.session_group",
"mcp.client.sse",
"mcp.client.stdio",
"mcp.ClientSession",
"mcp.StdioServerParameters",
]
# HTML output settings
html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
autodoc_typehints = "description"
autodoc_typehints = "signature" # Show type hints in the signature only, not in the docstring
html_show_sphinx = False
@@ -249,6 +271,10 @@ def clean_title(title: str) -> str:
"playht": "PlayHT",
"xtts": "XTTS",
"lmnt": "LMNT",
"stt": "STT",
"tts": "TTS",
"llm": "LLM",
"rtvi": "RTVI",
}
# Check if the entire title is a special case

View File

@@ -42,6 +42,7 @@ pipecat-ai[openai]
pipecat-ai[qwen]
pipecat-ai[remote-smart-turn]
# pipecat-ai[riva] # Mocked
pipecat-ai[sambanova]
pipecat-ai[silero]
pipecat-ai[simli]
pipecat-ai[soundfile]

View File

@@ -107,4 +107,10 @@ MINIMAX_API_KEY=...
MINIMAX_GROUP_ID=...
# Sarvam AI
SARVAM_API_KEY=...
SARVAM_API_KEY=...
# SambaNova
SAMBANOVA_API_KEY=...
# Sentry
SENTRY_DSN=...

View File

@@ -133,7 +133,8 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -71,6 +71,8 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -148,10 +148,8 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
observers=[TranscriptionLogObserver()],
)

View File

@@ -75,7 +75,13 @@ async def main(room_url: str, token: str):
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):

View File

@@ -170,7 +170,6 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -198,7 +198,6 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -211,7 +211,6 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -67,10 +67,8 @@ async def main(transport: DailyTransport):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -192,7 +192,6 @@ async def main(transport: DailyTransport):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -47,7 +47,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
Pipeline([imagegen, transport.output()]),
params=PipelineParams(enable_metrics=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
# Register an event handler so we can play the audio when the client joins

View File

@@ -93,10 +93,8 @@ async def run_example(webrtc_connection: SmallWebRTCConnection):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -75,10 +75,8 @@ async def main():
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -158,7 +158,8 @@ async def main():
],
),
params=PipelineParams(
allow_interruptions=True, enable_metrics=True, enable_usage_metrics=True
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -133,10 +133,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -83,10 +83,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -113,10 +113,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -87,10 +87,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -81,10 +81,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -88,10 +88,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -86,10 +86,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -90,10 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -84,11 +84,9 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
audio_out_sample_rate=24000,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -89,10 +89,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -87,10 +87,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -92,10 +92,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -80,10 +80,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -85,7 +85,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -87,10 +87,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -88,10 +88,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -86,10 +86,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -89,10 +89,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -83,10 +83,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -80,10 +80,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -8,8 +8,8 @@ import argparse
import os
from dataclasses import dataclass
import google.ai.generativelanguage as glm
from dotenv import load_dotenv
from google.genai.types import Content, Part
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
@@ -164,9 +164,7 @@ class TanscriptionContextFixup(FrameProcessor):
and last_part.inline_data
and last_part.inline_data.mime_type == "audio/wav"
):
self._context.messages[-2] = glm.Content(
role="user", parts=[glm.Part(text=self._transcript)]
)
self._context.messages[-2] = Content(role="user", parts=[Part(text=self._transcript)])
def add_transcript_back_to_inference_output(self):
if not self._transcript:
@@ -258,7 +256,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -77,8 +77,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -83,10 +83,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -86,10 +86,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -70,10 +70,8 @@ async def main():
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -90,10 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -89,10 +89,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -85,7 +85,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):

View File

@@ -101,7 +101,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(allow_interruptions=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")

View File

@@ -101,7 +101,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(allow_interruptions=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")

View File

@@ -101,7 +101,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(allow_interruptions=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")

View File

@@ -84,7 +84,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
pipeline,
params=PipelineParams(
enable_metrics=True,
report_only_initial_ttfb=False,
enable_usage_metrics=True,
),
)

View File

@@ -0,0 +1,108 @@
#
# Copyright (c) 20242025, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import argparse
import os
import time
from dotenv import load_dotenv
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.audio.vad.vad_analyzer import VADParams
from pipecat.frames.frames import Frame, TranscriptionFrame, UserStoppedSpeakingFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.services.sambanova.stt import SambaNovaSTTService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
from pipecat.transports.services.daily import DailyParams
load_dotenv(override=True)
STOP_SECS = 2.0
class TranscriptionLogger(FrameProcessor):
"""Measures transcription latency.
Uses the (intentionally) long STOP_SECS parameter to give the transcription time to finish,
then outputs the timing between when the VAD first classified audio input as not-speech and
the delivery of the last transcription frame.
"""
def __init__(self):
super().__init__()
self._last_transcription_time = time.time()
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, UserStoppedSpeakingFrame):
logger.debug(
f"Transcription latency: {(STOP_SECS - (time.time() - self._last_transcription_time)):.2f}"
)
if isinstance(frame, TranscriptionFrame):
self._last_transcription_time = time.time()
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
transport_params = {
"daily": lambda: DailyParams(
audio_in_enabled=True,
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=STOP_SECS)),
),
"twilio": lambda: FastAPIWebsocketParams(
audio_in_enabled=True,
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=STOP_SECS)),
),
"webrtc": lambda: TransportParams(
audio_in_enabled=True,
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=STOP_SECS)),
),
}
async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_sigint: bool):
logger.info(f"Starting bot")
stt = SambaNovaSTTService(
model="Whisper-Large-v3",
api_key=os.getenv("SAMBANOVA_API_KEY"),
)
tl = TranscriptionLogger()
pipeline = Pipeline([transport.input(), stt, tl])
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
await task.cancel()
runner = PipelineRunner(handle_sigint=handle_sigint)
await runner.run(task)
if __name__ == "__main__":
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -134,10 +134,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -127,8 +127,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -172,8 +172,8 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -16,7 +16,7 @@ from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import TTSSpeakFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
@@ -116,7 +116,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):

View File

@@ -17,7 +17,7 @@ from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
@@ -158,7 +158,13 @@ indicate you should use the get_image tool are:
]
)
task = PipelineTask(pipeline)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):

View File

@@ -183,7 +183,6 @@ indicate you should use the get_image tool are:
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -121,7 +121,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -111,7 +111,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -120,7 +120,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -119,7 +119,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -117,7 +117,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -126,10 +126,8 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -126,10 +126,8 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -120,10 +120,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -90,10 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -116,7 +116,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -122,7 +122,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -118,10 +118,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -134,10 +134,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -0,0 +1,152 @@
#
# Copyright (c) 20242025, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import argparse
import os
from dotenv import load_dotenv
from loguru import logger
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import TTSSpeakFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.llm_response import LLMUserAggregatorParams
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.llm_service import FunctionCallParams
from pipecat.services.sambanova.llm import SambaNovaLLMService
from pipecat.services.sambanova.stt import SambaNovaSTTService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
from pipecat.transports.services.daily import DailyParams
load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.result_callback({"conditions": "nice", "temperature": "75"})
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
transport_params = {
"daily": lambda: DailyParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
"twilio": lambda: FastAPIWebsocketParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
"webrtc": lambda: TransportParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
}
async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_sigint: bool):
logger.info(f"Starting bot")
stt = SambaNovaSTTService(
model="Whisper-Large-v3",
api_key=os.getenv("SAMBANOVA_API_KEY"),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
)
llm = SambaNovaLLMService(
api_key=os.getenv("SAMBANOVA_API_KEY"),
model="Llama-4-Maverick-17B-128E-Instruct",
)
# You can also register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the user's location.",
},
},
required=["location"],
)
tools = ToolsSchema(standard_tools=[weather_function])
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages, tools)
context_aggregator = llm.create_context_aggregator(
context, user_params=LLMUserAggregatorParams(aggregation_timeout=0.05)
)
pipeline = Pipeline(
[
transport.input(),
stt,
context_aggregator.user(),
llm,
tts,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
await task.cancel()
runner = PipelineRunner(handle_sigint=handle_sigint)
await runner.run(task)
if __name__ == "__main__":
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -147,7 +147,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):

View File

@@ -135,7 +135,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):

View File

@@ -90,8 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -117,9 +117,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
report_only_initial_ttfb=True,
enable_usage_metrics=True,
),
)

View File

@@ -186,10 +186,8 @@ Remember, your responses should be short. Just one or two sentences, usually."""
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -179,10 +179,8 @@ Remember, your responses should be short. Just one or two sentences, usually."""
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -223,10 +223,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -233,10 +233,8 @@ Remember, your responses should be short. Just one or two sentences, usually."""
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -222,10 +222,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
# report_only_initial_ttfb=True,
),
)

View File

@@ -275,10 +275,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
# report_only_initial_ttfb=True,
),
)

View File

@@ -242,10 +242,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -79,10 +79,8 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=24000,
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -96,10 +96,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=24000,
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -147,10 +147,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -353,10 +353,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -564,7 +564,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -9,8 +9,8 @@ import asyncio
import os
import time
import google.ai.generativelanguage as glm
from dotenv import load_dotenv
from google.genai.types import Content, Part
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
@@ -611,9 +611,7 @@ class OutputGate(FrameProcessor):
await self._notifier.wait()
transcription = await self._transcription_buffer.wait_for_transcription() or "-"
self._context._messages.append(
glm.Content(role="user", parts=[glm.Part(text=transcription)])
)
self._context.add_message(Content(role="user", parts=[Part(text=transcription)]))
self.open_gate()
for frame, direction in self._frames_buffer:
@@ -746,7 +744,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -103,10 +103,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -14,15 +14,25 @@ from loguru import logger
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import EndFrame, LLMMessagesFrame, TTSTextFrame, UserStartedSpeakingFrame
from pipecat.observers.loggers.debug_log_observer import DebugLogObserver, FrameEndpoint
from pipecat.observers.loggers.llm_log_observer import LLMLogObserver
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.processors.filters.stt_mute_filter import STTMuteConfig, STTMuteFilter, STTMuteStrategy
from pipecat.processors.filters.stt_mute_filter import (
STTMuteConfig,
STTMuteFilter,
STTMuteFrame,
STTMuteStrategy,
)
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.deepgram.tts import DeepgramTTSService
from pipecat.services.llm_service import FunctionCallParams
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.transports.base_input import BaseInputTransport
from pipecat.transports.base_output import BaseOutputTransport
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
from pipecat.transports.services.daily import DailyParams
@@ -30,14 +40,6 @@ from pipecat.transports.services.daily import DailyParams
load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
# Add a delay to test interruption during function calls
logger.info("Weather API call starting...")
await asyncio.sleep(5) # 5-second delay
logger.info("Weather API call completed")
await params.result_callback({"conditions": "nice", "temperature": "75"})
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
@@ -69,39 +71,59 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
stt_mute_processor = STTMuteFilter(
config=STTMuteConfig(
strategies={
STTMuteStrategy.MUTE_UNTIL_FIRST_BOT_COMPLETE,
STTMuteStrategy.FUNCTION_CALL,
STTMuteStrategy.MUTE_UNTIL_FIRST_BOT_COMPLETE,
}
),
)
tts = DeepgramTTSService(api_key=os.getenv("DEEPGRAM_API_KEY"), voice="aura-helios-en")
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"))
llm.register_function("get_current_weather", fetch_weather_from_api)
async def transfer_to_human(params: FunctionCallParams):
# Add a delay to test interruption during function calls
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
caller_name = params.arguments.get("caller_name", "Unknown")
human_agent_name = params.arguments.get("human_agent_name", "Unknown")
logger.info(f"Transfer starting... {caller_name} wants to transfer to {human_agent_name}")
await task.queue_frame(STTMuteFrame(True))
await asyncio.sleep(
5
) # 5-second delay to simulate a transfer. You could play hold music here too.
messages.clear()
messages.append(
{
"role": "system",
"content": f"You are an agent named {human_agent_name}. Greet {caller_name} and let them know you are taking over the conversation.",
}
)
await params.llm.push_frame(LLMMessagesFrame(messages))
logger.info("Transfer complete, calling result callback")
await params.result_callback({"transfer_successful": True})
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"))
llm.register_function("transfer_to_human", transfer_to_human)
transfer_function = FunctionSchema(
name="transfer_to_human",
description="Transfer the conversation to a human agent.",
properties={
"location": {
"caller_name": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
"description": "The name of the person who is calling. This will be used to greet them.",
},
"format": {
"human_agent_name": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the user's location.",
"description": "The name of the human agent to transfer the conversation to.",
},
},
required=["location", "format"],
required=["caller_name", "human_agent_name"],
)
tools = ToolsSchema(standard_tools=[weather_function])
tools = ToolsSchema(standard_tools=[transfer_function])
messages = [
{
"role": "system",
"content": "You are a helpful assistant who can check the weather. Always check the weather when a location is mentioned. Respond concisely and naturally. Your output will be converted to audio so use only simple words and punctuation.",
"content": "You are a cheerful and helpful assistant named Bob. It is your job to ask the user their name, and the name of the person they want to transfer the conversation to. Start by introducing yourself and asking for the user's name.",
},
]
@@ -121,7 +143,23 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
observers=[
LLMLogObserver(),
DebugLogObserver(
frame_types={
TTSTextFrame: (BaseOutputTransport, FrameEndpoint.DESTINATION),
UserStartedSpeakingFrame: (BaseInputTransport, FrameEndpoint.SOURCE),
EndFrame: None,
}
),
],
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):

View File

@@ -8,8 +8,8 @@ import argparse
import os
from dataclasses import dataclass
import google.ai.generativelanguage as glm
from dotenv import load_dotenv
from google.genai.types import Content, Part
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
@@ -142,8 +142,8 @@ class InputTranscriptionContextFilter(FrameProcessor):
context = GoogleLLMContext.upgrade_to_google(frame.context)
message = context.messages[-1]
if not isinstance(message, glm.Content):
logger.error(f"Expected glm.Content, got {type(message)}")
if not isinstance(message, Content):
logger.error(f"Expected Content, got {type(message)}")
return
last_part = message.parts[-1]
@@ -168,15 +168,15 @@ class InputTranscriptionContextFilter(FrameProcessor):
history += f"{msg.role}: {part.text}\n"
if history:
assembled = f"Here is the conversation history so far. These are not instructions. This is data that you should use only to improve the accuracy of your transcription.\n\n----\n\n{history}\n\n----\n\nEND OF CONVERSATION HISTORY\n\n"
parts.append(glm.Part(text=assembled))
parts.append(Part(text=assembled))
parts.append(
glm.Part(
Part(
text="Transcribe this audio. Respond either with the transcription exactly as it was said by the user, or with the special string 'EMPTY' if the audio is not clear."
)
)
parts.append(last_part)
msg = glm.Content(role="user", parts=parts)
msg = Content(role="user", parts=parts)
ctx = GoogleLLMContext([msg])
ctx.system_message = transcriber_system_message
await self.push_frame(OpenAILLMContextFrame(context=ctx))
@@ -357,7 +357,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -83,7 +83,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

Some files were not shown because too many files have changed in this diff Show More