Compare commits

...

230 Commits

Author SHA1 Message Date
Aleix Conchillo Flaqué
5a682f8c1f AudioBufferProcessor: record with lowest sample rate
Fixes #1653
2025-06-19 14:18:54 -07:00
Aleix Conchillo Flaqué
d34ebfc126 Merge pull request #2027 from pipecat-ai/aleix/task-on-idle-timeout-repeated
PipelineTask: fix repeated on_idle_timeout
2025-06-19 14:13:10 -07:00
Aleix Conchillo Flaqué
028f7b2d65 PipelineTask: fix repeated on_idle_timeout 2025-06-19 09:14:10 -07:00
Mark Backman
0aa3ec50f2 Merge pull request #2023 from pipecat-ai/mb/allow-interruptions-true
allow_interruptions=True
2025-06-19 10:24:53 -04:00
Mark Backman
9146def21b Update examples to use default allow_interruptions, fixes to align examples 2025-06-19 10:07:32 -04:00
Aleix Conchillo Flaqué
ebb23a5a8c Merge pull request #2024 from pipecat-ai/aleix/audio-buffer-processor-sync-issues
AudioBufferProcessor: treat all streams as intermittent
2025-06-18 18:26:38 -07:00
Aleix Conchillo Flaqué
b118082984 AudioBufferProcessor: treat all streams as intermittent
This fixes an issue with STTMuteFilter that prevents user audio to be pushed
downstream.
2025-06-18 18:23:31 -07:00
Mark Backman
b5c0ac5f25 allow_interruptions=True 2025-06-18 20:33:40 -04:00
Filipi da Silva Fuchter
dc78e874af Merge pull request #2021 from pipecat-ai/gladia_stt_improvements_changelog
Adding the GladiaSTTService improvements in the changelog.
2025-06-18 18:25:36 -03:00
Filipi Fuchter
c30bde0a2b Adding the GladiaSTTService improvements in the changelog. 2025-06-18 16:19:58 -03:00
Filipi da Silva Fuchter
171597fbe9 Merge pull request #1952 from jqueguiner/feat/gladia-auto-reconnect
feat: Enhance GladiaSTTService with reconnection and audio buffer management features
2025-06-18 16:14:58 -03:00
Mark Backman
f5d028f3b3 Merge pull request #2017 from pipecat-ai/mb/fix-11labs-voice-settings
fix: ElevenLabsTTSService voice settings not being sent
2025-06-18 09:56:46 -04:00
Mark Backman
e5b7dbba90 fix: ElevenLabsTTSService voice settings not being sent 2025-06-18 09:49:17 -04:00
Filipi da Silva Fuchter
7ffba1e0b3 Merge pull request #1950 from pipecat-ai/filipi/tavus_custom_tracks
Sending audio to Tavus using custom tracks
2025-06-18 07:57:19 -03:00
Filipi Fuchter
72cdbf0b78 Mentioning the Tavus improvements in the changelog. 2025-06-18 07:46:04 -03:00
Filipi Fuchter
8b4a86f629 Ignoring the audio level when creating the custom tracks. 2025-06-18 07:45:54 -03:00
Filipi Fuchter
fa15e64fc9 Test script that mimics the behavior expected to be supported by Tavus. 2025-06-18 07:45:38 -03:00
Filipi Fuchter
564f064c71 Refactoring TavusVideoService to send audio using WebRTC audio tracks instead of app-messages. 2025-06-18 07:44:51 -03:00
Filipi Fuchter
4062c7afa0 Refactoring TavusTransport to send audio using WebRTC audio tracks instead of app-messages. 2025-06-18 07:44:38 -03:00
Jean-Louis Queguiner
8071c4ba1c Merge branch 'main' into feat/gladia-auto-reconnect 2025-06-18 08:57:21 +02:00
jqueguiner
3d0ffbc832 🐛 (stt.py): handle websocket connection closure gracefully and log warnings
♻️ (stt.py): refactor reconnection logic into a separate method for clarity
 (stt.py): implement exponential backoff for reconnection attempts to improve reliability
2025-06-18 08:52:43 +02:00
Filipi da Silva Fuchter
1cac94bf97 Merge pull request #1925 from pipecat-ai/filipi/websocket_transport_example_twilio
Websocket client web app to test Twilio.
2025-06-17 16:24:18 -03:00
Mark Backman
96958933af Merge pull request #2016 from pipecat-ai/aleix/example-params-allow-async-objects
examples: create transport params async
2025-06-17 15:08:37 -04:00
Filipi Fuchter
2300c2632e Refactoring how we are organizing the twilio chatbot examples and improving the readmes 2025-06-17 16:08:35 -03:00
Filipi Fuchter
cbd0529674 Merge branch 'main' into filipi/websocket_transport_example_twilio 2025-06-17 15:54:31 -03:00
Filipi da Silva Fuchter
5614e35ac4 Merge pull request #2015 from pipecat-ai/bumping_pipecat_required_versions
Bumping pipecat-ai-krisp required version
2025-06-17 15:42:20 -03:00
Aleix Conchillo Flaqué
c11172caba examples: create transport params async 2025-06-17 11:37:42 -07:00
Filipi Fuchter
11b6e409bb Bumping pipecat-ai-krisp required version 2025-06-17 15:22:31 -03:00
Aleix Conchillo Flaqué
3dca95aa3c Merge pull request #2014 from pipecat-ai/aleix/daily-python-0.19.3
update daily-python to 0.19.3
2025-06-17 10:10:23 -07:00
Aleix Conchillo Flaqué
7ddc706434 update daily-python to 0.19.3 2025-06-17 09:30:28 -07:00
Aleix Conchillo Flaqué
20eebb08e9 update CHANGELOG with AWSTranscribeSTTService Polish support 2025-06-16 10:34:56 -07:00
Aleix Conchillo Flaqué
4abf41b85a Merge pull request #2011 from wuodar/wuodar/polish-lang-aws-transcribe
Support polish language in Amazon Transcribe
2025-06-16 10:33:55 -07:00
Aleix Conchillo Flaqué
e426f7ee7c Merge pull request #2012 from pipecat-ai/aleix/frame-pause-resume-frames
FrameProcessor: handle new FrameProcessorPauseFrame/FrameProcessorResumeFrame
2025-06-16 10:32:38 -07:00
Aleix Conchillo Flaqué
14dc6a7984 FrameProcessor: handle new FrameProcessorPauseFrame/FrameProcessorResumeFrame 2025-06-16 10:31:33 -07:00
Mark Backman
e0a24a3f07 Merge pull request #2006 from pipecat-ai/mb/expose-function-calls-in-progress
Expose has_function_calls_in_progress property
2025-06-16 12:49:07 -04:00
Mark Backman
d1bee22d73 Expose has_function_calls_in_progress property 2025-06-16 12:45:16 -04:00
Jon Taylor
d73f7908f2 Merge pull request #2008 from pipecat-ai/khk/groq-audio
fix groq wav file header parsing
2025-06-16 14:09:09 +01:00
Aleix Conchillo Flaqué
a4ea0d2b82 dev-requirements: update pyright 1.1.400 and ruff 0.11.13 2025-06-15 21:05:03 -07:00
Kacper Włodarczyk
e2c15169b8 feat: support polish language in Amazon Transcribe 2025-06-15 21:44:06 +02:00
Kwindla Hultman Kramer
fe16ed3c73 added changelog entry 2025-06-15 10:49:40 -07:00
Filipi Fuchter
80ce097f90 Using relative URL for the websocket. 2025-06-15 10:49:25 -03:00
Filipi Fuchter
eceaf8a46b Making the path to the web client relative 2025-06-14 21:07:15 -03:00
Kwindla Hultman Kramer
1e3fa4a9c7 fix groq wav file header parsing 2025-06-14 17:41:44 -04:00
Filipi da Silva Fuchter
dc640a7591 Merge pull request #2001 from pipecat-ai/filipi/google_stt_reconnection_issue
Fixed an issue with `GoogleSTTService` where it was constantly reconnecting
2025-06-13 08:29:18 -03:00
Filipi Fuchter
1f072d182c Merge branch 'main' into filipi/google_stt_reconnection_issue
# Conflicts:
#	CHANGELOG.md
2025-06-13 08:26:00 -03:00
Mark Backman
1d64e04ed5 Merge pull request #2002 from pipecat-ai/mb/google-fix-ttfb
Fix: GoogleLLMService TTFB
2025-06-12 12:10:01 -04:00
Mark Backman
22f4f0b79e Update 14e example name 2025-06-12 11:45:59 -04:00
Mark Backman
69c63293fb fix: GoogleLLMService TTFB value 2025-06-12 11:43:27 -04:00
Filipi Fuchter
c1db13ceeb Fixed an issue with GoogleSTTService where it was constantly reconnecting before starting to receive audio from the user. 2025-06-12 12:07:33 -03:00
Filipi Fuchter
70eadee0aa Bumping the @pipecat-ai/websocket-transport dependency. 2025-06-11 18:30:16 -03:00
Aleix Conchillo Flaqué
228afe01ed Merge pull request #1993 from pipecat-ai/aleix/pipecat-0.0.71
update CHANGELOG for 0.0.71
2025-06-10 14:42:09 -07:00
Aleix Conchillo Flaqué
61a5154e49 update CHANGELOG for 0.0.71 2025-06-10 14:34:30 -07:00
Sunah Suh
d3df75aaa0 Add additional_span_attributes param to PipelineTask for extra otel… (#1992) 2025-06-10 17:32:24 -04:00
Aleix Conchillo Flaqué
c59180dd6e udpate CHANGELOG 2025-06-10 14:23:02 -07:00
Mark Backman
e4c2310632 Merge pull request #1990 from pipecat-ai/mb/more-cartesia-stt
Add Cartesia STT docs link to README, fix set_model error
2025-06-10 17:19:11 -04:00
Aleix Conchillo Flaqué
e1735a2da1 Merge pull request #1991 from pipecat-ai/aleix/pipecat-0.0.70
update CHANGELOG for 0.0.70
2025-06-10 14:08:52 -07:00
Aleix Conchillo Flaqué
c101c9c8e1 update CHANGELOG for 0.0.70 2025-06-10 13:37:28 -07:00
Aleix Conchillo Flaqué
96dc162de5 Merge pull request #1988 from pipecat-ai/aleix/update-examples-22b
examples(22b): remove unnecessary parallel pipeline branch
2025-06-10 12:58:37 -07:00
Mark Backman
257dbe3104 Fix model param error 2025-06-10 15:14:47 -04:00
Mark Backman
cd98657e3c Add Cartesia STT docs link to README 2025-06-10 15:09:13 -04:00
Aleix Conchillo Flaqué
03eb22fe0a examples(22b): remove unnecessary parallel pipeline branch 2025-06-10 09:05:58 -07:00
Filipi Fuchter
0073a868d4 Websocket client web app to test Twilio. 2025-06-10 11:34:02 -03:00
Mark Backman
0bb61d72ab Merge pull request #1984 from pipecat-ai/mb/cartesia-stt-cleanup
CartesiaSTTService cleanup
2025-06-10 10:30:18 -04:00
Mark Backman
f758508a82 Merge pull request #1978 from pipecat-ai/mb/rime-languages
Add languages to RimeHttpTTSService, extend lang support to German an…
2025-06-10 10:27:15 -04:00
Mark Backman
69d0218d7e Add languages to RimeHttpTTSService, extend lang support to German and French 2025-06-10 10:20:41 -04:00
Aleix Conchillo Flaqué
eb5e5ab1df update CHANGELOG 2025-06-09 20:22:39 -07:00
Aleix Conchillo Flaqué
093697906c Merge pull request #1954 from WebinarGeek/wg/gladia-informal-translations
Gladia informal translations
2025-06-09 20:21:40 -07:00
Aleix Conchillo Flaqué
efe96b7ed1 Merge pull request #1986 from pipecat-ai/aleix/daily-python0.19.2
pyproject: update daily-python to 0.19.2
2025-06-09 19:46:14 -07:00
Aleix Conchillo Flaqué
7ecdd41ab9 pyproject: update daily-python to 0.19.2 2025-06-09 17:29:07 -07:00
Mark Backman
aec70d61e9 CartesiaSTTService cleanup 2025-06-09 15:20:57 -04:00
Mark Backman
2efac13344 Merge pull request #1983 from pipecat-ai/mb/exotel-resampling
Resample audio in ExotelFrameSerializer
2025-06-09 14:41:08 -04:00
Mark Backman
15aeb11c36 Resample audio in ExotelFrameSerializer 2025-06-09 14:02:25 -04:00
Mark Backman
e705f4d984 Merge pull request #1972 from Vaibhav159/vl_add_exotel_serializer
adding exotel serializer
2025-06-09 13:54:26 -04:00
Shrey Gupta
96fa62fdfe [Add] Support for Cartesia AI STT (#1982) 2025-06-09 14:51:01 -03:00
Mark Backman
845c70797a Merge pull request #1975 from pipecat-ai/mb/11labs-flush-context-reset
fix: ElevenLabsTTSService reset context when flushing audio
2025-06-09 13:21:25 -04:00
kompfner
16048956c3 Merge pull request #1956 from pipecat-ai/pk/make-add-observer-sync
Make `PipelineTask.add_observer()` synchronous. This allows callers t…
2025-06-09 13:19:34 -04:00
Mark Backman
cf2f4b5902 fix: ElevenLabsTTSService reset context when flushing audio 2025-06-09 13:17:55 -04:00
marcus-daily
db46f33f34 Update to Android transports 0.3.7 2025-06-09 17:09:59 +01:00
Aleix Conchillo Flaqué
25d1515daf Merge pull request #1979 from pipecat-ai/aleix/buffer-tts-before-playback
buffer audio from TTS service before pushing frames
2025-06-09 08:43:55 -07:00
Paul Kompfner
a3469cd59f Add CHANGELOG entry describing PipelineTask.add_observer() being made synchronous 2025-06-09 11:37:30 -04:00
Paul Kompfner
513ce26200 Add unit test exercising synchronous usage of PipelineTask.add_observer() right after initializing the PipelineTask (before anything else is done with it) 2025-06-09 11:30:24 -04:00
Paul Kompfner
1cd96f94ff Make PipelineTask.add_observer() synchronous. This allows callers to call it before run()ning the PipelineTask first. Without this change, if they tried to do that, they would get an error because the TaskManager's event loop hadn't been set yet. 2025-06-09 11:30:24 -04:00
Aleix Conchillo Flaqué
901dd041f0 buffer audio from TTS service before pushing frames 2025-06-09 07:29:09 -07:00
Vaibhav159
a2ee94651e removing resampling 2025-06-07 12:53:55 +05:30
Aleix Conchillo Flaqué
abdce063f1 Merge pull request #1973 from pipecat-ai/aleix/assemblyai-yield-none
AssemblyAISTTService: yield None instead of Frame()
2025-06-06 15:12:16 -07:00
Aleix Conchillo Flaqué
a33ce5e4bf AssemblyAISTTService: yield None instead of Frame() 2025-06-06 14:41:01 -07:00
Filipi da Silva Fuchter
c9575eaef9 Merge pull request #1911 from pipecat-ai/filipi/websocket_transport_example
Adding support to WebsocketTransport
2025-06-06 17:25:07 -03:00
Filipi Fuchter
1e74476a71 Refactoring to use the observer inside the pipelinetask, and moving to start the bot inside on_client_ready. 2025-06-06 17:22:50 -03:00
Filipi Fuchter
82935884c4 Mentioning the new websocket example in the changelog. 2025-06-06 17:17:11 -03:00
Filipi Fuchter
d774a23768 Improving the readme to mention that can choose which server websocket to use. 2025-06-06 17:12:05 -03:00
Filipi Fuchter
e9f041e170 Removing the old websocket-server example 2025-06-06 17:09:01 -03:00
Filipi Fuchter
1f51b6e4f1 A Pipecat example demonstrating how to use WebsocketTransport 2025-06-06 17:08:43 -03:00
Filipi Fuchter
028650249c Adding support in ProtobufFrameSerializer to deserialize MessageFrame. 2025-06-06 17:07:39 -03:00
Vaibhav159
534197239f updating changelog 2025-06-07 00:24:54 +05:30
Vaibhav159
d2f4bb574c adding exotel serializer 2025-06-07 00:22:41 +05:30
jqueguiner
25ff8ef37b (config.py): add new configuration options for lip-sync optimization, context adaptation, and additional context to enhance translation accuracy
♻️ (stt.py): increase default max buffer size from 5MB to 20MB to accommodate larger audio data
♻️ (stt.py): simplify audio sending logic by removing chunking and sending the entire buffered audio at once for improved performance
2025-06-05 16:51:29 -07:00
Aleix Conchillo Flaqué
07fb1a2c39 Merge pull request #1967 from counterleft/unused-http-client-session
Remove instantiation of unused http client session from certain examples
2025-06-05 12:59:01 -07:00
Aleix Conchillo Flaqué
581b800c43 Merge pull request #1961 from ken-kuro/patch-1
fix(piper-tts): typo
2025-06-05 12:57:58 -07:00
Aleix Conchillo Flaqué
30ca39287f Merge pull request #1962 from ken-kuro/patch-2
fix(fastapi_websocket): typo
2025-06-05 12:57:22 -07:00
Aleix Conchillo Flaqué
01fa9698de Merge pull request #1960 from pipecat-ai/aleix/disable-uvloop
disable uvloop by default and just let the user set it
2025-06-05 12:12:47 -07:00
Brian Mathiyakom
10bd969636 Remove instantiation of unused http client session
These examples don't make any HTTP requests with `session` so there
doesn't seem be a need to create one in the first place. Probably a
copy-paste from a previous example.
2025-06-05 11:45:13 -07:00
Kendrick Ha
f7761f2b61 fix(fastapi_websocket): typo 2025-06-05 13:55:28 +07:00
Kendrick Ha
49ff38a21f fix(piper-tts): typo 2025-06-05 13:50:56 +07:00
Aleix Conchillo Flaqué
8d161306c7 disable uvloop by default and just let the user set it 2025-06-04 21:25:06 -07:00
Vanessa Pyne
027a82dff1 Merge pull request #1958 from pipecat-ai/vp-livekit-fix
fix: transports/services/livekit.py typo
2025-06-04 12:27:47 -05:00
vipyne
cb409d58e0 fix: transports/services/livekit.py typo 2025-06-04 11:14:21 -05:00
Dan Berg
094e2f8151 Fix formatting 2025-06-03 17:21:51 +02:00
Dan Berg
71d121aeb9 Update CHANGELOG.md explaining informal on Gladia TranslationConfig 2025-06-03 17:15:29 +02:00
Dan Berg
b1a88af43c Add informal to Gladia TranslationConfig 2025-06-03 17:10:52 +02:00
Filipi da Silva Fuchter
f73eb4ebd9 Merge pull request #1949 from pipecat-ai/filipi/transport_destination_issue
Fixed transport destination issue
2025-06-03 08:41:34 -03:00
Filipi Fuchter
31ca9be299 Fixing missing await to self.reset. 2025-06-03 08:37:47 -03:00
jqueguiner
02cc6f3d56 Enhance GladiaSTTService with reconnection and audio buffer management features
- Added parameters for maximum reconnection attempts, reconnection delay, and maximum audio buffer size.
- Implemented automatic reconnection logic with exponential backoff.
- Introduced audio buffer management to handle audio data efficiently, including trimming excess data.
- Updated connection handling to ensure proper cleanup and management of WebSocket connections.
- Enhanced audio sending logic to support buffered audio transmission after reconnections.
2025-06-03 03:16:57 -07:00
Filipi Fuchter
1642c082d1 Describing the fix in the changelog. 2025-06-02 22:28:31 -03:00
Filipi Fuchter
892d213442 Fixing issue to keep the transport_destination. 2025-06-02 22:16:10 -03:00
Filipi Fuchter
fc24267e09 Waiting for the LLM response to reset. 2025-06-02 22:15:53 -03:00
Aleix Conchillo Flaqué
9b71bdc608 Merge pull request #1947 from pipecat-ai/aleix/pipecat-0.0.69
update CHANGELOG for 0.0.69
2025-06-02 12:51:51 -07:00
Aleix Conchillo Flaqué
310be89895 update CHANGELOG for 0.0.69 2025-06-02 12:07:50 -07:00
Aleix Conchillo Flaqué
71fbd57e12 Merge pull request #1938 from pipecat-ai/aleix/custom-interruption-strategies
allow custom interruption strategies
2025-06-02 12:05:50 -07:00
Aleix Conchillo Flaqué
ab4b48c823 examples(04a): fix daily_runner import 2025-06-02 12:01:26 -07:00
Aleix Conchillo Flaqué
532767cfa1 LLMUserContextAggregator: reset strategies when reseting the aggregator 2025-06-02 12:01:26 -07:00
Aleix Conchillo Flaqué
5512de3221 allow custom interruption strategies 2025-06-02 12:01:26 -07:00
Mark Backman
13546d5e8f Merge pull request #1946 from pipecat-ai/mb/fix-11labs-context
fix: Use AudioContextWordTTSService context methods in ElevenLabsTTSS…
2025-06-02 14:55:49 -04:00
Mark Backman
c6f1aa8086 fix: Use AudioContextWordTTSService context methods in ElevenLabsTTSService 2025-06-02 14:49:05 -04:00
Mark Backman
5606c47cb7 Merge pull request #1945 from pipecat-ai/mb/gemini-2.0
Reverting Gemini Live model back to gemini-2.0-flash-live-001
2025-06-02 14:25:30 -04:00
Filipi da Silva Fuchter
7f7cd96211 Merge pull request #1944 from pipecat-ai/fixing_tavus_transport
Adding the direction when pushing the frame.
2025-06-02 15:21:58 -03:00
Filipi Fuchter
b828bfd890 Adding the direction when pushing the BotStartedSpeaking and BotStoppedSpeaking frames. 2025-06-02 15:05:56 -03:00
Mark Backman
31d084eb78 Reverting Gemini Live model back to gemini-2.0-flash-live-001 2025-06-02 13:29:05 -04:00
Mark Backman
ab18b280e9 Merge pull request #1943 from pipecat-ai/mb/add-transcription-19-openai
Add a TranscriptProcessor to 19-openai-realtime-beta.py
2025-06-02 13:01:52 -04:00
Mark Backman
24e89c4081 Merge pull request #1936 from pipecat-ai/mb/fix-01-quickstart
Add daily to the foundational examples requirements.txt
2025-06-02 12:55:36 -04:00
Mark Backman
e129390f56 Add a TranscriptProcessor to 19-openai-realtime-beta.py 2025-06-02 11:38:49 -04:00
Mark Backman
4d7c87bb4c Merge pull request #1941 from pipecat-ai/vp-observer-up
chore: move observers arg in p2p-webrtc example; add deprecated to in…
2025-06-02 11:17:48 -04:00
Mark Backman
dac3f82a75 Merge pull request #1934 from counterleft/use-disconnect-on-small-webrtc-connection-example
Fix type checker error with missing function call in the small WebRTC transport example
2025-06-02 11:17:30 -04:00
Mark Backman
fd860921f1 Add daily to the foundational examples requirements.txt 2025-06-02 11:10:49 -04:00
vipyne
0482ccd48b chore: move observers arg in p2p-webrtc example; add deprecated to in line comments 2025-06-02 09:41:09 -05:00
Dominic Stewart
b8b1990617 Fix example env file (#1939)
* Fixed typo in the example env files
2025-06-02 15:12:43 +09:00
Dominic Stewart
70951b1198 Add simplified pstn examples (#1822)
* Add simplified pstn examples
* Add daily_twilio_sip_dial_out example
2025-06-02 14:50:21 +09:00
Mark Backman
6d24514ace Merge pull request #1937 from pipecat-ai/mb/fix-message-for-logging
fix: correctly display non-roman characters
2025-06-01 12:49:48 -04:00
Aleix Conchillo Flaqué
49915ceb84 Merge pull request #1683 from pipecat-ai/aleix/run-function-calls-sequentially
run function calls sequentially or in parallel
2025-06-01 09:47:35 -07:00
Mark Backman
925b13e337 fix: correctly display non-roman characters 2025-06-01 12:29:26 -04:00
Aleix Conchillo Flaqué
ef3143d558 LLMService: don't run function calls if none are given 2025-05-31 14:01:56 -07:00
Mark Backman
ed84637b55 Add additional function call for testing to 14e, 14r, 19, 19a, 26b 2025-05-31 13:57:52 -07:00
Aleix Conchillo Flaqué
897a944478 examples(14,14a): add restaurant recommendation function call 2025-05-31 13:57:52 -07:00
Aleix Conchillo Flaqué
d86343c38d examples: update to use on_function_calls_started event 2025-05-31 13:57:52 -07:00
Aleix Conchillo Flaqué
297afdd126 LLMService: add new FunctionCallsStartedFrame 2025-05-31 13:57:52 -07:00
Aleix Conchillo Flaqué
f0cbdc4e68 LLMService: add on_function_calls_started event 2025-05-31 13:57:52 -07:00
Aleix Conchillo Flaqué
40b52cadde LLMService: s/FunctionCallLLM/FunctionCallFromLLM/ s/FunctionCallRunner/FunctionCallRunnerItem/ 2025-05-31 13:57:52 -07:00
Aleix Conchillo Flaqué
04bf85ddfe LLMService: allow executing tasks sequentially and in parallel 2025-05-31 13:57:52 -07:00
Aleix Conchillo Flaqué
4809684a13 LLMService: cancel function calls on interruptions by default 2025-05-31 13:57:50 -07:00
Aleix Conchillo Flaqué
1eb50ad88f LLMService: pass LLM function calls all at once 2025-05-31 13:57:36 -07:00
Aleix Conchillo Flaqué
52569bcdb2 LLMService: don't allow running functions concurrently for now 2025-05-31 13:57:36 -07:00
Aleix Conchillo Flaqué
a50a407415 LLMService: run function calls sequentially 2025-05-31 13:57:35 -07:00
Aleix Conchillo Flaqué
9f223442c2 Merge pull request #1935 from pipecat-ai/aleix/script-evals
improve evals
2025-05-30 19:14:59 -07:00
Mark Backman
c647114bb9 Merge pull request #1927 from pipecat-ai/mb/gemini-tracing 2025-05-30 22:01:44 -04:00
Mark Backman
43719ec737 Update CHANGELOG 2025-05-30 21:36:39 -04:00
Mark Backman
8602557985 Refactor Gemini tracing to more closely match OpenAI Realtime, add TTFB metrics 2025-05-30 21:36:00 -04:00
Mark Backman
dd1f7d0875 Add tracking to OpenAI Realtime 2025-05-30 21:36:00 -04:00
Mark Backman
ec39e794d3 _handle_transcription 2025-05-30 21:36:00 -04:00
Mark Backman
7b1a937d4c Add tracing for Gemini Live 2025-05-30 21:36:00 -04:00
Mark Backman
0fd38d8115 Merge pull request #1931 from pipecat-ai/mb/num-words
Add support for interruption strategies
2025-05-30 21:14:02 -04:00
Mark Backman
7a4efc6212 Code review feedback 2025-05-30 21:09:15 -04:00
Brian Mathiyakom
2eb2c5a413 Use disconnect() because close() doesn't exist
SmallWebRTCConnection doesn't have a `close()`. There's a `_close()` but I assume that's private due to its naming. The closest function that uses `_close()` is `disconnect()`. I assume then, that the intended resource freeing function call should be to `disconnect()`.
2025-05-30 17:14:53 -07:00
Aleix Conchillo Flaqué
2fcfb0aa9f evals: don't use Deepgram's smart formatting 2025-05-30 16:55:55 -07:00
Aleix Conchillo Flaqué
f1df079512 evals: allow running a single eval 2025-05-30 16:55:55 -07:00
Aleix Conchillo Flaqué
d77bedbafb evals: move scripts/release to script/evals and add README 2025-05-30 15:04:05 -07:00
Mark Backman
b34c593c54 Add changelog entry 2025-05-30 16:48:42 -04:00
Mark Backman
62efbc3342 Add foundational example 42 2025-05-30 16:48:42 -04:00
Mark Backman
2d609a0bde Update LLmUserContextAggregator to conditionally push_aggregation 2025-05-30 16:48:42 -04:00
Mark Backman
6bc4b4a17f Update BaseInputTransport to not push StartInterruptionFrame when InterruptionConfig is set and bot is speaking 2025-05-30 16:48:42 -04:00
Mark Backman
b489e52080 Add InterruptionConfig 2025-05-30 16:15:20 -04:00
Aleix Conchillo Flaqué
a8aaeec52b Merge pull request #1926 from pipecat-ai/aleix/pause-base-input-transport
handle StopFrame in base input transport and stop pushing frames
2025-05-30 08:27:20 -07:00
Mark Backman
ad7eec181e Merge pull request #1923 from philipp-eisen/philipp/fix-ttfb_ms-implementation
Fix implementation of ttfb_ms metric
2025-05-30 11:03:42 -04:00
Aleix Conchillo Flaqué
b33897ffb9 SmallWebRTCTransport: don't initialize if a second StartFrame is received 2025-05-30 07:51:51 -07:00
Aleix Conchillo Flaqué
1c3d3f2f4b WebsocketClientTransport: don't initialize if a second StartFrame is received 2025-05-30 07:51:51 -07:00
Aleix Conchillo Flaqué
9a5a1edb6b WebsocketServerTransport: don't initialize if a second StartFrame is received 2025-05-30 07:51:51 -07:00
Aleix Conchillo Flaqué
f2eb869b02 FastAPIWebsocketTransport: don't initialize if a second StartFrame is received 2025-05-30 07:51:51 -07:00
Aleix Conchillo Flaqué
0c7e3cfcb2 LiveKitTransport: don't initialize if a second StartFrame is received 2025-05-30 07:51:51 -07:00
Aleix Conchillo Flaqué
24e19db29e TavusTransport: don't initialize if a second StartFrame is received 2025-05-30 07:51:51 -07:00
Aleix Conchillo Flaqué
bc6d7b7bbd examples(phone-chatbot): don't show dangling tasks in first pipeline 2025-05-30 07:51:51 -07:00
Aleix Conchillo Flaqué
cad271068e BaseInputTransport: handle StopFrame and pause pushing frames 2025-05-30 07:51:49 -07:00
Philipp Eisen
3425293115 Ensure correct formatting 2025-05-30 15:41:45 +02:00
Mark Backman
20dbfec3a9 Merge pull request #1876 from m-ods/m-ods/assemblyai-universal-streaming
Update AssemblyAI Streaming STT
2025-05-30 08:55:43 -04:00
marcus-daily
170057a75a Updating dependency version 2025-05-30 12:38:32 +01:00
marcus-daily
b86b761e0b Fixing Yaml syntax 2025-05-30 12:38:32 +01:00
marcus-daily
da0d2f0266 Small WebRTC transport demo app for Android 2025-05-30 12:38:32 +01:00
Martin Schweiger
321ea27c34 changelog entry 2025-05-30 17:15:58 +08:00
Philipp Eisen
b712e6b9aa Switch ttfb metric to use seconds instead 2025-05-30 11:13:26 +02:00
Martin Schweiger
b3652e6527 set vad_force_turn_endpoint to True by default 2025-05-30 17:07:54 +08:00
Philipp Eisen
bc97f397ef Switch to using float instead of int, but keep ms 2025-05-30 10:27:20 +02:00
Martin Schweiger
e5da3f6e68 add tracing 2025-05-30 10:55:19 +08:00
Martin Schweiger
8400539acf set formatted_finals true by default 2025-05-30 10:40:01 +08:00
Martin Schweiger
b5eac8dfed add message to TranscriptionFrame result 2025-05-30 10:39:07 +08:00
Martin Schweiger
ba312b5591 Merge branch 'main' into m-ods/assemblyai-universal-streaming 2025-05-30 10:29:49 +08:00
Martin Schweiger
f23572b318 Merge branch 'main' into m-ods/assemblyai-universal-streaming 2025-05-30 10:11:02 +08:00
Martin Schweiger
db838634e7 fix: double finals bug 2025-05-30 10:00:31 +08:00
Martin Schweiger
7f2e848a5c use FrameProcessor class methods 2025-05-30 09:40:46 +08:00
Martin Schweiger
096e854d50 remove .dict() 2025-05-30 09:31:20 +08:00
Martin Schweiger
3ffe8b3155 remove parse_obj 2025-05-30 09:29:51 +08:00
Mark Backman
a471f49b61 Merge pull request #1889 from pipecat-ai/mb/add-dtmf-aggregator
Add DTMFAggregator
2025-05-29 17:13:28 -04:00
Mark Backman
4d2a02f318 Refactor to create task on StartFrame, also cleanup 2025-05-29 17:10:54 -04:00
Mark Backman
0bec7db03b Emit a BotInterruptionFrame when the first keypress of a sequence is received 2025-05-29 17:07:18 -04:00
Mark Backman
74827f983f Add tests, improve frame timing 2025-05-29 17:07:18 -04:00
Mark Backman
0ed46f457e Add DTMFAggregator 2025-05-29 17:07:17 -04:00
Aleix Conchillo Flaqué
36b731be73 Merge pull request #1915 from pipecat-ai/aleix/uvloop-event-loop
use uvloop as the new event loop on Linux and macOS
2025-05-29 14:06:44 -07:00
Aleix Conchillo Flaqué
62fbdd4e81 Merge pull request #1922 from pipecat-ai/aleix/output-transport-cleanup
output transports cleanup
2025-05-29 14:06:17 -07:00
Aleix Conchillo Flaqué
ca7b0650c2 examples: capture camera or screen. allow setting framerate 2025-05-29 13:16:44 -07:00
Aleix Conchillo Flaqué
67dd146038 output transports cleanup 2025-05-29 13:16:44 -07:00
Aleix Conchillo Flaqué
fb66df2efd use uvloop as the new event loop on Linux and macOS 2025-05-29 11:24:21 -07:00
Aleix Conchillo Flaqué
2395ca0057 Merge pull request #1921 from pipecat-ai/aleix/transcription-frame-result
add STT result field to TranscriptionFrame/InterimTranscriptionFrame
2025-05-29 11:22:38 -07:00
Aleix Conchillo Flaqué
d203789490 add STT result field to TranscriptionFrame/InterimTranscriptionFrame 2025-05-29 11:21:44 -07:00
Aleix Conchillo Flaqué
7ea0e31cd4 Merge pull request #1924 from pipecat-ai/aleix/new-examples-package
add new pipecat.examples package and make runner public
2025-05-29 11:18:22 -07:00
Mark Backman
d3bf13a503 Merge pull request #1917 from pipecat-ai/mb/fix-twilio-chatbot-client
fix: update websocket_client to use FrameProcessorSetup.task_manager
2025-05-29 14:05:27 -04:00
Mark Backman
ea91970499 Update the twilio-chatbot client to work with the updated server, which requires call_sid 2025-05-29 14:02:49 -04:00
Mark Backman
803b3f2cc4 fix: update websocket_client to use FrameProcessorSetup 2025-05-29 14:02:47 -04:00
Aleix Conchillo Flaqué
1788ba6c5c add new pipecat.examples package and make runner public 2025-05-29 10:43:12 -07:00
Philipp Eisen
5209bd3d9f Fix implemetation of ttfb_ms metric 2025-05-29 19:24:04 +02:00
Aleix Conchillo Flaqué
cb9178f1ec Merge pull request #1914 from pipecat-ai/aleix/base-output-daily-handle-dtmf-frames
output transport can now handle DTMF keypress
2025-05-29 09:35:01 -07:00
Aleix Conchillo Flaqué
5676920a6a BaseOutputTransport/DailyTransport: allow sending DTMF keypress 2025-05-29 09:33:29 -07:00
Aleix Conchillo Flaqué
513221d9fd added OutputDTMFUrgentFrame 2025-05-29 09:32:57 -07:00
Mark Backman
a33d0b4b53 Merge pull request #1904 from pipecat-ai/mb/aws-bedrock-no-op-tool
Add no_op tool to AWSBedrockLLMService
2025-05-29 10:29:19 -04:00
Mark Backman
bee242b781 Safer check when using the no_operation tool 2025-05-29 10:25:20 -04:00
Mark Backman
fa1c98ff29 Add no_op tool to AWSBedrockLLMService 2025-05-29 10:25:19 -04:00
Mark Backman
ae3a7d9bed Merge pull request #1920 from alexflorensa/alexflorensa/set-deepgram-model-name
fix(deepgram-stt): set model name to Deepgram STT
2025-05-29 09:41:56 -04:00
alexflorensa
0c2efb312c fix(deepgram-stt) Set model name to Deepgram STT 2025-05-29 12:08:16 +02:00
Martin Schweiger
f1d7eb8565 final touches 2025-05-27 21:29:50 +08:00
Martin Schweiger
320b52eb1e Merge branch 'm-ods/assemblyai-universal-streaming' of https://github.com/m-ods/pipecat into m-ods/assemblyai-universal-streaming 2025-05-26 09:13:08 +08:00
Martin Schweiger
428cee75c5 Add User-Agent header to AssemblyAI websocket connection 2025-05-26 09:10:55 +08:00
Martin Schweiger
5479a55b2c Add websockets dependency to assemblyai extra 2025-05-26 09:08:56 +08:00
Martin Schweiger
7f568e3e7e Merge branch 'main' into m-ods/assemblyai-universal-streaming 2025-05-23 17:39:00 +08:00
Martin Schweiger
9b8800ac1d update stt.py 2025-05-23 17:32:31 +08:00
Martin Schweiger
fd53712567 add models for new streaming service 2025-05-23 17:32:12 +08:00
387 changed files with 15579 additions and 4400 deletions

View File

@@ -6,11 +6,13 @@ on:
- main
paths:
- "examples/simple-chatbot/client/android/**"
- "examples/p2p-webrtc/video-transform/client/android/**"
pull_request:
branches:
- "**"
paths:
- "examples/simple-chatbot/client/android/**"
- "examples/p2p-webrtc/video-transform/client/android/**"
workflow_dispatch:
inputs:
sdk_git_ref:
@@ -23,7 +25,7 @@ concurrency:
jobs:
sdk:
name: "Simple chatbot demo"
name: "Demo apps"
runs-on: ubuntu-latest
steps:
- name: Checkout repo
@@ -37,12 +39,22 @@ jobs:
distribution: 'temurin'
java-version: '17'
- name: Build demo app
- name: "Example app: Simple Chatbot"
working-directory: examples/simple-chatbot/client/android
run: ./gradlew :simple-chatbot-client:assembleDebug
- name: Upload demo APK
- name: Upload Simple Chatbot APK
uses: actions/upload-artifact@v4
with:
name: Simple Chatbot Android Client
path: examples/simple-chatbot/client/android/simple-chatbot-client/build/outputs/apk/debug/simple-chatbot-client-debug.apk
- name: "Example app: Small WebRTC Client"
working-directory: examples/p2p-webrtc/video-transform/client/android
run: ./gradlew :small-webrtc-client:assembleDebug
- name: Upload Small WebRTC APK
uses: actions/upload-artifact@v4
with:
name: Small WebRTC Android Client
path: examples/p2p-webrtc/video-transform/client/android/small-webrtc-client/build/outputs/apk/debug/small-webrtc-client-debug.apk

View File

@@ -9,10 +9,234 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Added reconnection logic and audio buffer management to `GladiaSTTService`.
- Added Polish support to `AWSTranscribeSTTService`.
- Added new frames `FrameProcessorPauseFrame` and `FrameProcessorResumeFrame`
which allow pausing and resuming frame processing for a given frame
processor. These are control frames, so they are ordered. Pausing frame
processor will keep old frames in the internal queues until resume takes
place. Frames being pushed while a frame processor is paused will be pushed to
the queues. When frame processing is resumed all queued frames will be
processed in order. Also added `FrameProcessorPauseUrgentFrame` and
`FrameProcessorResumeUrgentFrame` which are system frames and therefore they
have high priority.
- Added a property called `has_function_calls_in_progress` in
`LLMAssistantContextAggregator` that exposes whether a function call is in
progress.
### Changed
- The `PipelineParams` arg `allow_interruptions` now defaults to `True`.
- `TavusTransport` and `TavusVideoService` now send audio to Tavus using WebRTC
audio tracks instead of `app-messages` over WebSocket. This should improve the
overall audio quality.
- Upgraded `daily-python` to 0.19.3.
### Deprecated
- `AudioBufferProcessor` parameter `user_continuos_stream` is deprecated.
### Fixed
- Fixed an `AudioBufferProcessor` issue that was causing crackling on the audio
stream with lower sample rate (due to upsampling the other stream). We now
record with the lowest sample rate to avoid upsampling.
- Fixed an issue that would cause multiple `PipelineTask.on_idle_timeout`
events to be triggered repeatedly.
- Fixed an `AudioBufferProcessor` issue that was causing user and bot speech to
not be synchronized during recordings.
- Fixed an issue where voice settings weren't applied to ElevenLabsTTSService.
- Fixed an issue with `GroqTTSService` where it was not properly parsing the
WAV file header.
- Fixed an issue with `GoogleSTTService` where it was constantly reconnecting
before starting to receive audio from the user.
- Fixed an issue where `GoogleLLMService`'s TTFB value was incorrect.
### Other
- Rename `14e-function-calling-gemini.py` to `14e-function-calling-google.py`.
## [0.0.71] - 2025-06-10
### Added
- Adds a parameter called `additional_span_attributes` to PipelineTask that
lets you add any additional attributes you'd like to the conversation span.
### Fixed
- Fixed an issue with `CartesiaSTTService` initialization.
## [0.0.70] - 2025-06-10
### Added
- Added `ExotelFrameSerializer` to handle telephony calls via Exotel.
- Added the option `informal` to `TranslationConfig` on Gladia config.
Allowing to force informal language forms when available.
- Added `CartesiaSTTService` which is a websocket based implementation to
transcribe audio. Added a foundational example in
`13f-cartesia-transcription.py`
- Added an `websocket` example, showing how to use the new Pipecat client
`WebsocketTransport` to connect with Pipecat `FastAPIWebsocketTransport` or
`WebsocketServerTransport`.
- Added language support to `RimeHttpTTSService`. Extended languages to include
German and French for both `RimeTTSService` and `RimeHttpTTSService`.
### Changed
- Upgraded `daily-python` to 0.19.2.
- Make `PipelineTask.add_observer()` synchronous. This allows callers to call it
before doing the work of running the `PipelineTask` (i.e. without invoking
`PipelineTask.set_event_loop()` first).
- Pipecat 0.0.69 forced `uvloop` event loop on Linux on macOS. Unfortunately,
this is causing issue in some systems. So, `uvloop` is not enabled by default
anymore. If you want to use `uvloop` you can just set the `asyncio` event
policy before starting your agent with:
```python
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
```
### Fixed
- Fixed an issue with various TTS services that would cause audio glitches at
the start of every bot turn.
- Fixed an `ElevenLabsTTSService` issue where a context warning was printed
when pushing a `TTSSpeakFrame`.
- Fixed an `AssemblyAISTTService` issue that could cause unexpected behavior
when yielding empty `Frame()`s.
- Fixed an issue where `OutputAudioRawFrame.transport_destination` was being
reset to `None` instead of retaining its intended value before sending the
audio frame to `write_audio_frame`.
- Fixed a typo in Livekit transport that prevented initialization.
## [0.0.69] - 2025-06-02 "AI Engineer World's Fair release" ✨
### Added
- Added a new frame `FunctionCallsStartedFrame`. This frame is pushed both
upstream and downstream from the LLM service to indicate that one or more
function calls are going to be executed.
- Added LLM services `on_function_calls_started` event. This event will be
triggered when the LLM service receives function calls from the model and is
going to start executing them.
- Function calls can now be executed sequentially (in the order received in the
completion) by passing `run_in_parallel=False` when creating your LLM
service. By default, if the LLM completion returns 2 or more function calls
they run concurrently. In both cases, concurrently and sequentially, a new LLM
completion will run when the last function call finishes.
- Added OpenTelemetry tracing for `GeminiMultimodalLiveLLMService` and
`OpenAIRealtimeBetaLLMService`.
- Added initial support for interruption strategies, which determine if the user
should interrupt the bot while the bot is speaking. Interruption strategies
can be based on factors such as audio volume or the number of words spoken by
the user. These can be specified via the new `interruption_strategies` field
in `PipelineParams`. A new `MinWordsInterruptionStrategy` strategy has been
introduced which triggers an interruption if the user has spoken a minimum
number of words. If no interruption strategies are specified, the normal
interruption behavior applies. If multiple strategies are provided, the first
one that evaluates to true will trigger the interruption.
- `BaseInputTransport` now handles `StopFrame`. When a `StopFrame` is received
the transport will pause sending frames downstream until a new `StartFrame` is
received. This allows the transport to be reused (keeping the same connection)
in a different pipeline.
- Updated AssemblyAI STT service to support their latest streaming
speech-to-text model with improved transcription latency and endpointing.
- You can now access STT service results through the new
`TranscriptionFrame.result` and `InterimTranscriptionFrame.result` field. This
is useful in case you use some specific settings for the STT and you want to
access the STT results.
- The examples runner is now public from the `pipecat.examples` package. This
allows everyone to build their own examples and run them easily.
- It is now possible to push `OutputDTMFFrame` or `OutputDTMFUrgentFrame` with
`DailyTransport`. This will be sent properly if a Daily dial-out connection
has been established.
- Added `OutputDTMFUrgentFrame` to send a DTMF keypress quickly. The previous
`OutputDTMFFrame` queues the keypress with the rest of data frames.
- Added `DTMFAggregator`, which aggregates keypad presses into
`TranscriptionFrame`s. Aggregation occurs after a timeout, termination key
press, or user interruption. You can specify the prefix of the
`TranscriptionFrame`.
- Added new functions `DailyTransport.start_transcription()` and
`DailyTransport.stop_transcription()` to be able to start and stop Daily
transcription dynamically (maybe with different settings).
### Changed
- Reverted the default model for `GeminiMultimodalLiveLLMService` back to
`models/gemini-2.0-flash-live-001`.
`gemini-2.5-flash-preview-native-audio-dialog` has inconsistent performance.
You can opt in to using this model by setting the `model` arg.
- Function calls are now cancelled by default if there's an interruption. To
disable this behavior you can set `cancel_on_interruption=False` when
registering the function call. Since function calls are executed as tasks you
can tell if a function call has been cancelled by catching the
`asyncio.CancelledError` exception (and don't forget to raise it again!).
- Updated OpenTelemetry tracing attribute `metrics.ttfb_ms` to `metrics.ttfb`.
The attribute reports TTFB in seconds.
### Deprecated
- `DailyTransport.send_dtmf()` is deprecated, push an `OutputDTMFFrame` or an
`OutputDTMFUrgentFrame` instead.
### Fixed
- Fixed an issue with `ElevenLabsTTSService` where long responses would
continue generating output even after an interruption.
- Fixed an issue with the `OpenAILLMContext` where non-Roman characters were
being incorrectly encoded as Unicode escape sequences. This was a logging
issue and did not impact the actual conversation.
- In `AWSBedrockLLMService`, worked around a possible bug in AWS Bedrock where
a `toolConfig` is required if there has been previous tool use in the
messages array. This workaround includes a no_op factory function call is
used to satisfy the requirement.
- Fixed `WebsocketClientTransport` to use `FrameProcessorSetup.task_manager`
instead of `StartFrame.task_manager`.
### Performance
- Use `uvloop` as the new event loop on Linux and macOS systems.
## [0.0.68] - 2025-05-28
### Added

View File

@@ -53,7 +53,7 @@ You can connect to Pipecat from any platform using our official SDKs:
| Category | Services |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Parakeet (NVIDIA)](https://docs.pipecat.ai/server/services/stt/parakeet), [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Cartesia](https://docs.pipecat.ai/server/services/stt/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Parakeet (NVIDIA)](https://docs.pipecat.ai/server/services/stt/parakeet), [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
| LLMs | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [AWS](https://docs.pipecat.ai/server/services/llm/aws), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Qwen](https://docs.pipecat.ai/server/services/llm/qwen), [Together AI](https://docs.pipecat.ai/server/services/llm/together) |
| Text-to-Speech | [AWS](https://docs.pipecat.ai/server/services/tts/aws), [Azure](https://docs.pipecat.ai/server/services/tts/azure), [Cartesia](https://docs.pipecat.ai/server/services/tts/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/tts/deepgram), [ElevenLabs](https://docs.pipecat.ai/server/services/tts/elevenlabs), [FastPitch (NVIDIA)](https://docs.pipecat.ai/server/services/tts/fastpitch), [Fish](https://docs.pipecat.ai/server/services/tts/fish), [Google](https://docs.pipecat.ai/server/services/tts/google), [LMNT](https://docs.pipecat.ai/server/services/tts/lmnt), [MiniMax](https://docs.pipecat.ai/server/services/tts/minimax), [Neuphonic](https://docs.pipecat.ai/server/services/tts/neuphonic), [OpenAI](https://docs.pipecat.ai/server/services/tts/openai), [Piper](https://docs.pipecat.ai/server/services/tts/piper), [PlayHT](https://docs.pipecat.ai/server/services/tts/playht), [Rime](https://docs.pipecat.ai/server/services/tts/rime), [Sarvam](https://docs.pipecat.ai/server/services/tts/sarvam), [XTTS](https://docs.pipecat.ai/server/services/tts/xtts) |
| Speech-to-Speech | [AWS Nova Sonic](https://docs.pipecat.ai/server/services/s2s/aws), [Gemini Multimodal Live](https://docs.pipecat.ai/server/services/s2s/gemini), [OpenAI Realtime](https://docs.pipecat.ai/server/services/s2s/openai) |

View File

@@ -3,11 +3,11 @@ coverage~=7.6.12
grpcio-tools~=1.67.1
pip-tools~=7.4.1
pre-commit~=4.0.1
pyright~=1.1.397
pyright~=1.1.400
pytest~=8.3.4
pytest-asyncio~=0.25.3
pytest-aiohttp==1.1.0
ruff~=0.11.1
ruff~=0.11.13
setuptools~=70.0.0
setuptools_scm~=8.1.0
python-dotenv~=1.0.1

View File

@@ -133,7 +133,8 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -71,6 +71,8 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -148,10 +148,8 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
observers=[TranscriptionLogObserver()],
)

View File

@@ -75,7 +75,13 @@ async def main(room_url: str, token: str):
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):

View File

@@ -170,7 +170,6 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -198,7 +198,6 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -211,7 +211,6 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -67,10 +67,8 @@ async def main(transport: DailyTransport):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -192,7 +192,6 @@ async def main(transport: DailyTransport):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -55,6 +55,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -56,6 +56,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -53,6 +53,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -77,37 +77,36 @@ async def configure_livekit():
async def main():
async with aiohttp.ClientSession() as session:
(url, token, room_name) = await configure_livekit()
(url, token, room_name) = await configure_livekit()
transport = LiveKitTransport(
url=url,
token=token,
room_name=room_name,
params=LiveKitParams(audio_out_enabled=True),
)
transport = LiveKitTransport(
url=url,
token=token,
room_name=room_name,
params=LiveKitParams(audio_out_enabled=True),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
)
runner = PipelineRunner()
runner = PipelineRunner()
task = PipelineTask(Pipeline([tts, transport.output()]))
task = PipelineTask(Pipeline([tts, transport.output()]))
# Register an event handler so we can play the audio when the
# participant joins.
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant_id):
await asyncio.sleep(1)
await task.queue_frame(
TextFrame(
"Hello there! How are you doing today? Would you like to talk about the weather?"
)
# Register an event handler so we can play the audio when the
# participant joins.
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant_id):
await asyncio.sleep(1)
await task.queue_frame(
TextFrame(
"Hello there! How are you doing today? Would you like to talk about the weather?"
)
)
await runner.run(task)
await runner.run(task)
if __name__ == "__main__":

View File

@@ -50,6 +50,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -63,6 +63,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -68,6 +68,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -47,7 +47,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
Pipeline([imagegen, transport.output()]),
params=PipelineParams(enable_metrics=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
# Register an event handler so we can play the audio when the client joins
@@ -68,6 +71,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -93,10 +93,8 @@ async def run_example(webrtc_connection: SmallWebRTCConnection):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -156,7 +154,7 @@ async def offer(request: dict, background_tasks: BackgroundTasks):
@asynccontextmanager
async def lifespan(app: FastAPI):
yield # Run app
coros = [pc.close() for pc in pcs_map.values()]
coros = [pc.disconnect() for pc in pcs_map.values()]
await asyncio.gather(*coros)
pcs_map.clear()

View File

@@ -9,11 +9,11 @@ import os
import sys
import aiohttp
from daily_runner import configure
from dotenv import load_dotenv
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.daily_runner import configure
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
@@ -75,10 +75,8 @@ async def main():
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -158,7 +158,8 @@ async def main():
],
),
params=PipelineParams(
allow_interruptions=True, enable_metrics=True, enable_usage_metrics=True
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -174,6 +174,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -137,6 +137,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -133,10 +133,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -156,6 +154,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +107,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -83,10 +83,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -108,6 +106,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -113,10 +113,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -141,6 +139,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,10 +87,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -120,6 +118,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -81,10 +81,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -106,6 +104,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -88,10 +88,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -113,6 +111,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +107,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +107,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -86,10 +86,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -111,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -90,10 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -115,6 +113,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,11 +84,9 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
audio_out_sample_rate=24000,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -110,6 +108,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -89,10 +89,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -114,6 +112,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,10 +87,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -112,6 +110,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -92,10 +92,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -116,6 +114,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -80,10 +80,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -105,6 +103,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -85,7 +85,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -109,6 +108,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,10 +87,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -112,6 +110,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -88,10 +88,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -113,6 +111,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -86,10 +86,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -111,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +107,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -89,10 +89,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -114,6 +112,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -83,10 +83,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -108,6 +106,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -80,10 +80,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -105,6 +103,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -258,7 +258,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -282,6 +281,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +107,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -77,8 +77,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@@ -97,6 +97,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,10 +84,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +107,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -83,10 +83,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -108,6 +106,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -86,10 +86,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -111,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -70,10 +70,8 @@ async def main():
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -90,10 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -115,6 +113,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -89,10 +89,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -114,6 +112,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -96,6 +96,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -11,6 +11,7 @@ import tkinter as tk
from dotenv import load_dotenv
from loguru import logger
from pipecat.examples.run import maybe_capture_participant_camera
from pipecat.frames.frames import (
Frame,
InputAudioRawFrame,
@@ -107,6 +108,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
await maybe_capture_participant_camera(transport, client, framerate=30)
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
@@ -119,6 +121,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -85,7 +85,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
@@ -104,6 +110,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -162,6 +162,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -105,7 +105,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_video(transport, client)
await maybe_capture_participant_camera(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -125,6 +125,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -101,14 +101,17 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(allow_interruptions=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_video(transport, client)
await maybe_capture_participant_camera(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -128,6 +131,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -101,14 +101,17 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(allow_interruptions=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_video(transport, client)
await maybe_capture_participant_camera(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -128,6 +131,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -101,14 +101,17 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(allow_interruptions=True),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_video(transport, client)
await maybe_capture_participant_camera(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -128,6 +131,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -72,6 +72,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -66,6 +66,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -66,6 +66,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,6 +87,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -65,6 +65,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,7 +84,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
pipeline,
params=PipelineParams(
enable_metrics=True,
report_only_initial_ttfb=False,
enable_usage_metrics=True,
),
)
@@ -99,6 +99,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -0,0 +1,71 @@
#
# Copyright (c) 20242025, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import argparse
import os
from dotenv import load_dotenv
from loguru import logger
from pipecat.frames.frames import Frame, TranscriptionFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.services.cartesia.stt import CartesiaSTTService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
from pipecat.transports.services.daily import DailyParams
load_dotenv(override=True)
class TranscriptionLogger(FrameProcessor):
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TranscriptionFrame):
print(f"Transcription: {frame.text}")
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
transport_params = {
"daily": lambda: DailyParams(audio_in_enabled=True),
"twilio": lambda: FastAPIWebsocketParams(audio_in_enabled=True),
"webrtc": lambda: TransportParams(audio_in_enabled=True),
}
async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_sigint: bool):
logger.info(f"Starting bot")
stt = CartesiaSTTService(
api_key=os.getenv("CARTESIA_API_KEY"),
base_url=os.getenv("CARTESIA_BASE_URL"),
)
tl = TranscriptionLogger()
pipeline = Pipeline([transport.input(), stt, tl])
task = PipelineTask(pipeline)
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
await task.cancel()
runner = PipelineRunner(handle_sigint=handle_sigint)
await runner.run(task)
if __name__ == "__main__":
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,10 +30,13 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
@@ -71,6 +74,11 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# You can also register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
@@ -88,7 +96,18 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["location", "format"],
)
tools = ToolsSchema(standard_tools=[weather_function])
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
tools = ToolsSchema(standard_tools=[weather_function, restaurant_function])
messages = [
{
@@ -115,10 +134,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -139,6 +156,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -33,6 +33,10 @@ async def get_weather(params: FunctionCallParams):
await params.result_callback(f"The weather in {location} is currently 72 degrees and sunny.")
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
@@ -66,9 +70,11 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
)
llm = AnthropicLLMService(
api_key=os.getenv("ANTHROPIC_API_KEY"), model="claude-3-7-sonnet-latest"
api_key=os.getenv("ANTHROPIC_API_KEY"),
model="claude-3-7-sonnet-latest",
)
llm.register_function("get_weather", get_weather)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
weather_function = FunctionSchema(
name="get_weather",
@@ -81,7 +87,18 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["location"],
)
tools = ToolsSchema(standard_tools=[weather_function])
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
tools = ToolsSchema(standard_tools=[weather_function, restaurant_function])
# todo: test with very short initial user message
@@ -110,8 +127,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@@ -132,6 +149,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,11 +10,11 @@ import os
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
@@ -172,8 +172,8 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@@ -181,7 +181,7 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_video(transport, client)
await maybe_capture_participant_camera(transport, client)
global client_id
client_id = get_transport_client_id(transport, client)
@@ -200,6 +200,6 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -16,7 +16,7 @@ from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import TTSSpeakFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -74,6 +73,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -113,7 +116,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
@@ -132,6 +141,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,14 +10,14 @@ import os
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
@@ -158,13 +158,19 @@ indicate you should use the get_image tool are:
]
)
task = PipelineTask(pipeline)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
await maybe_capture_participant_video(transport, client)
await maybe_capture_participant_camera(transport, client)
global client_id
client_id = get_transport_client_id(transport, client)
@@ -183,6 +189,6 @@ indicate you should use the get_image tool are:
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,11 +10,11 @@ import os
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import TTSSpeakFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -35,11 +35,14 @@ client_id = ""
async def get_weather(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
location = params.arguments["location"]
await params.result_callback(f"The weather in {location} is currently 72 degrees and sunny.")
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
async def get_image(params: FunctionCallParams):
question = params.arguments["question"]
logger.debug(f"Requesting image with user_id={client_id}, question={question}")
@@ -93,6 +96,11 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
llm = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"), model="gemini-2.0-flash-001")
llm.register_function("get_weather", get_weather)
llm.register_function("get_image", get_image)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_weather",
@@ -110,6 +118,17 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["location", "format"],
)
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
get_image_function = FunctionSchema(
name="get_image",
description="Get an image from the video stream.",
@@ -121,14 +140,14 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["question"],
)
tools = ToolsSchema(standard_tools=[weather_function, get_image_function])
tools = ToolsSchema(standard_tools=[weather_function, get_image_function, restaurant_function])
system_prompt = """\
You are a helpful assistant who converses with a user and answers questions. Respond concisely to general questions.
Your response will be turned into speech so use only simple words and punctuation.
You have access to two tools: get_weather and get_image.
You have access to three tools: get_weather, get_restaurant_recommendation, and get_image.
You can respond to questions about the weather using the get_weather tool.
@@ -164,7 +183,6 @@ indicate you should use the get_image tool are:
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -174,7 +192,7 @@ indicate you should use the get_image tool are:
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_video(transport, client)
await maybe_capture_participant_camera(transport, client)
global client_id
client_id = get_transport_client_id(transport, client)
@@ -193,6 +211,6 @@ indicate you should use the get_image tool are:
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -31,7 +31,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -74,6 +73,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -118,7 +121,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -141,6 +143,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -111,7 +111,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -134,6 +133,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -75,6 +74,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -117,7 +120,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -140,6 +142,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -74,6 +73,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -116,7 +119,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -139,6 +141,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -72,6 +71,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -114,7 +117,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -137,6 +139,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -71,6 +70,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -123,10 +126,8 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -147,6 +148,6 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -71,6 +70,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -123,10 +126,8 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -147,6 +148,6 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -75,6 +74,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -117,10 +120,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -141,6 +142,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -90,10 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -114,6 +112,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -71,6 +70,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -113,7 +116,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -136,6 +138,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -76,6 +75,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -119,7 +122,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -142,6 +144,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,7 +30,6 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -72,6 +71,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -115,10 +118,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -139,6 +140,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -32,6 +32,10 @@ async def fetch_weather_from_api(params: FunctionCallParams):
await params.result_callback({"conditions": "nice", "temperature": "75"})
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
@@ -74,6 +78,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# You can also register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
weather_function = FunctionSchema(
name="get_current_weather",
@@ -91,7 +96,18 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["location", "format"],
)
tools = ToolsSchema(standard_tools=[weather_function])
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
tools = ToolsSchema(standard_tools=[weather_function, restaurant_function])
messages = [
{
@@ -118,10 +134,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -143,6 +157,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -147,7 +147,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
@@ -172,6 +178,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -135,7 +135,13 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
@@ -160,6 +166,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -90,8 +90,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@@ -143,6 +143,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -117,9 +117,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
report_only_initial_ttfb=True,
enable_usage_metrics=True,
),
)
@@ -141,6 +140,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -66,7 +66,7 @@ async def run_example(transport: BaseTransport, args: argparse.Namespace, handle
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
parser = argparse.ArgumentParser(description="Pipecat Bot Runner")
parser.add_argument("-i", "--input", type=str, required=True, help="Input video file")

View File

@@ -62,6 +62,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -14,10 +14,12 @@ from loguru import logger
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import TranscriptionMessage
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.processors.transcript_processor import TranscriptProcessor
from pipecat.services.llm_service import FunctionCallParams
from pipecat.services.openai_realtime_beta import (
InputAudioNoiseReduction,
@@ -45,6 +47,10 @@ async def fetch_weather_from_api(params: FunctionCallParams):
)
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -62,8 +68,20 @@ weather_function = FunctionSchema(
required=["location", "format"],
)
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
# Create tools schema
tools = ToolsSchema(standard_tools=[weather_function])
tools = ToolsSchema(standard_tools=[weather_function, restaurant_function])
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
@@ -100,7 +118,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# turn_detection=False,
input_audio_noise_reduction=InputAudioNoiseReduction(type="near_field"),
# tools=tools,
instructions="""Your knowledge cutoff is 2023-10. You are a helpful and friendly AI.
instructions="""You are a helpful and friendly AI.
Act like a human, but remember that you aren't a human and that you can't do human
things in the real world. Your voice and personality should be warm and engaging, with a lively and
@@ -109,10 +127,14 @@ playful tone.
If interacting in a non-English language, start by using the standard accent or dialect familiar to
the user. Talk quickly. You should always call a function if you can. Do not refer to these rules,
even if you're asked about them.
-
You are participating in a voice conversation. Keep your responses concise, short, and to the point
unless specifically asked to elaborate on a topic.
You have access to the following tools:
- get_current_weather: Get the current weather for a given location.
- get_restaurant_recommendation: Get a restaurant recommendation for a given location.
Remember, your responses should be short. Just one or two sentences, usually.""",
)
@@ -125,6 +147,9 @@ Remember, your responses should be short. Just one or two sentences, usually."""
# you can either register a single function for all function calls, or specific functions
# llm.register_function(None, fetch_weather_from_api)
llm.register_function("get_current_weather", fetch_weather_from_api)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
transcript = TranscriptProcessor()
# Create a standard OpenAI LLM context object using the normal messages format. The
# OpenAIRealtimeBetaLLMService will convert this internally to messages that the
@@ -151,7 +176,9 @@ Remember, your responses should be short. Just one or two sentences, usually."""
transport.input(), # Transport user input
context_aggregator.user(),
llm, # LLM
transcript.user(), # Placed after the LLM, as LLM pushes TranscriptionFrames downstream
transport.output(), # Transport bot output
transcript.assistant(), # After the transcript output, to time with the audio output
context_aggregator.assistant(),
]
)
@@ -159,10 +186,8 @@ Remember, your responses should be short. Just one or two sentences, usually."""
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -177,12 +202,21 @@ Remember, your responses should be short. Just one or two sentences, usually."""
logger.info(f"Client disconnected")
await task.cancel()
# Register event handler for transcript updates
@transcript.event_handler("on_transcript_update")
async def on_transcript_update(processor, frame):
for msg in frame.messages:
if isinstance(msg, TranscriptionMessage):
timestamp = f"[{msg.timestamp}] " if msg.timestamp else ""
line = f"{timestamp}{msg.role}: {msg.content}"
logger.info(f"Transcript: {line}")
runner = PipelineRunner(handle_sigint=handle_sigint)
await runner.run(task)
if __name__ == "__main__":
from run import main
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

Some files were not shown because too many files have changed in this diff Show More