Aleix Conchillo Flaqué
337f048864
introduce synchronous and asynchronous frame processors
...
Pipecat has a pipeline-based architecture. The pipeline consists of frame
processors linked to each other. The elements travelling across the pipeline are
called frames.
To have a deterministic behavior the frames travelling through the pipeline
should always be ordered, except system frames which are out-of-band frames. To
achieve that, each frame processor should only output frames from a single task.
There are synchronous and asynchronous frame processors. The synchronous
processors push output frames from the same task that they receive input frames,
and therefore only pushing frames from one task. Asynchrnous frame processors
can have internal tasks to perform things asynchrnously (e.g. receiving data
from a websocket) but they also have a single task where they push frames from.
2024-09-19 01:31:10 -07:00
Mark Backman
eadd68d40b
Add sample_rate setting to TTS services
2024-09-18 14:50:20 -04:00
Aleix Conchillo Flaqué
20c019ae16
services(cartesia,elevenlabs): close websocket before the receiving task
2024-09-16 23:54:21 -07:00
Kunal Shah
540cad4844
Undo sorting
2024-09-16 16:07:19 -07:00
Kunal Shah
0a26b650c0
Undo sorting
2024-09-16 16:06:25 -07:00
Kunal Shah
adaac003e5
[Cartesia] Fix streaming truncation bug with Twilio Fast API WS
2024-09-16 15:59:06 -07:00
Aleix Conchillo Flaqué
1fe940bd6b
servceis(cartesia,elevenlabs): use word start times instead
2024-09-13 13:10:44 -07:00
Aleix Conchillo Flaqué
cb36a71381
fix some linting
2024-09-13 09:56:12 -07:00
Aleix Conchillo Flaqué
434493b8aa
services(elevenlabs): implement word-by-word support through websockets
2024-09-13 09:31:35 -07:00
Aleix Conchillo Flaqué
3665734972
transports(output): initial sink clock synchronization
2024-09-12 00:37:34 -07:00
Aleix Conchillo Flaqué
a98d78cdea
services(lmnt): change to subclass of AsyncTTSService
2024-09-12 00:37:34 -07:00
Aleix Conchillo Flaqué
80f6d74e80
services(cartesia): change to subclass of AsyncWordTTSService
2024-09-12 00:37:34 -07:00
Aleix Conchillo Flaqué
02d926e9bd
services: create AsyncTTSService and AsyncWordTTSService
2024-09-12 00:31:48 -07:00
Aleix Conchillo Flaqué
7749692f72
processors: get pipeline clock from StartFrame
2024-09-12 00:31:48 -07:00
Aleix Conchillo Flaqué
7807cbeb39
pipeline(task): add a clock to the pipeline task
2024-09-12 00:31:48 -07:00
Aleix Conchillo Flaqué
72f231b327
frames: add a presentation timestamp (pts) to each frame
2024-09-12 00:31:48 -07:00
Aleix Conchillo Flaqué
3cbe97d346
clocks: added new BaseClock and SystemClock
2024-09-12 00:31:48 -07:00
Kwindla Hultman Kramer
b880e1a60e
Merge pull request #448 from pipecat-ai/khk/aggregation-leading-space
...
fix for leading space in context aggregator strings
2024-09-10 09:57:35 -07:00
Aleix Conchillo Flaqué
98286336bf
transports(daily): allow setting audio output bitrate (default 96kpbs)
...
Fixes #388
2024-09-08 19:39:17 -07:00
Kwindla Hultman Kramer
081b001c8b
fix for leading space in context aggregator strings
2024-09-07 16:42:52 -07:00
Aleix Conchillo Flaqué
e405d7af9f
services(elevenlabs): add elevenlabs package and use streaming
2024-09-05 11:20:01 -07:00
Aashraya
51cd7fd285
twiliohandle interruption ( #422 )
...
* add interuption handler in twilio serializer
* fix autopep8
* revert ruff autoformatting
* address pr comments
* change interruption frame to user started frame in serializer
* remove overrrident handle interrupt
* remove unused import
* change userstarted to interuption frame
2024-09-02 11:06:38 -07:00
Soof Golan
5c0f5a1613
Generate ids with itertools.count
...
Avoids the critical section with threading.Lock in favor of itertools.count.
`count` objects are threadsafe, and their critical section is implemented in C and provide better performance that Python level locking.
2024-09-02 15:39:58 +02:00
Aleix Conchillo Flaqué
37e2388758
StartFrame should be the first frame every processor receives
...
Fixes #427
2024-08-29 22:43:44 -07:00
Aleix Conchillo Flaqué
c0ac5c6ae8
services(lmnt): fix example and update README and CHANGELOG
2024-08-29 11:11:24 -07:00
Aleix Conchillo Flaqué
be923687fb
processors(rtvi): user decices if bot interrupts on update config
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
ae6fbb3146
services: just set model, voice, language independently
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
864768635a
services: add voice and language to set_model()
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
d7c9679977
services: allow TTSModelUpdateFrame to also update language and voice
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
fedfc366f6
services(deepgram): fix strenum values
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
b3b39626e1
services: allow switching STT language and mdoel at the same time
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
4e0ece17b6
services: added support for setting STT model and language
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
fd3fdacdee
transcriptions: added more languages
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
a253606d50
services(daily): on_joined now returns all data not only participant
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
568d9dc0a3
services(whisper): inherit from SegmentedSTTService
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
6629b853c5
services(deepgram): inherit from STTService instead of AsyncAIService
2024-08-29 11:00:03 -07:00
Aleix Conchillo Flaqué
3931cb3235
services(cartesia): allow setting language and language voices
2024-08-29 11:00:01 -07:00
Aleix Conchillo Flaqué
38cd86ad52
services: added language to transcription frames
2024-08-29 10:59:02 -07:00
Aleix Conchillo Flaqué
c0cdabf61d
frames: adde TTSLanguageUpdateFrame and TTSLanguageVoicesUpdateFrame
2024-08-29 10:59:02 -07:00
Aleix Conchillo Flaqué
51270a96c5
frames: add language to transcription frames
2024-08-29 10:59:02 -07:00
Kwindla Hultman Kramer
84d72c0d5c
Merge pull request #425 from pipecat-ai/khk/rtvi-together-function-calling
...
fixup type mismatches between rtvi data structures and together.py
2024-08-28 13:11:52 -07:00
Kwindla Hultman Kramer
b9d362bd62
fixup type mismatches between rtvi data structures and together.py
2024-08-27 17:39:21 -07:00
Sharvil Nanavati
87c4a1bee1
Move stop frame task creation into TTSService.start
2024-08-27 04:45:21 +00:00
Sharvil Nanavati
c979762b70
Handle cancellation, stopping, and restarting
2024-08-27 01:24:00 +00:00
Sharvil Nanavati
1d92fc3199
Merge branch 'main' into pr/add-lmnt
2024-08-24 10:07:52 -07:00
Sharvil Nanavati
8ac7fb1a67
Use a single long-lived Task to push TTSStoppedFrame
2024-08-24 16:18:07 +00:00
Sharvil Nanavati
60c3d33def
Default LMNT to 24kHz, add example
2024-08-24 15:40:29 +00:00
Sharvil Nanavati
8a39d3f4eb
services: add a generic mechanism to produce TTSStoppedFrames
2024-08-24 15:40:12 +00:00
Aleix Conchillo Flaqué
873d5dc23f
added livekit dependency
2024-08-22 10:54:18 -07:00
Aleix Conchillo Flaqué
6d141fd47f
Merge pull request #396 from nulyang/feat/livekit-serializers
...
Add livekit audio serializers
2024-08-22 10:44:24 -07:00
