Merge branch 'main' into google-streaming-tts

This commit is contained in:
aristid
2025-05-24 17:16:22 +02:00
committed by GitHub
40 changed files with 1795 additions and 824 deletions

View File

@@ -11,6 +11,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added `GoogleHttpTTSService` which uses Google's HTTP TTS API.
- Added `TavusTransport`, a new transport implementation compatible with any
Pipecat pipeline. When using the `TavusTransport`the Pipecat bot will
connect in the same room as the Tavus Avatar and the user.
- Added `UserBotLatencyLogObserver`. This is an observer that logs the latency
between when the user stops speaking and when the bot starts speaking. This
gives you an initial idea on how quickly the AI services respond.
- Added `SarvamTTSService`, which implements Sarvam AI's TTS API:
https://docs.sarvam.ai/api-reference-docs/text-to-speech/convert.
- Added `PipelineTask.add_observer()` and `PipelineTask.remove_observer()` to
allow mangaging observers at runtime. This is useful for cases where the task
is passed around to other code components that might want to observe the
@@ -77,6 +88,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Updated `GoogleTTSService` to use Google's streaming TTS API. The default voice also updated to `en-US-Chirp3-HD-Charon`.
- ⚠Refactored the `TavusVideoService`, so it acts like a proxy, sending audio to
Tavus and receiving both audio and video. This will make `TavusVideoService` usable
with any Pipecat pipeline and with any transport. This is a **breaking change**,
check the `examples/foundational/21a-tavus-layer-small-webrtc.py` to see how to
use it.
- `DailyTransport` now uses custom microphone audio tracks instead of virtual
microphones. Now, multiple Daily transports can be used in the same process.
- `DailyTransport` now captures audio from individual participants instead of
the whole room. This allows identifying audio frames per participant.
- Updated the default model for `AnthropicLLMService` to
`claude-sonnet-4-20250514`.
- Updated the default model for `GeminiMultimodalLiveLLMService` to
`models/gemini-2.5-flash-preview-native-audio-dialog`.
- `BaseTextFilter` methods `filter()`, `update_settings()`,
`handle_interruption()` and `reset_interruption()` are now async.
@@ -112,6 +141,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Fixed a `DailyTransport` issue that would cause images needing resize to block
the event loop.
- Fixed an issue with `ElevenLabsTTSService` where changing the model or voice
while the service is running wasn't working.
- Fixed an issue that would cause multiple instances of the same class to behave
incorrectly if any of the given constructor arguments defaulted to a mutable
value (e.g. lists, dictionaries, objects).
@@ -122,13 +157,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Performance
- `DailyTransport`: process audio, video and events in separate tasks.
- Don't create event handler tasks if no user event handlers have been
registered.
### Other
- Added foundation example `07y-minimax-http.py` to show how to use the
`MiniMaxHttpTTSService`.
- Added foundation examples `07y-interruptible-minimax.py` and
`07z-interruptible-sarvam.py`to show how to use the `MiniMaxHttpTTSService`
and `SarvamTTSService`, respectively.
- Added an `open-telemetry-tracing` example, showing how to setup tracing. The
example also includes Jaeger as an open source OpenTelemetry client to review