wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #4249 from sathwikareddy02/nvidia-tts-update

Browse Source 
Add stitching support and enhancements for NvidiaTTSService
This commit is contained in:
Filipi da Silva Fuchter
2026-04-13 09:39:48 -03:00
committed by GitHub
parent 4254c1f0e0 1d5dcf1698
commit e248c4c049
3 changed files with 361 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
changelog/4249.added.md Normal file
View File

@@ -0,0 +1,6 @@
- Added enhancements to `NvidiaTTSService`:
- Cross-sentence stitching: multiple sentences within an LLM turn are fed into a single `SynthesizeOnline` gRPC stream for seamless audio across sentence boundaries (requires Magpie TTS model v1.7.0+).
- `custom_dictionary` and `encoding` parameters for IPA-based custom pronunciation and output audio encoding.
- Metrics generation (`can_generate_metrics` returns true) and `stop_all_metrics()` when an audio context is interrupted.
- gRPC error handling around synthesis config retrieval (`GetRivaSynthesisConfig`).

8
changelog/4249.changed.md Normal file
View File

@@ -0,0 +1,8 @@
- Updated `NvidiaTTSService`:
- Made `api_key` optional for local NIM deployments.
- Voice, language, and quality can be updated without reconnecting the gRPC client; new values take effect on the next synthesis turn, not for the current turn's in-flight requests.
- Replaced per-sentence synchronous `synthesize_online` calls with async queue-backed gRPC streaming.
- Streaming now uses asyncio tasks with explicit gRPC cancellation on interruption and stale-response filtering when a stream is aborted or replaced.
- Renamed Riva references to Nemotron Speech in docs and messages.
- Disabled automatic TTS start frames at the service level (`push_start_frame=False`) and emit `TTSStartedFrame` when a stitched synthesis stream is started for a context.