wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve user turn stop timing by triggering timeout from VAD stop

Browse Source 
Refactor TranscriptionUserTurnStopStrategy and TurnAnalyzerUserTurnStopStrategy
to use VADUserStoppedSpeakingFrame as the ground truth for when speech ended,
rather than triggering timeouts from transcription frames.
This commit is contained in:
Mark Backman
2026-02-03 22:32:34 -05:00
parent 05e2a013b3
commit 34b068d657
47 changed files with 933 additions and 292 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
changelog/3637.added.3.md Normal file
View File

@@ -0,0 +1 @@
- Added `RequestMetadataFrame` and metadata handling for `ServiceSwitcher` to ensure STT services correctly emit `STTMetadataFrame` when switching between services. Only the active service's metadata is propagated downstream, switching services triggers the newly active service to re-emit its metadata, and proper frame ordering is maintained at startup.

6
changelog/3637.added.md Normal file
View File

@@ -0,0 +1,6 @@
- Added `STTMetadataFrame` to broadcast STT service latency information at pipeline start.
- STT services broadcast P99 time-to-final-segment (`ttfs_p99_latency`) to downstream processors
- Turn stop strategies automatically configure their STT timeout from this metadata
- Developers can override `ttfs_p99_latency` via constructor argument for custom deployments
- Added measured P99 values for STT providers.
- See [stt-benchmark](https://github.com/pipecat-ai/stt-benchmark) to measure latency for your configuration

5
changelog/3637.changed.2.md Normal file
View File

@@ -0,0 +1,5 @@
- Improved user turn stop timing in `TranscriptionUserTurnStopStrategy` and `TurnAnalyzerUserTurnStopStrategy`.
- Timeout now starts on `VADUserStoppedSpeakingFrame` for tighter, more predictable timing
- Added support for finalized transcripts (`TranscriptionFrame.finalized=True`) to trigger earlier
- Added fallback timeout for edge cases where transcripts arrive without VAD events
- Removed `InterimTranscriptionFrame` handling (no longer affects timing)

1
changelog/3637.changed.3.md Normal file
View File

@@ -0,0 +1 @@
- Updated the `VADUserStartedSpeakingFrame` to include `start_secs` and `timestamp` and `VADUserStoppedSpeakingFrame` to include `stop_secs` and `timestamp`, removing the need to separately handle the `SpeechControlParamsFrame` for VADParams values.

1
changelog/3637.changed.4.md Normal file
View File

@@ -0,0 +1 @@
- ⚠️ Renamed `TranscriptionUserTurnStopStrategy` to `SpeechTimeoutUserTurnStopStrategy`. The old name is deprecated and will be removed in a future release.

1
changelog/3637.changed.md Normal file
View File

@@ -0,0 +1 @@
- ⚠️ Renamed `timeout` parameter to `user_speech_timeout` in `TranscriptionUserTurnStopStrategy`.

1
changelog/3637.removed.md Normal file
View File

@@ -0,0 +1 @@
- ⚠️ Removed `timeout` parameter from `TurnAnalyzerUserTurnStopStrategy`. The timeout is now managed internally based on STT latency.