Update changelog for version 0.0.102
This commit is contained in:
aconchillo
committed by
Aleix Conchillo Flaqué
Aleix Conchillo Flaqué
parent
12dc6c0b9e
commit
f1e2001a4e
@@ -1 +0,0 @@
|
||||
- Added `ResembleAITTSService` for text-to-speech using Resemble AI's streaming WebSocket API with word-level timestamps and jitter buffering for smooth audio playback.
|
||||
@@ -1 +0,0 @@
|
||||
- Added `UserBotLatencyObserver` for tracking user-to-bot response latency. When tracing is enabled, latency measurements are automatically recorded as `turn.user_bot_latency_seconds` attributes on OpenTelemetry turn spans.
|
||||
@@ -1 +0,0 @@
|
||||
- Deprecated `UserBotLatencyLogObserver`. Use `UserBotLatencyObserver` directly with its `on_latency_measured` event handler instead.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed pipeline freeze when `InterruptionFrame` discards `EndFrame` or `StopFrame` by making terminal frames uninterruptible.
|
||||
@@ -1,3 +0,0 @@
|
||||
- Added `append_to_context` parameter to `TTSSpeakFrame` for conditional LLM context addition.
|
||||
- Allows fine-grained control over whether text should be added to conversation context
|
||||
- Defaults to `True` to maintain backward compatibility
|
||||
@@ -1,4 +0,0 @@
|
||||
- Added TTS context tracking system with `context_id` field to trace audio generation through the pipeline.
|
||||
- `TTSAudioRawFrame`, `TTSStartedFrame`, `TTSStoppedFrame` now include `context_id`
|
||||
- `AggregatedTextFrame` and `TTSTextFrame` now include `context_id`
|
||||
- Enables tracking which TTS request generated specific audio chunks
|
||||
@@ -1,3 +0,0 @@
|
||||
- Simplified context aggregators to use `frame.append_to_context` flag instead of tracking internal state.
|
||||
- Cleaner logic in `LLMResponseAggregator` and `LLMResponseUniversalAggregator`
|
||||
- More consistent behavior across aggregator implementations
|
||||
@@ -1,4 +0,0 @@
|
||||
- ⚠️ `TTSService.run_tts()` now requires a `context_id` parameter for context tracking.
|
||||
- Custom TTS service implementations must update their `run_tts()` signature
|
||||
- Before: `async def run_tts(self, text: str) -> AsyncGenerator[Frame, None]:`
|
||||
- After: `async def run_tts(self, text: str, context_id: str) -> AsyncGenerator[Frame, None]:`
|
||||
@@ -1,3 +0,0 @@
|
||||
- Updated all 30+ TTS service implementations to support context tracking with `context_id`.
|
||||
- Services now generate and propagate context IDs through TTS frames
|
||||
- Enables end-to-end tracing of TTS requests through the pipeline
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed OpenAI LLM stream not being closed on cancellation/exception, which could leak sockets.
|
||||
@@ -1 +0,0 @@
|
||||
- Added support for Inworld TTS Websocket Auto Mode for improved latency
|
||||
@@ -1 +0,0 @@
|
||||
- Updated timestamps to be cumulative within an agent turn, using flushCompleted message as an indication of when timestamps from the server are reset to 0
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed `PipelineTask` adding duplicate `RTVIProcessor` and `RTVIObserver` when they were already provided in the pipeline or observers list. They are now detected and skipped, with appropriate warnings and errors logged for mismatched configurations.
|
||||
@@ -1 +0,0 @@
|
||||
- Changed `KokoroTTSService` to use `kokoro-onnx` instead of `kokoro` as the underlying TTS engine.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed function call timeout task not being cancelled when the handler completes without calling `result_callback` or is cancelled externally, which caused `RuntimeWarning: coroutine was never awaited`.
|
||||
@@ -1,5 +0,0 @@
|
||||
- Fixed sentence splitting for Japanese, Chinese, Korean, and other non-Latin
|
||||
languages in TTS pipeline. NLTK's sentence tokenizer does not support CJK
|
||||
languages, causing text to accumulate until flush instead of being split at
|
||||
sentence boundaries. Added fallback detection for unambiguous non-Latin
|
||||
sentence-ending punctuation (e.g., `。`, `？`, `！`).
|
||||
@@ -1 +0,0 @@
|
||||
- Added new frames for context summarization: `LLMContextSummaryRequestFrame` and `LLMContextSummaryResultFrame`.
|
||||
@@ -1,5 +0,0 @@
|
||||
- Added context summarization feature to automatically compress conversation history when conversation length limits (by token or message count) are reached, enabling efficient long-running conversations.
|
||||
- Configure via `enable_context_summarization=True` in `LLMAssistantAggregatorParams`
|
||||
- Customize behavior with `LLMContextSummarizationConfig` (max tokens, thresholds, etc.)
|
||||
- Automatically preserves incomplete function call sequences during summarization
|
||||
- See new examples: `examples/foundational/54-context-summarization-openai.py` and `examples/foundational/54a-context-summarization-google.py`
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed `PipelineTask` to also call `set_bot_ready()` when an external `RTVIProcessor` is provided.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed `VADController` not broadcasting `SpeechControlParamsFrame` on startup, which prevented STT services from receiving VAD params needed for TTFB measurement.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed `StopAsyncIteration` exceptions in `parse_telephony_websocket()` when WebSocket connections close before sending expected messages.
|
||||
@@ -1 +0,0 @@
|
||||
- Added RTVI function call lifecycle events (`llm-function-call-started`, `llm-function-call-in-progress`, `llm-function-call-stopped`) with configurable security levels via `RTVIObserverParams.function_call_report_level`. Supports per-function control over what information is exposed (`DISABLED`, `NONE`, `NAME`, or `FULL`).
|
||||
@@ -1 +0,0 @@
|
||||
- Deprecated `RTVILLMFunctionCallMessage`, `RTVILLMFunctionCallMessageData`, and `RTVIProcessor.handle_function_call()`. Use the new `llm-function-call-in-progress` event sent automatically by `RTVIObserver` instead.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed WebSocket transport error when broadcasting `InputTransportMessageFrame` by correctly instantiating the frame with its message parameter.
|
||||
@@ -1 +0,0 @@
|
||||
- Added `RequestMetadataFrame` and metadata handling for `ServiceSwitcher` to ensure STT services correctly emit `STTMetadataFrame` when switching between services. Only the active service's metadata is propagated downstream, switching services triggers the newly active service to re-emit its metadata, and proper frame ordering is maintained at startup.
|
||||
@@ -1,6 +0,0 @@
|
||||
- Added `STTMetadataFrame` to broadcast STT service latency information at pipeline start.
|
||||
- STT services broadcast P99 time-to-final-segment (`ttfs_p99_latency`) to downstream processors
|
||||
- Turn stop strategies automatically configure their STT timeout from this metadata
|
||||
- Developers can override `ttfs_p99_latency` via constructor argument for custom deployments
|
||||
- Added measured P99 values for STT providers.
|
||||
- See [stt-benchmark](https://github.com/pipecat-ai/stt-benchmark) to measure latency for your configuration
|
||||
@@ -1,5 +0,0 @@
|
||||
- Improved user turn stop timing in `TranscriptionUserTurnStopStrategy` and `TurnAnalyzerUserTurnStopStrategy`.
|
||||
- Timeout now starts on `VADUserStoppedSpeakingFrame` for tighter, more predictable timing
|
||||
- Added support for finalized transcripts (`TranscriptionFrame.finalized=True`) to trigger earlier
|
||||
- Added fallback timeout for edge cases where transcripts arrive without VAD events
|
||||
- Removed `InterimTranscriptionFrame` handling (no longer affects timing)
|
||||
@@ -1 +0,0 @@
|
||||
- Updated the `VADUserStartedSpeakingFrame` to include `start_secs` and `timestamp` and `VADUserStoppedSpeakingFrame` to include `stop_secs` and `timestamp`, removing the need to separately handle the `SpeechControlParamsFrame` for VADParams values.
|
||||
@@ -1 +0,0 @@
|
||||
- ⚠️ Renamed `TranscriptionUserTurnStopStrategy` to `SpeechTimeoutUserTurnStopStrategy`. The old name is deprecated and will be removed in a future release.
|
||||
@@ -1 +0,0 @@
|
||||
- Improved the accuracy of the `UserBotLatencyObserver` and `UserBotLatencyLogObserver` by measuring from the time when the user actually starts speaking.
|
||||
@@ -1 +0,0 @@
|
||||
- ⚠️ Renamed `timeout` parameter to `user_speech_timeout` in `TranscriptionUserTurnStopStrategy`.
|
||||
@@ -1 +0,0 @@
|
||||
- ⚠️ Removed `timeout` parameter from `TurnAnalyzerUserTurnStopStrategy`. The timeout is now managed internally based on STT latency.
|
||||
@@ -1 +0,0 @@
|
||||
- `AssemblyAISTTService` now automatically configures optimal settings for manual turn detection when `vad_force_turn_endpoint=True`. This sets `end_of_turn_confidence_threshold=1.0` and `max_turn_silence=2000` by default, which disables model-based turn detection and reduces latency by relying on external VAD for turn endpoints. Warnings are logged if conflicting settings are detected.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed orphan OpenTelemetry spans during flow initialization and transitions in tracing.
|
||||
@@ -1 +0,0 @@
|
||||
- Upgraded the `pipecat-ai-small-webrtc-prebuilt` package to v2.1.0.
|
||||
@@ -1 +0,0 @@
|
||||
- Added support for `video_settings` parameter in `LiveAvatarNewSessionRequest` to configure video encoding (H264/VP8) and quality levels.
|
||||
@@ -1 +0,0 @@
|
||||
- Added support for `is_sandbox` parameter in `LiveAvatarNewSessionRequest` to enable sandbox mode for HeyGen LiveAvatar sessions.
|
||||
@@ -1 +0,0 @@
|
||||
- Changed default session mode from "CUSTOM" to "LITE" in HeyGen LiveAvatar integration, with VP8 as the default video encoding.
|
||||
@@ -1 +0,0 @@
|
||||
- Added `OpenAIRealtimeSTTService` for real-time streaming speech-to-text using OpenAI's Realtime API WebSocket transcription sessions. Supports local VAD and server-side VAD modes, noise reduction, and automatic reconnection.
|
||||
@@ -1,10 +0,0 @@
|
||||
- ⚠️ The default `VADParams` `stop_secs` default is changing from `0.8` seconds
|
||||
to `0.2` seconds. This change both simplifies the developer experience and
|
||||
improves the performance of STT services. With a shorter `stop_secs` value,
|
||||
STT services using a local VAD can finalize sooner, resulting in faster
|
||||
transcription.
|
||||
|
||||
- `SpeechTimeoutUserTurnStopStrategy`: control how long to wait for
|
||||
additional user speech using `user_speech_timeout` (default: 0.6 sec).
|
||||
- `TurnAnalyzerUserTurnStopStrategy`: the turn analyzer automatically adjusts
|
||||
the user wait time based on the audio input.
|
||||
@@ -1 +0,0 @@
|
||||
- Moved interruption wait event from per-processor instance state to `InterruptionFrame` itself. Added `InterruptionFrame.complete()` to signal when the interruption has fully traversed the pipeline. Custom processors that block or consume an `InterruptionFrame` before it reaches the pipeline sink must call `frame.complete()` to avoid stalling `push_interruption_task_frame_and_wait()`. A warning is logged if completion does not happen within 2 seconds.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed `SambaNovaLLMService` and `GoogleLLMOpenAIBetaService` streams not being closed on cancellation/exception, which could leak sockets.
|
||||
@@ -1 +0,0 @@
|
||||
- Update the default model to `scribe_v2` for `ElevenLabsSTTService`.
|
||||
@@ -1 +0,0 @@
|
||||
- Changed the `DeepgramSTTService` default setting for `smart_format` to `False`, as agents don't need smart formatting. Disabling this setting provides a small performance improvement, as well.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed an issue in `InworldTTSService` where punctuation was pronounced. Now, the `InworldTTSService` ensures proper spacing between sentences, resolving pronunciation issues.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed `ParallelPipeline` allowing frames pushed by internal processors to escape during lifecycle frame (`StartFrame`/`EndFrame`/`CancelFrame`) synchronization. These frames are now buffered and flushed after all branches complete.
|
||||
@@ -1 +0,0 @@
|
||||
- Added `bulbul:v3-beta` TTS model support for Sarvam AI with temperature control and 25 new speaker voices.
|
||||
@@ -1 +0,0 @@
|
||||
- Added `saaras:v3` STT model support for Sarvam AI with new `mode` parameter (transcribe, translate, verbatim, translit, codemix) and prompt support.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed issues in Sarvam STT and TTS services: missing event handler registration for VAD signals, `Optional[bool]` type annotations, WebSocket state cleanup on API errors, and TTS disconnect/reconnection state management.
|
||||
@@ -1 +0,0 @@
|
||||
- Changed `FunctionCallCancelFrame` to broadcast in both directions for consistency with other function call frames.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed `RTVIObserver` sending duplicate client messages for frames that are broadcast in both directions (e.g. `UserStartedSpeakingFrame`, `FunctionCallResultFrame`).
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed WebSocket STT services (ElevenLabs, Cartesia, Gladia, Soniox) disconnecting due to idle timeout when no audio is being sent (e.g. when inactive behind a `ServiceSwitcher`). `WebsocketSTTService` now provides opt-in silence-based keepalive via `keepalive_timeout` and `keepalive_interval` parameters.
|
||||
@@ -1 +0,0 @@
|
||||
- Added new OpenAI TTS voice options `marin` and `cedar`.
|
||||
@@ -1 +0,0 @@
|
||||
- Added `UserMuteStartedFrame` and `UserMuteStoppedFrame` system frames, and corresponding `user-mute-started` / `user-mute-stopped` RTVI messages, so clients can observe when mute strategies activate or deactivate.
|
||||
@@ -1 +0,0 @@
|
||||
- Changed default user turn stop strategy from `TranscriptionUserTurnStopStrategy` to `TurnAnalyzerUserTurnStopStrategy` with `LocalSmartTurnAnalyzerV3`.
|
||||
@@ -1 +0,0 @@
|
||||
- Renamed `RequestMetadataFrame` to `ServiceSwitcherRequestMetadataFrame` and added a `service` field to target a specific service. The frame is now pushed downstream by services after handling instead of being silently consumed.
|
||||
@@ -1 +0,0 @@
|
||||
- Update `SonioxSTTService` to set `vad_force_turn_endpoint` to `True`. This setting disabled the turn detection logic available natively in Soniox. Instead, Soniox relies on a local VAD to finalize the transcript. This configuration meaningfully reduces the time to final segment for Soniox. With this setting enabled, Soniox outputs a transcript in ~250ms (median). Pipecat enables smart-turn detection by default using the `LocalSmartTurnAnalyzerV3`. To use the native turn detection logic in Soniox, just set `vad_force_turn_endpoint` to `False`.
|
||||
@@ -1 +0,0 @@
|
||||
- Update `SonioxSTTService` default model to `stt-rt-v4`.
|
||||
@@ -1 +0,0 @@
|
||||
- Updated the default model to `async_flash_v1.0` and base URL to `https://api.async.com` for `AsyncAITTSService`.
|
||||
@@ -1 +0,0 @@
|
||||
- Fixed stream closing compatibility for OpenAI-compatible providers (e.g. OpenPipe) that return async generators instead of `AsyncStream`.
|
||||
Reference in New Issue
Block a user