Merge branch 'main' into filipi/deepgram

# Conflicts:
#	src/pipecat/services/deepgram/stt_sagemaker.py

changelog/3696.added.md

@@ -1 +0,0 @@
-- Added `TextAggregationMetricsData` metric measuring the time from the first LLM token to the first complete sentence, representing the latency cost of sentence aggregation in the TTS pipeline.

changelog/3696.changed.md

@@ -1 +0,0 @@
-- Added `text_aggregation_mode` parameter to `TTSService` and all TTS subclasses with a new `TextAggregationMode` enum (`SENTENCE`, `TOKEN`). All text now flows through text aggregators regardless of mode, enabling pattern detection and tag handling in TOKEN mode.

changelog/3696.deprecated.md

@@ -1 +0,0 @@
-- ⚠️ Deprecated `aggregate_sentences` parameter on `TTSService` and all TTS subclasses. Use `text_aggregation_mode=TextAggregationMode.SENTENCE` or `text_aggregation_mode=TextAggregationMode.TOKEN` instead.

changelog/3714.added.md

@@ -1,19 +0,0 @@
-- Added support for using strongly-typed objects instead of dicts for updating service settings at runtime.
-
-  Instead of, say:
-
-  ```python
-  await task.queue_frame(
-      STTUpdateSettingsFrame(settings={"language": Language.ES})
-  )
-  ```
-
-  you'd do:
-
-  ```python
-  await task.queue_frame(
-      STTUpdateSettingsFrame(delta=DeepgramSTTSettings(language=Language.ES))
-  )
-  ```
-
-  Each service now vends strongly-typed classes like `DeepgramSTTSettings` representing the service's runtime-updatable settings.

changelog/3714.changed.md

@@ -1 +0,0 @@
-- ⚠️ Refactored runtime-updatable service settings to use strongly-typed classes (`TTSSettings`, `STTSettings`, `LLMSettings`, and service-specific subclasses) instead of plain dicts. Each service's `_settings` now holds these strongly-typed objects. For service maintainers, see changes in COMMUNITY_INTEGRATIONS.md.

changelog/3714.deprecated.2.md

@@ -1 +0,0 @@
-- Dict-based `*UpdateSettingsFrame(settings={...})` is deprecated in favor of passing typed settings delta objects with `*UpdateSettingsFrame(delta={...})`.

changelog/3714.deprecated.md

@@ -1,3 +0,0 @@
-- Deprecated `set_model()`, `set_voice()`, and `set_language()` on AI services in favor of runtime updates via `TTSUpdateSettingsFrame`, `STTUpdateSettingsFrame`, and `LLMUpdateSettingsFrame`.
-
-  ⚠️ Note, too, a subtle behavior change in these deprecated methods. Whereas previously only `set_language()` caused the service to actually react to the update (e.g. by reconnecting to a remote service so it an pick up the change), now all these methods do. This change was made as part of a refactor making them all work the same way under the hood.

changelog/3759.performance.md

@@ -1 +0,0 @@
-- Switched `GradiumTTSService` from `InterruptibleWordTTSService` to `AudioContextWordTTSService`, eliminating websocket disconnect/reconnect on every interruption by using `client_req_id`-based multiplexing.

changelog/3764.added.md

@@ -1 +0,0 @@
-- Added support for specifying private endpoints for Azure Speech-to-Text, enabling use in private networks behind firewalls.

changelog/3786.changed.md

@@ -1 +0,0 @@
-- Word timestamp support has been moved from `WordTTSService` into `TTSService` via a new `supports_word_timestamps` parameter. Services that previously extended `WordTTSService`, `AudioContextWordTTSService`, or `WebsocketWordTTSService` now pass `supports_word_timestamps=True` to their parent `__init__` instead.

changelog/3786.deprecated.md

@@ -1,5 +0,0 @@
-- Deprecated `WordTTSService`, `WebsocketWordTTSService`, `AudioContextWordTTSService`, and `InterruptibleWordTTSService`. Use their non-word counterparts with `supports_word_timestamps=True` instead:
-  - `WordTTSService` → `TTSService(supports_word_timestamps=True)`
-  - `WebsocketWordTTSService` → `WebsocketTTSService(supports_word_timestamps=True)`
-  - `AudioContextWordTTSService` → `AudioContextTTSService(supports_word_timestamps=True)`
-  - `InterruptibleWordTTSService` → `InterruptibleTTSService(supports_word_timestamps=True)`

changelog/3794.fixed.md

@@ -1 +0,0 @@
-- Added `LLMSpecificMessage` handling in `LLMContextSummarizationUtil` to skip provider-specific messages during context summarization.

changelog/3795.fixed.md

@@ -1 +0,0 @@
-- Treated `response_cancel_not_active` as a non-fatal error in realtime services (`OpenAIRealtimeLLMService`, `GrokRealtimeLLMService`, `OpenAIRealtimeBetaLLMService`) to prevent WebSocket disconnection when cancelling an inactive response.

changelog/3803.fixed.md

@@ -1 +0,0 @@
-- Fixed Poetry compatibility by inlining `local-smart-turn-v3` dependencies (`transformers`, `onnxruntime`) into core dependencies instead of using a self-referential extra.

changelog/3803.removed.md

@@ -1 +0,0 @@
-- Removed `local-smart-turn-v3` optional extra from `pyproject.toml`. The `transformers` and `onnxruntime` packages are now always installed as core dependencies since they are required by the default turn stop strategy, `TurnAnalyzerUserTurnStopStrategy` which uses `LocalSmartTurnAnalyzerV3`.

changelog/3806.added.md

@@ -1 +0,0 @@
-- Added `output_medium` parameter to `AgentInputParams` and `OneShotInputParams` in Ultravox service to control initial output medium (text or voice) at call creation time.

changelog/3806.changed.2.md

@@ -1 +0,0 @@
-- Improved Ultravox TTFB measurement accuracy by using VAD speech end time instead of `UserStoppedSpeakingFrame` timing.

changelog/3806.changed.md

@@ -1 +0,0 @@
-- Aligned `UltravoxRealtimeLLMService` frame handling with OpenAI/Gemini realtime services: added `InterruptionFrame` handling with metrics cleanup, processing metrics at response boundaries, and improved agent transcript handling for both voice and text output modalities.

changelog/3807.changed.md

@@ -1 +0,0 @@
-- Updated `OpenAIRealtimeLLMService` default model to `gpt-realtime-1.5`.

changelog/3808.fixed.md

@@ -1 +0,0 @@
-- Fixed `SentryMetrics` method signatures to match updated `FrameProcessorMetrics` base class, resolving `TypeError` when using `start_time`/`end_time` keyword arguments.

changelog/3809.added.md

@@ -1 +0,0 @@
-- Added `TurnMetricsData` as a generic metrics class for turn detection, with e2e processing time measurement. `KrispVivaTurn` now emits `TurnMetricsData` with `e2e_processing_time_ms` tracking the interval from VAD speech-to-silence transition to turn completion.

changelog/3809.changed.md

@@ -1 +0,0 @@
-- Added `api_key` parameter to `KrispVivaSDKManager`, `KrispVivaTurn`, and `KrispVivaFilter` for Krisp SDK v1.6.1+ licensing. Falls back to `KRISP_VIVA_API_KEY` environment variable.

changelog/3809.deprecated.md

@@ -1 +0,0 @@
-- Deprecated `SmartTurnMetricsData` in favor of `TurnMetricsData`. `BaseSmartTurn` now emits `TurnMetricsData` directly.

changelog/3811.changed.md

@@ -1 +0,0 @@
-- Bumped `nltk` minimum version from 3.9.1 to 3.9.3 to resolve a security vulnerability.

changelog/3813.fixed.md

@@ -1 +0,0 @@
-- Fixed STT TTFB metrics not being reported for `SonioxSTTService` and `AWSTranscribeSTTService` due to missing `can_generate_metrics()` override.

changelog/3814.added.md

@@ -1 +0,0 @@
-- Added `on_audio_context_interrupted()` and `on_audio_context_completed()` callbacks to `AudioContextTTSService`. Subclasses can override these to perform provider-specific cleanup instead of overriding `_handle_interruption()`.

changelog/3814.fixed.md

@@ -1 +0,0 @@
-- Fixed an issue where `AudioContextTTSService`-based providers (AsyncAI, ElevenLabs, Inworld, Rime) did not close or clean up their server-side audio contexts after normal speech completion, only on interruption.

changelog/3819.changed.md

@@ -1,4 +0,0 @@
-- `ServiceSettingsUpdateFrame`s are now `UninterruptibleFrame`s. Generally speaking, you don't want a user interruption to prevent a service setting change from going into effect. Note that you usually don't use `ServiceSettingsUpdateFrame` directly, you use one of its subclasses:
-  - `LLMUpdateSettingsFrame`
-  - `TTSUpdateSettingsFrame`
-  - `STTUpdateSettingsFrame`

changelog/3822.fixed.md

@@ -1 +0,0 @@
-- Fixed STT TTFB metrics measuring timeout expiry time instead of actual transcript arrival time.

changelog/3825.fixed.md

@@ -1 +0,0 @@
-- Fixed `InterimTranscriptionFrame` and `TranslationFrame` being unintentionally pushed downstream in `LLMUserAggregator`. They are now consumed like `TranscriptionFrame`.

changelog/3828.fixed.md

@@ -1 +0,0 @@
-- Fixed misleading "Empty audio frame received for STT service" warnings when using audio filters (e.g. `RNNoiseFilter`, `KrispVivaFilter`, `AICFilter`) that buffer audio internally.

changelog/3837.fixed.md

@@ -1 +0,0 @@
-- Fixed issues with `RimeNonJsonTTSService` where trailing punctuation is sometimes vocalized

changelog/3838.removed.md

@@ -1 +0,0 @@
-- ⚠️ Removed `PlayHTTTSService` and `PlayHTHttpTTSService`. PlayHT has been shut down and is no longer available.

changelog/3845.fixed.md

@@ -1 +0,0 @@
-- Fixed `TTSSpeakFrame` not committing spoken text to the conversation context when used outside of an LLM response (e.g., bot greetings or injected speech).

changelog/3850.fixed.md

@@ -1 +0,0 @@
-- Removed verbose per-chunk audio logging from `GenesysAudioHookSerializer` that flooded production logs.

changelog/3852.deprecated.md

@@ -1 +0,0 @@
-- Deprecated `ProcessingMetricsData` and `start_processing_metrics()`/`stop_processing_metrics()` on `FrameProcessor` and `FrameProcessorMetrics`. These metrics don't accurately depict a service's performance. Instead, TTFB metrics are recommended. Processing metrics will be removed in the 1.0.0 version.

changelog/3855.added.2.md

@@ -1 +0,0 @@
-- Added optional `llm` field to `LLMContextSummarizationConfig` for routing summarization to a dedicated LLM service (e.g., a cheaper/faster model) instead of the pipeline's primary model.

changelog/3855.added.3.md

@@ -1 +0,0 @@
-- Added `summarization_timeout` to `LLMContextSummarizationConfig` (default 120s) to prevent hung LLM calls from permanently blocking future summarizations.

changelog/3855.added.4.md

@@ -1 +0,0 @@
-- Added `on_summary_applied` event to `LLMContextSummarizer` for observability, providing message counts before and after context summarization.

changelog/3855.added.md

@@ -1 +0,0 @@
-- Added `summary_message_template` to `LLMContextSummarizationConfig` for customizing how summaries are formatted when injected into context (e.g., wrapping in XML tags).

changelog/3855.changed.md

@@ -1 +0,0 @@
-- Updated context summarization to use `user` role instead of `assistant` for summary messages.

changelog/3857.fixed.md

@@ -1 +0,0 @@
-- Fixed `LocalSmartTurnAnalyzerV3` producing incorrect end-of-turn predictions at non-16kHz sample rates (e.g. 8kHz Twilio telephony) by adding automatic resampling to 16kHz before Whisper feature extraction.

changelog/3863.added.2.md

@@ -1 +0,0 @@
-- Added `LLMContextSummaryConfig` (summary generation params: `target_context_tokens`, `min_messages_after_summary`, `summarization_prompt`) and `LLMAutoContextSummarizationConfig` (auto-trigger thresholds: `max_context_tokens`, `max_unsummarized_messages`, plus a nested `summary_config`). These replace the monolithic `LLMContextSummarizationConfig`.

changelog/3863.added.md

@@ -1 +0,0 @@
-- Added `LLMSummarizeContextFrame` to trigger on-demand context summarization from anywhere in the pipeline (e.g. a function call tool). Accepts an optional `config: LLMContextSummaryConfig` to override summary generation settings per request.

changelog/3863.changed.md

@@ -1 +0,0 @@
-- ⚠️ Renamed `LLMAssistantAggregatorParams` fields: `enable_context_summarization` → `enable_auto_context_summarization` and `context_summarization_config` → `auto_context_summarization_config` (now accepts `LLMAutoContextSummarizationConfig`). The old names still work with a `DeprecationWarning` for one release cycle.

changelog/3863.deprecated.md

@@ -1 +0,0 @@
-- Deprecated `LLMContextSummarizationConfig`. Use `LLMAutoContextSummarizationConfig` with a nested `LLMContextSummaryConfig` instead. The old class emits a `DeprecationWarning`.

changelog/3865.changed.md

@@ -1 +0,0 @@
-- `ElevenLabsRealtimeSTTService` now sets `TranscriptionFrame.finalized` to `True` when using `CommitStrategy.MANUAL`.

changelog/3867.fixed.md

@@ -1 +0,0 @@
-- Fixed `PipelineTask` double-inserting `RTVIProcessor` into the frame chain when the user provides both an `RTVIProcessor` in the pipeline and a custom `RTVIObserver` subclass in observers.

changelog/3868.changed.md

@@ -1 +0,0 @@
-- Updated numba version pin from == to >=0.61.2

changelog/3879.changed.md

@@ -1 +0,0 @@
-- Updated tracing code to use `ServiceSettings` dataclass API (`given_fields()`, attribute access) instead of dict-style access (`.items()`, `in`, subscript).

changelog/3883.added.md

@@ -1 +0,0 @@
-- Added optional `direction` parameter to `PipelineTask.queue_frame()` and `PipelineTask.queue_frames()`, allowing frames to be pushed upstream from the end of the pipeline.

changelog/3888.fixed.md

@@ -1 +0,0 @@
-- Fixed turn completion instructions being lost when `LLMMessagesUpdateFrame` replaces the LLM context. When `filter_incomplete_user_turns` is enabled, the turn completion system message is now re-injected after context replacement.