From cb0335c82a30eaa92a88d9be60873c130b5defe9 Mon Sep 17 00:00:00 2001 From: aconchillo <951761+aconchillo@users.noreply.github.com> Date: Mon, 27 Apr 2026 20:55:17 +0000 Subject: [PATCH] Update changelog for version 1.1.0 --- CHANGELOG.md | 338 +++++++++++++++++++++++++++++++++++ changelog/4253.added.md | 1 - changelog/4269.changed.md | 2 - changelog/4270.changed.md | 1 - changelog/4301.fixed.md | 1 - changelog/4304.fixed.md | 1 - changelog/4311.changed.md | 1 - changelog/4311.fixed.md | 1 - changelog/4313.added.2.md | 1 - changelog/4313.added.3.md | 1 - changelog/4313.added.md | 1 - changelog/4314.changed.md | 1 - changelog/4320.fixed.md | 1 - changelog/4323.fixed.md | 1 - changelog/4324.added.md | 1 - changelog/4324.changed.md | 1 - changelog/4325.fixed.md | 1 - changelog/4326.added.md | 1 - changelog/4326.changed.md | 1 - changelog/4330.changed.md | 2 - changelog/4334.added.md | 1 - changelog/4334.changed.md | 1 - changelog/4337.changed.2.md | 1 - changelog/4337.changed.md | 1 - changelog/4340.added.md | 1 - changelog/4341.added.md | 1 - changelog/4344.fixed.md | 1 - changelog/4347.fixed.md | 1 - changelog/4349.changed.md | 1 - changelog/4352.changed.md | 1 - changelog/4352.fixed.2.md | 1 - changelog/4352.fixed.md | 1 - changelog/4355.fixed.md | 1 - changelog/4358.fixed.md | 1 - changelog/4360.added.md | 1 - changelog/4362.changed.md | 1 - changelog/4370.added.2.md | 12 -- changelog/4370.added.md | 16 -- changelog/4370.deprecated.md | 1 - changelog/4371.added.md | 1 - changelog/4372.changed.md | 3 - changelog/4374.changed.md | 1 - changelog/4379.changed.md | 1 - 43 files changed, 338 insertions(+), 72 deletions(-) delete mode 100644 changelog/4253.added.md delete mode 100644 changelog/4269.changed.md delete mode 100644 changelog/4270.changed.md delete mode 100644 changelog/4301.fixed.md delete mode 100644 changelog/4304.fixed.md delete mode 100644 changelog/4311.changed.md delete mode 100644 changelog/4311.fixed.md delete mode 100644 changelog/4313.added.2.md delete mode 100644 changelog/4313.added.3.md delete mode 100644 changelog/4313.added.md delete mode 100644 changelog/4314.changed.md delete mode 100644 changelog/4320.fixed.md delete mode 100644 changelog/4323.fixed.md delete mode 100644 changelog/4324.added.md delete mode 100644 changelog/4324.changed.md delete mode 100644 changelog/4325.fixed.md delete mode 100644 changelog/4326.added.md delete mode 100644 changelog/4326.changed.md delete mode 100644 changelog/4330.changed.md delete mode 100644 changelog/4334.added.md delete mode 100644 changelog/4334.changed.md delete mode 100644 changelog/4337.changed.2.md delete mode 100644 changelog/4337.changed.md delete mode 100644 changelog/4340.added.md delete mode 100644 changelog/4341.added.md delete mode 100644 changelog/4344.fixed.md delete mode 100644 changelog/4347.fixed.md delete mode 100644 changelog/4349.changed.md delete mode 100644 changelog/4352.changed.md delete mode 100644 changelog/4352.fixed.2.md delete mode 100644 changelog/4352.fixed.md delete mode 100644 changelog/4355.fixed.md delete mode 100644 changelog/4358.fixed.md delete mode 100644 changelog/4360.added.md delete mode 100644 changelog/4362.changed.md delete mode 100644 changelog/4370.added.2.md delete mode 100644 changelog/4370.added.md delete mode 100644 changelog/4370.deprecated.md delete mode 100644 changelog/4371.added.md delete mode 100644 changelog/4372.changed.md delete mode 100644 changelog/4374.changed.md delete mode 100644 changelog/4379.changed.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 7fb1006b0..4bb2c63dd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,344 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 +## [1.1.0] - 2026-04-27 + +### Added + +- Added `MistralSTTService` for real-time speech-to-text using Mistral's + Voxtral Realtime API (`voxtral-mini-transcribe-realtime-2602`). Supports + streaming transcription with interim results, automatic language detection, + and VAD-driven utterance lifecycle. + (PR [#4253](https://github.com/pipecat-ai/pipecat/pull/4253)) + +- Added `buttons` field to `OutputDTMFFrame` and `OutputDTMFUrgentFrame` for + sending multi-key DTMF sequences as a `list[KeypadEntry]`. Use + `OutputDTMFFrame.from_string("123#")` (or the equivalent on + `OutputDTMFUrgentFrame`) to build one from a dial string, and `to_string()` + to convert back. + (PR [#4313](https://github.com/pipecat-ai/pipecat/pull/4313)) + +- Added `DailyTransport.send_dtmf()` to expose the Daily call client's DTMF + sending capability, enabling applications to send tones during a call (e.g. + IVR navigation). + (PR [#4313](https://github.com/pipecat-ai/pipecat/pull/4313)) + +- Added `DailyOutputDTMFFrame` and `DailyOutputDTMFUrgentFrame` frames. In + addition to the inherited `buttons`, they accept `session_id`, + `digit_duration_ms` and `method`, which are forwarded to Daily's `send_dtmf` + as `sessionId`, `digitDurationMs` and `method`. + (PR [#4313](https://github.com/pipecat-ai/pipecat/pull/4313)) + +- Added incremental `pyright` type checking. A `pyrightconfig.json` at the repo + root uses `typeCheckingMode: "basic"` with an explicit `include` list of + modules that pass cleanly (`clocks`, `metrics`, `transcriptions`, `frames`, + `observers`, `extensions`, `turns`, `pipeline`, `runner`). Remaining modules + will be added in subsequent PRs. CI enforces the checked set via `uv run + pyright` in the format workflow. + (PR [#4324](https://github.com/pipecat-ai/pipecat/pull/4324)) + +- Added multilingual support to `DeepgramFluxSTTService` via a new + `language_hints: list[Language]` setting. Works with Deepgram's new + `flux-general-multi` model to bias transcription across English, Spanish, + French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. + Omit the hints to use auto-detection, or pass a subset to bias toward + expected languages. Hints can be updated mid-stream via + `STTUpdateSettingsFrame` (sent as a Deepgram `Configure` control message, no + reconnect) to support detect-then-lock flows. + (PR [#4326](https://github.com/pipecat-ai/pipecat/pull/4326)) + +- Added fine-grained server-side VAD tuning options to + `SarvamSTTService.Settings` for the `saaras:v3` model, including speech + thresholds, frame-count controls, pre-speech padding, interruption + sensitivity, and initial-frame skipping. + (PR [#4334](https://github.com/pipecat-ai/pipecat/pull/4334)) + +- Added `XAISTTService` for real-time speech-to-text using xAI's voice STT + WebSocket API (`wss://api.x.ai/v1/stt`). Streams raw audio (PCM, µ-law, or + A-law) and emits interim and final transcription frames driven by the + server's `is_final` / `speech_final` flags. Settings expose + `interim_results`, `endpointing`, `language`, `multichannel`, `channels`, and + `diarize`. Requires the `xai` optional extra (`pip install + "pipecat-ai[xai]"`). + (PR [#4340](https://github.com/pipecat-ai/pipecat/pull/4340)) + +- Added `XAITTSService` for streaming text-to-speech using xAI's WebSocket TTS + endpoint (`wss://api.x.ai/v1/tts`). Streams `text.delta` chunks up and base64 + `audio.delta` chunks down on the same connection so audio begins flowing + before the full utterance finishes synthesizing; complements the batch-HTTP + `XAIHttpTTSService`. Defaults to raw PCM output so `TTSAudioRawFrame` needs + no decoding. The `xai` optional extra now pulls in + `pipecat-ai[websockets-base]`. + (PR [#4341](https://github.com/pipecat-ai/pipecat/pull/4341)) + +- Added `SonioxTTSService`, a real-time WebSocket TTS service that streams text + in and audio out over a persistent connection. Install with `pip install + "pipecat-ai[soniox]"`. + (PR [#4360](https://github.com/pipecat-ai/pipecat/pull/4360)) + +- Added support for Daily's built-in `screenVideo` destination in + `DailyTransport`. When `"screenVideo"` is included in + `video_out_destinations` transport parameter, a dedicated screen video track + is created at join time and frames with `transport_destination="screenVideo"` + are routed to it. + + ```python + params = DailyParams( + video_out_enabled=True, + video_out_is_live=True, + video_out_width=1280, + video_out_height=720, + video_out_destinations=["screenVideo"] + ) + + ... + + frame = OutputImageRawFrame(...) + frame.transport_destination = "screenVideo" + ``` + (PR [#4370](https://github.com/pipecat-ai/pipecat/pull/4370)) + +- Added `camera_out_send_settings` to `DailyParams`. This dict is passed + verbatim to the Daily client's camera publishing settings, allowing + applications to fully control encoding, codec, bitrate, and framerate. + + ```python + params = DailyParams( + camera_out_send_settings={ + "maxQuality": "high", + "encodings": { + "high": {"maxBitrate": 2_000_000, "maxFramerate": 30} + }, + }, + ) + ``` + (PR [#4370](https://github.com/pipecat-ai/pipecat/pull/4370)) + +- Added `tool_resources` to `PipelineTask` and `FunctionCallParams`. Pass an + application-defined object (DB handles, clients, state, etc.) to + `PipelineTask(..., tool_resources=...)` and access it from any tool handler + via `params.tool_resources`. Passed by reference; the caller retains their + handle and can read mutations after the task finishes. Resolves #4256. + (PR [#4371](https://github.com/pipecat-ai/pipecat/pull/4371)) + +### Changed + +- Updated NVIDIA STT services to align with Nemotron Speech defaults and + configuration: `api_key` is now optional for local deployments, additional + recognition settings are available (including alternatives, word offsets, and + diarization), and streaming/segmented docs now reflect Nemotron Speech APIs. + - NVIDIA streaming STT now sets `TranscriptionFrame.finalized=True` when the + provider marks a result as final, and preserves `language` on both + `TranscriptionFrame` and `InterimTranscriptionFrame`. + (PR [#4269](https://github.com/pipecat-ai/pipecat/pull/4269)) + +- Updated `NvidiaLLMService` to emit model reasoning as `LLMThought*Frame`s + (from both `reasoning_content` and `...` output), avoid mixing + reasoning text into normal assistant content, and allow keyless local NIM + endpoints while warning when the cloud endpoint is used without an API key. + (PR [#4270](https://github.com/pipecat-ai/pipecat/pull/4270)) + +- STT services now reconnect safely when settings change: reconnection is + deferred until the current user turn ends (i.e., until + `UserStoppedSpeakingFrame` is received) rather than interrupting an active + speech session. Audio frames received while the reconnect is in progress are + buffered and replayed once the new connection is ready. `CartesiaSTTService` + and `DeepgramSTTService` both use this new behavior. + (PR [#4311](https://github.com/pipecat-ai/pipecat/pull/4311)) + +- Reduced debug log noise for LLM services. The system instruction is now + logged once when composed (e.g. when turn completion is enabled) instead of + on every LLM call. Per-call logs now show only the conversation messages, + consistent across Google, Anthropic, AWS, and OpenAI services. + (PR [#4314](https://github.com/pipecat-ai/pipecat/pull/4314)) + +- `LiveKitRunnerArguments.token` is now a required `str` (previously `str | + None` with a default of `None`). LiveKit requires a token to join a room, so + the type now reflects reality. This only affects custom runners that + construct `LiveKitRunnerArguments` directly; code consuming the argument from + the standard runner is unaffected. + (PR [#4324](https://github.com/pipecat-ai/pipecat/pull/4324)) + +- `TranscriptionFrame.language` and `InterimTranscriptionFrame.language` + emitted by `DeepgramFluxSTTService` now reflect the language Deepgram + detected for each turn (read from the `languages` field on Flux's `TurnInfo` + event). On `flux-general-multi` this gives per-turn accuracy for downstream + consumers (e.g. TTS voice selection). `flux-general-en` continues to emit + `Language.EN`. + (PR [#4326](https://github.com/pipecat-ai/pipecat/pull/4326)) + +- Added `includes_inter_frame_spaces` parameter to + `TTSService.add_word_timestamps` and `_add_word_timestamps` (default `None`). + When `True`, downstream consumers will not inject additional spaces between + tokens; `None` leaves each frame's own default unchanged. + - `InworldTTSService` now passes `includes_inter_frame_spaces=True` when + reporting word timestamps, since Inworld tokens already include inter-word + spacing. + (PR [#4330](https://github.com/pipecat-ai/pipecat/pull/4330)) + +- `SarvamSTTService` now uses `saaras:v3` as its default model instead of + `saarika:v2.5`. Applications that relied on the previous default should set + `settings=SarvamSTTService.Settings(model="saarika:v2.5")` explicitly. + (PR [#4334](https://github.com/pipecat-ai/pipecat/pull/4334)) + +- `SpeechTimeoutUserTurnStopStrategy` now waits only `user_speech_timeout` when + a transcript arrives without a VAD stop event, rather than + `max(ttfs_p99_latency, user_speech_timeout)`. If you had `ttfs_p99_latency > + user_speech_timeout`, turn detection in that path is slightly faster than + before. + (PR [#4337](https://github.com/pipecat-ai/pipecat/pull/4337)) + +- If you use an STT service that emits finalized transcripts (Speechmatics, + Soniox, Deepgram Flux, AssemblyAI) with `SpeechTimeoutUserTurnStopStrategy`, + user turns now end as soon as `user_speech_timeout` elapses after VAD stop. + Previously the strategy also waited for the STT P99 latency + (`ttfs_p99_latency`) even when the transcript was already marked final. + `user_speech_timeout` is still honored as a floor — STT finalization never + shortens it. + (PR [#4337](https://github.com/pipecat-ai/pipecat/pull/4337)) + +- ⚠️ `PlivoFrameSerializer` and `TelnyxFrameSerializer` now raise `ValueError` + at construction when `auto_hang_up=True` (the default) but required + credentials are missing, matching `TwilioFrameSerializer`. Previously they + constructed successfully and the hangup failed silently at call-end, leaving + phantom billable sessions on the provider. If you relied on the old silent + behavior, pass `auto_hang_up=False` explicitly or provide the credentials. + The specific fields checked are `call_id`/`auth_id`/`auth_token` for Plivo + and `call_control_id`/`api_key` for Telnyx. + (PR [#4349](https://github.com/pipecat-ai/pipecat/pull/4349)) + +- `ToolsSchema(standard_tools=...)` now accepts any `Sequence[FunctionSchema | + DirectFunction]` rather than requiring an exact `list` of the union. Callers + can pass a narrower `list[FunctionSchema]` (or any other `Sequence`) without + the type checker complaining about list invariance. + (PR [#4352](https://github.com/pipecat-ai/pipecat/pull/4352)) + +- Updated `aic-sdk` dependency to `~=2.2.0`. The `AIC_LICENSE_KEY` environment + variable replaces the previous `AICOUSTICS_LICENSE_KEY`. + (PR [#4362](https://github.com/pipecat-ai/pipecat/pull/4362)) + +- Loosened the `protobuf` dependency to `>=5.29.6,<7`, so projects pinned to + protobuf 5.x can install `pipecat-ai` again. The previous `>=6.31.1,<7` pin + (introduced in 1.0.8 alongside the `nvidia-riva-client 2.25.1` upgrade) + silently blocked any environment whose dependency graph already constrained + protobuf to the 5.x line. The bundled `frames_pb2.py` is now compiled with + protoc 5.x so it imports cleanly on both 5.x and 6.x runtimes. + + Installing the `nvidia` extra still pulls protobuf 6.x: `nvidia-riva-client + 2.25.1` ships gencode that requires a 6.x runtime, so `pipecat-ai[nvidia]` + now declares `protobuf>=6.31.1,<7` explicitly to cover an upstream packaging + gap (https://github.com/nvidia-riva/python-clients/issues/172). + (PR [#4372](https://github.com/pipecat-ai/pipecat/pull/4372)) + +- Daily rooms created by the development runner (`pipecat.runner.run`) now + expire after 4 hours with `eject_at_room_exp=True`, mirroring Pipecat Cloud's + max session limit. Previously, runner-created rooms inherited a 2-hour + expiration on the default code paths and had no expiration at all when + callers posted partial `dailyRoomProperties` (e.g. `{"start_video_off": + true}`) to `/start`, causing rooms to accumulate indefinitely. Explicit `exp` + and `eject_at_room_exp` values in `dailyRoomProperties` are still respected. + (PR [#4374](https://github.com/pipecat-ai/pipecat/pull/4374)) + +- Updated `daily-python` dependency to `~=0.28.0`. + (PR [#4379](https://github.com/pipecat-ai/pipecat/pull/4379)) + +### Deprecated + +- Deprecated `TransportParams.video_out_bitrate` for the Daily transport. Use + `DailyParams.camera_out_send_settings` instead to configure camera publishing + encodings (bitrate, framerate, codec, etc.). + (PR [#4370](https://github.com/pipecat-ai/pipecat/pull/4370)) + +### Fixed + +- Fixed missing tool handlers so unregistered tool calls fail with a normal + final tool result instead of leaving tool-call state hanging. + (PR [#4301](https://github.com/pipecat-ai/pipecat/pull/4301)) + +- Fixed `pipecat-ai[tavus]` not installing the required `daily-python` + dependency. Installing the `tavus` extra now correctly pulls in + `pipecat-ai[daily]`. + (PR [#4304](https://github.com/pipecat-ai/pipecat/pull/4304)) + +- Fixed audio loss and potential errors when STT settings were updated + mid-speech. Previously, `CartesiaSTTService` and `DeepgramSTTService` would + immediately disconnect and reconnect when settings changed, dropping any + in-flight audio. Reconnection is now deferred until the user stops speaking, + and audio arriving during the reconnect window is buffered and replayed. +(PR [#4311](https://github.com/pipecat-ai/pipecat/pull/4311)) + +- Fixed `SmallestTTSService` WebSocket endpoint URL to match Smallest AI v4.0.0 + API (`wss://waves-api.smallest.ai` → `wss://api.smallest.ai`) and restored + keepalive using a silent space message instead of the unsupported flush + command. + (PR [#4320](https://github.com/pipecat-ai/pipecat/pull/4320)) + +- Fixed whitespace handling in TTS token streaming mode. Inter-token whitespace + (e.g., spaces between words) is now preserved for correct prosody, while + leading whitespace before the first non-whitespace token is still stripped to + avoid issues with TTS models that are sensitive to leading spaces. + (PR [#4323](https://github.com/pipecat-ai/pipecat/pull/4323)) + +- Fixed `SentryMetrics` silently dropping `MetricsFrame`s from + `stop_ttfb_metrics` and `stop_processing_metrics`. `SentryMetrics` called the + base `FrameProcessorMetrics` implementation but discarded its return value, + so `FrameProcessor` never pushed the `MetricsFrame` downstream. This + prevented observers (e.g. `UserBotLatencyObserver`, `MetricsLogObserver`) + from seeing TTFB and processing metrics for any service using + `metrics=SentryMetrics()`. The metrics were still calculated and Sentry + transactions still completed — only the downstream frame push was affected. + (PR [#4325](https://github.com/pipecat-ai/pipecat/pull/4325)) + +- Fixed `ElevenLabsTTSService` and `ElevenLabsHttpTTSService` emitting word + timestamps and `TTSTextFrame` content that matched the input text instead of + the spoken audio when a pronunciation dictionary + (`pronunciation_dictionary_locators`) or text normalization rewrote the + input. Both services now consume ElevenLabs' normalized alignment, so + downstream consumers (captions, transcripts, context aggregation) reflect + what the listener actually hears. + (PR [#4344](https://github.com/pipecat-ai/pipecat/pull/4344)) + +- Fixed a crash in `DeepgramSTTService` when an `STTUpdateSettingsFrame` + arrived before the WebSocket handshake completed (for example, when pushing + an update upstream on `StartFrame`). The settings-triggered reconnect + cancelled the in-flight connection task before its keepalive task was + created, causing an `UnboundLocalError: cannot access local variable + 'keepalive_task'` in the handler's `finally` block. + (PR [#4347](https://github.com/pipecat-ai/pipecat/pull/4347)) + +- Fixed direct-function registration crashing for functions without a + docstring. `DirectFunctionWrapper` passed `inspect.getdoc()`'s result to + `docstring_parser.parse()`, which raises when the docstring is `None`. + Functions now register cleanly whether or not they have a docstring; an empty + docstring produces empty description and parameter metadata as expected. + (PR [#4352](https://github.com/pipecat-ai/pipecat/pull/4352)) + +- Fixed `AssemblyAISTTService`, `CartesiaSTTService`, `GradiumSTTService`, and + `SonioxSTTService` crashing the pipeline on transient WebSocket send + failures. Each `run_stt` sent audio directly without catching errors, so a + single network hiccup mid-stream raised an uncaught exception through + `process_frame`. The guards now log a warning and let the connection-state + check on the next call handle recovery, matching the pattern used by + Deepgram, xAI, Azure, and other push-based STTs. + (PR [#4352](https://github.com/pipecat-ai/pipecat/pull/4352)) + +- Fixed Gemini Live losing conversation history in the (rare) case of a + WebSocket reconnect before any session resumption handle is received. When + the session reconnects (e.g. on system instruction change), conversation + history is now re-seeded into the new session before it is marked ready for + input. + (PR [#4355](https://github.com/pipecat-ai/pipecat/pull/4355)) + +- Fixed SmallWebRTC data channel silently stalling on networks with a 1280-byte + MTU (IPv6, Tailscale overlays, many consumer VPNs). aiortc's default SCTP + chunk size of 1200 bytes produces ~1305-byte UDP datagrams after headers, + which the kernel rejects with EMSGSIZE; aiortc has no path-MTU discovery so + it retransmits forever at the same oversized size. The chunk size is now + clamped to 1100 bytes (~1205-byte datagrams, ~75 bytes of slack). Override + with `PIPECAT_SCTP_MAX_CHUNK_SIZE` if your path MTU requires a different + value. + (PR [#4358](https://github.com/pipecat-ai/pipecat/pull/4358)) + ## [1.0.0] - 2026-04-14 Migration guide: https://docs.pipecat.ai/pipecat/migration/migration-1.0 diff --git a/changelog/4253.added.md b/changelog/4253.added.md deleted file mode 100644 index c9a9f91de..000000000 --- a/changelog/4253.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added `MistralSTTService` for real-time speech-to-text using Mistral's Voxtral Realtime API (`voxtral-mini-transcribe-realtime-2602`). Supports streaming transcription with interim results, automatic language detection, and VAD-driven utterance lifecycle. diff --git a/changelog/4269.changed.md b/changelog/4269.changed.md deleted file mode 100644 index e88c45b2d..000000000 --- a/changelog/4269.changed.md +++ /dev/null @@ -1,2 +0,0 @@ -- Updated NVIDIA STT services to align with Nemotron Speech defaults and configuration: `api_key` is now optional for local deployments, additional recognition settings are available (including alternatives, word offsets, and diarization), and streaming/segmented docs now reflect Nemotron Speech APIs. -- NVIDIA streaming STT now sets `TranscriptionFrame.finalized=True` when the provider marks a result as final, and preserves `language` on both `TranscriptionFrame` and `InterimTranscriptionFrame`. diff --git a/changelog/4270.changed.md b/changelog/4270.changed.md deleted file mode 100644 index eb9f54972..000000000 --- a/changelog/4270.changed.md +++ /dev/null @@ -1 +0,0 @@ -- Updated `NvidiaLLMService` to emit model reasoning as `LLMThought*Frame`s (from both `reasoning_content` and `...` output), avoid mixing reasoning text into normal assistant content, and allow keyless local NIM endpoints while warning when the cloud endpoint is used without an API key. diff --git a/changelog/4301.fixed.md b/changelog/4301.fixed.md deleted file mode 100644 index 1ee4b092d..000000000 --- a/changelog/4301.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed missing tool handlers so unregistered tool calls fail with a normal final tool result instead of leaving tool-call state hanging. diff --git a/changelog/4304.fixed.md b/changelog/4304.fixed.md deleted file mode 100644 index f10b5bc54..000000000 --- a/changelog/4304.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed `pipecat-ai[tavus]` not installing the required `daily-python` dependency. Installing the `tavus` extra now correctly pulls in `pipecat-ai[daily]`. diff --git a/changelog/4311.changed.md b/changelog/4311.changed.md deleted file mode 100644 index 2a3dab454..000000000 --- a/changelog/4311.changed.md +++ /dev/null @@ -1 +0,0 @@ -- STT services now reconnect safely when settings change: reconnection is deferred until the current user turn ends (i.e., until `UserStoppedSpeakingFrame` is received) rather than interrupting an active speech session. Audio frames received while the reconnect is in progress are buffered and replayed once the new connection is ready. `CartesiaSTTService` and `DeepgramSTTService` both use this new behavior. diff --git a/changelog/4311.fixed.md b/changelog/4311.fixed.md deleted file mode 100644 index a407306d1..000000000 --- a/changelog/4311.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed audio loss and potential errors when STT settings were updated mid-speech. Previously, `CartesiaSTTService` and `DeepgramSTTService` would immediately disconnect and reconnect when settings changed, dropping any in-flight audio. Reconnection is now deferred until the user stops speaking, and audio arriving during the reconnect window is buffered and replayed. diff --git a/changelog/4313.added.2.md b/changelog/4313.added.2.md deleted file mode 100644 index 2093e0a8a..000000000 --- a/changelog/4313.added.2.md +++ /dev/null @@ -1 +0,0 @@ -- Added `buttons` field to `OutputDTMFFrame` and `OutputDTMFUrgentFrame` for sending multi-key DTMF sequences as a `list[KeypadEntry]`. Use `OutputDTMFFrame.from_string("123#")` (or the equivalent on `OutputDTMFUrgentFrame`) to build one from a dial string, and `to_string()` to convert back. diff --git a/changelog/4313.added.3.md b/changelog/4313.added.3.md deleted file mode 100644 index 8dc2c89f1..000000000 --- a/changelog/4313.added.3.md +++ /dev/null @@ -1 +0,0 @@ -- Added `DailyOutputDTMFFrame` and `DailyOutputDTMFUrgentFrame` frames. In addition to the inherited `buttons`, they accept `session_id`, `digit_duration_ms` and `method`, which are forwarded to Daily's `send_dtmf` as `sessionId`, `digitDurationMs` and `method`. diff --git a/changelog/4313.added.md b/changelog/4313.added.md deleted file mode 100644 index 36e4b4f2e..000000000 --- a/changelog/4313.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added `DailyTransport.send_dtmf()` to expose the Daily call client's DTMF sending capability, enabling applications to send tones during a call (e.g. IVR navigation). diff --git a/changelog/4314.changed.md b/changelog/4314.changed.md deleted file mode 100644 index 733958b55..000000000 --- a/changelog/4314.changed.md +++ /dev/null @@ -1 +0,0 @@ -- Reduced debug log noise for LLM services. The system instruction is now logged once when composed (e.g. when turn completion is enabled) instead of on every LLM call. Per-call logs now show only the conversation messages, consistent across Google, Anthropic, AWS, and OpenAI services. diff --git a/changelog/4320.fixed.md b/changelog/4320.fixed.md deleted file mode 100644 index 12e49c957..000000000 --- a/changelog/4320.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed `SmallestTTSService` WebSocket endpoint URL to match Smallest AI v4.0.0 API (`wss://waves-api.smallest.ai` → `wss://api.smallest.ai`) and restored keepalive using a silent space message instead of the unsupported flush command. diff --git a/changelog/4323.fixed.md b/changelog/4323.fixed.md deleted file mode 100644 index 26929a095..000000000 --- a/changelog/4323.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed whitespace handling in TTS token streaming mode. Inter-token whitespace (e.g., spaces between words) is now preserved for correct prosody, while leading whitespace before the first non-whitespace token is still stripped to avoid issues with TTS models that are sensitive to leading spaces. diff --git a/changelog/4324.added.md b/changelog/4324.added.md deleted file mode 100644 index d38887585..000000000 --- a/changelog/4324.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added incremental `pyright` type checking. A `pyrightconfig.json` at the repo root uses `typeCheckingMode: "basic"` with an explicit `include` list of modules that pass cleanly (`clocks`, `metrics`, `transcriptions`, `frames`, `observers`, `extensions`, `turns`, `pipeline`, `runner`). Remaining modules will be added in subsequent PRs. CI enforces the checked set via `uv run pyright` in the format workflow. diff --git a/changelog/4324.changed.md b/changelog/4324.changed.md deleted file mode 100644 index 672c60b62..000000000 --- a/changelog/4324.changed.md +++ /dev/null @@ -1 +0,0 @@ -- `LiveKitRunnerArguments.token` is now a required `str` (previously `str | None` with a default of `None`). LiveKit requires a token to join a room, so the type now reflects reality. This only affects custom runners that construct `LiveKitRunnerArguments` directly; code consuming the argument from the standard runner is unaffected. diff --git a/changelog/4325.fixed.md b/changelog/4325.fixed.md deleted file mode 100644 index b87a97bfb..000000000 --- a/changelog/4325.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed `SentryMetrics` silently dropping `MetricsFrame`s from `stop_ttfb_metrics` and `stop_processing_metrics`. `SentryMetrics` called the base `FrameProcessorMetrics` implementation but discarded its return value, so `FrameProcessor` never pushed the `MetricsFrame` downstream. This prevented observers (e.g. `UserBotLatencyObserver`, `MetricsLogObserver`) from seeing TTFB and processing metrics for any service using `metrics=SentryMetrics()`. The metrics were still calculated and Sentry transactions still completed — only the downstream frame push was affected. diff --git a/changelog/4326.added.md b/changelog/4326.added.md deleted file mode 100644 index 856c7b5d7..000000000 --- a/changelog/4326.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added multilingual support to `DeepgramFluxSTTService` via a new `language_hints: list[Language]` setting. Works with Deepgram's new `flux-general-multi` model to bias transcription across English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. Omit the hints to use auto-detection, or pass a subset to bias toward expected languages. Hints can be updated mid-stream via `STTUpdateSettingsFrame` (sent as a Deepgram `Configure` control message, no reconnect) to support detect-then-lock flows. diff --git a/changelog/4326.changed.md b/changelog/4326.changed.md deleted file mode 100644 index 4cc6e9bb7..000000000 --- a/changelog/4326.changed.md +++ /dev/null @@ -1 +0,0 @@ -- `TranscriptionFrame.language` and `InterimTranscriptionFrame.language` emitted by `DeepgramFluxSTTService` now reflect the language Deepgram detected for each turn (read from the `languages` field on Flux's `TurnInfo` event). On `flux-general-multi` this gives per-turn accuracy for downstream consumers (e.g. TTS voice selection). `flux-general-en` continues to emit `Language.EN`. diff --git a/changelog/4330.changed.md b/changelog/4330.changed.md deleted file mode 100644 index 839357a16..000000000 --- a/changelog/4330.changed.md +++ /dev/null @@ -1,2 +0,0 @@ -- Added `includes_inter_frame_spaces` parameter to `TTSService.add_word_timestamps` and `_add_word_timestamps` (default `None`). When `True`, downstream consumers will not inject additional spaces between tokens; `None` leaves each frame's own default unchanged. -- `InworldTTSService` now passes `includes_inter_frame_spaces=True` when reporting word timestamps, since Inworld tokens already include inter-word spacing. \ No newline at end of file diff --git a/changelog/4334.added.md b/changelog/4334.added.md deleted file mode 100644 index e3dc45543..000000000 --- a/changelog/4334.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added fine-grained server-side VAD tuning options to `SarvamSTTService.Settings` for the `saaras:v3` model, including speech thresholds, frame-count controls, pre-speech padding, interruption sensitivity, and initial-frame skipping. \ No newline at end of file diff --git a/changelog/4334.changed.md b/changelog/4334.changed.md deleted file mode 100644 index ff8d75147..000000000 --- a/changelog/4334.changed.md +++ /dev/null @@ -1 +0,0 @@ -- `SarvamSTTService` now uses `saaras:v3` as its default model instead of `saarika:v2.5`. Applications that relied on the previous default should set `settings=SarvamSTTService.Settings(model="saarika:v2.5")` explicitly. \ No newline at end of file diff --git a/changelog/4337.changed.2.md b/changelog/4337.changed.2.md deleted file mode 100644 index a1da061c8..000000000 --- a/changelog/4337.changed.2.md +++ /dev/null @@ -1 +0,0 @@ -- `SpeechTimeoutUserTurnStopStrategy` now waits only `user_speech_timeout` when a transcript arrives without a VAD stop event, rather than `max(ttfs_p99_latency, user_speech_timeout)`. If you had `ttfs_p99_latency > user_speech_timeout`, turn detection in that path is slightly faster than before. diff --git a/changelog/4337.changed.md b/changelog/4337.changed.md deleted file mode 100644 index 67c1ddbfe..000000000 --- a/changelog/4337.changed.md +++ /dev/null @@ -1 +0,0 @@ -- If you use an STT service that emits finalized transcripts (Speechmatics, Soniox, Deepgram Flux, AssemblyAI) with `SpeechTimeoutUserTurnStopStrategy`, user turns now end as soon as `user_speech_timeout` elapses after VAD stop. Previously the strategy also waited for the STT P99 latency (`ttfs_p99_latency`) even when the transcript was already marked final. `user_speech_timeout` is still honored as a floor — STT finalization never shortens it. diff --git a/changelog/4340.added.md b/changelog/4340.added.md deleted file mode 100644 index 5abcc0f0e..000000000 --- a/changelog/4340.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added `XAISTTService` for real-time speech-to-text using xAI's voice STT WebSocket API (`wss://api.x.ai/v1/stt`). Streams raw audio (PCM, µ-law, or A-law) and emits interim and final transcription frames driven by the server's `is_final` / `speech_final` flags. Settings expose `interim_results`, `endpointing`, `language`, `multichannel`, `channels`, and `diarize`. Requires the `xai` optional extra (`pip install "pipecat-ai[xai]"`). diff --git a/changelog/4341.added.md b/changelog/4341.added.md deleted file mode 100644 index 9301bcc5f..000000000 --- a/changelog/4341.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added `XAITTSService` for streaming text-to-speech using xAI's WebSocket TTS endpoint (`wss://api.x.ai/v1/tts`). Streams `text.delta` chunks up and base64 `audio.delta` chunks down on the same connection so audio begins flowing before the full utterance finishes synthesizing; complements the batch-HTTP `XAIHttpTTSService`. Defaults to raw PCM output so `TTSAudioRawFrame` needs no decoding. The `xai` optional extra now pulls in `pipecat-ai[websockets-base]`. diff --git a/changelog/4344.fixed.md b/changelog/4344.fixed.md deleted file mode 100644 index 4e786f65c..000000000 --- a/changelog/4344.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed `ElevenLabsTTSService` and `ElevenLabsHttpTTSService` emitting word timestamps and `TTSTextFrame` content that matched the input text instead of the spoken audio when a pronunciation dictionary (`pronunciation_dictionary_locators`) or text normalization rewrote the input. Both services now consume ElevenLabs' normalized alignment, so downstream consumers (captions, transcripts, context aggregation) reflect what the listener actually hears. diff --git a/changelog/4347.fixed.md b/changelog/4347.fixed.md deleted file mode 100644 index e5c8206e5..000000000 --- a/changelog/4347.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed a crash in `DeepgramSTTService` when an `STTUpdateSettingsFrame` arrived before the WebSocket handshake completed (for example, when pushing an update upstream on `StartFrame`). The settings-triggered reconnect cancelled the in-flight connection task before its keepalive task was created, causing an `UnboundLocalError: cannot access local variable 'keepalive_task'` in the handler's `finally` block. diff --git a/changelog/4349.changed.md b/changelog/4349.changed.md deleted file mode 100644 index d41e1edb1..000000000 --- a/changelog/4349.changed.md +++ /dev/null @@ -1 +0,0 @@ -- ⚠️ `PlivoFrameSerializer` and `TelnyxFrameSerializer` now raise `ValueError` at construction when `auto_hang_up=True` (the default) but required credentials are missing, matching `TwilioFrameSerializer`. Previously they constructed successfully and the hangup failed silently at call-end, leaving phantom billable sessions on the provider. If you relied on the old silent behavior, pass `auto_hang_up=False` explicitly or provide the credentials. The specific fields checked are `call_id`/`auth_id`/`auth_token` for Plivo and `call_control_id`/`api_key` for Telnyx. diff --git a/changelog/4352.changed.md b/changelog/4352.changed.md deleted file mode 100644 index ed77eec98..000000000 --- a/changelog/4352.changed.md +++ /dev/null @@ -1 +0,0 @@ -- `ToolsSchema(standard_tools=...)` now accepts any `Sequence[FunctionSchema | DirectFunction]` rather than requiring an exact `list` of the union. Callers can pass a narrower `list[FunctionSchema]` (or any other `Sequence`) without the type checker complaining about list invariance. diff --git a/changelog/4352.fixed.2.md b/changelog/4352.fixed.2.md deleted file mode 100644 index 5ea5de888..000000000 --- a/changelog/4352.fixed.2.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed direct-function registration crashing for functions without a docstring. `DirectFunctionWrapper` passed `inspect.getdoc()`'s result to `docstring_parser.parse()`, which raises when the docstring is `None`. Functions now register cleanly whether or not they have a docstring; an empty docstring produces empty description and parameter metadata as expected. diff --git a/changelog/4352.fixed.md b/changelog/4352.fixed.md deleted file mode 100644 index 6e66fb122..000000000 --- a/changelog/4352.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed `AssemblyAISTTService`, `CartesiaSTTService`, `GradiumSTTService`, and `SonioxSTTService` crashing the pipeline on transient WebSocket send failures. Each `run_stt` sent audio directly without catching errors, so a single network hiccup mid-stream raised an uncaught exception through `process_frame`. The guards now log a warning and let the connection-state check on the next call handle recovery, matching the pattern used by Deepgram, xAI, Azure, and other push-based STTs. diff --git a/changelog/4355.fixed.md b/changelog/4355.fixed.md deleted file mode 100644 index a9b6c932f..000000000 --- a/changelog/4355.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed Gemini Live losing conversation history in the (rare) case of a WebSocket reconnect before any session resumption handle is received. When the session reconnects (e.g. on system instruction change), conversation history is now re-seeded into the new session before it is marked ready for input. diff --git a/changelog/4358.fixed.md b/changelog/4358.fixed.md deleted file mode 100644 index 719111d8e..000000000 --- a/changelog/4358.fixed.md +++ /dev/null @@ -1 +0,0 @@ -- Fixed SmallWebRTC data channel silently stalling on networks with a 1280-byte MTU (IPv6, Tailscale overlays, many consumer VPNs). aiortc's default SCTP chunk size of 1200 bytes produces ~1305-byte UDP datagrams after headers, which the kernel rejects with EMSGSIZE; aiortc has no path-MTU discovery so it retransmits forever at the same oversized size. The chunk size is now clamped to 1100 bytes (~1205-byte datagrams, ~75 bytes of slack). Override with `PIPECAT_SCTP_MAX_CHUNK_SIZE` if your path MTU requires a different value. diff --git a/changelog/4360.added.md b/changelog/4360.added.md deleted file mode 100644 index 1c312b2b5..000000000 --- a/changelog/4360.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added `SonioxTTSService`, a real-time WebSocket TTS service that streams text in and audio out over a persistent connection. Install with `pip install "pipecat-ai[soniox]"`. diff --git a/changelog/4362.changed.md b/changelog/4362.changed.md deleted file mode 100644 index a34761e47..000000000 --- a/changelog/4362.changed.md +++ /dev/null @@ -1 +0,0 @@ -- Updated `aic-sdk` dependency to `~=2.2.0`. The `AIC_LICENSE_KEY` environment variable replaces the previous `AICOUSTICS_LICENSE_KEY`. diff --git a/changelog/4370.added.2.md b/changelog/4370.added.2.md deleted file mode 100644 index 355a2ce40..000000000 --- a/changelog/4370.added.2.md +++ /dev/null @@ -1,12 +0,0 @@ -- Added `camera_out_send_settings` to `DailyParams`. This dict is passed verbatim to the Daily client's camera publishing settings, allowing applications to fully control encoding, codec, bitrate, and framerate. - - ```python - params = DailyParams( - camera_out_send_settings={ - "maxQuality": "high", - "encodings": { - "high": {"maxBitrate": 2_000_000, "maxFramerate": 30} - }, - }, - ) - ``` diff --git a/changelog/4370.added.md b/changelog/4370.added.md deleted file mode 100644 index 620ba721e..000000000 --- a/changelog/4370.added.md +++ /dev/null @@ -1,16 +0,0 @@ -- Added support for Daily's built-in `screenVideo` destination in `DailyTransport`. When `"screenVideo"` is included in `video_out_destinations` transport parameter, a dedicated screen video track is created at join time and frames with `transport_destination="screenVideo"` are routed to it. - - ```python - params = DailyParams( - video_out_enabled=True, - video_out_is_live=True, - video_out_width=1280, - video_out_height=720, - video_out_destinations=["screenVideo"] - ) - - ... - - frame = OutputImageRawFrame(...) - frame.transport_destination = "screenVideo" - ``` diff --git a/changelog/4370.deprecated.md b/changelog/4370.deprecated.md deleted file mode 100644 index 6fea1c970..000000000 --- a/changelog/4370.deprecated.md +++ /dev/null @@ -1 +0,0 @@ -- Deprecated `TransportParams.video_out_bitrate` for the Daily transport. Use `DailyParams.camera_out_send_settings` instead to configure camera publishing encodings (bitrate, framerate, codec, etc.). diff --git a/changelog/4371.added.md b/changelog/4371.added.md deleted file mode 100644 index 92c7cc9e1..000000000 --- a/changelog/4371.added.md +++ /dev/null @@ -1 +0,0 @@ -- Added `tool_resources` to `PipelineTask` and `FunctionCallParams`. Pass an application-defined object (DB handles, clients, state, etc.) to `PipelineTask(..., tool_resources=...)` and access it from any tool handler via `params.tool_resources`. Passed by reference; the caller retains their handle and can read mutations after the task finishes. Resolves #4256. diff --git a/changelog/4372.changed.md b/changelog/4372.changed.md deleted file mode 100644 index d8483775e..000000000 --- a/changelog/4372.changed.md +++ /dev/null @@ -1,3 +0,0 @@ -- Loosened the `protobuf` dependency to `>=5.29.6,<7`, so projects pinned to protobuf 5.x can install `pipecat-ai` again. The previous `>=6.31.1,<7` pin (introduced in 1.0.8 alongside the `nvidia-riva-client 2.25.1` upgrade) silently blocked any environment whose dependency graph already constrained protobuf to the 5.x line. The bundled `frames_pb2.py` is now compiled with protoc 5.x so it imports cleanly on both 5.x and 6.x runtimes. - - Installing the `nvidia` extra still pulls protobuf 6.x: `nvidia-riva-client 2.25.1` ships gencode that requires a 6.x runtime, so `pipecat-ai[nvidia]` now declares `protobuf>=6.31.1,<7` explicitly to cover an upstream packaging gap (https://github.com/nvidia-riva/python-clients/issues/172). diff --git a/changelog/4374.changed.md b/changelog/4374.changed.md deleted file mode 100644 index 30ef7f2b6..000000000 --- a/changelog/4374.changed.md +++ /dev/null @@ -1 +0,0 @@ -- Daily rooms created by the development runner (`pipecat.runner.run`) now expire after 4 hours with `eject_at_room_exp=True`, mirroring Pipecat Cloud's max session limit. Previously, runner-created rooms inherited a 2-hour expiration on the default code paths and had no expiration at all when callers posted partial `dailyRoomProperties` (e.g. `{"start_video_off": true}`) to `/start`, causing rooms to accumulate indefinitely. Explicit `exp` and `eject_at_room_exp` values in `dailyRoomProperties` are still respected. diff --git a/changelog/4379.changed.md b/changelog/4379.changed.md deleted file mode 100644 index 666b9c7fd..000000000 --- a/changelog/4379.changed.md +++ /dev/null @@ -1 +0,0 @@ -- Updated `daily-python` dependency to `~=0.28.0`.