Commit Graph

4732 Commits

Author SHA1 Message Date
sysradium
ba0ebd5525 Treat conversation_already_has_active_response as non-fatal in Realtime API 2026-03-10 15:57:55 +01:00
Mark Backman
764c3c4f32 Merge pull request #3938 from koriyoshi2041/fix/replace-bare-except-handlers
fix: replace bare except handlers with specific exception types
2026-03-08 09:04:49 -04:00
radhikagpt1208
b14c8e0e94 Fix turn completion mixin not resetting state after each LLM response 2026-03-08 08:46:45 -04:00
kigland
57f0b6d75b fix: address review feedback on exception handling
- mcp_service.py: remove unnecessary try/except around debug log,
  use len(available_tools.tools) to match actual iteration target
- bedrock_adapter.py, aws/llm.py: add AttributeError to except tuple
  to handle None content (previously caught by bare except)
2026-03-08 12:28:03 +08:00
Mark Backman
fdf9fb6f02 Merge pull request #3946 from pipecat-ai/mb/tts-settings-review
Review TTS settings
2026-03-07 07:48:26 -05:00
Mark Backman
1cfaea2007 Address code review feedback 2026-03-07 07:42:42 -05:00
Paul Kompfner
622d9279cb Use exact service class names in LLMSettings docstrings 2026-03-06 22:35:55 -05:00
Paul Kompfner
6088e6eb52 Make budget_tokens optional in AnthropicThinkingConfig
budget_tokens is required when type is "enabled" and rejected when type is "disabled" (this is validated by the server)
2026-03-06 22:32:14 -05:00
Paul Kompfner
256c8f87b4 Add missing step 3 comment to LLM service init methods
Adds the explicit "no params object" step 3 comment to all
LLM services that skip from step 2 to step 4 in their
settings initialization sequence, matching the pattern
established in services that do have a params object.
2026-03-06 22:32:14 -05:00
Mark Backman
ec93cd1d51 Fix settings update handling in additional STT services 2026-03-06 21:52:45 -05:00
Mark Backman
9c42d27f4d Support runtime language updates in Azure STT
Extract recognizer setup/teardown into _connect/_disconnect so
_update_settings can reconnect when language changes at runtime.
2026-03-06 21:07:03 -05:00
Mark Backman
536f1e178a Fix race condition in Deepgram STT disconnect causing error flood
Clear self._connection before sending close stream so run_stt stops
sending audio immediately during the WebSocket close handshake.
2026-03-06 21:01:31 -05:00
Mark Backman
671e9a6846 TTS service and example updates 2026-03-06 20:53:22 -05:00
Mark Backman
a97a086dbd Fix GeminiLiveLLMService init referencing undefined _params variable
Replace references to undefined `_params` with `self._settings` for
language and VAD config. Add missing `system_instruction` to default
settings to satisfy validate_complete(). Remove redundant line that
read language from the deprecated `params` arg.
2026-03-06 18:41:54 -05:00
Mark Backman
4ed3480e4b Update TTSSettings docstrings with the corresponding class name(s) 2026-03-06 16:40:38 -05:00
Mark Backman
7d41049b35 Review feedback, clarify corresponding class in STTSettings docstrings 2026-03-06 15:17:02 -05:00
Mark Backman
6431ad8e2a Fix service settings init ordering and example bugs
- Speechmatics: move config build after super().__init__ and settings
  delta so turn_detection_mode (e.g. ADAPTIVE) takes effect
- Google STT: fix example passing bare Language enum instead of list
- Google TTS: add missing explicit defaults for all custom settings fields
- Soniox: fix accidental tuple wrapping of STT service in example
- Speechmatics examples: fix system->user role in kick-off messages
- Deepgram Flux: move tag from settings to __init__ (billing metadata)
- ElevenLabs STT: default tag_audio_events to None (use API default)
- Fal STT: simplify language default handling
- Google TTS: rename GoogleStreamTTSSettings to GoogleTTSSettings
2026-03-06 15:17:01 -05:00
Mark Backman
c3794956ef Add deprecation version, fix foundational example double system message 2026-03-06 15:16:58 -05:00
Mark Backman
940da9eeeb Add vad_threshold to AssemblyAISTTSettings
Wire vad_threshold through Settings, default_settings, the deprecated
connection_params path, and _build_ws_url query params.
2026-03-06 15:16:10 -05:00
kompfner
1a1c5668de Merge pull request #3942 from pipecat-ai/pk/aws-nova-sonic-audio-config
Add AudioConfig class to AWSNovaSonicLLMService for non-deprecated au…
2026-03-06 14:58:22 -05:00
Filipi da Silva Fuchter
4b9fc8a30c Merge pull request #3804 from pipecat-ai/filipi/concurrent_audio_contexts
Allowing concurrent audio contexts
2026-03-06 14:49:57 -05:00
Paul Kompfner
9b7a86bb12 Add AudioConfig class to AWSNovaSonicLLMService for non-deprecated audio configuration
The audio fields (sample rates, sample sizes, channel counts) on the deprecated `Params` class had no non-deprecated equivalent. This adds an `AudioConfig` class and `audio_config` init arg so users can specify audio configuration without relying on the deprecated `params` parameter.
2026-03-06 14:39:53 -05:00
filipi87
07abd3d60f Fixed BotStoppedSpeakingFrame emission: now emitted as soon as TTSStoppedFrame is received, with a fallback silence-based timeout increased to reduce false positives 2026-03-06 16:16:11 -03:00
filipi87
88ff7c451b Refactored all 25+ TTS service implementations to use the new push_start_frame=True pattern 2026-03-06 16:15:59 -03:00
filipi87
24430d8d45 Fixing Piper test. 2026-03-06 16:15:26 -03:00
filipi87
921e9e1fc9 Refactoring TTS services to allow concurrent audio contexts. 2026-03-06 16:15:10 -03:00
Paul Kompfner
f4c039048c Adopt the settings pattern for Grok Realtime session properties
Move `session_properties` into `GrokRealtimeLLMSettings`, making `settings` the canonical way to configure Grok Realtime — matching the pattern used across the rest of the codebase. The `session_properties` init arg is now deprecated in favor of `settings=GrokRealtimeLLMSettings(session_properties=...)`.

`system_instruction` is synced bidirectionally between the top-level settings field and `session_properties.instructions`, with top-level taking precedence on conflict. (Unlike OpenAI Realtime, Grok's `SessionProperties` has no `model` field, so no model sync is needed.)
2026-03-06 12:53:26 -05:00
kompfner
d84a250b62 Merge pull request #3939 from pipecat-ai/pk/openai-realtime-settings-pattern
Adopt the `settings` pattern for OpenAI Realtime session properties
2026-03-06 12:39:08 -05:00
mattie ruth backman
18494658c3 rename models_vX to models.py and models_deprecated.py 2026-03-06 11:49:59 -05:00
mattie ruth backman
da0975a4e0 Fix forward reference 2026-03-06 11:49:59 -05:00
mattie ruth backman
49fba5209c copilot feedback 2026-03-06 11:49:59 -05:00
mattie ruth backman
158424aa28 Convert RTVI framework into a structured package
Replace the monolithic rtvi.py with a proper package split by concern
protocol version:
  - models_v0.py: deprecated pre-1.0 Pydantic models
  - models_v1.py: current RTVI protocol v1 message models
  - frames.py: RTVI pipeline frame dataclasses
  - observer.py: RTVIObserver and RTVIObserverParams
  - processor.py: RTVIProcessor (now lean, imports from submodules)
  - __init__.py: re-exports full public API for backward compatability
2026-03-06 11:49:59 -05:00
Paul Kompfner
bd4229ea9d Adopt the settings pattern for OpenAI Realtime session properties
Move `session_properties` into `OpenAIRealtimeLLMSettings`, making `settings` the canonical way to configure OpenAI Realtime — matching the pattern used across the rest of the codebase. The `session_properties` init arg is now deprecated in favor of `settings=OpenAIRealtimeLLMSettings(session_properties=...)`.

`model` and `system_instruction` are synced bidirectionally between the top-level settings fields and `session_properties.model`/`.instructions`, with top-level taking precedence on conflict.
2026-03-06 11:46:21 -05:00
kigland
848f35f5df fix: replace bare except handlers with specific exception types 2026-03-06 23:05:02 +08:00
Paul Kompfner
a1641f3762 Add system_instruction to realtime service settings
Add `system_instruction=None` to `default_settings` for OpenAIRealtimeLLMService, GrokRealtimeLLMService, UltravoxRealtimeLLMService, AWSNovaSonicLLMService (Azure inherits from OpenAI), and OpenAIRealtimeBetaLLMService (Azure Beta inherits from OpenAI Beta).

Deprecate `system_instruction` init arg in AWSNovaSonicLLMService in favor of `settings=AWSNovaSonicLLMSettings(system_instruction=...)`. Use `self._settings.system_instruction` directly instead of storing a separate `self._system_instruction`.

Deprecation of `params` and `session_properties` in favor of `settings` for realtime services will be tackled in future work.
2026-03-06 09:57:34 -05:00
Paul Kompfner
78deaa735d Move system_instruction into LLMSettings
Add `system_instruction` field to `LLMSettings` so it is runtime-updatable via settings.
For Google (GoogleLLMService, GoogleVertexLLMService), deprecate the init-time arg since it was already shipped. For Anthropic, AWS Bedrock, and OpenAI, remove the init-time arg entirely since it was never shipped.

Still need to handle realtime services (OpenAI Realtime, Grok Realtime, Gemini Live).
2026-03-06 09:57:08 -05:00
Mark Backman
8a203dd98f Update more examples, misc services 2026-03-06 08:30:00 -05:00
Mark Backman
62554a2390 Update examples 2026-03-06 08:30:00 -05:00
Mark Backman
939d753c2b Update LLMs 2026-03-06 08:29:14 -05:00
Mark Backman
a4375274b2 Add Settings subclasses to all services and auto-discovered init tests
- Add dedicated Settings subclasses to 20 LLM services that were
  borrowing parent Settings classes (e.g. AzureLLMSettings,
  GroqLLMSettings) so users don't need cross-module imports
- Fix field defaults to NOT_GIVEN in BaseWhisperSTTSettings,
  OpenAIRealtimeSTTSettings, and NvidiaSegmentedSTTSettings for
  delta-mode safety
- Fix incomplete default_settings in AWS, Cartesia, ElevenLabs,
  Fish, and Whisper services so validate_complete() passes
- Add auto-discovered tests that verify all Settings classes default
  to NOT_GIVEN (delta safety) and all services initialize with
  complete settings (store completeness)
2026-03-06 08:29:14 -05:00
Mark Backman
034e81ff18 Update STT service settings 2026-03-06 08:29:14 -05:00
Mark Backman
3cb792a801 Update TTS service settings 2026-03-06 08:29:14 -05:00
Mark Backman
1274bb2c55 Update deprecation version to 0.0.105 2026-03-06 08:29:14 -05:00
Mark Backman
f31bfcf4ec Clean up CartesiaTTSSettings: separate init-only vs runtime-updatable fields
Move output_container, output_encoding, output_sample_rate out of
CartesiaTTSSettings into plain instance attributes since they cannot
change at runtime without breaking the audio pipeline. Remove deprecated
speed/emotion fields and their dead references in _build_msg() and
run_tts(). Remove the from_mapping override that only existed to
destructure those now-removed output format fields.
2026-03-06 08:29:14 -05:00
Mark Backman
07f1d0cd96 Change _warn_deprecated_param to accept type references instead of strings
Update all ~192 call sites across 84 service files to pass class references
(e.g. `CartesiaTTSSettings`) instead of string names (`"CartesiaTTSSettings"`)
to `_warn_deprecated_param()`. This enables better IDE refactoring support.

Also fix `from_mapping` return type annotations in 5 settings subclasses to
use `typing.Self` instead of forward reference strings.
2026-03-06 08:29:14 -05:00
Mark Backman
bc2843e30a Fix deprecation version 2026-03-06 08:29:14 -05:00
Paul Kompfner
5dc312ce0c Add settings as canonical init arg for all AIService descendants, deprecate redundant model/voice/params args
ServiceSettings types were introduced for runtime updates via ServiceUpdateSettingsFrame, but there was tension between init-time and runtime APIs: overlapping-but-different InputParams vs ServiceSettings classes, and runtime-updatable fields like `model` and `voice` scattered as direct init args rather than living in a settings object. This unifies them so developers use the same settings type at both init and runtime, improving ergonomics and consistency.

Every concrete AIService subclass (LLM, TTS, STT, ImageGen, Vision, Video) now accepts a `settings` parameter for runtime-updatable config. Old init args (`model`, `voice_id`, `params`/`InputParams`) still work but emit DeprecationWarnings pointing to the new API. When both are provided, `settings` takes precedence. Leaf classes emit warnings; base classes do not, avoiding double warnings in inheritance chains.
2026-03-06 08:29:14 -05:00
Aleix Conchillo Flaqué
1221e2dd76 Fix Daily transport log level and eval script import
Change participant_updated log from debug to trace (too noisy).
Fix deepgram LiveOptions import in eval script.
2026-03-05 16:37:02 -08:00
Mark Backman
79131dd6c6 Merge pull request #3930 from dakshdua/main
Add `push_empty_transcripts` param to `BaseWhisperSTTService` to push received empty transcripts downstream
2026-03-05 19:25:15 -05:00
Daksh Dua
789ce2fd5e Add param to push empty transcripts 2026-03-05 16:16:24 -08:00