Commit Graph

4758 Commits

Author SHA1 Message Date
Mark Backman
2ce9179662 Merge pull request #3958 from pipecat-ai/mb/deepgram-tts-audio-context
Route Deepgram WebSocket TTS audio through audio context queue
2026-03-10 11:37:39 -04:00
Mark Backman
50cc01a578 Guard against None context ID in append_to_audio_context
After interruption, both _playing_context_id and _turn_context_id are
None. If a subclass calls append_to_audio_context(None, frame), the
recovery path matches (None == None) and creates a bogus audio context
that blocks the handler from ever processing the real context.

Early-return when context_id is falsy to prevent this.
2026-03-10 11:34:03 -04:00
Mark Backman
92b5185165 Route Deepgram WebSocket TTS audio through audio context queue
The Deepgram TTS service was bypassing pipecats audio context management
system, pushing audio frames directly via push_frame() instead of routing
them through append_to_audio_context(). This caused stale audio to leak
into the pipeline after interruptions and missed ordered playback
guarantees.

- Route audio frames through append_to_audio_context() with context
  availability checks to discard stale post-interruption frames
- Handle Flushed responses by appending TTSStoppedFrame and removing
  the audio context to signal completion
- Replace _handle_interruption override with on_audio_context_interrupted
  hook (the recommended pattern used by ElevenLabs and Cartesia)
- Remove redundant process_frame override that caused double-flush
  (base class already flushes via on_turn_context_completed)
- Remove redundant start_tts_usage_metrics call (base class handles
  aggregated usage metrics)
2026-03-10 11:34:03 -04:00
yukiobata1
c7ef23dd22 Update Fish Audio default model from s1 to s2-pro 2026-03-10 18:22:20 +09:00
Mark Backman
f533dc3203 Fix Azure STT SpeechConfig failing when private_endpoint is provided
SpeechConfig does not accept both `region` and `endpoint` simultaneously —
they are mutually exclusive. The previous code always passed both, which
raises ValueError when a user supplies a private_endpoint URL. Now we
conditionally pass either `endpoint` or `region`, never both.
2026-03-09 15:05:20 -04:00
Mark Backman
786279f143 Remove unused imports, 2026-03-07 2026-03-09 12:44:47 -04:00
Filipi da Silva Fuchter
0c5e936aa5 Merge pull request #3936 from pipecat-ai/filipi/fix_push_aggregation
Fixed TTS context not being appended to the assistant message history
2026-03-09 11:14:38 -04:00
filipi87
f0c5925a79 Fixing Piper test. 2026-03-09 12:07:45 -03:00
Mark Backman
c16e534f73 Merge pull request #3952 from pipecat-ai/mb/settings-alias
Add Settings class attribute alias to all service classes
2026-03-09 10:45:10 -04:00
Filipi da Silva Fuchter
1e615cd095 Merge pull request #3962 from pipecat-ai/filipi/smallwebrtc_queue
Queuing the messages received before the data channel is ready
2026-03-09 10:29:05 -04:00
filipi87
ba87d1609c Only marking self._is_yielding_frames_synchronously if receiving TTSAudioRawFrame 2026-03-09 11:24:36 -03:00
filipi87
c5ce667387 Retrieving the context_id from the TTSStartedFrame 2026-03-09 11:10:42 -03:00
filipi87
097f9c0896 Fixed to push LLMAssistantPushAggregationFrame when the base TTSService class is responsible for pushing the TTSStoppedFrame. 2026-03-09 11:04:09 -03:00
Filipi da Silva Fuchter
16336a3ea4 Merge pull request #3937 from pipecat-ai/filipi/fix_orphan_function_call
Fix context summarization leaving orphaned tool responses in kept context.
2026-03-09 09:19:17 -04:00
Mark Backman
9eaa99c8e2 Merge pull request #3957 from pipecat-ai/mb/user-turn-completion-system-instruction
Move turn completion instructions to system_instruction
2026-03-09 09:17:06 -04:00
filipi87
4557ef8c42 Renaming method to _get_earliest_function_call_not_resolved_in_range 2026-03-09 10:16:02 -03:00
filipi87
74a06a6968 Adding extra comment. 2026-03-09 10:06:38 -03:00
filipi87
322e317a00 Adding guardrails in case the data channel is never established. 2026-03-09 10:04:33 -03:00
filipi87
25165d6e2b Queuing the messages received before the data channel is ready to send them. 2026-03-09 09:47:45 -03:00
Mark Backman
8a02e6fbc5 Merge pull request #3959 from ajmeraharsh/fix/livekit-call-state-updated-args
fix(livekit): remove redundant self arg in on_call_state_updated
2026-03-09 08:38:12 -04:00
Mark Backman
d85ba75dda Merge pull request #3953 from pipecat-ai/mb/deepgram-flux-on-the-fly
Add on-the-fly Configure support for Deepgram Flux STT
2026-03-09 08:36:00 -04:00
Aleix Conchillo Flaqué
1f8cc3d216 Expose on_summary_applied event on LLMAssistantAggregator
Forward the on_summary_applied event from the internal summarizer to
the aggregator so users can listen for it without accessing private
members. Update summarization examples to use the new public event.
2026-03-08 19:02:51 -07:00
ajmeraharsh
57c4d72bf0 fix(livekit): remove redundant self arg in on_call_state_updated event
_on_call_state_updated passes (self, state) to _call_event_handler,
but _run_handler already prepends self when invoking the handler.
This causes handlers to receive 3 positional arguments instead of 2,
making the on_call_state_updated event unusable.

This aligns with how _on_first_participant_joined correctly passes
only the data arg without self.
2026-03-09 02:51:35 +04:00
Mark Backman
efda57de5c Move turn completion instructions to system_instruction
Turn completion instructions were being injected as a system message in
the LLM context, which caused warning spam when system_instruction was
also set, did not persist across full context updates, and broke LLMs
that do not support consecutive system messages.

Instead, compose the turn completion instructions into the LLM service
system_instruction field. This is managed via _base_system_instruction
which stores the original value for restoration when turn completion is
disabled.
2026-03-08 10:41:40 -04:00
Mark Backman
764c3c4f32 Merge pull request #3938 from koriyoshi2041/fix/replace-bare-except-handlers
fix: replace bare except handlers with specific exception types
2026-03-08 09:04:49 -04:00
radhikagpt1208
b14c8e0e94 Fix turn completion mixin not resetting state after each LLM response 2026-03-08 08:46:45 -04:00
kigland
57f0b6d75b fix: address review feedback on exception handling
- mcp_service.py: remove unnecessary try/except around debug log,
  use len(available_tools.tools) to match actual iteration target
- bedrock_adapter.py, aws/llm.py: add AttributeError to except tuple
  to handle None content (previously caught by bare except)
2026-03-08 12:28:03 +08:00
Mark Backman
c5da3cf2bd Add on-the-fly Configure support for Deepgram Flux STT
Wire up the existing settings update infrastructure to send a Configure
WebSocket message when keyterm, eot_threshold, eager_eot_threshold, or
eot_timeout_ms change mid-stream, avoiding a full reconnect.
2026-03-07 08:37:27 -05:00
Mark Backman
26631a9c31 Add Settings class attribute alias to all service classes
Add a `Settings` class-level alias on every STT, LLM, TTS, image,
vision, and video service class pointing to its settings dataclass.
This lets developers discover the right settings class via the service
class itself (e.g. `GoogleSTTService.Settings(...)`) without needing
to know or import the separate settings class name.
2026-03-07 08:17:40 -05:00
Mark Backman
fdf9fb6f02 Merge pull request #3946 from pipecat-ai/mb/tts-settings-review
Review TTS settings
2026-03-07 07:48:26 -05:00
Mark Backman
1cfaea2007 Address code review feedback 2026-03-07 07:42:42 -05:00
Paul Kompfner
622d9279cb Use exact service class names in LLMSettings docstrings 2026-03-06 22:35:55 -05:00
Paul Kompfner
6088e6eb52 Make budget_tokens optional in AnthropicThinkingConfig
budget_tokens is required when type is "enabled" and rejected when type is "disabled" (this is validated by the server)
2026-03-06 22:32:14 -05:00
Paul Kompfner
256c8f87b4 Add missing step 3 comment to LLM service init methods
Adds the explicit "no params object" step 3 comment to all
LLM services that skip from step 2 to step 4 in their
settings initialization sequence, matching the pattern
established in services that do have a params object.
2026-03-06 22:32:14 -05:00
Mark Backman
ec93cd1d51 Fix settings update handling in additional STT services 2026-03-06 21:52:45 -05:00
Mark Backman
9c42d27f4d Support runtime language updates in Azure STT
Extract recognizer setup/teardown into _connect/_disconnect so
_update_settings can reconnect when language changes at runtime.
2026-03-06 21:07:03 -05:00
Mark Backman
536f1e178a Fix race condition in Deepgram STT disconnect causing error flood
Clear self._connection before sending close stream so run_stt stops
sending audio immediately during the WebSocket close handshake.
2026-03-06 21:01:31 -05:00
Mark Backman
671e9a6846 TTS service and example updates 2026-03-06 20:53:22 -05:00
Mark Backman
a97a086dbd Fix GeminiLiveLLMService init referencing undefined _params variable
Replace references to undefined `_params` with `self._settings` for
language and VAD config. Add missing `system_instruction` to default
settings to satisfy validate_complete(). Remove redundant line that
read language from the deprecated `params` arg.
2026-03-06 18:41:54 -05:00
Mark Backman
4ed3480e4b Update TTSSettings docstrings with the corresponding class name(s) 2026-03-06 16:40:38 -05:00
Mark Backman
7d41049b35 Review feedback, clarify corresponding class in STTSettings docstrings 2026-03-06 15:17:02 -05:00
Mark Backman
6431ad8e2a Fix service settings init ordering and example bugs
- Speechmatics: move config build after super().__init__ and settings
  delta so turn_detection_mode (e.g. ADAPTIVE) takes effect
- Google STT: fix example passing bare Language enum instead of list
- Google TTS: add missing explicit defaults for all custom settings fields
- Soniox: fix accidental tuple wrapping of STT service in example
- Speechmatics examples: fix system->user role in kick-off messages
- Deepgram Flux: move tag from settings to __init__ (billing metadata)
- ElevenLabs STT: default tag_audio_events to None (use API default)
- Fal STT: simplify language default handling
- Google TTS: rename GoogleStreamTTSSettings to GoogleTTSSettings
2026-03-06 15:17:01 -05:00
Mark Backman
c3794956ef Add deprecation version, fix foundational example double system message 2026-03-06 15:16:58 -05:00
Mark Backman
940da9eeeb Add vad_threshold to AssemblyAISTTSettings
Wire vad_threshold through Settings, default_settings, the deprecated
connection_params path, and _build_ws_url query params.
2026-03-06 15:16:10 -05:00
kompfner
1a1c5668de Merge pull request #3942 from pipecat-ai/pk/aws-nova-sonic-audio-config
Add AudioConfig class to AWSNovaSonicLLMService for non-deprecated au…
2026-03-06 14:58:22 -05:00
Filipi da Silva Fuchter
4b9fc8a30c Merge pull request #3804 from pipecat-ai/filipi/concurrent_audio_contexts
Allowing concurrent audio contexts
2026-03-06 14:49:57 -05:00
Paul Kompfner
9b7a86bb12 Add AudioConfig class to AWSNovaSonicLLMService for non-deprecated audio configuration
The audio fields (sample rates, sample sizes, channel counts) on the deprecated `Params` class had no non-deprecated equivalent. This adds an `AudioConfig` class and `audio_config` init arg so users can specify audio configuration without relying on the deprecated `params` parameter.
2026-03-06 14:39:53 -05:00
filipi87
07abd3d60f Fixed BotStoppedSpeakingFrame emission: now emitted as soon as TTSStoppedFrame is received, with a fallback silence-based timeout increased to reduce false positives 2026-03-06 16:16:11 -03:00
filipi87
88ff7c451b Refactored all 25+ TTS service implementations to use the new push_start_frame=True pattern 2026-03-06 16:15:59 -03:00
filipi87
24430d8d45 Fixing Piper test. 2026-03-06 16:15:26 -03:00