Commit Graph

4402 Commits

Author SHA1 Message Date
Chad Bailey
794811fbdb Updated WSS endpoint for Rime Arcana v3 support 2026-02-12 13:24:29 -05:00
Mark Backman
d99a256715 Merge pull request #3706 from ianbbqzy/ian/inworld-user-agent
[Inworld] add User-Agent and X-Request-Id for better traceability
2026-02-11 19:38:26 -05:00
Ian Lee
dcbcab1542 [Inworld] add User-Agent and X-Request-Id for better traceability 2026-02-11 15:47:20 -08:00
filipi87
ec2b38dc29 Fixing smallwebrtc transport input audio resampling logic. 2026-02-11 13:01:25 -03:00
Aleix Conchillo Flaqué
18aad05a7c fix(openai): use compatible stream closing for non-OpenAI providers
OpenAI's AsyncStream uses close() while async generators (e.g. from
OpenPipe) use aclose(). Replace direct async-with on the stream with a
helper that handles both protocols.
2026-02-10 17:59:21 -08:00
Mark Backman
17ab9c425f Merge pull request #3675 from pipecat-ai/mb/elevenlabs-realtime-send-silence
Add silence-based keepalive to WebsocketSTTService
2026-02-10 18:03:38 -05:00
Mark Backman
2f5e61ac55 Add silence-based keepalive to WebsocketSTTService
Adds opt-in keepalive_timeout and keepalive_interval params to
WebsocketSTTService. When enabled, a background task sends silent audio
(or a service-specific protocol message) when the connection has been
idle, preventing server-side timeout disconnects.

Subclasses override _send_keepalive(silence) to wrap the silence in
their wire format. The default sends raw PCM bytes.

Enables keepalive for ElevenLabs (10s), Gladia (20s), and Soniox (1s),
replacing their per-service custom keepalive tasks.
2026-02-10 17:58:47 -05:00
filipi87
314d074c61 Context summarization feature implementation. 2026-02-10 18:58:12 -03:00
Filipi da Silva Fuchter
9c627e7292 Merge pull request #3653 from pipecat-ai/filipi/heygen_lite
HeyGen improvements.
2026-02-10 12:12:22 -05:00
Filipi da Silva Fuchter
ad179b0852 Merge pull request #3584 from pipecat-ai/filipi/speak_frame
TTS services improvements.
2026-02-10 12:11:47 -05:00
filipi87
24f90715e3 Use LITE as the default mode, and add support for video_settings and is_sandbox in LiveAvatarNewSessionRequest. 2026-02-10 14:02:09 -03:00
filipi87
a47d7f98ee Refactored all 30+ TTS service implementations to support context tracking 2026-02-10 11:28:08 -03:00
filipi87
19cd242261 Added TTS context tracking system to trace audio generation through the pipeline. 2026-02-10 11:27:58 -03:00
filipi87
9bb712a47b Simplified universal context aggregators, _handle_text() to only check frame.append_to_context instead of also checking self._started 2026-02-10 11:27:30 -03:00
filipi87
1dccbe7c0b Simplified context aggregators, _handle_text() to only check frame.append_to_context instead of also checking self._started 2026-02-10 11:27:13 -03:00
Mark Backman
2dd3e2f1e7 Merge pull request #3697 from pipecat-ai/mb/soniox-rt-4
Update SonioxSTTService default model to stt-rt-v4
2026-02-10 09:24:39 -05:00
filipi87
f206aaa28d - Added context_id field to all TTS-related frames (TTSAudioRawFrame, TTSStartedFrame, TTSStoppedFrame, AggregatedTextFrame, TTSTextFrame)
- Added append_to_context parameter to TTSSpeakFrame for conditional LLM context addition
2026-02-10 11:22:26 -03:00
Mark Backman
88e981c013 Set vad_force_turn_endpoint to False in SonioxSTTService 2026-02-10 09:16:03 -05:00
Ashot
28e8b61eb4 chore: update Async API URL and default model 2026-02-10 15:23:51 +04:00
Mark Backman
d47d95e1f0 Update SonioxSTTService default model to stt-rt-v4 2026-02-09 23:48:08 -05:00
Mark Backman
79b9d929c5 Merge pull request #3682 from eoinoreilly30/patch-1
Add new voice options 'marin' and 'cedar'
2026-02-09 23:47:39 -05:00
Eoin
f3c1cd4cd6 Lint 2026-02-10 12:31:26 +09:00
Eoin
18d91d6df3 Add new voice options 'marin' and 'cedar' 2026-02-10 12:31:26 +09:00
Mark Backman
7684a94c33 AssemblyAISTTService: Disable turn detection when setting vad_force_turn_endpoint to True 2026-02-09 22:20:35 -05:00
Aleix Conchillo Flaqué
d32c4b2f5f Merge pull request #3693 from pipecat-ai/aleix/update-examples-remove-default-turn-analyzer
remove the now default turn analyzer from examples
2026-02-09 16:04:19 -08:00
Mark Backman
77a5d16a10 Merge pull request #3692 from pipecat-ai/mb/request-metadata-updates
Rename RequestMetadataFrame to ServiceSwitcherRequestMetadataFrame with service targeting
2026-02-09 18:19:29 -05:00
Aleix Conchillo Flaqué
3867bc6302 LLMUserAggregator: update turn analyzer warning 2026-02-09 14:33:38 -08:00
mattie ruth backman
f2688deb0d Update args field in RTVILLMFunctionCallInProgressMessageData to match API of existing RTVILLMFunctionCallResultData. 2026-02-09 17:17:01 -05:00
Mark Backman
981253c703 Rename RequestMetadataFrame to ServiceSwitcherRequestMetadataFrame with service targeting
Add a `service` field so the frame targets a specific service, allowing
ServiceSwitcher.push_frame to consume it only when the targeted service
matches the active service. STTService and test mocks now push the frame
downstream after handling instead of silently consuming it.
2026-02-09 16:48:34 -05:00
Mark Backman
6305e04569 Clean up on Sarvam STT and TTS classes 2026-02-09 15:53:05 -05:00
Mark Backman
3ff9b7b5ad Merge pull request #3687 from pipecat-ai/mb/rtvi-mute-events
Emit RTVI events for user mute/unmute
2026-02-09 15:18:28 -05:00
Aleix Conchillo Flaqué
631463e573 Use TurnAnalyzerUserTurnStopStrategy as default stop strategy
Change the default user turn stop strategy from
TranscriptionUserTurnStopStrategy to TurnAnalyzerUserTurnStopStrategy
with LocalSmartTurnAnalyzerV3. Also reduce AUDIO_INPUT_TIMEOUT_SECS
from 1.0 to 0.5 and remove its debug log.
2026-02-09 11:58:09 -08:00
Mark Backman
00ec6c77ea Emit RTVI events for user mute/unmute state changes
Add UserMuteStartedFrame/UserMuteStoppedFrame and corresponding RTVI
messages so clients can observe when mute strategies activate/deactivate.
2026-02-09 14:44:32 -05:00
Aleix Conchillo Flaqué
2a572aedba Simplify ServiceSwitcher with closure-based filters
- Make ServiceSwitcherStrategy inherit from BaseObject with properties
  for services and active_service, and move initial service selection
  into the base class
- Add on_service_switched event to ServiceSwitcherStrategy
- handle_frame now returns the switched-to service (or None), allowing
  ServiceSwitcher to swallow ManuallySwitchServiceFrame on switch and
  request metadata from the new active service
- Override push_frame to suppress RequestMetadataFrame and
  ServiceMetadataFrame from inactive services
- Remove ServiceSwitcherFilter and ServiceSwitcherFilterFrame in favor
  of plain FunctionFilter instances with closures that check the
  strategy's active service directly
- FunctionFilter: add FilterType alias
- FunctionFilter: when direction is None, frames in both directions
  are filtered instead of just one
- Add docstrings to ServiceSwitcher and its components
2026-02-09 14:12:33 -05:00
Mark Backman
5e66702cf5 Improved the accuracy of the UserBotLatencyObserver and UserBotLatencyLogObserver 2026-02-09 14:12:33 -05:00
Mark Backman
34b068d657 Improve user turn stop timing by triggering timeout from VAD stop
Refactor TranscriptionUserTurnStopStrategy and TurnAnalyzerUserTurnStopStrategy
to use VADUserStoppedSpeakingFrame as the ground truth for when speech ended,
rather than triggering timeouts from transcription frames.
2026-02-09 14:12:33 -05:00
Mark Backman
05e2a013b3 Merge pull request #3672 from pipecat-ai/mb/rtvi-duplicate-events
Filter RTVIObserver to downstream frames only and broadcast FunctionCallCancelFrame
2026-02-09 12:58:28 -05:00
Mark Backman
5f64dae0cf Filter RTVIObserver to downstream frames only and broadcast FunctionCallCancelFrame
RTVIObserver now skips upstream frames to prevent duplicate RTVI messages
when frames are broadcast in both directions. Also changed
FunctionCallCancelFrame to use broadcast_frame for consistency with
other function call frames.
2026-02-09 12:39:25 -05:00
Mark Backman
1bf8b54502 Merge pull request #3683 from dhruvladia-sarvam/sarvam-v3-update 2026-02-09 06:49:59 -05:00
dhruvladia-sarvam
947ff03c9f v3 addition 2026-02-09 13:04:45 +05:30
Mark Backman
90ad2a4e81 Remove SequentialMergePipeline 2026-02-08 14:44:48 -05:00
Mark Backman
570f2d7fc0 Merge pull request #3667 from ianbbqzy/ian/fix-auto-mode-space
[inworld] aggregate_sentence mode needs trailing space
2026-02-07 18:22:32 -05:00
Ian Lee
f3d99adf8f [inworld] aggregate_sentence mode needs trailing space 2026-02-07 15:18:24 -08:00
Mark Backman
d34f416281 Merge pull request #3598 from dhruvladia-sarvam/sarvam-v3-update
ASR and TTS v3 update
2026-02-07 10:51:35 -05:00
Mark Backman
5a1deb7cb4 Merge pull request #3659 from pipecat-ai/mb/change-vad-defaults
Set VADParams stop_secs to 0.2 by default
2026-02-06 23:51:50 -05:00
Mark Backman
a5fc2b1650 Set VADParams stop_secs to 0.2 by default 2026-02-06 23:49:08 -05:00
Aleix Conchillo Flaqué
ce690848c0 Merge pull request #3616 from omChauhanDev/fix/function-call-timeout-task-cleanup
fix: ensure function call timeout task is always cancelled
2026-02-06 16:40:56 -08:00
Aleix Conchillo Flaqué
4945cfbd8f Buffer internal frames during ParallelPipeline lifecycle synchronization
Processors inside parallel sub-pipelines can push frames during
StartFrame/EndFrame/CancelFrame processing. Previously these frames
could escape the ParallelPipeline before all branches finished
processing the lifecycle frame. Now they are buffered and flushed
after synchronization completes.
2026-02-06 15:15:46 -08:00
Mark Backman
8d37d3bae7 Merge pull request #3666 from pipecat-ai/mb/deepgram-stt-smart-format
DeepgramSTTService: disable smart_format by default
2026-02-06 14:04:37 -05:00
Mark Backman
d7b1624d3c Merge pull request #3663 from lukepayyapilli/fix/stream-close-sambanova-google
fix: close stream on cancellation for SambaNova and Google OpenAI services
2026-02-06 14:02:31 -05:00