wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'pipecat-ai:main' into telemetry-fix-system-message

Browse Source
This commit is contained in:
Kinshuk Bairagi
2026-03-20 18:36:31 +05:30
committed by GitHub
parent 9cc2644719 08052d8880
commit 7caec9018b
447 changed files with 10044 additions and 3690 deletions
Download Patch File Download Diff File Expand all files Collapse all files

477
CHANGELOG.md
View File

@@ -7,6 +7,483 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
<!-- towncrier release notes start -->
## [0.0.106] - 2026-03-18
### Added
- Added optional `service` field to `ServiceUpdateSettingsFrame` (and its
subclasses `LLMUpdateSettingsFrame`, `TTSUpdateSettingsFrame`,
`STTUpdateSettingsFrame`) to target a specific service instance. When
`service` is set, only the matching service applies the settings; others
forward the frame unchanged. This enables updating a single service when
multiple services of the same type exist in the pipeline.
(PR [#4004](https://github.com/pipecat-ai/pipecat/pull/4004))
- Added `sip_provider` and `room_geo` parameters to `configure()` in the Daily
runner. These convenience parameters let callers specify a SIP provider name
and geographic region directly without manually constructing
`DailyRoomProperties` and `DailyRoomSipParams`.
(PR [#4005](https://github.com/pipecat-ai/pipecat/pull/4005))
- Added `PerplexityLLMAdapter` that automatically transforms conversation
messages to satisfy Perplexity's stricter API constraints (strict role
alternation, no non-initial system messages, last message must be user/tool).
Previously, certain conversation histories could cause Perplexity API errors
that didn't occur with OpenAI (`PerplexityLLMService` subclasses
`OpenAILLMService` since Perplexity uses an OpenAI-compatible API).
(PR [#4009](https://github.com/pipecat-ai/pipecat/pull/4009))
- Added DTMF input event support to the Daily transport. Incoming DTMF tones
are now received via Daily's `on_dtmf_event` callback and pushed into the
pipeline as `InputDTMFFrame`, enabling bots to react to keypad presses from
phone callers.
(PR [#4047](https://github.com/pipecat-ai/pipecat/pull/4047))
- Added `WakePhraseUserTurnStartStrategy` for triggering user turns based on
wake phrases, with support for `single_activation` mode. Deprecates
`WakeCheckFilter`.
(PR [#4064](https://github.com/pipecat-ai/pipecat/pull/4064))
- Added `default_user_turn_start_strategies()` and
`default_user_turn_stop_strategies()` helper functions for composing custom
strategy lists.
(PR [#4064](https://github.com/pipecat-ai/pipecat/pull/4064))
### Changed
- Changed tool result JSON serialization to use `ensure_ascii=False`,
preserving UTF-8 characters instead of escaping them. This reduces context
size and token usage for non-English languages.
(PR [#3457](https://github.com/pipecat-ai/pipecat/pull/3457))
- `OpenAIRealtimeSTTService`'s `noise_reduction` parameter is now part of
`OpenAIRealtimeSTTSettings`, making it runtime-updatable via
`STTUpdateSettingsFrame`. The direct `noise_reduction` init argument is
deprecated as of 0.0.106.
(PR [#3991](https://github.com/pipecat-ai/pipecat/pull/3991))
- Updated `sarvamai` dependency from `0.1.26a2` (alpha) to `0.1.26` (stable
release).
(PR [#3997](https://github.com/pipecat-ai/pipecat/pull/3997))
- `SimliVideoService` now extends `AIService` instead of `FrameProcessor`,
aligning it with the HeyGen and Tavus video services. It supports
`SimliVideoService.Settings(...)` for configuration and uses
`start()`/`stop()`/`cancel()` lifecycle methods. Existing constructor usage
(`api_key`, `face_id`, etc.) remains unchanged.
(PR [#4001](https://github.com/pipecat-ai/pipecat/pull/4001))
- Update `pipecat-ai-small-webrtc-prebuilt` to `2.4.0`.
(PR [#4023](https://github.com/pipecat-ai/pipecat/pull/4023))
- Nova Sonic assistant text transcripts are now delivered in real-time using
speculative text events instead of delayed final text events. Previously,
assistant text only arrived after all audio had finished playing, causing
laggy transcripts in client UIs. Speculative text arrives before each audio
chunk, providing text synchronized with what the bot is saying. This also
simplifies the internal text handling by removing the interruption re-push
hack and assistant text buffer.
(PR [#4042](https://github.com/pipecat-ai/pipecat/pull/4042))
- Updated `daily-python` dependency to 0.25.0.
(PR [#4047](https://github.com/pipecat-ai/pipecat/pull/4047))
- Added `enable_dialout` parameter to `configure()` in `pipecat.runner.daily`
to support dial-out rooms. Also narrowed misleading `Optional` type hints and
deduplicated token expiry calculation.
(PR [#4048](https://github.com/pipecat-ai/pipecat/pull/4048))
- Extended `ProcessFrameResult` to stop strategies, allowing a stop strategy to
short-circuit evaluation of subsequent strategies by returning `STOP`.
(PR [#4064](https://github.com/pipecat-ai/pipecat/pull/4064))
- `GradiumSTTService` now takes both an `encoding` and `sample_rate`
constructor argument which is assmebled in the class to form the
`input_format`. PCM accepts `8000`, `16000`, and `24000` Hz sample rates.
(PR [#4066](https://github.com/pipecat-ai/pipecat/pull/4066))
- Improved `GradiumSTTService` transcription accuracy by reworking how text
fragments are accumulated and finalized. Previously, trailing words could be
dropped when the server's `flushed` response arrived before all text tokens
were delivered. The service now uses a short aggregation delay after flush to
capture trailing tokens, producing complete utterances.
(PR [#4066](https://github.com/pipecat-ai/pipecat/pull/4066))
### Deprecated
- `SimliVideoService.InputParams` is deprecated. Use the direct constructor
parameters `max_session_length`, `max_idle_time`, and `enable_logging`
instead.
(PR [#4001](https://github.com/pipecat-ai/pipecat/pull/4001))
- Deprecated `LocalSmartTurnAnalyzerV2` and `LocalCoreMLSmartTurnAnalyzer`. Use
`LocalSmartTurnAnalyzerV3` instead. Instantiating these analyzers will now
emit a `DeprecationWarning`.
(PR [#4012](https://github.com/pipecat-ai/pipecat/pull/4012))
- Deprecated `WakeCheckFilter` in favor of `WakePhraseUserTurnStartStrategy`.
(PR [#4064](https://github.com/pipecat-ai/pipecat/pull/4064))
### Fixed
- Fixed an issue where the default model for `OpenAILLMService` and
`AzureLLMService` was mistakenly reverted to `gpt-4o`. The defaults are now
restored to `gpt-4.1`.
(PR [#4000](https://github.com/pipecat-ai/pipecat/pull/4000))
- Fixed a race condition where `EndTaskFrame` could cause the pipeline to shut
down before in-flight frames (e.g. LLM function call responses) finished
processing. `EndTaskFrame` and `StopTaskFrame` now flow through the pipeline
as `ControlFrame`s, ensuring all pending work is flushed before shutdown
begins. `CancelTaskFrame` and `InterruptionTaskFrame` remain immediate
(`SystemFrame`).
(PR [#4006](https://github.com/pipecat-ai/pipecat/pull/4006))
- Fixed `ParallelPipeline` dropping or misordering frames during lifecycle
synchronization. Buffered frames are now flushed in the correct order
relative to synchronization frames (`StartFrame` goes first,
`EndFrame`/`CancelFrame` go after), and frames added to the buffer during
flush are also drained.
(PR [#4007](https://github.com/pipecat-ai/pipecat/pull/4007))
- Fixed `TTSService` potentially canceling in-flight audio during shutdown. The
stop sequence now waits for all queued audio contexts to finish processing
before canceling the stop frame task.
(PR [#4007](https://github.com/pipecat-ai/pipecat/pull/4007))
- Fixed `Language` enum values (e.g. `Language.ES`) not being converted to
service-specific codes when passed via
`settings=Service.Settings(language=Language.ES)` at init time. This caused
API errors (e.g. 400 from Rime) because the raw enum was sent instead of the
expected language code (e.g. `"spa"`). Runtime updates via
`UpdateSettingsFrame` were unaffected. The fix centralizes conversion in the
base `TTSService` and `STTService` classes so all services handle this
consistently.
(PR [#4024](https://github.com/pipecat-ai/pipecat/pull/4024))
- Fixed `DeepgramSTTService` ignoring the `base_url` scheme when using `ws://`
or `http://`. Previously these were silently overwritten with `wss://` /
`https://`, breaking air-gapped or private deployments that don't use TLS.
All scheme choices (`wss://`, `https://`, `ws://`, `http://`, or bare
hostname) are now respected.
(PR [#4026](https://github.com/pipecat-ai/pipecat/pull/4026))
- Fixed `LLMSwitcher.register_function()` and `register_direct_function()` not
accepting or forwarding the `timeout_secs` parameter.
(PR [#4037](https://github.com/pipecat-ai/pipecat/pull/4037))
- Fixed empty user transcriptions in Nova Sonic causing spurious interruptions.
Previously, an empty transcription could trigger an interruption of the
assistant's response even though the user hadn't actually spoken.
(PR [#4042](https://github.com/pipecat-ai/pipecat/pull/4042))
- Fixed `SonioxSTTService` and `OpenAIRealtimeSTTService` crash when language
parameters contain plain strings instead of `Language` enum values.
(PR [#4046](https://github.com/pipecat-ai/pipecat/pull/4046))
- Fixed premature user turn stops caused by late transcriptions arriving
between turns. A stale transcript from the previous turn could persist into
the next turn and trigger a stop before the current turn's real transcript
arrived. Stop strategies are now reset at both turn start and turn stop to
prevent state from leaking across turn boundaries.
(PR [#4057](https://github.com/pipecat-ai/pipecat/pull/4057))
- Fixed raw language strings like `"de-DE"` silently failing when passed to
TTS/STT services (e.g. ElevenLabs producing no audio). Raw strings now go
through the same `Language` enum resolution as enum values, so regional codes
like `"de-DE"` are properly converted to service-expected formats like
`"de"`. Unrecognized strings log a warning instead of failing silently.
(PR [#4058](https://github.com/pipecat-ai/pipecat/pull/4058))
- Fixed Deepgram STT list-type settings (`keyterm`, `keywords`, `search`,
`redact`, `replace`) being stringified instead of passed as lists to the SDK,
which caused them to be sent as literal strings (e.g. `"['pipecat']"`) in the
WebSocket query params.
(PR [#4063](https://github.com/pipecat-ai/pipecat/pull/4063))
- Fixed `MinWordsUserTurnStartStrategy` including text below the word threshold
in the output by resetting aggregation when the minimum word count is not
met.
(PR [#4064](https://github.com/pipecat-ai/pipecat/pull/4064))
- Fixed audio overlap and potential dropped TTS content when multiple assistant
turns occur in quick succession. `TTSService` now flushes remaining text
before pausing frame processing on `LLMFullResponseEndFrame`/`EndFrame`,
instead of pausing first.
(PR [#4071](https://github.com/pipecat-ai/pipecat/pull/4071))
### Security
- Bumped PyJWT minimum version from 2.10.1 to 2.12.0 in the `livekit` extra to
address CVE-2026-32597 (GHSA-752w-5fwx-jx9f), where PyJWT <= 2.11.0 accepted
unknown `crit` header extensions.
(PR [#4035](https://github.com/pipecat-ai/pipecat/pull/4035))
## [0.0.105] - 2026-03-10
### Added
- Added concurrent audio context support: `CartesiaTTSService` can now
synthesize the next sentence while the previous one is still playing, by
setting `pause_frame_processing=False` and routing each sentence through its
own audio context queue.
(PR [#3804](https://github.com/pipecat-ai/pipecat/pull/3804))
- Added custom video track support to Daily transport. Use
`video_out_destinations` in `DailyParams` to publish multiple video tracks
simultaneously, mirroring the existing `audio_out_destinations` feature.
(PR [#3831](https://github.com/pipecat-ai/pipecat/pull/3831))
- Added `ServiceSwitcherStrategyFailover` that automatically switches to the
next service when the active service reports a non-fatal error. Recovery
policies can be implemented via the `on_service_switched` event handler.
(PR [#3861](https://github.com/pipecat-ai/pipecat/pull/3861))
- Added optional `timeout_secs` parameter to `register_function()` and
`register_direct_function()` for per-tool function call timeout control,
overriding the global `function_call_timeout_secs` default.
(PR [#3915](https://github.com/pipecat-ai/pipecat/pull/3915))
- Added `cloud-audio-only` recording option to Daily transport's
`enable_recording` property.
(PR [#3916](https://github.com/pipecat-ai/pipecat/pull/3916))
- Wired up `system_instruction` in `BaseOpenAILLMService`,
`AnthropicLLMService`, and `AWSBedrockLLMService` so it works as a default
system prompt, matching the behavior of the Google services. This enables
sharing a single `LLMContext` across multiple LLM services, where each
service provides its own system instruction independently.
```python
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
system_instruction="You are a helpful assistant.",
)
context = LLMContext()
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
context.add_message({"role": "user", "content": "Please introduce yourself."})
await task.queue_frames([LLMRunFrame()])
```
(PR [#3918](https://github.com/pipecat-ai/pipecat/pull/3918))
- Added `vad_threshold` parameter to `AssemblyAIConnectionParams` for
configuring voice activity detection sensitivity in U3 Pro. Aligning this
with external VAD thresholds (e.g., Silero VAD) prevents the "dead zone"
where AssemblyAI transcribes speech that VAD hasn't detected yet.
(PR [#3927](https://github.com/pipecat-ai/pipecat/pull/3927))
- Added `push_empty_transcripts` parameter to `BaseWhisperSTTService` and
`OpenAISTTService` to allow empty transcripts to be pushed downstream as
`TranscriptionFrame` instead of discarding them (the default behavior). This
is intended for situations where VAD fires even though the user did not
speak. In these cases, it is useful to know that nothing was transcribed so
that the agent can resume speaking, instead of waiting longer for a
transcription.
(PR [#3930](https://github.com/pipecat-ai/pipecat/pull/3930))
- LLM services (`BaseOpenAILLMService`, `AnthropicLLMService`,
`AWSBedrockLLMService`) now log a warning when both `system_instruction` and
a system message in the context are set. The constructor's
`system_instruction` takes precedence.
(PR [#3932](https://github.com/pipecat-ai/pipecat/pull/3932))
- Runtime settings updates (via `STTUpdateSettingsFrame`) now work for AWS
Transcribe, Azure, Cartesia, Deepgram, ElevenLabs Realtime, Gradium, and
Soniox STT services. Previously, changing settings at runtime only stored the
new values without reconnecting.
(PR [#3946](https://github.com/pipecat-ai/pipecat/pull/3946))
- Exposed `on_summary_applied` event on `LLMAssistantAggregator`, allowing
users to listen for context summarization events without accessing private
members.
(PR [#3947](https://github.com/pipecat-ai/pipecat/pull/3947))
- Deepgram Flux STT settings (`keyterm`, `eot_threshold`,
`eager_eot_threshold`, `eot_timeout_ms`) can now be updated mid-stream via
`STTUpdateSettingsFrame` without triggering a reconnect. The new values are
sent to Deepgram as a Configure WebSocket message on the existing connection.
(PR [#3953](https://github.com/pipecat-ai/pipecat/pull/3953))
- Added `system_instruction` parameter to `run_inference` across all LLM
services, allowing callers to override the system prompt for one-shot
inference calls. Used by `_generate_summary` to pass the summarization prompt
cleanly.
(PR [#3968](https://github.com/pipecat-ai/pipecat/pull/3968))
### Changed
- Audio context management (previously in `AudioContextTTSService`) is now
built into `TTSService`. All WebSocket providers (`cartesia`, `elevenlabs`,
`asyncai`, `inworld`, `rime`, `gradium`, `resembleai`) now inherit from
`WebsocketTTSService` directly. Word-timestamp baseline is set automatically
on the first audio chunk of each context instead of requiring each provider
to call `start_word_timestamps()` in their receive loop.
(PR [#3804](https://github.com/pipecat-ai/pipecat/pull/3804))
- Daily transport now uses `CustomVideoSource`/`CustomVideoTrack` instead of
`VirtualCameraDevice` for the default camera output, mirroring how audio
already works with `CustomAudioSource`/`CustomAudioTrack`.
(PR [#3831](https://github.com/pipecat-ai/pipecat/pull/3831))
- ⚠️ Updated `DeepgramSTTService` to use `deepgram-sdk` v6. The `LiveOptions`
class was removed from the SDK and is now provided by pipecat directly;
import it from `pipecat.services.deepgram.stt` instead of `deepgram`.
(PR [#3848](https://github.com/pipecat-ai/pipecat/pull/3848))
- `ServiceSwitcherStrategy` base class now provides a `handle_error()` hook for
subclasses to implement error-based switching. `ServiceSwitcher` defaults to
`ServiceSwitcherStrategyManual` and `strategy_type` is now optional.
(PR [#3861](https://github.com/pipecat-ai/pipecat/pull/3861))
- Support for Voice Focus 2.0 models.
- Updated `aic-sdk` to `~=2.1.0` to support Voice Focus 2.0 models.
- Cleaned unused `ParameterFixedError` exception handling in `AICFilter`
parameter setup.
(PR [#3889](https://github.com/pipecat-ai/pipecat/pull/3889))
- `max_context_tokens` and `max_unsummarized_messages` in
`LLMAutoContextSummarizationConfig` (and deprecated
`LLMContextSummarizationConfig`) can now be set to `None` independently to
disable that summarization threshold. At least one must remain set.
(PR [#3914](https://github.com/pipecat-ai/pipecat/pull/3914))
- ⚠️ Removed `formatted_finals` and `word_finalization_max_wait_time` from
`AssemblyAIConnectionParams` as these were v2 API parameters not supported in
v3. Clarified that `format_turns` only applies to Universal-Streaming models;
U3 Pro has automatic formatting built-in.
(PR [#3927](https://github.com/pipecat-ai/pipecat/pull/3927))
- Changed `DeepgramTTSService` to send a Clear message on interruption instead
of disconnecting and reconnecting the WebSocket, allowing the connection to
persist throughout the session.
(PR [#3958](https://github.com/pipecat-ai/pipecat/pull/3958))
- Re-added `enhancement_level` support to `AICFilter` with runtime
`FilterEnableFrame` control, applying `ProcessorParameter.Bypass` and
`ProcessorParameter.EnhancementLevel` together.
(PR [#3961](https://github.com/pipecat-ai/pipecat/pull/3961))
- Updated `daily-python` dependency from `~=0.23.0` to `~=0.24.0`.
(PR [#3970](https://github.com/pipecat-ai/pipecat/pull/3970))
- Updated `FishAudioTTSService` default model from `s1` to `s2-pro`, matching
Fish Audio's latest recommended model for improved quality and speed.
(PR [#3973](https://github.com/pipecat-ai/pipecat/pull/3973))
- `AzureSTTService` `region` parameter is now optional when `private_endpoint`
is provided. A `ValueError` is raised if neither is given, and a warning is
logged if both are provided (`private_endpoint` takes priority).
(PR [#3974](https://github.com/pipecat-ai/pipecat/pull/3974))
### Deprecated
- Deprecated `AudioContextTTSService` and `AudioContextWordTTSService`.
Subclass `WebsocketTTSService` directly instead; audio context management is
now part of the base `TTSService`.
- Deprecated `WordTTSService`, `WebsocketWordTTSService`, and
`InterruptibleWordTTSService`. Word timestamp logic is now always active in
`TTSService` and no longer needs to be opted into via a subclass.
(PR [#3804](https://github.com/pipecat-ai/pipecat/pull/3804))
- Deprecated `pipecat.services.google.llm_vertex`,
`pipecat.services.google.llm_openai`, and
`pipecat.services.google.gemini_live.llm_vertex` modules. Use
`pipecat.services.google.vertex.llm`, `pipecat.services.google.openai.llm`,
and `pipecat.services.google.gemini_live.vertex.llm` instead. The old import
paths still work but will emit a `DeprecationWarning`.
(PR [#3980](https://github.com/pipecat-ai/pipecat/pull/3980))
### Removed
- ⚠️ Removed `supports_word_timestamps` parameter from `TTSService.__init__()`.
Word timestamp logic is now always active. Remove this argument from any
custom subclass `super().__init__()` calls.
(PR [#3804](https://github.com/pipecat-ai/pipecat/pull/3804))
### Fixed
- Fixed `DeepgramSTTService` keepalive ping timeout disconnections. The
deepgram-sdk v6 removed automatic keepalive; pipecat now sends explicit
`KeepAlive` messages every 5 seconds, within the recommended 3–5 second
interval before Deepgram's 10-second inactivity timeout.
(PR [#3848](https://github.com/pipecat-ai/pipecat/pull/3848))
- Fixed `BufferError: Existing exports of data: object cannot be re-sized` in
`AICFilter` caused by holding a `memoryview` on the mutable audio buffer
across async yield points.
(PR [#3889](https://github.com/pipecat-ai/pipecat/pull/3889))
- Fixed TTS context not being appended to the assistant message history when
using `TTSSpeakFrame` with `append_to_context=True` with some TTS providers.
(PR [#3936](https://github.com/pipecat-ai/pipecat/pull/3936))
- Fixed context summarization leaving orphaned tool responses in the kept
context when tool calls were moved to the summarized portion.
(PR [#3937](https://github.com/pipecat-ai/pipecat/pull/3937))
- Fixed turn completion state not resetting at end of LLM responses.
`LLMFullResponseEndFrame` is pushed (not received) by the LLM service, so the
mixin now handles it in `push_frame` instead of `process_frame`.
(PR [#3956](https://github.com/pipecat-ai/pipecat/pull/3956))
- Fixed turn completion instructions being injected as a context system message
instead of using `system_instruction`. This caused warning spam when
`system_instruction` was also set and didn't persist across full context
updates.
(PR [#3957](https://github.com/pipecat-ai/pipecat/pull/3957))
- Fixed `TTSService` audio context queue getting blocked when
`append_to_audio_context()` was called with a `None` context ID, which
prevented subsequent audio from being delivered.
(PR [#3958](https://github.com/pipecat-ai/pipecat/pull/3958))
- Fixed `on_call_state_updated` event handler in LiveKit transport receiving
incorrect number of arguments due to redundant `self` passed to
`_call_event_handler`.
(PR [#3959](https://github.com/pipecat-ai/pipecat/pull/3959))
- Fixed OpenAI Realtime, OpenAI Realtime Beta, and Grok realtime services
treating `conversation_already_has_active_response` as a fatal error. These
services now log it as a non-fatal debug event when a response is already in
progress.
(PR [#3960](https://github.com/pipecat-ai/pipecat/pull/3960))
- Fixed `SmallWebRTCConnection` silently discarding messages sent before the
data channel is open by queuing them and flushing once the channel is ready.
A bounded queue (`MAX_MESSAGE_QUEUE_SIZE = 50`) prevents unbounded memory
growth, and a 10-second timeout after connection clears the queue and falls
back to discard mode if the data channel never opens.
(PR [#3962](https://github.com/pipecat-ai/pipecat/pull/3962))
- Fixed `AzureSTTService` failing to initialize when `private_endpoint` is
provided. The Azure Speech SDK's `SpeechConfig` does not accept both `region`
and `endpoint` simultaneously, so they are now passed conditionally.
(PR [#3967](https://github.com/pipecat-ai/pipecat/pull/3967))
- Fixed `GoogleLLMService` ignoring the `system_instruction` set via
constructor or `GoogleLLMSettings` when a system message was also present in
the context. The settings value now correctly takes priority, and a warning
is logged when both are set.
(PR [#3976](https://github.com/pipecat-ai/pipecat/pull/3976))
### Other
- Updated foundational examples to use `system_instruction` on LLM services
instead of adding system messages to `LLMContext`.
(PR [#3918](https://github.com/pipecat-ai/pipecat/pull/3918))
- Updated AssemblyAI turn detection example to use `keyterms_prompt` list
format instead of `prompt` string for improved clarity.
(PR [#3929](https://github.com/pipecat-ai/pipecat/pull/3929))
- Updated foundational examples and eval scripts to use `"user"` role instead
of `"system"` when adding messages to `LLMContext`, since system prompts
should be set via `system_instruction` on the LLM service.
(PR [#3931](https://github.com/pipecat-ai/pipecat/pull/3931))
## [0.0.104] - 2026-03-02
### Added

6
COMMUNITY_INTEGRATIONS.md
View File

@@ -280,17 +280,17 @@ from typing import Optional
class MyTTSService(TTSService):
Settings = MyTTSSettings
_settings: MyTTSSettings
_settings: Settings
def __init__(
self,
*,
api_key: str,
settings: Optional[MyTTSSettings] = None,
settings: Optional[Settings] = None,
**kwargs,
):
# 1. Defaults — every field has a real value (store mode).
default_settings = MyTTSSettings(
default_settings = self.Settings(
model="my-model-v1",
voice="default-voice",
language="en",

1
changelog/3804.added.md
View File

@@ -1 +0,0 @@
- Added concurrent audio context support: `CartesiaTTSService` can now synthesize the next sentence while the previous one is still playing, by setting `pause_frame_processing=False` and routing each sentence through its own audio context queue.

1
changelog/3804.changed.md
View File

@@ -1 +0,0 @@
- Audio context management (previously in `AudioContextTTSService`) is now built into `TTSService`. All WebSocket providers (`cartesia`, `elevenlabs`, `asyncai`, `inworld`, `rime`, `gradium`, `resembleai`) now inherit from `WebsocketTTSService` directly. Word-timestamp baseline is set automatically on the first audio chunk of each context instead of requiring each provider to call `start_word_timestamps()` in their receive loop.

2
changelog/3804.deprecated.md
View File

@@ -1,2 +0,0 @@
- Deprecated `AudioContextTTSService` and `AudioContextWordTTSService`. Subclass `WebsocketTTSService` directly instead; audio context management is now part of the base `TTSService`.
- Deprecated `WordTTSService`, `WebsocketWordTTSService`, and `InterruptibleWordTTSService`. Word timestamp logic is now always active in `TTSService` and no longer needs to be opted into via a subclass.

1
changelog/3804.removed.md
View File

@@ -1 +0,0 @@
- ⚠️ Removed `supports_word_timestamps` parameter from `TTSService.__init__()`. Word timestamp logic is now always active. Remove this argument from any custom subclass `super().__init__()` calls.

1
changelog/3848.changed.md
View File

@@ -1 +0,0 @@
- ⚠️ Updated `DeepgramSTTService` to use `deepgram-sdk` v6. The `LiveOptions` class was removed from the SDK and is now provided by pipecat directly; import it from `pipecat.services.deepgram.stt` instead of `deepgram`.

1
changelog/3848.fixed.md
View File

@@ -1 +0,0 @@
- Fixed `DeepgramSTTService` keepalive ping timeout disconnections. The deepgram-sdk v6 removed automatic keepalive; pipecat now sends explicit `KeepAlive` messages every 5 seconds, within the recommended 3–5 second interval before Deepgram's 10-second inactivity timeout.

3
changelog/3889.changed.md
View File

@@ -1,3 +0,0 @@
- Support for Voice Focus 2.0 models.
- Updated `aic-sdk` to `~=2.1.0` to support Voice Focus 2.0 models.
- Cleaned unused `ParameterFixedError` exception handling in `AICFilter` parameter setup.

1
changelog/3889.fixed.md
View File

@@ -1 +0,0 @@
- Fixed `BufferError: Existing exports of data: object cannot be re-sized` in `AICFilter` caused by holding a `memoryview` on the mutable audio buffer across async yield points.

1
changelog/3914.changed.md
View File

@@ -1 +0,0 @@
- `max_context_tokens` and `max_unsummarized_messages` in `LLMAutoContextSummarizationConfig` (and deprecated `LLMContextSummarizationConfig`) can now be set to `None` independently to disable that summarization threshold. At least one must remain set.

1
changelog/3915.added.md
View File

@@ -1 +0,0 @@
- Added optional `timeout_secs` parameter to `register_function()` and `register_direct_function()` for per-tool function call timeout control, overriding the global `function_call_timeout_secs` default.

1
changelog/3916.added.md
View File

@@ -1 +0,0 @@
- Added `cloud-audio-only` recording option to Daily transport's `enable_recording` property.

15
changelog/3918.added.md
View File

@@ -1,15 +0,0 @@
- Wired up `system_instruction` in `BaseOpenAILLMService`, `AnthropicLLMService`, and `AWSBedrockLLMService` so it works as a default system prompt, matching the behavior of the Google services. This enables sharing a single `LLMContext` across multiple LLM services, where each service provides its own system instruction independently.
```python
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
system_instruction="You are a helpful assistant.",
)
context = LLMContext()
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
context.add_message({"role": "user", "content": "Please introduce yourself."})
await task.queue_frames([LLMRunFrame()])
```

1
changelog/3918.other.md
View File

@@ -1 +0,0 @@
- Updated foundational examples to use `system_instruction` on LLM services instead of adding system messages to `LLMContext`.

1
changelog/3927.added.md
View File

@@ -1 +0,0 @@
- Added `vad_threshold` parameter to `AssemblyAIConnectionParams` for configuring voice activity detection sensitivity in U3 Pro. Aligning this with external VAD thresholds (e.g., Silero VAD) prevents the "dead zone" where AssemblyAI transcribes speech that VAD hasn't detected yet.

1
changelog/3927.changed.md
View File

@@ -1 +0,0 @@
- ⚠️ Removed `formatted_finals` and `word_finalization_max_wait_time` from `AssemblyAIConnectionParams` as these were v2 API parameters not supported in v3. Clarified that `format_turns` only applies to Universal-Streaming models; U3 Pro has automatic formatting built-in.

1
changelog/3929.other.md
View File

@@ -1 +0,0 @@
- Updated AssemblyAI turn detection example to use `keyterms_prompt` list format instead of `prompt` string for improved clarity.

1
changelog/3930.added.md
View File

@@ -1 +0,0 @@
- Added `push_empty_transcripts` parameter to `BaseWhisperSTTService` and `OpenAISTTService` to allow empty transcripts to be pushed downstream as `TranscriptionFrame` instead of discarding them (the default behavior). This is intended for situations where VAD fires even though the user did not speak. In these cases, it is useful to know that nothing was transcribed so that the agent can resume speaking, instead of waiting longer for a transcription.

1
changelog/3931.other.md
View File

@@ -1 +0,0 @@
- Updated foundational examples and eval scripts to use `"user"` role instead of `"system"` when adding messages to `LLMContext`, since system prompts should be set via `system_instruction` on the LLM service.

1
changelog/3932.added.md
View File

@@ -1 +0,0 @@
- LLM services (`BaseOpenAILLMService`, `AnthropicLLMService`, `AWSBedrockLLMService`) now log a warning when both `system_instruction` and a system message in the context are set. The constructor's `system_instruction` takes precedence.

1
changelog/3936.fixed.md
View File

@@ -1 +0,0 @@
- Fixed TTS context not being appended to the assistant message history when using `TTSSpeakFrame` with `append_to_context=True` with some TTS providers.

1
changelog/3937.fixed.md
View File

@@ -1 +0,0 @@
- Fixed context summarization leaving orphaned tool responses in the kept context when tool calls were moved to the summarized portion.

1
changelog/3946.added.md
View File

@@ -1 +0,0 @@
- Runtime settings updates (via `STTUpdateSettingsFrame`) now work for AWS Transcribe, Azure, Cartesia, Deepgram, ElevenLabs Realtime, Gradium, and Soniox STT services. Previously, changing settings at runtime only stored the new values without reconnecting.

1
changelog/3947.added.md
View File

@@ -1 +0,0 @@
- Exposed `on_summary_applied` event on `LLMAssistantAggregator`, allowing users to listen for context summarization events without accessing private members.

1
changelog/3953.added.md
View File

@@ -1 +0,0 @@
- Deepgram Flux STT settings (`keyterm`, `eot_threshold`, `eager_eot_threshold`, `eot_timeout_ms`) can now be updated mid-stream via `STTUpdateSettingsFrame` without triggering a reconnect. The new values are sent to Deepgram as a Configure WebSocket message on the existing connection.

1
changelog/3956.fixed.md
View File

@@ -1 +0,0 @@
- Fixed turn completion state not resetting at end of LLM responses. `LLMFullResponseEndFrame` is pushed (not received) by the LLM service, so the mixin now handles it in `push_frame` instead of `process_frame`.

1
changelog/3957.fixed.md
View File

@@ -1 +0,0 @@
- Fixed turn completion instructions being injected as a context system message instead of using `system_instruction`. This caused warning spam when `system_instruction` was also set and didn't persist across full context updates.

1
changelog/3959.fixed.md
View File

@@ -1 +0,0 @@
- Fixed `on_call_state_updated` event handler in LiveKit transport receiving incorrect number of arguments due to redundant `self` passed to `_call_event_handler`.

1
changelog/3962.fixed.md
View File

@@ -1 +0,0 @@
- Fixed `SmallWebRTCConnection` silently discarding messages sent before the data channel is open by queuing them and flushing once the channel is ready. A bounded queue (`MAX_MESSAGE_QUEUE_SIZE = 50`) prevents unbounded memory growth, and a 10-second timeout after connection clears the queue and falls back to discard mode if the data channel never opens.

1
changelog/3967.fixed.md
View File

@@ -1 +0,0 @@
- Fixed `AzureSTTService` failing to initialize when `private_endpoint` is provided. The Azure Speech SDK's `SpeechConfig` does not accept both `region` and `endpoint` simultaneously, so they are now passed conditionally.

1
changelog/4029.added.2.md Normal file
View File

@@ -0,0 +1 @@
- Added `frame_order` parameter to `SyncParallelPipeline`. Set `frame_order=FrameOrder.PIPELINE` to push synchronized output frames in pipeline definition order (all frames from the first pipeline, then the second, etc.) instead of the default arrival order.

1
changelog/4029.added.md Normal file
View File

@@ -0,0 +1 @@
- Added `sync_with_audio` field to `OutputImageRawFrame`. When set to `True`, the output transport queues image frames with audio so they are displayed only after all preceding audio has been sent, enabling synchronized audio/image playback.

1
changelog/4029.fixed.3.md Normal file
View File

@@ -0,0 +1 @@
- Fixed `SyncParallelPipeline` breaking the Whisker debugger.

1
changelog/4029.fixed.md Normal file
View File

@@ -0,0 +1 @@
- Fixed `SyncParallelPipeline` race condition where concurrent SystemFrame processing (e.g. from RTVI) could corrupt sink queues and cause deadlocks. SystemFrames now take a fast path that passes them through without draining queued output.

1
changelog/4074.added.md Normal file
View File

@@ -0,0 +1 @@
- Added `OpenAIResponsesLLMService`, a new LLM service that uses the OpenAI Responses API. Supports streaming text, function calling, usage metrics, and out-of-band inference. Works with the universal `LLMContext` and `LLMContextAggregatorPair`. See `examples/foundational/07-interruptible-openai-responses.py` and `14-function-calling-openai-responses.py`.

1
changelog/4075.fixed.md Normal file
View File

@@ -0,0 +1 @@
- Fixed TTS frame ordering so that non-system frames always arrive in correct order relative to the `TTSStartedFrame`/`TTSAudioRawFrame`/`TTSStoppedFrame` sequence. Previously these frames could race ahead of or behind audio context frames, producing out-of-order output downstream.

2
examples/foundational/02-llm-say-one-thing.py
View File

@@ -47,7 +47,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are an LLM in a WebRTC session, and this is a 'hello world' demo.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/04-transports-small-webrtc.py
View File

@@ -75,7 +75,7 @@ async def run_example(webrtc_connection: SmallWebRTCConnection):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

3
examples/foundational/04a-transports-daily.py
View File

@@ -58,8 +58,7 @@ async def main():
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
model="gpt-4o",
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/04b-transports-livekit.py
View File

@@ -58,7 +58,7 @@ async def main():
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

33
examples/foundational/05-sync-speech-and-image.py
View File

@@ -16,11 +16,12 @@ from pipecat.frames.frames import (
Frame,
LLMContextFrame,
LLMFullResponseStartFrame,
OutputImageRawFrame,
TextFrame,
)
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.sync_parallel_pipeline import SyncParallelPipeline
from pipecat.pipeline.sync_parallel_pipeline import FrameOrder, SyncParallelPipeline
from pipecat.pipeline.task import PipelineTask
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.sentence import SentenceAggregator
@@ -30,6 +31,7 @@ from pipecat.runner.utils import create_transport
from pipecat.services.cartesia.tts import CartesiaHttpTTSService
from pipecat.services.fal.image import FalImageGenService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.services.tts_service import TextAggregationMode
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.daily.transport import DailyParams
@@ -44,6 +46,18 @@ class MonthFrame(DataFrame):
return f"{self.name}(month: {self.month})"
class MarkImageForPlaybackSync(FrameProcessor):
"""Marks output image frames to be synchronized with audio playback."""
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, OutputImageRawFrame):
frame.sync_with_audio = True
await self.push_frame(frame, direction)
class MonthPrepender(FrameProcessor):
def __init__(self):
super().__init__()
@@ -101,6 +115,10 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
settings=CartesiaHttpTTSService.Settings(
voice="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
),
# No need to aggregate by sentences (the default), as we already know we're getting full sentences
# (Otherwise the service will unnecessarily wait for follow-up input to confirm the sentence is complete,
# which, sadly, actually breaks the synchronization mechanism)
text_aggregation_mode=TextAggregationMode.TOKEN,
)
imagegen = FalImageGenService(
@@ -119,17 +137,26 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
# that, each pipeline runs concurrently and `SyncParallelPipeline` will
# wait for the input frame to be processed.
#
# We use `FrameOrder.PIPELINE` so that each synchronized batch of output
# frames is pushed in the order the pipelines are listed: image first,
# then audio. This ensures the transport receives the image before the
# audio frames it should accompany.
#
# Note that `SyncParallelPipeline` requires the last processor in each
# of the pipelines to be synchronous. In this case, we use
# `CartesiaHttpTTSService` and `FalImageGenService` which make HTTP
# `FalImageGenService` and `CartesiaHttpTTSService` which make HTTP
# requests and wait for the response.
pipeline = Pipeline(
[
llm, # LLM
sentence_aggregator, # Aggregates LLM output into full sentences
SyncParallelPipeline( # Run pipelines in parallel aggregating the result
[
imagegen, # Generate image
MarkImageForPlaybackSync(), # Mark image as needing sync w/audio during playback
],
[month_prepender, tts], # Create "Month: sentence" and output audio
[imagegen], # Generate image
frame_order=FrameOrder.PIPELINE,
),
transport.output(), # Transport output
]

202
examples/foundational/05a-local-sync-speech-and-image.py
View File

@@ -1,202 +0,0 @@
#
# Copyright (c) 2024-2026, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import tkinter as tk
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from pipecat.frames.frames import (
Frame,
LLMContextFrame,
OutputAudioRawFrame,
TextFrame,
TTSAudioRawFrame,
URLImageRawFrame,
)
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.sync_parallel_pipeline import SyncParallelPipeline
from pipecat.pipeline.task import PipelineTask
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.sentence import SentenceAggregator
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.services.cartesia.tts import CartesiaHttpTTSService
from pipecat.services.fal.image import FalImageGenService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.transports.local.tk import TkLocalTransport, TkTransportParams
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
tk_root = tk.Tk()
tk_root.title("Calendar")
runner = PipelineRunner()
async def get_month_data(month):
messages = [
{
"role": "user",
"content": f"Describe a nature photograph suitable for use in a calendar, for the month of {month}. Include only the image description with no preamble. Limit the description to one sentence, please.",
}
]
class ImageDescription(FrameProcessor):
def __init__(self):
super().__init__()
self.text = ""
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TextFrame):
self.text = frame.text
await self.push_frame(frame, direction)
class AudioGrabber(FrameProcessor):
def __init__(self):
super().__init__()
self.audio = bytearray()
self.frame = None
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TTSAudioRawFrame):
self.audio.extend(frame.audio)
self.frame = OutputAudioRawFrame(
bytes(self.audio), frame.sample_rate, frame.num_channels
)
await self.push_frame(frame, direction)
class ImageGrabber(FrameProcessor):
def __init__(self):
super().__init__()
self.frame = None
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, URLImageRawFrame):
self.frame = frame
await self.push_frame(frame, direction)
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"))
tts = CartesiaHttpTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
settings=CartesiaHttpTTSService.Settings(
voice="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
),
)
imagegen = FalImageGenService(
settings=FalImageGenService.Settings(
image_size="square_hd",
),
aiohttp_session=session,
key=os.getenv("FAL_KEY"),
)
sentence_aggregator = SentenceAggregator()
description = ImageDescription()
audio_grabber = AudioGrabber()
image_grabber = ImageGrabber()
# With `SyncParallelPipeline` we synchronize audio and images by
# pushing them basically in order (e.g. I1 A1 A1 A1 I2 A2 A2 A2 A2
# I3 A3). To do that, each pipeline runs concurrently and
# `SyncParallelPipeline` will wait for the input frame to be
# processed.
#
# Note that `SyncParallelPipeline` requires the last processor in
# each of the pipelines to be synchronous. In this case, we use
# `CartesiaHttpTTSService` and `FalImageGenService` which make HTTP
# requests and wait for the response.
pipeline = Pipeline(
[
llm, # LLM
sentence_aggregator, # Aggregates LLM output into full sentences
description, # Store sentence
SyncParallelPipeline(
[tts, audio_grabber], # Generate and store audio for the given sentence
[imagegen, image_grabber], # Generate and storeimage for the given sentence
),
]
)
task = PipelineTask(pipeline)
await task.queue_frame(LLMContextFrame(LLMContext(messages)))
await task.stop_when_done()
await runner.run(task)
return {
"month": month,
"text": description.text,
"image": image_grabber.frame,
"audio": audio_grabber.frame,
}
transport = TkLocalTransport(
tk_root,
TkTransportParams(
audio_out_enabled=True,
video_out_enabled=True,
video_out_width=1024,
video_out_height=1024,
),
)
pipeline = Pipeline([transport.output()])
task = PipelineTask(pipeline)
# We only specify a few months as we create tasks all at once and we
# might get rate limited otherwise.
months: list[str] = [
"January",
"February",
]
# We create one task per month. This will be executed concurrently.
month_tasks = [asyncio.create_task(get_month_data(month)) for month in months]
# Now we wait for each month task in the order they're completed. The
# benefit is we'll have as little delay as possible before the first
# month, and likely no delay between months, but the months won't
# display in order.
async def show_images(month_tasks):
for month_data_task in asyncio.as_completed(month_tasks):
data = await month_data_task
await task.queue_frames([data["image"], data["audio"]])
await runner.stop_when_done()
async def run_tk():
while not task.has_finished():
tk_root.update()
tk_root.update_idletasks()
await asyncio.sleep(0.1)
await asyncio.gather(runner.run(task), show_images(month_tasks), run_tk())
if __name__ == "__main__":
asyncio.run(main())

2
examples/foundational/06-listen-and-respond.py
View File

@@ -91,7 +91,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/06a-image-sync.py
View File

@@ -108,7 +108,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07-interruptible-cartesia-http.py
View File

@@ -67,7 +67,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

125
examples/foundational/07-interruptible-openai-responses.py Normal file
View File

@@ -0,0 +1,125 @@
#
# Copyright (c) 2024-2026, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import os
from dotenv import load_dotenv
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import (
LLMContextAggregatorPair,
LLMUserAggregatorParams,
)
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.responses.llm import OpenAIResponsesLLMService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.daily.transport import DailyParams
from pipecat.transports.websocket.fastapi import FastAPIWebsocketParams
load_dotenv(override=True)
# We use lambdas to defer transport parameter creation until the transport
# type is selected at runtime.
transport_params = {
"daily": lambda: DailyParams(
audio_in_enabled=True,
audio_out_enabled=True,
),
"twilio": lambda: FastAPIWebsocketParams(
audio_in_enabled=True,
audio_out_enabled=True,
),
"webrtc": lambda: TransportParams(
audio_in_enabled=True,
audio_out_enabled=True,
),
}
async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
logger.info(f"Starting bot")
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
settings=CartesiaTTSService.Settings(
voice="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
),
)
llm = OpenAIResponsesLLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAIResponsesLLMService.Settings(
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)
context = LLMContext()
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
context,
user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
)
pipeline = Pipeline(
[
transport.input(), # Transport user input
stt,
user_aggregator, # User responses
llm, # LLM
tts, # TTS
transport.output(), # Transport bot output
assistant_aggregator, # Assistant spoken responses
]
)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
idle_timeout_secs=runner_args.pipeline_idle_timeout_secs,
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
# Kick off the conversation.
context.add_message(
{"role": "developer", "content": "Please introduce yourself to the user."}
)
await task.queue_frames([LLMRunFrame()])
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
await task.cancel()
runner = PipelineRunner(handle_sigint=runner_args.handle_sigint)
await runner.run(task)
async def bot(runner_args: RunnerArguments):
"""Main bot entry point compatible with Pipecat Cloud."""
transport = await create_transport(runner_args, transport_params)
await run_bot(transport, runner_args)
if __name__ == "__main__":
from pipecat.runner.run import main
main()

2
examples/foundational/07-interruptible.py
View File

@@ -63,7 +63,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07a-interruptible-speechmatics-vad.py
View File

@@ -113,7 +113,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
temperature=0.75,
system_instruction="You are a helpful British assistant called Sarah. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Always include punctuation in your responses. Give very short replies - do not give longer replies unless strictly necessary. Respond to what the user said in a concise, funny, creative and helpful way. Use `<Sn/>` tags to identify different speakers - do not use tags in your replies. Do not respond to speakers within `<PASSIVE/>` tags unless explicitly asked to.",
system_instruction="You are a helpful British assistant called Sarah in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Always include punctuation in your responses. Give very short replies - do not give longer replies unless strictly necessary. Respond to what the user said in a concise, funny, creative and helpful way. Use `<Sn/>` tags to identify different speakers - do not use tags in your replies. Do not respond to speakers within `<PASSIVE/>` tags unless explicitly asked to.",
),
)

2
examples/foundational/07a-interruptible-speechmatics.py
View File

@@ -93,7 +93,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
temperature=0.75,
system_instruction="You are a helpful British assistant called Sarah. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Always include punctuation in your responses. Give very short replies - do not give longer replies unless strictly necessary. Respond to what the user said in a concise, funny, creative and helpful way. Use `<Sn/>` tags to identify different speakers - do not use tags in your replies. Do not respond to speakers within `<PASSIVE/>` tags unless explicitly asked to.",
system_instruction="You are a helpful British assistant called Sarah in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Always include punctuation in your responses. Give very short replies - do not give longer replies unless strictly necessary. Respond to what the user said in a concise, funny, creative and helpful way. Use `<Sn/>` tags to identify different speakers - do not use tags in your replies. Do not respond to speakers within `<PASSIVE/>` tags unless explicitly asked to.",
),
)

3
examples/foundational/07b-interruptible-langchain.py
View File

@@ -80,8 +80,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
[
(
"system",
"Be nice and helpful. Answer very briefly and without special characters like `#` or `*`. "
"Your response will be synthesized to voice and those characters will create unnatural sounds.",
"You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
MessagesPlaceholder("chat_history"),
("human", "{input}"),

2
examples/foundational/07c-interruptible-deepgram-flux.py
View File

@@ -71,7 +71,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07c-interruptible-deepgram-http.py
View File

@@ -68,7 +68,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07c-interruptible-deepgram-sagemaker.py
View File

@@ -79,7 +79,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
settings=AWSBedrockLLMSettings(
model="us.amazon.nova-pro-v1:0",
temperature=0.8,
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07c-interruptible-deepgram-vad.py
View File

@@ -71,7 +71,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07c-interruptible-deepgram.py
View File

@@ -65,7 +65,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07d-interruptible-elevenlabs-http.py
View File

@@ -72,7 +72,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07d-interruptible-elevenlabs.py
View File

@@ -65,7 +65,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07f-interruptible-azure-http.py
View File

@@ -67,7 +67,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
endpoint=os.getenv("AZURE_CHATGPT_ENDPOINT"),
settings=AzureLLMService.Settings(
model=os.getenv("AZURE_CHATGPT_MODEL"),
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07f-interruptible-azure.py
View File

@@ -67,7 +67,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
endpoint=os.getenv("AZURE_CHATGPT_ENDPOINT"),
settings=AzureLLMService.Settings(
model=os.getenv("AZURE_CHATGPT_MODEL"),
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07h-interruptible-openpipe.py
View File

@@ -68,7 +68,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
openpipe_api_key=os.getenv("OPENPIPE_API_KEY"),
tags={"conversation_id": f"pipecat-{timestamp}"},
settings=OpenPipeLLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07i-interruptible-xtts.py
View File

@@ -68,7 +68,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07j-interruptible-gladia-vad.py
View File

@@ -76,7 +76,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY", ""),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07j-interruptible-gladia.py
View File

@@ -74,7 +74,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY", ""),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07k-interruptible-lmnt.py
View File

@@ -64,7 +64,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

4
examples/foundational/07l-interruptible-groq.py
View File

@@ -57,8 +57,8 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = GroqLLMService(
api_key=os.getenv("GROQ_API_KEY"),
settings=GroqLLMService.Settings(
model="meta-llama/llama-4-maverick-17b-128e-instruct",
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
model="llama-3.1-8b-instant",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07m-interruptible-aws.py
View File

@@ -66,7 +66,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
settings=AWSBedrockLLMService.Settings(
model="us.anthropic.claude-sonnet-4-6",
temperature=0.8,
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07n-interruptible-gemini-image.py
View File

@@ -89,7 +89,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
settings=GoogleLLMService.Settings(
model="gemini-2.5-flash-image",
# model="gemini-3-pro-image-preview", # A more powerful model, but slower,
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

4
examples/foundational/07n-interruptible-gemini.py
View File

@@ -74,7 +74,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
api_key=os.getenv("GOOGLE_API_KEY"),
model="gemini-2.5-flash",
settings=GoogleLLMService.Settings(
system_instruction="""You are a helpful AI assistant in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way.
system_instruction="""You are a helpful assistant in a voice conversation.
IMPORTANT: You're using Gemini TTS which supports expressive markup tags. You can use these tags in your responses:
- [sigh] - Insert a sigh sound
@@ -91,7 +91,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
- "[whispering] Let me tell you a secret."
- "The answer is... [long pause] ...42!"
Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.""",
Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Keep responses concise. Respond to what the user said in a creative and helpful way.""",
),
)

4
examples/foundational/07n-interruptible-google-http.py
View File

@@ -73,11 +73,11 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = GoogleLLMService(
api_key=os.getenv("GOOGLE_API_KEY"),
settings=GoogleLLMService.GoogleLLMSettings(
settings=GoogleLLMService.Settings(
model="gemini-2.5-flash",
# force a certain amount of thinking if you want it
# thinking=GoogleLLMService.ThinkingConfig(thinking_budget=4096)
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07n-interruptible-google.py
View File

@@ -77,7 +77,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
model="gemini-2.5-flash",
# force a certain amount of thinking if you want it
# thinking=GoogleLLMService.ThinkingConfig(thinking_budget=4096),
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07o-interruptible-assemblyai-turn-detection.py
View File

@@ -115,7 +115,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07o-interruptible-assemblyai.py
View File

@@ -67,7 +67,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07p-interruptible-krisp-viva.py
View File

@@ -93,7 +93,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07p-interruptible-krisp.py
View File

@@ -68,7 +68,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07q-interruptible-rime-http.py
View File

@@ -71,7 +71,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07q-interruptible-rime.py
View File

@@ -64,7 +64,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07r-interruptible-nvidia.py
View File

@@ -58,7 +58,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
api_key=os.getenv("NVIDIA_API_KEY"),
settings=NvidiaLLMService.Settings(
model="meta/llama-3.3-70b-instruct",
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07s-interruptible-google-audio-in.py
View File

@@ -48,7 +48,7 @@ load_dotenv(override=True)
marker = "|----|"
system_message = f"""
You are a helpful LLM in a WebRTC call. Your goals are to be helpful and brief in your responses.
You are a helpful LLM in a voice call. Your goals are to be helpful and brief in your responses.
You are expert at transcribing audio to text. You will receive a mixture of audio and text input. When
asked to transcribe what the user said, output an exact, word-for-word transcription.

2
examples/foundational/07t-interruptible-fish.py
View File

@@ -65,7 +65,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07v-interruptible-neuphonic-http.py
View File

@@ -69,7 +69,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07v-interruptible-neuphonic.py
View File

@@ -64,7 +64,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07w-interruptible-fal.py
View File

@@ -70,7 +70,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07x-interruptible-local.py
View File

@@ -52,7 +52,7 @@ async def main():
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07y-interruptible-minimax.py
View File

@@ -71,7 +71,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07z-interruptible-sarvam-http.py
View File

@@ -75,7 +75,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07z-interruptible-sarvam.py
View File

@@ -69,7 +69,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07za-interruptible-soniox.py
View File

@@ -71,7 +71,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07zc-interruptible-asyncai-http.py
View File

@@ -69,7 +69,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07zc-interruptible-asyncai.py
View File

@@ -65,7 +65,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07zd-interruptible-aicoustics.py
View File

@@ -85,7 +85,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07ze-interruptible-hume.py
View File

@@ -67,7 +67,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07zf-interruptible-gradium.py
View File

@@ -71,7 +71,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07zg-interruptible-camb.py
View File

@@ -64,7 +64,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful voice assistant powered by Camb AI text-to-speech. ",
system_instruction="You are a helpful voice assistant powered by Camb AI text-to-speech. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Keep responses concise.",
),
)

2
examples/foundational/07zi-interruptible-piper.py
View File

@@ -63,7 +63,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/07zj-interruptible-kokoro.py
View File

@@ -63,7 +63,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

10
examples/foundational/07zk-interruptible-resemble.py
View File

@@ -30,24 +30,20 @@ from pipecat.transports.websocket.fastapi import FastAPIWebsocketParams
load_dotenv(override=True)
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
# We use lambdas to defer transport parameter creation until the transport
# type is selected at runtime.
transport_params = {
"daily": lambda: DailyParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
"twilio": lambda: FastAPIWebsocketParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
"webrtc": lambda: TransportParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
}
@@ -67,7 +63,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

2
examples/foundational/08-custom-frame-processor.py
View File

@@ -103,7 +103,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

41
examples/foundational/10-wake-phrase.py
View File

@@ -19,7 +19,6 @@ from pipecat.processors.aggregators.llm_response_universal import (
LLMContextAggregatorPair,
LLMUserAggregatorParams,
)
from pipecat.processors.filters.wake_check_filter import WakeCheckFilter
from pipecat.runner.types import RunnerArguments
from pipecat.runner.utils import create_transport
from pipecat.services.cartesia.tts import CartesiaTTSService
@@ -28,6 +27,11 @@ from pipecat.services.openai.llm import OpenAILLMService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.daily.transport import DailyParams
from pipecat.transports.websocket.fastapi import FastAPIWebsocketParams
from pipecat.turns.user_start import WakePhraseUserTurnStartStrategy
from pipecat.turns.user_turn_strategies import (
UserTurnStrategies,
default_user_turn_start_strategies,
)
load_dotenv(override=True)
@@ -52,7 +56,12 @@ transport_params = {
async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
logger.info(f"Starting bot")
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
stt = DeepgramSTTService(
api_key=os.getenv("DEEPGRAM_API_KEY"),
settings=DeepgramSTTService.Settings(
keyterm=["pipecat"],
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
@@ -64,23 +73,32 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful assistant. Respond to what the user said in a creative and helpful way. Keep your responses brief.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)
hey_robot_filter = WakeCheckFilter(["hey robot", "hey, robot"])
context = LLMContext()
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
context,
user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
user_params=LLMUserAggregatorParams(
user_turn_strategies=UserTurnStrategies(
start=[
WakePhraseUserTurnStartStrategy(
phrases=["pipecat"],
# Timeout before wake phrase must be spoken again
timeout=5.0,
),
*default_user_turn_start_strategies(),
]
),
vad_analyzer=SileroVADAnalyzer(),
),
)
pipeline = Pipeline(
[
transport.input(), # Transport user input
stt, # STT
hey_robot_filter, # Filter out speech not directed at the robot
stt,
user_aggregator, # User responses
llm, # LLM
tts, # TTS
@@ -102,12 +120,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
async def on_client_connected(transport, client):
logger.info(f"Client connected")
# Kick off the conversation.
context.add_message(
{
"role": "user",
"content": "Please introduce yourself. Tell the user they should say 'Hey Robot' before talking to you.",
}
)
context.add_message({"role": "user", "content": "Please introduce yourself to the user."})
await task.queue_frames([LLMRunFrame()])
@transport.event_handler("on_client_disconnected")

2
examples/foundational/11-sound-effects.py
View File

@@ -107,7 +107,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
settings=OpenAILLMService.Settings(
system_instruction="You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way.",
system_instruction="You are a helpful assistant in a voice conversation. Your responses will be spoken aloud, so avoid emojis, bullet points, or other formatting that can't be spoken. Respond to what the user said in a creative, helpful, and brief way.",
),
)

Some files were not shown because too many files have changed in this diff Show More