Compare commits

..

1 Commits

Author SHA1 Message Date
Mark Backman
c522a1ad1a Fix OTel examples to use new runner 2025-05-29 00:59:45 -04:00
588 changed files with 7275 additions and 37705 deletions

View File

@@ -6,13 +6,11 @@ on:
- main
paths:
- "examples/simple-chatbot/client/android/**"
- "examples/p2p-webrtc/video-transform/client/android/**"
pull_request:
branches:
- "**"
paths:
- "examples/simple-chatbot/client/android/**"
- "examples/p2p-webrtc/video-transform/client/android/**"
workflow_dispatch:
inputs:
sdk_git_ref:
@@ -25,7 +23,7 @@ concurrency:
jobs:
sdk:
name: "Demo apps"
name: "Simple chatbot demo"
runs-on: ubuntu-latest
steps:
- name: Checkout repo
@@ -39,22 +37,12 @@ jobs:
distribution: 'temurin'
java-version: '17'
- name: "Example app: Simple Chatbot"
- name: Build demo app
working-directory: examples/simple-chatbot/client/android
run: ./gradlew :simple-chatbot-client:assembleDebug
- name: Upload Simple Chatbot APK
- name: Upload demo APK
uses: actions/upload-artifact@v4
with:
name: Simple Chatbot Android Client
path: examples/simple-chatbot/client/android/simple-chatbot-client/build/outputs/apk/debug/simple-chatbot-client-debug.apk
- name: "Example app: Small WebRTC Client"
working-directory: examples/p2p-webrtc/video-transform/client/android
run: ./gradlew :small-webrtc-client:assembleDebug
- name: Upload Small WebRTC APK
uses: actions/upload-artifact@v4
with:
name: Small WebRTC Android Client
path: examples/p2p-webrtc/video-transform/client/android/small-webrtc-client/build/outputs/apk/debug/small-webrtc-client-debug.apk

View File

@@ -17,7 +17,7 @@ concurrency:
jobs:
ruff-format:
name: "Code quality checks"
name: "Formatting checker"
runs-on: ubuntu-latest
steps:
- name: Checkout repo
@@ -39,8 +39,8 @@ jobs:
run: |
source .venv/bin/activate
ruff format --diff
- name: Ruff linter (all rules)
- name: Ruff import linter
id: ruff-check
run: |
source .venv/bin/activate
ruff check
ruff check --select I

View File

@@ -9,382 +9,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Added `run_llm` field to `LLMMessagesAppendFrame` and `LLMMessagesUpdateFrame`
frames. If true, a context frame will be pushed triggering the LLM to respond.
- Added a new `SOXRStreamAudioResampler` for processing audio in chunks or
streams. If you write your own processor and need to use an audio resampler,
use the new `create_stream_resampler()`.
- Added new `DailyParams.audio_in_user_tracks` to allow receiving one track per
user (default) or a single track from the room (all participants mixed).
- Added support for providing "direct" functions, which don't need an
accompanying `FunctionSchema` or function definition dict. Instead, metadata
(i.e. `name`, `description`, `properties`, and `required`) are automatically
extracted from a combination of the function signature and docstring.
Usage:
```python
# "Direct" function
# `params` must be the first parameter
async def do_something(params: FunctionCallParams, foo: int, bar: str = ""):
"""
Do something interesting.
Args:
foo (int): The foo to do something interesting with.
bar (string): The bar to do something interesting with.
"""
result = await process(foo, bar)
await params.result_callback({"result": result})
# ...
llm.register_direct_function(do_something)
# ...
tools = ToolsSchema(standard_tools=[do_something])
```
- `user_id` is now populated in the `TranscriptionFrame` and
`InterimTranscriptionFrame` when using a transport that provides a
`user_id`, like `DailyTransport` or `LiveKitTransport`.
- Added `watchdog_coroutine()`. This is a watchdog helper for couroutines. So,
if you have a coroutine that is waiting for a result and that takes a long
time, you will need to wrap it with `watchdog_coroutine()` so the watchdog
timers are reset regularly.
- Added `session_token` parameter to `AWSNovaSonicLLMService`.
### Changed
- Updated all the services to use the new `SOXRStreamAudioResampler`, ensuring smooth
transitions and eliminating clicks.
- Upgraded `daily-python` to 0.19.4.
- Updated `google` optional dependency to use `google-genai` version `1.24.0`.
### Fixed
- Fixed an issue where audio would get stuck in the queue when an interrupt occurs
during Azure TTS synthesis.
- Fixed a race condition that occurs in Python 3.10+ where the task could miss
the `CancelledError` and continue running indefinitely, freezing the pipeline.
- Fixed a `AWSNovaSonicLLMService` issue introduced in 0.0.72.
## [0.0.73] - 2025-06-26
### Fixed
- Fixed an issue introduced in 0.0.72 that would cause `ElevenLabsTTSService`,
`GladiaSTTService`, `NeuphonicTTSService` and `OpenAIRealtimeBetaLLMService`
to throw an error.
## [0.0.72] - 2025-06-26
### Added
- Added logging and improved error handling to help diagnose and prevent potential
Pipeline freezes.
- Added `WatchdogQueue`, `WatchdogPriorityQueue`, `WatchdogEvent` and
`WatchdogAsyncIterator`. These helper utilities reset watchdog timers
appropriately before they expire. When watchdog timers are disabled, the
utilities behave as standard counterparts without side effects.
- Introduce task watchdog timers. Watchdog timers are used to detect if a
Pipecat task is taking longer than expected (by default 5 seconds). Watchdog
timers are disabled by default and can be enabled globally by passing
`enable_watchdog_timers` argument to `PipelineTask` constructor. It is
possible to change the default watchdog timer timeout by using the
`watchdog_timeout` argument. You can also log how long it takes to reset the
watchdog timers which is done with the `enable_watchdog_logging`. You can
control all these settings per each frame processor or even per task. That is,
you can set `enable_watchdog_timers`, `enable_watchdog_logging` and
`watchdog_timeout` when creating any frame processor through their constructor
arguments or when you create a task with `FrameProcessor.create_task()`. Note
that watchdog timers only work with Pipecat tasks and will not work if you use
`asycio.create_task()` or similar.
- Added `lexicon_names` parameter to `AWSPollyTTSService.InputParams`.
- Added reconnection logic and audio buffer management to `GladiaSTTService`.
- The `TurnTrackingObserver` now ends a turn upon observing an `EndFrame` or
`CancelFrame`.
- Added Polish support to `AWSTranscribeSTTService`.
- Added new frames `FrameProcessorPauseFrame` and `FrameProcessorResumeFrame`
which allow pausing and resuming frame processing for a given frame
processor. These are control frames, so they are ordered. Pausing frame
processor will keep old frames in the internal queues until resume takes
place. Frames being pushed while a frame processor is paused will be pushed to
the queues. When frame processing is resumed all queued frames will be
processed in order. Also added `FrameProcessorPauseUrgentFrame` and
`FrameProcessorResumeUrgentFrame` which are system frames and therefore they
have high priority.
- Added a property called `has_function_calls_in_progress` in
`LLMAssistantContextAggregator` that exposes whether a function call is in
progress.
- Added `SambaNovaLLMService` which provides llm api integration with an
OpenAI-compatible interface.
- Added `SambaNovaTTSService` which provides speech-to-text functionality using
SambaNovas's (whisper) API.
- Add fundational examples for function calling and transcription
`14s-function-calling-sambanova.py`, `13g-sambanova-transcription.py`
### Changed
- `HeartbeatFrame`s are now control frames. This will make it easier to detect
pipeline freezes. Previously, heartbeat frames were system frames which meant
they were not get queued with other frames, making it difficult to detect
pipeline stalls.
- Updated `OpenAIRealtimeBetaLLMService` to accept `language` in the
`InputAudioTranscription` class for all models.
- Updated the default model for `OpenAIRealtimeBetaLLMService` to
`gpt-4o-realtime-preview-2025-06-03`.
- The `PipelineParams` arg `allow_interruptions` now defaults to `True`.
- `TavusTransport` and `TavusVideoService` now send audio to Tavus using WebRTC
audio tracks instead of `app-messages` over WebSocket. This should improve the
overall audio quality.
- Upgraded `daily-python` to 0.19.3.
### Fixed
- Fixed an issue that would cause heartbeat frames to be sent before processors
were started.
- Fixed an event loop blocking issue when using `SentryMetrics`.
- Fixed an issue in `FastAPIWebsocketClient` to ensure proper disconnection
when the websocket is already closed.
- Fixed an issue where the `UserStoppedSpeakingFrame` was not received if the
transport was not receiving new audio frames.
- Fixed an edge case where if the user interrupted the bot but no new aggregation
was received, the bot would not resume speaking.
- Fixed an issue with `TelnyxFrameSerializer` where it would throw an exception
when the user hung up the call.
- Fixed an issue with `ElevenLabsTTSService` where the context was not being
closed.
- Fixed function calling in `AWSNovaSonicLLMService`.
- Fixed an issue that would cause multiple `PipelineTask.on_idle_timeout`
events to be triggered repeatedly.
- Fixed an issue that was causing user and bot speech to not be synchronized
during recordings.
- Fixed an issue where voice settings weren't applied to ElevenLabsTTSService.
- Fixed an issue with `GroqTTSService` where it was not properly parsing the
WAV file header.
- Fixed an issue with `GoogleSTTService` where it was constantly reconnecting
before starting to receive audio from the user.
- Fixed an issue where `GoogleLLMService`'s TTFB value was incorrect.
### Deprecated
- `AudioBufferProcessor` parameter `user_continuos_stream` is deprecated.
### Other
- Rename `14e-function-calling-gemini.py` to `14e-function-calling-google.py`.
## [0.0.71] - 2025-06-10
### Added
- Adds a parameter called `additional_span_attributes` to PipelineTask that
lets you add any additional attributes you'd like to the conversation span.
### Fixed
- Fixed an issue with `CartesiaSTTService` initialization.
## [0.0.70] - 2025-06-10
### Added
- Added `ExotelFrameSerializer` to handle telephony calls via Exotel.
- Added the option `informal` to `TranslationConfig` on Gladia config.
Allowing to force informal language forms when available.
- Added `CartesiaSTTService` which is a websocket based implementation to
transcribe audio. Added a foundational example in
`13f-cartesia-transcription.py`
- Added an `websocket` example, showing how to use the new Pipecat client
`WebsocketTransport` to connect with Pipecat `FastAPIWebsocketTransport` or
`WebsocketServerTransport`.
- Added language support to `RimeHttpTTSService`. Extended languages to include
German and French for both `RimeTTSService` and `RimeHttpTTSService`.
### Changed
- Upgraded `daily-python` to 0.19.2.
- Make `PipelineTask.add_observer()` synchronous. This allows callers to call it
before doing the work of running the `PipelineTask` (i.e. without invoking
`PipelineTask.set_event_loop()` first).
- Pipecat 0.0.69 forced `uvloop` event loop on Linux on macOS. Unfortunately,
this is causing issue in some systems. So, `uvloop` is not enabled by default
anymore. If you want to use `uvloop` you can just set the `asyncio` event
policy before starting your agent with:
```python
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
```
### Fixed
- Fixed an issue with various TTS services that would cause audio glitches at
the start of every bot turn.
- Fixed an `ElevenLabsTTSService` issue where a context warning was printed
when pushing a `TTSSpeakFrame`.
- Fixed an `AssemblyAISTTService` issue that could cause unexpected behavior
when yielding empty `Frame()`s.
- Fixed an issue where `OutputAudioRawFrame.transport_destination` was being
reset to `None` instead of retaining its intended value before sending the
audio frame to `write_audio_frame`.
- Fixed a typo in Livekit transport that prevented initialization.
## [0.0.69] - 2025-06-02 "AI Engineer World's Fair release" ✨
### Added
- Added a new frame `FunctionCallsStartedFrame`. This frame is pushed both
upstream and downstream from the LLM service to indicate that one or more
function calls are going to be executed.
- Added LLM services `on_function_calls_started` event. This event will be
triggered when the LLM service receives function calls from the model and is
going to start executing them.
- Function calls can now be executed sequentially (in the order received in the
completion) by passing `run_in_parallel=False` when creating your LLM
service. By default, if the LLM completion returns 2 or more function calls
they run concurrently. In both cases, concurrently and sequentially, a new LLM
completion will run when the last function call finishes.
- Added OpenTelemetry tracing for `GeminiMultimodalLiveLLMService` and
`OpenAIRealtimeBetaLLMService`.
- Added initial support for interruption strategies, which determine if the user
should interrupt the bot while the bot is speaking. Interruption strategies
can be based on factors such as audio volume or the number of words spoken by
the user. These can be specified via the new `interruption_strategies` field
in `PipelineParams`. A new `MinWordsInterruptionStrategy` strategy has been
introduced which triggers an interruption if the user has spoken a minimum
number of words. If no interruption strategies are specified, the normal
interruption behavior applies. If multiple strategies are provided, the first
one that evaluates to true will trigger the interruption.
- `BaseInputTransport` now handles `StopFrame`. When a `StopFrame` is received
the transport will pause sending frames downstream until a new `StartFrame` is
received. This allows the transport to be reused (keeping the same connection)
in a different pipeline.
- Updated AssemblyAI STT service to support their latest streaming
speech-to-text model with improved transcription latency and endpointing.
- You can now access STT service results through the new
`TranscriptionFrame.result` and `InterimTranscriptionFrame.result` field. This
is useful in case you use some specific settings for the STT and you want to
access the STT results.
- The examples runner is now public from the `pipecat.examples` package. This
allows everyone to build their own examples and run them easily.
- It is now possible to push `OutputDTMFFrame` or `OutputDTMFUrgentFrame` with
`DailyTransport`. This will be sent properly if a Daily dial-out connection
has been established.
- Added `OutputDTMFUrgentFrame` to send a DTMF keypress quickly. The previous
`OutputDTMFFrame` queues the keypress with the rest of data frames.
- Added `DTMFAggregator`, which aggregates keypad presses into
`TranscriptionFrame`s. Aggregation occurs after a timeout, termination key
press, or user interruption. You can specify the prefix of the
`TranscriptionFrame`.
- Added new functions `DailyTransport.start_transcription()` and
`DailyTransport.stop_transcription()` to be able to start and stop Daily
transcription dynamically (maybe with different settings).
### Changed
- Reverted the default model for `GeminiMultimodalLiveLLMService` back to
`models/gemini-2.0-flash-live-001`.
`gemini-2.5-flash-preview-native-audio-dialog` has inconsistent performance.
You can opt in to using this model by setting the `model` arg.
- Function calls are now cancelled by default if there's an interruption. To
disable this behavior you can set `cancel_on_interruption=False` when
registering the function call. Since function calls are executed as tasks you
can tell if a function call has been cancelled by catching the
`asyncio.CancelledError` exception (and don't forget to raise it again!).
- Updated OpenTelemetry tracing attribute `metrics.ttfb_ms` to `metrics.ttfb`.
The attribute reports TTFB in seconds.
### Deprecated
- `DailyTransport.send_dtmf()` is deprecated, push an `OutputDTMFFrame` or an
`OutputDTMFUrgentFrame` instead.
### Fixed
- Fixed an issue with `ElevenLabsTTSService` where long responses would
continue generating output even after an interruption.
- Fixed an issue with the `OpenAILLMContext` where non-Roman characters were
being incorrectly encoded as Unicode escape sequences. This was a logging
issue and did not impact the actual conversation.
- In `AWSBedrockLLMService`, worked around a possible bug in AWS Bedrock where
a `toolConfig` is required if there has been previous tool use in the
messages array. This workaround includes a no_op factory function call is
used to satisfy the requirement.
- Fixed `WebsocketClientTransport` to use `FrameProcessorSetup.task_manager`
instead of `StartFrame.task_manager`.
### Performance
- Use `uvloop` as the new event loop on Linux and macOS systems.
## [0.0.68] - 2025-05-28
### Added

View File

@@ -41,150 +41,36 @@ We use Ruff for code linting and formatting. Please ensure your code passes all
We follow Google-style docstrings with these specific conventions:
**Regular Classes:**
- Class docstrings should fully document all parameters used in `__init__`
- We don't require separate docstrings for `__init__` methods when parameters are documented in the class docstring
- Property methods should have docstrings explaining their purpose and return value
- Class docstring describes the class purpose and key functionality
- `__init__` method has its own docstring with complete `Args:` section documenting all parameters
- All public methods must have docstrings with `Args:` and `Returns:` sections as appropriate
**Dataclasses:**
- Class docstring describes the purpose and documents all fields in a `Parameters:` section
- No `__init__` docstring (auto-generated)
**Properties:**
- Must have docstrings with `Returns:` section
**Abstract Methods:**
- Must have docstrings explaining what subclasses should implement
**`__init__.py` Files:**
- **Skip docstrings** for pure import/re-export modules
- **Add brief docstrings** for top-level packages or those with initialization logic
**Enums:**
- Class docstring describes the enumeration purpose
- Use `Parameters:` section to document each enum value and its meaning
- No `__init__` docstring (Enums don't have custom constructors)
**Code Examples in Docstrings:**
- Use `Examples:` as a section header for multiple examples
- Use descriptive text followed by double colons (`::`) for each example
- **Always include a blank line after the `::"`**
- Indent all code consistently within each block
- Separate multiple examples with blank lines for readability
**Lists and Bullets in Docstrings:**
- Use dashes (`-`) for bullet points, not asterisks (`*`)
- **Add a blank line before bullet lists** when they follow a colon
- Use section headers like "Supported features:" or "Behavior:" before lists
- For complex nested information, consider using paragraph format instead
**Deprecations:**
- Use `warnings.warn()` in code for runtime deprecation warnings
- Add `.. deprecated::` directive in docstrings for documentation visibility
- Include version information and describe current status
- Describe parameters in present tense, use directive to indicate deprecation status
#### Examples:
Example of correctly documented class:
```python
# Regular class
class MyService(BaseService):
"""Description of what the service does.
class MyClass:
"""Class description.
Provides detailed explanation of the service's functionality,
key features, and usage patterns.
Additional details about the class.
Supported features:
- Feature one with detailed explanation
- Feature two with additional context
- Feature three for advanced use cases
Args:
param1: Description of first parameter.
param2: Description of second parameter.
"""
def __init__(self, param1: str, old_param: str = None, **kwargs):
"""Initialize the service.
Args:
param1: Description of param1.
old_param: Controls legacy behavior.
.. deprecated:: 1.2.0
This parameter no longer has any effect and will be removed in version 2.0.
**kwargs: Additional arguments passed to parent.
"""
if old_param is not None:
import warnings
warnings.warn(
"Parameter 'old_param' is deprecated and will be removed in version 2.0.",
DeprecationWarning,
)
super().__init__(**kwargs)
def __init__(self, param1, param2):
# No docstring required here as parameters are documented above
self.param1 = param1
self.param2 = param2
@property
def sample_rate(self) -> int:
"""Get the current sample rate.
def some_property(self) -> str:
"""Get the formatted property value.
Returns:
The sample rate in Hz.
A string representation of the property.
"""
return self._sample_rate
async def process_data(self, data: str) -> bool:
"""Process the provided data.
Args:
data: The data to process.
Returns:
True if processing succeeded.
"""
pass
# Dataclass with code examples
@dataclass
class MessageFrame:
"""Frame containing messages in OpenAI format.
Supports both simple and content list message formats.
Example::
[
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there!"}
]
Parameters:
messages: List of messages in OpenAI format.
"""
messages: List[dict]
# Enum class
class Status(Enum):
"""Status codes for processing operations.
Parameters:
PENDING: Operation is queued but not started.
RUNNING: Operation is currently in progress.
COMPLETED: Operation finished successfully.
FAILED: Operation encountered an error.
"""
PENDING = "pending"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
return f"Property: {self.param1}"
```
# Contributor Covenant Code of Conduct

View File

@@ -53,8 +53,8 @@ You can connect to Pipecat from any platform using our official SDKs:
| Category | Services |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Cartesia](https://docs.pipecat.ai/server/services/stt/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Parakeet (NVIDIA)](https://docs.pipecat.ai/server/services/stt/parakeet), [SambaNova (Whisper)](https://docs.pipecat.ai/server/services/stt/sambanova) [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
| LLMs | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [AWS](https://docs.pipecat.ai/server/services/llm/aws), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Qwen](https://docs.pipecat.ai/server/services/llm/qwen), [SambaNova](https://docs.pipecat.ai/server/services/llm/sambanova) [Together AI](https://docs.pipecat.ai/server/services/llm/together) |
| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Parakeet (NVIDIA)](https://docs.pipecat.ai/server/services/stt/parakeet), [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
| LLMs | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [AWS](https://docs.pipecat.ai/server/services/llm/aws), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Qwen](https://docs.pipecat.ai/server/services/llm/qwen), [Together AI](https://docs.pipecat.ai/server/services/llm/together) |
| Text-to-Speech | [AWS](https://docs.pipecat.ai/server/services/tts/aws), [Azure](https://docs.pipecat.ai/server/services/tts/azure), [Cartesia](https://docs.pipecat.ai/server/services/tts/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/tts/deepgram), [ElevenLabs](https://docs.pipecat.ai/server/services/tts/elevenlabs), [FastPitch (NVIDIA)](https://docs.pipecat.ai/server/services/tts/fastpitch), [Fish](https://docs.pipecat.ai/server/services/tts/fish), [Google](https://docs.pipecat.ai/server/services/tts/google), [LMNT](https://docs.pipecat.ai/server/services/tts/lmnt), [MiniMax](https://docs.pipecat.ai/server/services/tts/minimax), [Neuphonic](https://docs.pipecat.ai/server/services/tts/neuphonic), [OpenAI](https://docs.pipecat.ai/server/services/tts/openai), [Piper](https://docs.pipecat.ai/server/services/tts/piper), [PlayHT](https://docs.pipecat.ai/server/services/tts/playht), [Rime](https://docs.pipecat.ai/server/services/tts/rime), [Sarvam](https://docs.pipecat.ai/server/services/tts/sarvam), [XTTS](https://docs.pipecat.ai/server/services/tts/xtts) |
| Speech-to-Speech | [AWS Nova Sonic](https://docs.pipecat.ai/server/services/s2s/aws), [Gemini Multimodal Live](https://docs.pipecat.ai/server/services/s2s/gemini), [OpenAI Realtime](https://docs.pipecat.ai/server/services/s2s/openai) |
| Transport | [Daily (WebRTC)](https://docs.pipecat.ai/server/services/transport/daily), [FastAPI Websocket](https://docs.pipecat.ai/server/services/transport/fastapi-websocket), [SmallWebRTCTransport](https://docs.pipecat.ai/server/services/transport/small-webrtc), [WebSocket Server](https://docs.pipecat.ai/server/services/transport/websocket-server), Local |

View File

@@ -1,13 +1,13 @@
build~=1.2.2
coverage~=7.9.1
coverage~=7.6.12
grpcio-tools~=1.67.1
pip-tools~=7.4.1
pre-commit~=4.2.0
pyright~=1.1.402
pytest~=8.4.1
pytest-asyncio~=1.0.0
pre-commit~=4.0.1
pyright~=1.1.397
pytest~=8.3.4
pytest-asyncio~=0.25.3
pytest-aiohttp==1.1.0
ruff~=0.12.1
setuptools~=78.1.1
setuptools_scm~=8.3.1
python-dotenv~=1.1.1
ruff~=0.11.1
setuptools~=70.0.0
setuptools_scm~=8.1.0
python-dotenv~=1.0.1

View File

@@ -1,6 +1,5 @@
import logging
import sys
from datetime import datetime
from pathlib import Path
# Configure logging
@@ -14,8 +13,7 @@ sys.path.insert(0, str(project_root / "src"))
# Project information
project = "pipecat-ai"
current_year = datetime.now().year
copyright = f"2024-{current_year}, Daily" if current_year > 2024 else "2024, Daily"
copyright = "2024, Daily"
author = "Daily"
# General configuration
@@ -26,20 +24,19 @@ extensions = [
"sphinx.ext.intersphinx",
]
suppress_warnings = [
"autodoc.mocked_object",
]
# Napoleon settings
napoleon_google_docstring = True
napoleon_numpy_docstring = False
napoleon_include_init_with_doc = True
# AutoDoc settings
autodoc_default_options = {
"members": True,
"member-order": "bysource",
"undoc-members": False,
"exclude-members": "__weakref__,model_config",
"special-members": "__init__",
"undoc-members": True,
"exclude-members": "__weakref__",
"no-index": True,
"show-inheritance": True,
}
@@ -74,6 +71,7 @@ autodoc_mock_imports = [
"langchain",
"lmnt",
"noisereduce",
"openai",
"openpipe",
"simli",
"soundfile",
@@ -83,6 +81,10 @@ autodoc_mock_imports = [
"tkinter",
"daily",
"daily_python",
"pydantic.BaseModel",
"pydantic.Field",
"pydantic._internal._model_construction",
"pydantic._internal._fields",
# Moondream dependencies
"torch",
"transformers",
@@ -143,76 +145,85 @@ autodoc_mock_imports = [
"transformers.AutoFeatureExtractor",
# Also add specific classes that are imported
"AutoFeatureExtractor",
# Sentry dependencies
"sentry_sdk",
# AWS Nova Sonic dependencies
"aws_sdk_bedrock_runtime",
"aws_sdk_bedrock_runtime.client",
"aws_sdk_bedrock_runtime.config",
"aws_sdk_bedrock_runtime.models",
"smithy_aws_core",
"smithy_aws_core.credentials_resolvers",
"smithy_aws_core.credentials_resolvers.static",
"smithy_aws_core.identity",
"smithy_core",
"smithy_core.aio",
"smithy_core.aio.eventstream",
# MCP dependencies (you may already have these)
"mcp",
"mcp.client",
"mcp.client.session_group",
"mcp.client.sse",
"mcp.client.stdio",
"mcp.ClientSession",
"mcp.StdioServerParameters",
# gstreamer
"gi",
"gi.require_version",
"gi.repository",
# Protobuf mocks
"pipecat.frames.protobufs.frames_pb2",
"pipecat.serializers.protobuf",
"google.protobuf",
"google.protobuf.descriptor",
"google.protobuf.descriptor_pool",
"google.protobuf.runtime_version",
"google.protobuf.symbol_database",
"google.protobuf.internal.builder",
]
# HTML output settings
html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
autodoc_typehints = "signature" # Show type hints in the signature only, not in the docstring
autodoc_typehints = "description"
html_show_sphinx = False
def import_core_modules():
"""Import core pipecat modules for autodoc to discover."""
core_modules = [
"pipecat",
"pipecat.frames",
"pipecat.pipeline",
"pipecat.processors",
"pipecat.services",
"pipecat.transports",
"pipecat.audio",
"pipecat.adapters",
"pipecat.clocks",
"pipecat.metrics",
"pipecat.observers",
"pipecat.serializers",
"pipecat.sync",
"pipecat.transcriptions",
"pipecat.utils",
]
def verify_modules():
"""Verify that required modules are available."""
required_modules = {
"services": [
"assemblyai",
"aws",
"cartesia",
"deepgram",
"google",
"lmnt",
"riva",
"simli",
],
"serializers": ["livekit"],
"vad": ["silero", "vad_analyzer"],
"transports": {
"services": ["daily", "livekit"],
"local": ["audio", "tk"],
"network": ["fastapi_websocket", "websocket_server"],
},
}
for module_name in core_modules:
try:
__import__(module_name)
logger.info(f"Successfully imported {module_name}")
except ImportError as e:
logger.warning(f"Failed to import {module_name}: {e}")
# Skip importing modules that are in autodoc_mock_imports
skipped_modules = set(autodoc_mock_imports)
missing = []
for category, modules in required_modules.items():
if isinstance(modules, dict):
# Handle nested structure
for subcategory, submodules in modules.items():
for module in submodules:
# Check if module is in autodoc_mock_imports
if (
f"pipecat.{category}.{subcategory}.{module}" in skipped_modules
or module in skipped_modules
):
logger.info(
f"Skipping import of mocked module: pipecat.{category}.{subcategory}.{module}"
)
continue
try:
__import__(f"pipecat.{category}.{subcategory}.{module}")
logger.info(
f"Successfully imported pipecat.{category}.{subcategory}.{module}"
)
except (ImportError, TypeError, NameError) as e:
missing.append(f"pipecat.{category}.{subcategory}.{module}")
logger.warning(
f"Optional module not available: pipecat.{category}.{subcategory}.{module} - {str(e)}"
)
else:
# Handle flat structure
for module in modules:
# Check if module is in autodoc_mock_imports
if f"pipecat.{category}.{module}" in skipped_modules or module in skipped_modules:
logger.info(f"Skipping import of mocked module: pipecat.{category}.{module}")
continue
try:
__import__(f"pipecat.{category}.{module}")
logger.info(f"Successfully imported pipecat.{category}.{module}")
except (ImportError, TypeError, NameError) as e:
missing.append(f"pipecat.{category}.{module}")
logger.warning(
f"Optional module not available: pipecat.{category}.{module} - {str(e)}"
)
if missing:
logger.warning(f"Some optional modules are not available: {missing}")
def clean_title(title: str) -> str:
@@ -224,7 +235,36 @@ def clean_title(title: str) -> str:
parts = title.split(".")
title = parts[-1]
return title
# Special cases for service names and common acronyms
special_cases = {
"ai": "AI",
"aws": "AWS",
"api": "API",
"vad": "VAD",
"assemblyai": "AssemblyAI",
"deepgram": "Deepgram",
"elevenlabs": "ElevenLabs",
"openai": "OpenAI",
"openpipe": "OpenPipe",
"playht": "PlayHT",
"xtts": "XTTS",
"lmnt": "LMNT",
}
# Check if the entire title is a special case
if title.lower() in special_cases:
return special_cases[title.lower()]
# Otherwise, capitalize each word
words = title.split("_")
cleaned_words = []
for word in words:
if word.lower() in special_cases:
cleaned_words.append(special_cases[word.lower()])
else:
cleaned_words.append(word.capitalize())
return " ".join(cleaned_words)
def setup(app):
@@ -249,8 +289,9 @@ def setup(app):
excludes = [
str(project_root / "src/pipecat/pipeline/to_be_updated"),
str(project_root / "src/pipecat/examples"),
str(project_root / "src/pipecat/tests"),
str(project_root / "src/pipecat/processors/gstreamer"),
str(project_root / "src/pipecat/services/to_be_updated"),
str(project_root / "src/pipecat/vad"), # deprecated
"**/test_*.py",
"**/tests/*.py",
]
@@ -291,4 +332,5 @@ def setup(app):
logger.error(f"Error generating API documentation: {e}", exc_info=True)
import_core_modules()
# Run module verification
verify_modules()

View File

@@ -1,17 +1,57 @@
Pipecat API Reference
=====================
Pipecat API Reference Docs
==========================
Welcome to the Pipecat API reference.
Welcome to Pipecat's API reference documentation!
Use the navigation on the left to browse modules, or search using the search box.
**New to Pipecat?** Check out the `main documentation <https://docs.pipecat.ai>`_ for tutorials, guides, and client SDK information.
Pipecat is an open source framework for building voice and multimodal assistants.
It provides a flexible pipeline architecture for connecting various AI services,
audio processing, and transport layers.
Quick Links
-----------
* `GitHub Repository <https://github.com/pipecat-ai/pipecat>`_
* `Join our Community <https://discord.gg/pipecat>`_
* `Website <https://pipecat.ai>`_
API Reference
-------------
Core Components
~~~~~~~~~~~~~~~
* :mod:`Frames <pipecat.frames>`
* :mod:`Processors <pipecat.processors>`
* :mod:`Pipeline <pipecat.pipeline>`
Audio Processing
~~~~~~~~~~~~~~~~
* :mod:`Audio <pipecat.audio>`
Services
~~~~~~~~
* :mod:`Services <pipecat.services>`
Transport & Serialization
~~~~~~~~~~~~~~~~~~~~~~~~~
* :mod:`Transports <pipecat.transports>`
* :mod:`Local <pipecat.transports.local>`
* :mod:`Network <pipecat.transports.network>`
* :mod:`Services <pipecat.transports.services>`
* :mod:`Serializers <pipecat.serializers>`
Utilities
~~~~~~~~~
* :mod:`Adapters <pipecat.adapters>`
* :mod:`Clocks <pipecat.clocks>`
* :mod:`Metrics <pipecat.metrics>`
* :mod:`Observers <pipecat.observers>`
* :mod:`Sync <pipecat.sync>`
* :mod:`Transcriptions <pipecat.transcriptions>`
* :mod:`Utils <pipecat.utils>`
.. toctree::
:maxdepth: 3
@@ -31,4 +71,11 @@ Quick Links
Sync <api/pipecat.sync>
Transcriptions <api/pipecat.transcriptions>
Transports <api/pipecat.transports>
Utils <api/pipecat.utils>
Utils <api/pipecat.utils>
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

View File

@@ -42,7 +42,6 @@ pipecat-ai[openai]
pipecat-ai[qwen]
pipecat-ai[remote-smart-turn]
# pipecat-ai[riva] # Mocked
pipecat-ai[sambanova]
pipecat-ai[silero]
pipecat-ai[simli]
pipecat-ai[soundfile]

View File

@@ -107,10 +107,4 @@ MINIMAX_API_KEY=...
MINIMAX_GROUP_ID=...
# Sarvam AI
SARVAM_API_KEY=...
# SambaNova
SAMBANOVA_API_KEY=...
# Sentry
SENTRY_DSN=...
SARVAM_API_KEY=...

View File

@@ -4364,9 +4364,9 @@
}
},
"node_modules/brace-expansion": {
"version": "1.1.12",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz",
"integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==",
"version": "1.1.11",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz",
"integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==",
"dependencies": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -6081,9 +6081,9 @@
}
},
"node_modules/glob/node_modules/brace-expansion": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
"integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.1.tgz",
"integrity": "sha512-XnAIvQ8eM+kC6aULx6wuQiwVsnzsi9d3WxzV3FpWTGA19F621kwdbsAcFKXgKUHZWsy+mY6iL1sHTxWEFCytDA==",
"dependencies": {
"balanced-match": "^1.0.0"
}

View File

@@ -133,8 +133,7 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
enable_metrics=True,
enable_usage_metrics=True,
allow_interruptions=True,
),
)

View File

@@ -71,8 +71,6 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
enable_metrics=True,
enable_usage_metrics=True,
),
)

View File

@@ -148,8 +148,10 @@ async def main():
params=PipelineParams(
audio_in_sample_rate=16000,
audio_out_sample_rate=16000,
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
observers=[TranscriptionLogObserver()],
)

View File

@@ -2,4 +2,4 @@ aiofiles
python-dotenv
fastapi[all]
uvicorn
pipecat-ai[daily,deepgram,openai,silero,cartesia,soundfile]
pipecat-ai[daily,deepgram,openai,silero,cartesia]

View File

@@ -75,13 +75,7 @@ async def main(room_url: str, token: str):
]
)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):

View File

@@ -170,6 +170,7 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -198,6 +198,7 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -211,6 +211,7 @@ async def run_bot(room_url: str, token: str):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -215,9 +215,10 @@
}
},
"node_modules/@next/env": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/env/-/env-14.2.30.tgz",
"integrity": "sha512-KBiBKrDY6kxTQWGzKjQB7QirL3PiiOkV7KW98leHFjtVRKtft76Ra5qSA/SL75xT44dp6hOcqiiJ6iievLOYug=="
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/env/-/env-14.2.26.tgz",
"integrity": "sha512-vO//GJ/YBco+H7xdQhzJxF7ub3SUwft76jwaeOyVVQFHCi5DCnkP16WHB+JBylo4vOKPoZBlR94Z8xBxNBdNJA==",
"license": "MIT"
},
"node_modules/@next/eslint-plugin-next": {
"version": "14.2.25",
@@ -230,12 +231,13 @@
}
},
"node_modules/@next/swc-darwin-arm64": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-14.2.30.tgz",
"integrity": "sha512-EAqfOTb3bTGh9+ewpO/jC59uACadRHM6TSA9DdxJB/6gxOpyV+zrbqeXiFTDy9uV6bmipFDkfpAskeaDcO+7/g==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-darwin-arm64/-/swc-darwin-arm64-14.2.26.tgz",
"integrity": "sha512-zDJY8gsKEseGAxG+C2hTMT0w9Nk9N1Sk1qV7vXYz9MEiyRoF5ogQX2+vplyUMIfygnjn9/A04I6yrUTRTuRiyQ==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
@@ -245,12 +247,13 @@
}
},
"node_modules/@next/swc-darwin-x64": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-14.2.30.tgz",
"integrity": "sha512-TyO7Wz1IKE2kGv8dwQ0bmPL3s44EKVencOqwIY69myoS3rdpO1NPg5xPM5ymKu7nfX4oYJrpMxv8G9iqLsnL4A==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-darwin-x64/-/swc-darwin-x64-14.2.26.tgz",
"integrity": "sha512-U0adH5ryLfmTDkahLwG9sUQG2L0a9rYux8crQeC92rPhi3jGQEY47nByQHrVrt3prZigadwj/2HZ1LUUimuSbg==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
@@ -260,12 +263,13 @@
}
},
"node_modules/@next/swc-linux-arm64-gnu": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-14.2.30.tgz",
"integrity": "sha512-I5lg1fgPJ7I5dk6mr3qCH1hJYKJu1FsfKSiTKoYwcuUf53HWTrEkwmMI0t5ojFKeA6Vu+SfT2zVy5NS0QLXV4Q==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-gnu/-/swc-linux-arm64-gnu-14.2.26.tgz",
"integrity": "sha512-SINMl1I7UhfHGM7SoRiw0AbwnLEMUnJ/3XXVmhyptzriHbWvPPbbm0OEVG24uUKhuS1t0nvN/DBvm5kz6ZIqpg==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
@@ -275,12 +279,13 @@
}
},
"node_modules/@next/swc-linux-arm64-musl": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-14.2.30.tgz",
"integrity": "sha512-8GkNA+sLclQyxgzCDs2/2GSwBc92QLMrmYAmoP2xehe5MUKBLB2cgo34Yu242L1siSkwQkiV4YLdCnjwc/Micw==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-linux-arm64-musl/-/swc-linux-arm64-musl-14.2.26.tgz",
"integrity": "sha512-s6JaezoyJK2DxrwHWxLWtJKlqKqTdi/zaYigDXUJ/gmx/72CrzdVZfMvUc6VqnZ7YEvRijvYo+0o4Z9DencduA==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
@@ -290,12 +295,13 @@
}
},
"node_modules/@next/swc-linux-x64-gnu": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-14.2.30.tgz",
"integrity": "sha512-8Ly7okjssLuBoe8qaRCcjGtcMsv79hwzn/63wNeIkzJVFVX06h5S737XNr7DZwlsbTBDOyI6qbL2BJB5n6TV/w==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-gnu/-/swc-linux-x64-gnu-14.2.26.tgz",
"integrity": "sha512-FEXeUQi8/pLr/XI0hKbe0tgbLmHFRhgXOUiPScz2hk0hSmbGiU8aUqVslj/6C6KA38RzXnWoJXo4FMo6aBxjzg==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
@@ -305,12 +311,13 @@
}
},
"node_modules/@next/swc-linux-x64-musl": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-14.2.30.tgz",
"integrity": "sha512-dBmV1lLNeX4mR7uI7KNVHsGQU+OgTG5RGFPi3tBJpsKPvOPtg9poyav/BYWrB3GPQL4dW5YGGgalwZ79WukbKQ==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-linux-x64-musl/-/swc-linux-x64-musl-14.2.26.tgz",
"integrity": "sha512-BUsomaO4d2DuXhXhgQCVt2jjX4B4/Thts8nDoIruEJkhE5ifeQFtvW5c9JkdOtYvE5p2G0hcwQ0UbRaQmQwaVg==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
@@ -320,12 +327,13 @@
}
},
"node_modules/@next/swc-win32-arm64-msvc": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-14.2.30.tgz",
"integrity": "sha512-6MMHi2Qc1Gkq+4YLXAgbYslE1f9zMGBikKMdmQRHXjkGPot1JY3n5/Qrbg40Uvbi8//wYnydPnyvNhI1DMUW1g==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-win32-arm64-msvc/-/swc-win32-arm64-msvc-14.2.26.tgz",
"integrity": "sha512-5auwsMVzT7wbB2CZXQxDctpWbdEnEW/e66DyXO1DcgHxIyhP06awu+rHKshZE+lPLIGiwtjo7bsyeuubewwxMw==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
@@ -335,12 +343,13 @@
}
},
"node_modules/@next/swc-win32-ia32-msvc": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-win32-ia32-msvc/-/swc-win32-ia32-msvc-14.2.30.tgz",
"integrity": "sha512-pVZMnFok5qEX4RT59mK2hEVtJX+XFfak+/rjHpyFh7juiT52r177bfFKhnlafm0UOSldhXjj32b+LZIOdswGTg==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-win32-ia32-msvc/-/swc-win32-ia32-msvc-14.2.26.tgz",
"integrity": "sha512-GQWg/Vbz9zUGi9X80lOeGsz1rMH/MtFO/XqigDznhhhTfDlDoynCM6982mPCbSlxJ/aveZcKtTlwfAjwhyxDpg==",
"cpu": [
"ia32"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
@@ -350,12 +359,13 @@
}
},
"node_modules/@next/swc-win32-x64-msvc": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-14.2.30.tgz",
"integrity": "sha512-4KCo8hMZXMjpTzs3HOqOGYYwAXymXIy7PEPAXNEcEOyKqkjiDlECumrWziy+JEF0Oi4ILHGxzgQ3YiMGG2t/Lg==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-14.2.26.tgz",
"integrity": "sha512-2rdB3T1/Gp7bv1eQTTm9d1Y1sv9UuJ2LAwOE0Pe2prHKe32UNscj7YS13fRB37d0GAiGNR+Y7ZcW8YjDI8Ns0w==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
@@ -610,10 +620,11 @@
}
},
"node_modules/@typescript-eslint/typescript-estree/node_modules/brace-expansion": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
"integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.1.tgz",
"integrity": "sha512-XnAIvQ8eM+kC6aULx6wuQiwVsnzsi9d3WxzV3FpWTGA19F621kwdbsAcFKXgKUHZWsy+mY6iL1sHTxWEFCytDA==",
"dev": true,
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0"
}
@@ -1213,10 +1224,11 @@
"license": "MIT"
},
"node_modules/brace-expansion": {
"version": "1.1.12",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz",
"integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==",
"version": "1.1.11",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz",
"integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==",
"dev": true,
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -2602,10 +2614,11 @@
}
},
"node_modules/glob/node_modules/brace-expansion": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
"integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.1.tgz",
"integrity": "sha512-XnAIvQ8eM+kC6aULx6wuQiwVsnzsi9d3WxzV3FpWTGA19F621kwdbsAcFKXgKUHZWsy+mY6iL1sHTxWEFCytDA==",
"dev": true,
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0"
}
@@ -3600,11 +3613,12 @@
"license": "MIT"
},
"node_modules/next": {
"version": "14.2.30",
"resolved": "https://registry.npmjs.org/next/-/next-14.2.30.tgz",
"integrity": "sha512-+COdu6HQrHHFQ1S/8BBsCag61jZacmvbuL2avHvQFbWa2Ox7bE+d8FyNgxRLjXQ5wtPyQwEmk85js/AuaG2Sbg==",
"version": "14.2.26",
"resolved": "https://registry.npmjs.org/next/-/next-14.2.26.tgz",
"integrity": "sha512-b81XSLihMwCfwiUVRRja3LphLo4uBBMZEzBBWMaISbKTwOmq3wPknIETy/8000tr7Gq4WmbuFYPS7jOYIf+ZJw==",
"license": "MIT",
"dependencies": {
"@next/env": "14.2.30",
"@next/env": "14.2.26",
"@swc/helpers": "0.5.5",
"busboy": "1.6.0",
"caniuse-lite": "^1.0.30001579",
@@ -3619,15 +3633,15 @@
"node": ">=18.17.0"
},
"optionalDependencies": {
"@next/swc-darwin-arm64": "14.2.30",
"@next/swc-darwin-x64": "14.2.30",
"@next/swc-linux-arm64-gnu": "14.2.30",
"@next/swc-linux-arm64-musl": "14.2.30",
"@next/swc-linux-x64-gnu": "14.2.30",
"@next/swc-linux-x64-musl": "14.2.30",
"@next/swc-win32-arm64-msvc": "14.2.30",
"@next/swc-win32-ia32-msvc": "14.2.30",
"@next/swc-win32-x64-msvc": "14.2.30"
"@next/swc-darwin-arm64": "14.2.26",
"@next/swc-darwin-x64": "14.2.26",
"@next/swc-linux-arm64-gnu": "14.2.26",
"@next/swc-linux-arm64-musl": "14.2.26",
"@next/swc-linux-x64-gnu": "14.2.26",
"@next/swc-linux-x64-musl": "14.2.26",
"@next/swc-win32-arm64-msvc": "14.2.26",
"@next/swc-win32-ia32-msvc": "14.2.26",
"@next/swc-win32-x64-msvc": "14.2.26"
},
"peerDependencies": {
"@opentelemetry/api": "^1.1.0",

View File

@@ -67,8 +67,10 @@ async def main(transport: DailyTransport):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -192,6 +192,7 @@ async def main(transport: DailyTransport):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),

View File

@@ -55,6 +55,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -56,6 +56,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -53,6 +53,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -77,36 +77,37 @@ async def configure_livekit():
async def main():
(url, token, room_name) = await configure_livekit()
async with aiohttp.ClientSession() as session:
(url, token, room_name) = await configure_livekit()
transport = LiveKitTransport(
url=url,
token=token,
room_name=room_name,
params=LiveKitParams(audio_out_enabled=True),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
)
runner = PipelineRunner()
task = PipelineTask(Pipeline([tts, transport.output()]))
# Register an event handler so we can play the audio when the
# participant joins.
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant_id):
await asyncio.sleep(1)
await task.queue_frame(
TextFrame(
"Hello there! How are you doing today? Would you like to talk about the weather?"
)
transport = LiveKitTransport(
url=url,
token=token,
room_name=room_name,
params=LiveKitParams(audio_out_enabled=True),
)
await runner.run(task)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
)
runner = PipelineRunner()
task = PipelineTask(Pipeline([tts, transport.output()]))
# Register an event handler so we can play the audio when the
# participant joins.
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant_id):
await asyncio.sleep(1)
await task.queue_frame(
TextFrame(
"Hello there! How are you doing today? Would you like to talk about the weather?"
)
)
await runner.run(task)
if __name__ == "__main__":

View File

@@ -50,6 +50,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -63,6 +63,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -68,6 +68,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -47,10 +47,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
Pipeline([imagegen, transport.output()]),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
params=PipelineParams(enable_metrics=True),
)
# Register an event handler so we can play the audio when the client joins
@@ -71,6 +68,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -93,8 +93,10 @@ async def run_example(webrtc_connection: SmallWebRTCConnection):
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -154,7 +156,7 @@ async def offer(request: dict, background_tasks: BackgroundTasks):
@asynccontextmanager
async def lifespan(app: FastAPI):
yield # Run app
coros = [pc.disconnect() for pc in pcs_map.values()]
coros = [pc.close() for pc in pcs_map.values()]
await asyncio.gather(*coros)
pcs_map.clear()

View File

@@ -9,11 +9,11 @@ import os
import sys
import aiohttp
from daily_runner import configure
from dotenv import load_dotenv
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.daily_runner import configure
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
@@ -75,8 +75,10 @@ async def main():
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -158,8 +158,7 @@ async def main():
],
),
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
allow_interruptions=True, enable_metrics=True, enable_usage_metrics=True
),
)

View File

@@ -174,6 +174,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -137,6 +137,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -133,8 +133,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -154,6 +156,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,8 +84,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -107,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -83,8 +83,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -106,6 +108,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -113,8 +113,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -139,6 +141,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,8 +87,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -118,6 +120,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -81,8 +81,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -104,6 +106,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -88,8 +88,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -111,6 +113,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,8 +84,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -107,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,8 +84,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -107,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -86,8 +86,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +111,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -90,8 +90,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -113,6 +115,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,9 +84,11 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
audio_out_sample_rate=24000,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -108,6 +110,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -89,8 +89,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -112,6 +114,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,8 +87,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -110,6 +112,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -92,8 +92,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -114,6 +116,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -80,8 +80,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -103,6 +105,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -85,6 +85,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -108,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,8 +87,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -110,6 +112,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -61,12 +61,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
credentials=os.getenv("GOOGLE_TEST_CREDENTIALS"),
)
llm = GoogleLLMService(
api_key=os.getenv("GOOGLE_API_KEY"),
model="gemini-2.5-flash",
# turn on thinking if you want it
# params=GoogleLLMService.InputParams(extra={"thinking_config": {"thinking_budget": 4096}}),)
)
llm = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"))
messages = [
{
@@ -93,8 +88,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -116,6 +113,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -86,8 +86,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +111,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,8 +84,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -107,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -89,8 +89,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -112,6 +114,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -83,8 +83,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -106,6 +108,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -80,8 +80,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -103,6 +105,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -8,8 +8,8 @@ import argparse
import os
from dataclasses import dataclass
import google.ai.generativelanguage as glm
from dotenv import load_dotenv
from google.genai.types import Content, Part
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
@@ -164,7 +164,9 @@ class TanscriptionContextFixup(FrameProcessor):
and last_part.inline_data
and last_part.inline_data.mime_type == "audio/wav"
):
self._context.messages[-2] = Content(role="user", parts=[Part(text=self._transcript)])
self._context.messages[-2] = glm.Content(
role="user", parts=[glm.Part(text=self._transcript)]
)
def add_transcript_back_to_inference_output(self):
if not self._transcript:
@@ -214,12 +216,7 @@ transport_params = {
async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_sigint: bool):
logger.info(f"Starting bot")
llm = GoogleLLMService(
api_key=os.getenv("GOOGLE_API_KEY"),
model="gemini-2.5-flash",
# turn on thinking if you want it
# params=GoogleLLMService.InputParams(extra={"thinking_config": {"thinking_budget": 4096}}),
)
llm = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"), model="gemini-2.0-flash-001")
tts = GoogleTTSService(
voice_id="en-US-Chirp3-HD-Charon",
@@ -261,6 +258,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -284,6 +282,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,8 +84,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -107,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -77,8 +77,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@@ -97,6 +97,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,8 +84,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -107,6 +109,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -83,8 +83,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -106,6 +108,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -86,8 +86,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -109,6 +111,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -70,8 +70,10 @@ async def main():
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -90,8 +90,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -113,6 +115,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -89,8 +89,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -112,6 +114,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -96,6 +96,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -11,7 +11,6 @@ import tkinter as tk
from dotenv import load_dotenv
from loguru import logger
from pipecat.examples.run import maybe_capture_participant_camera
from pipecat.frames.frames import (
Frame,
InputAudioRawFrame,
@@ -108,7 +107,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
await maybe_capture_participant_camera(transport, client, framerate=30)
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
@@ -121,6 +119,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -85,13 +85,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
@@ -110,6 +104,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -162,6 +162,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -105,7 +105,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_camera(transport, client)
await maybe_capture_participant_video(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -125,6 +125,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -101,17 +101,14 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
params=PipelineParams(allow_interruptions=True),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_camera(transport, client)
await maybe_capture_participant_video(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -131,6 +128,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -101,17 +101,14 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
params=PipelineParams(allow_interruptions=True),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_camera(transport, client)
await maybe_capture_participant_video(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -131,6 +128,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,9 +10,9 @@ from typing import Optional
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import Frame, TextFrame, UserImageRequestFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -101,17 +101,14 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
params=PipelineParams(allow_interruptions=True),
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_camera(transport, client)
await maybe_capture_participant_video(transport, client)
# Set the participant ID in the image requester
client_id = get_transport_client_id(transport, client)
@@ -131,6 +128,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -72,6 +72,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -66,6 +66,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -66,6 +66,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -87,6 +87,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -65,6 +65,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -84,7 +84,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=False,
),
)
@@ -99,6 +99,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -1,71 +0,0 @@
#
# Copyright (c) 2024–2025, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import argparse
import os
from dotenv import load_dotenv
from loguru import logger
from pipecat.frames.frames import Frame, TranscriptionFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.services.cartesia.stt import CartesiaSTTService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
from pipecat.transports.services.daily import DailyParams
load_dotenv(override=True)
class TranscriptionLogger(FrameProcessor):
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TranscriptionFrame):
print(f"Transcription: {frame.text}")
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
transport_params = {
"daily": lambda: DailyParams(audio_in_enabled=True),
"twilio": lambda: FastAPIWebsocketParams(audio_in_enabled=True),
"webrtc": lambda: TransportParams(audio_in_enabled=True),
}
async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_sigint: bool):
logger.info(f"Starting bot")
stt = CartesiaSTTService(
api_key=os.getenv("CARTESIA_API_KEY"),
base_url=os.getenv("CARTESIA_BASE_URL"),
)
tl = TranscriptionLogger()
pipeline = Pipeline([transport.input(), stt, tl])
task = PipelineTask(pipeline)
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
await task.cancel()
runner = PipelineRunner(handle_sigint=handle_sigint)
await runner.run(task)
if __name__ == "__main__":
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -1,108 +0,0 @@
#
# Copyright (c) 2024–2025, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import argparse
import os
import time
from dotenv import load_dotenv
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.audio.vad.vad_analyzer import VADParams
from pipecat.frames.frames import Frame, TranscriptionFrame, UserStoppedSpeakingFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.services.sambanova.stt import SambaNovaSTTService
from pipecat.transports.base_transport import BaseTransport, TransportParams
from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
from pipecat.transports.services.daily import DailyParams
load_dotenv(override=True)
STOP_SECS = 2.0
class TranscriptionLogger(FrameProcessor):
"""Measures transcription latency.
Uses the (intentionally) long STOP_SECS parameter to give the transcription time to finish,
then outputs the timing between when the VAD first classified audio input as not-speech and
the delivery of the last transcription frame.
"""
def __init__(self):
super().__init__()
self._last_transcription_time = time.time()
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, UserStoppedSpeakingFrame):
logger.debug(
f"Transcription latency: {(STOP_SECS - (time.time() - self._last_transcription_time)):.2f}"
)
if isinstance(frame, TranscriptionFrame):
self._last_transcription_time = time.time()
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
transport_params = {
"daily": lambda: DailyParams(
audio_in_enabled=True,
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=STOP_SECS)),
),
"twilio": lambda: FastAPIWebsocketParams(
audio_in_enabled=True,
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=STOP_SECS)),
),
"webrtc": lambda: TransportParams(
audio_in_enabled=True,
vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=STOP_SECS)),
),
}
async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_sigint: bool):
logger.info(f"Starting bot")
stt = SambaNovaSTTService(
model="Whisper-Large-v3",
api_key=os.getenv("SAMBANOVA_API_KEY"),
)
tl = TranscriptionLogger()
pipeline = Pipeline([transport.input(), stt, tl])
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
await task.cancel()
runner = PipelineRunner(handle_sigint=handle_sigint)
await runner.run(task)
if __name__ == "__main__":
from pipecat.examples.run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,13 +30,10 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
@@ -74,11 +71,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# You can also register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
@@ -96,18 +88,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["location", "format"],
)
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
tools = ToolsSchema(standard_tools=[weather_function, restaurant_function])
tools = ToolsSchema(standard_tools=[weather_function])
messages = [
{
@@ -134,8 +115,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -156,6 +139,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -33,10 +33,6 @@ async def get_weather(params: FunctionCallParams):
await params.result_callback(f"The weather in {location} is currently 72 degrees and sunny.")
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
# We store functions so objects (e.g. SileroVADAnalyzer) don't get
# instantiated. The function will be called when the desired transport gets
# selected.
@@ -70,11 +66,9 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
)
llm = AnthropicLLMService(
api_key=os.getenv("ANTHROPIC_API_KEY"),
model="claude-3-7-sonnet-latest",
api_key=os.getenv("ANTHROPIC_API_KEY"), model="claude-3-7-sonnet-latest"
)
llm.register_function("get_weather", get_weather)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
weather_function = FunctionSchema(
name="get_weather",
@@ -87,18 +81,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["location"],
)
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
tools = ToolsSchema(standard_tools=[weather_function, restaurant_function])
tools = ToolsSchema(standard_tools=[weather_function])
# todo: test with very short initial user message
@@ -127,8 +110,8 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@@ -149,6 +132,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,11 +10,11 @@ import os
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
@@ -172,8 +172,8 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@@ -181,7 +181,7 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_camera(transport, client)
await maybe_capture_participant_video(transport, client)
global client_id
client_id = get_transport_client_id(transport, client)
@@ -200,6 +200,6 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -16,7 +16,7 @@ from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import TTSSpeakFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.pipeline.task import PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -73,10 +74,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -116,13 +113,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
]
)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
task = PipelineTask(pipeline)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
@@ -141,6 +132,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,14 +10,14 @@ import os
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.pipeline.task import PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
@@ -158,19 +158,13 @@ indicate you should use the get_image tool are:
]
)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
)
task = PipelineTask(pipeline)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
await maybe_capture_participant_camera(transport, client)
await maybe_capture_participant_video(transport, client)
global client_id
client_id = get_transport_client_id(transport, client)
@@ -189,6 +183,6 @@ indicate you should use the get_image tool are:
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -10,11 +10,11 @@ import os
from dotenv import load_dotenv
from loguru import logger
from run import get_transport_client_id, maybe_capture_participant_video
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.examples.run import get_transport_client_id, maybe_capture_participant_camera
from pipecat.frames.frames import TTSSpeakFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -35,14 +35,11 @@ client_id = ""
async def get_weather(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
location = params.arguments["location"]
await params.result_callback(f"The weather in {location} is currently 72 degrees and sunny.")
async def fetch_restaurant_recommendation(params: FunctionCallParams):
await params.result_callback({"name": "The Golden Dragon"})
async def get_image(params: FunctionCallParams):
question = params.arguments["question"]
logger.debug(f"Requesting image with user_id={client_id}, question={question}")
@@ -96,11 +93,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
llm = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"), model="gemini-2.0-flash-001")
llm.register_function("get_weather", get_weather)
llm.register_function("get_image", get_image)
llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_weather",
@@ -118,17 +110,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["location", "format"],
)
restaurant_function = FunctionSchema(
name="get_restaurant_recommendation",
description="Get a restaurant recommendation",
properties={
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
},
required=["location"],
)
get_image_function = FunctionSchema(
name="get_image",
description="Get an image from the video stream.",
@@ -140,14 +121,14 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
},
required=["question"],
)
tools = ToolsSchema(standard_tools=[weather_function, get_image_function, restaurant_function])
tools = ToolsSchema(standard_tools=[weather_function, get_image_function])
system_prompt = """\
You are a helpful assistant who converses with a user and answers questions. Respond concisely to general questions.
Your response will be turned into speech so use only simple words and punctuation.
You have access to three tools: get_weather, get_restaurant_recommendation, and get_image.
You have access to two tools: get_weather and get_image.
You can respond to questions about the weather using the get_weather tool.
@@ -183,6 +164,7 @@ indicate you should use the get_image tool are:
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -192,7 +174,7 @@ indicate you should use the get_image tool are:
async def on_client_connected(transport, client):
logger.info(f"Client connected: {client}")
await maybe_capture_participant_camera(transport, client)
await maybe_capture_participant_video(transport, client)
global client_id
client_id = get_transport_client_id(transport, client)
@@ -211,6 +193,6 @@ indicate you should use the get_image tool are:
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -31,6 +31,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -73,10 +74,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -121,6 +118,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -143,6 +141,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -111,6 +111,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -133,6 +134,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -74,10 +75,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -120,6 +117,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -142,6 +140,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -73,10 +74,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -119,6 +116,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -141,6 +139,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -71,10 +72,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -117,6 +114,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -139,6 +137,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -70,10 +71,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -126,8 +123,10 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -148,6 +147,6 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -70,10 +71,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -126,8 +123,10 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -148,6 +147,6 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -74,10 +75,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -120,8 +117,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -142,6 +141,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -90,8 +90,10 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@@ -112,6 +114,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

View File

@@ -30,6 +30,7 @@ load_dotenv(override=True)
async def fetch_weather_from_api(params: FunctionCallParams):
await params.llm.push_frame(TTSSpeakFrame("Let me check on that."))
await params.result_callback({"conditions": "nice", "temperature": "75"})
@@ -70,10 +71,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
# sent to the same callback with an additional function_name parameter.
llm.register_function("get_current_weather", fetch_weather_from_api)
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
weather_function = FunctionSchema(
name="get_current_weather",
description="Get the current weather",
@@ -116,6 +113,7 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
task = PipelineTask(
pipeline,
params=PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
@@ -138,6 +136,6 @@ async def run_example(transport: BaseTransport, _: argparse.Namespace, handle_si
if __name__ == "__main__":
from pipecat.examples.run import main
from run import main
main(run_example, transport_params=transport_params)

Some files were not shown because too many files have changed in this diff Show More