wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add README to client-server-web, add phone-bot-twilio files

Browse Source
This commit is contained in:
Mark Backman
2025-07-29 18:13:07 -04:00
parent eca366bfac
commit 3ab9c15583
9 changed files with 524 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files

111
examples/client-server-web/README.md Normal file
View File

@@ -0,0 +1,111 @@
# Client Server Web Example
Learn how to build web applications using Pipecat's client/server architecture. This approach separates your bot logic from your user interface, giving you full control over the client experience while maintaining real-time voice communication.
This example demonstrates:
- Server-side bot running with Pipecat
- React client using [Pipecat's client SDK](https://docs.pipecat.ai/client/introduction)
- Real-time voice communication between client and server
- UI components from [voice-ui-kit](https://github.com/pipecat-ai/voice-ui-kit) for common voice interface patterns
This is the recommended architecture for web applications that need custom interfaces or client-side functionality.
## Prerequisites
- Python 3.10+
- `npm` installed
- AI Service API keys for: [Deepgram](https://console.deepgram.com/signup), [OpenAI](https://auth.openai.com/create-account), and [Cartesia](https://play.cartesia.ai/sign-up)
## Setup
This example requires running both a server and client in **two separate terminal windows**.
### Terminal 1: Server Setup
1. Set up a virtual environment
From the `examples/client-server-web` directory, run:
```bash
cd server
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
> Using `uv`? Create your venv using: `uv venv && source .venv/bin/activate`.
2. Install packages
Then, install the requirements:
```bash
pip install -r requirements.txt
```
> Using `uv`? Install requirements using: `uv pip install -r requirements.txt`.
3. Configure environment variables
Create a `.env` file:
```bash
cp env.example .env
```
Then, add your API keys:
```
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
CARTESIA_API_KEY=your_cartesia_api_key
```
4. Run the example
Run your bot using:
```bash
python bot.py
```
> Using `uv`? Run your bot using: `uv run bot.py`.
> 💡 First run note: The initial startup may take ~10 seconds as Pipecat downloads required models, like the Silero VAD model.
### Terminal 2: Client Setup
1. Open a new terminal window and navigate to the `client` folder:
From the `examples/client-server-web` directory, run:
```bash
cd client
```
2. Install dependencies:
```bash
npm i
```
3. Run the client:
```bash
npm run dev
```
4. **Open http://localhost:5173 in your browser** and click `Connect` to start talking to your bot.
## Troubleshooting
- **Browser permissions**: Make sure to allow microphone access when prompted by your browser.
- **Connection issues**: If the WebRTC connection fails, first try a different browser. If that fails, make sure you don't have a VPN or firewall rules blocking traffic. WebRTC uses UDP to communicate.
- **Audio issues**: Check that your microphone and speakers are working and not muted.
## Next Steps
- **Explore the client SDK**: Learn more about [Pipecat's client SDKs](https://docs.pipecat.ai/client/introduction) for web, mobile, and other platforms
- **Learn about the voice-ui-kit**: Explore [voice-ui-kit](https://github.com/pipecat-ai/voice-ui-kit) to simplify your front end development
- **Advanced examples**: Check out [pipecat-examples](https://github.com/pipecat-ai/pipecat-examples) for more complex client/server applications
- **Join Discord**: Connect with other developers on [Discord](https://discord.gg/pipecat)

90
examples/client-server-web/server/README.md
View File

@@ -1,90 +0,0 @@
# Pipecat Quickstart
Run your first Pipecat bot in under 5 minutes. This example creates a voice AI bot that you can talk to in your browser.
## Prerequisites
### Python 3.10+
Pipecat requires Python 3.10 or newer. Check your version:
```bash
python --version
```
If you need to upgrade Python, we recommend using a version manager like `uv` or `pyenv`.
### AI Service API keys
Pipecat orchestrates different AI services in a pipeline, ensuring low latency communication. In this quickstart example, we'll use:
- [Deepgram](https://console.deepgram.com/signup) for Speech-to-Text transcriptions
- [OpenAI](https://auth.openai.com/create-account) for LLM inference
- [Cartesia](https://play.cartesia.ai/sign-up) for Text-to-Speech audio generation
Have your API keys ready. We'll add them to your `.env` shortly.
## Setup
1. Set up a virtual environment
From the root directory of the `pipecat` repo, run:
```bash
cd examples/quickstart
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
> Using `uv`? Create your venv using: `uv venv && source .venv/bin/activate`.
2. Install packages
Then, install the requirements:
```bash
pip install -r requirements.txt
```
> Using `uv`? Install requirements using: `uv pip install -r requirements.txt`.
3. Configure environment variables
Create a `.env` file:
```bash
cp env.example .env
```
Then, add your API keys:
```
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
CARTESIA_API_KEY=your_cartesia_api_key
```
4. Run the example
Run your bot using:
```bash
python bot.py
```
> Using `uv`? Run your bot using: `uv run bot.py`.
Connect to your bot in a browser at http://localhost:7860.
> 💡 First run note: The initial startup may take ~10 seconds as Pipecat downloads required models, like the Silero VAD model.
## Troubleshooting
- **Browser permissions**: Make sure to allow microphone access when prompted by your browser.
- **Connection issues**: If the WebRTC connection fails, first try a different browser. If that fails, make sure you don't have a VPN or firewall rules blocking traffic. WebRTC uses UDP to communicate.
- **Audio issues**: Check that your microphone and speakers are working and not muted.
## Next Steps
- **Read the docs**: Check out [Pipecat's docs](https://docs.pipecat.ai/) for guides and reference information.
- **Join Discord**: Join [Pipecat's Discord server](https://discord.gg/pipecat) to get help and learn about what others are building.

164
examples/phone-bot-twilio/.gitignore vendored Normal file
View File

@@ -0,0 +1,164 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
runpod.toml
# Examples
templates/streams.xml

115
examples/phone-bot-twilio/README.md Normal file
View File

@@ -0,0 +1,115 @@
# Twilio Chatbot
This project is a FastAPI-based chatbot that integrates with Twilio to handle WebSocket connections and provide real-time communication. The project includes endpoints for starting a call and handling WebSocket connections.
## Table of Contents
- [Features](#features)
- [Requirements](#requirements)
- [Installation](#installation)
- [Configure Twilio URLs](#configure-twilio-urls)
- [Running the Application](#running-the-application)
- [Usage](#usage)
## Features
- **FastAPI**: A modern, fast (high-performance), web framework for building APIs with Python 3.6+.
- **WebSocket Support**: Real-time communication using WebSockets.
- **CORS Middleware**: Allowing cross-origin requests for testing.
- **Dockerized**: Easily deployable using Docker.
## Requirements
- Python 3.10
- Docker (for containerized deployment)
- ngrok (for tunneling)
- Twilio Account
## Installation
1. **Set up a virtual environment** (optional but recommended):
```sh
python -m venv venv
source venv/bin/activate # On Windows, use `venv\Scripts\activate`
```
2. **Install dependencies**:
```sh
pip install -r requirements.txt
```
3. **Create .env**:
Copy the example environment file and update with your settings:
```sh
cp env.example .env
```
4. **Install ngrok**:
Follow the instructions on the [ngrok website](https://ngrok.com/download) to download and install ngrok.
## Configure Twilio URLs
1. **Start ngrok**:
In a new terminal, start ngrok to tunnel the local server:
```sh
ngrok http 8765
```
2. **Update the Twilio Webhook**:
- Go to your Twilio phone number's configuration page
- Under "Voice Configuration", in the "A call comes in" section:
- Select "Webhook" from the dropdown
- Enter your ngrok URL (e.g., http://<ngrok_url>)
- Ensure "HTTP POST" is selected
- Click Save at the bottom of the page
3. **Configure streams.xml**:
- Copy the template file to create your local version:
```sh
cp templates/streams.xml.template templates/streams.xml
```
- In `templates/streams.xml`, replace `<your server url>` with your ngrok URL (without `https://`)
- The final URL should look like: `wss://abc123.ngrok.io/ws`
## Running the Application
Choose one of these two methods to run the application:
### Using Python (Option 1)
**Run the FastAPI application**:
```sh
# Make sure you’re in the project directory and your virtual environment is activated
python server.py
```
### Using Docker (Option 2)
1. **Build the Docker image**:
```sh
docker build -t twilio-chatbot .
```
2. **Run the Docker container**:
```sh
docker run -it --rm -p 8765:8765 twilio-chatbot
```
The server will start on port 8765. Keep this running while you test with Twilio.
## Usage
To start a call, simply make a call to your configured Twilio phone number. The webhook URL will direct the call to your FastAPI application, which will handle it accordingly.
## Testing
It is also possible to test the server without making phone calls by using one of these clients.
- [python](client/python/README.md): This Python client enables automated testing of the server via WebSocket without the need to make actual phone calls.
- [typescript](client/typescript/README.md): This typescript client enables manual testing of the server via WebSocket without the need to make actual phone calls.

121
examples/phone-bot-twilio/bot.py Normal file
View File

@@ -0,0 +1,121 @@
#
# Copyright (c) 2024–2025, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
"""Pipecat Twilio Phone Example.
The example runs a simple voice AI bot that you can connect to using a
phone via Twilio.
Required AI services:
- Deepgram (Speech-to-Text)
- OpenAI (LLM)
- Cartesia (Text-to-Speech)
The example connects between client and server using a Twilio websocket
connection.
Run the bot using::
python bot.py -t twilio
"""
import argparse
import os
from dotenv import load_dotenv
from loguru import logger
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIObserver, RTVIProcessor
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.transports.base_transport import BaseTransport
from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
load_dotenv(override=True)
async def run_bot(transport: BaseTransport, _: argparse.Namespace, handle_sigint: bool):
logger.info(f"Starting bot")
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
)
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"))
messages = [
{
"role": "system",
"content": "You are a friendly AI assistant. Respond naturally and keep your answers conversational.",
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
pipeline = Pipeline(
[
transport.input(), # Transport user input
rtvi, # RTVI processor
stt,
context_aggregator.user(), # User responses
llm, # LLM
tts, # TTS
transport.output(), # Transport bot output
context_aggregator.assistant(), # Assistant spoken responses
]
)
task = PipelineTask(
pipeline,
params=PipelineParams(
enable_metrics=True,
enable_usage_metrics=True,
),
observers=[RTVIObserver(rtvi)],
)
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
logger.info(f"Client connected")
# Kick off the conversation.
messages.append({"role": "system", "content": "Say hello and briefly introduce yourself."})
await task.queue_frames([context_aggregator.user().get_context_frame()])
@transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, client):
logger.info(f"Client disconnected")
await task.cancel()
runner = PipelineRunner(handle_sigint=handle_sigint)
await runner.run(task)
if __name__ == "__main__":
from pipecat.runner.local import main
# SmallWebRTCTransport for a P2P WebRTC connection
transport_params = {
"twilio": lambda: FastAPIWebsocketParams(
audio_in_enabled=True,
audio_out_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
}
main(run_bot, transport_params=transport_params)

3
examples/phone-bot-twilio/env.example Normal file
View File

@@ -0,0 +1,3 @@
OPENAI_API_KEY=
DEEPGRAM_API_KEY=
CARTESIA_API_KEY=

1
examples/phone-bot-twilio/requirements.txt Normal file
View File

@@ -0,0 +1 @@
pipecat-ai[cartesia,openai,silero,deepgram,websocket,runner]

7
examples/phone-bot-twilio/templates/streams.xml.template Normal file
View File

@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Connect>
<Stream url="wss://<your server url>/ws"></Stream>
</Connect>
<Pause length="40"/>
</Response>

5
examples/quickstart/README.md
View File

@@ -28,10 +28,9 @@ Have your API keys ready. We'll add them to your `.env` shortly.
1. Set up a virtual environment 1. Set up a virtual environment
From the root directory of the `pipecat` repo, run: From the `examples/quickstart` directory, run:
```bash ```bash
cd examples/quickstart
python -m venv .venv python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate source .venv/bin/activate # On Windows: .venv\Scripts\activate
``` ```
@@ -74,7 +73,7 @@ python bot.py
> Using `uv`? Run your bot using: `uv run bot.py`. > Using `uv`? Run your bot using: `uv run bot.py`.
Connect to your bot in a browser at http://localhost:7860. **Open http://localhost:7860 in your browser** and click `Connect` to start talking to your bot.
> 💡 First run note: The initial startup may take ~10 seconds as Pipecat downloads required models, like the Silero VAD model. > 💡 First run note: The initial startup may take ~10 seconds as Pipecat downloads required models, like the Silero VAD model.