Remove non-quickstart demos
This commit is contained in:
Mark Backman
@@ -1,111 +0,0 @@
|
||||
# Client Server Web Example
|
||||
|
||||
Learn how to build web applications using Pipecat's client/server architecture. This approach separates your bot logic from your user interface, giving you full control over the client experience while maintaining real-time voice communication.
|
||||
|
||||
This example demonstrates:
|
||||
|
||||
- Server-side bot running with Pipecat
|
||||
- React client using [Pipecat's client SDK](https://docs.pipecat.ai/client/introduction)
|
||||
- Real-time voice communication between client and server
|
||||
- UI components from [voice-ui-kit](https://github.com/pipecat-ai/voice-ui-kit) for common voice interface patterns
|
||||
|
||||
This is the recommended architecture for web applications that need custom interfaces or client-side functionality.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+
|
||||
- `npm` installed
|
||||
- AI Service API keys for: [Deepgram](https://console.deepgram.com/signup), [OpenAI](https://auth.openai.com/create-account), and [Cartesia](https://play.cartesia.ai/sign-up)
|
||||
|
||||
## Setup
|
||||
|
||||
This example requires running both a server and client in **two separate terminal windows**.
|
||||
|
||||
### Terminal 1: Server Setup
|
||||
|
||||
1. Set up a virtual environment
|
||||
|
||||
From the `examples/client-server-web` directory, run:
|
||||
|
||||
```bash
|
||||
cd server
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
||||
```
|
||||
|
||||
> Using `uv`? Create your venv using: `uv venv && source .venv/bin/activate`.
|
||||
|
||||
2. Install dependencies
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
> Using `uv`? Install requirements using: `uv pip install -r requirements.txt`.
|
||||
|
||||
3. Configure environment variables
|
||||
|
||||
Create a `.env` file:
|
||||
|
||||
```bash
|
||||
cp env.example .env
|
||||
```
|
||||
|
||||
Then, add your API keys:
|
||||
|
||||
```
|
||||
DEEPGRAM_API_KEY=your_deepgram_api_key
|
||||
OPENAI_API_KEY=your_openai_api_key
|
||||
CARTESIA_API_KEY=your_cartesia_api_key
|
||||
```
|
||||
|
||||
4. Run the example
|
||||
|
||||
Run your bot using:
|
||||
|
||||
```bash
|
||||
python bot.py
|
||||
```
|
||||
|
||||
> Using `uv`? Run your bot using: `uv run bot.py`.
|
||||
|
||||
> 💡 First run note: The initial startup may take ~10 seconds as Pipecat downloads required models, like the Silero VAD model.
|
||||
|
||||
### Terminal 2: Client Setup
|
||||
|
||||
1. Open a new terminal window and navigate to the `client` folder:
|
||||
|
||||
From the `examples/client-server-web` directory, run:
|
||||
|
||||
```bash
|
||||
cd client
|
||||
```
|
||||
|
||||
2. Install dependencies:
|
||||
|
||||
```bash
|
||||
npm i
|
||||
```
|
||||
|
||||
3. Run the client:
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
4. **Open http://localhost:5173 in your browser** and click `Connect` to start talking to your bot.
|
||||
|
||||
> 💡 **Tip**: Check your server terminal for debug logs showing Pipecat's internal workings.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Browser permissions**: Make sure to allow microphone access when prompted by your browser.
|
||||
- **Connection issues**: If the WebRTC connection fails, first try a different browser. If that fails, make sure you don't have a VPN or firewall rules blocking traffic. WebRTC uses UDP to communicate.
|
||||
- **Audio issues**: Check that your microphone and speakers are working and not muted.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **Explore the client SDK**: Learn more about [Pipecat's client SDKs](https://docs.pipecat.ai/client/introduction) for web, mobile, and other platforms
|
||||
- **Learn about the voice-ui-kit**: Explore [voice-ui-kit](https://github.com/pipecat-ai/voice-ui-kit) to simplify your front end development
|
||||
- **Advanced examples**: Check out [pipecat-examples](https://github.com/pipecat-ai/pipecat-examples) for more complex client/server applications
|
||||
- **Join Discord**: Connect with other developers on [Discord](https://discord.gg/pipecat)
|
||||
24
examples/client-server-web/client/.gitignore
vendored
24
examples/client-server-web/client/.gitignore
vendored
@@ -1,24 +0,0 @@
|
||||
# Logs
|
||||
logs
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
lerna-debug.log*
|
||||
|
||||
node_modules
|
||||
dist
|
||||
dist-ssr
|
||||
*.local
|
||||
|
||||
# Editor directories and files
|
||||
.vscode/*
|
||||
!.vscode/extensions.json
|
||||
.idea
|
||||
.DS_Store
|
||||
*.suo
|
||||
*.ntvs*
|
||||
*.njsproj
|
||||
*.sln
|
||||
*.sw?
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"hash": "08bb9189",
|
||||
"configHash": "9d0f4652",
|
||||
"lockfileHash": "24c7a346",
|
||||
"browserHash": "f819d9a2",
|
||||
"optimized": {},
|
||||
"chunks": {}
|
||||
}
|
||||
@@ -1,3 +0,0 @@
|
||||
{
|
||||
"type": "module"
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Pipecat UI</title>
|
||||
<link rel="icon" href="/favicon.svg" type="image/svg+xml">
|
||||
<link rel="stylesheet" href="/src/style.css" />
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="/src/index.tsx"></script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
2635
examples/client-server-web/client/package-lock.json
generated
2635
examples/client-server-web/client/package-lock.json
generated
File diff suppressed because it is too large
Load Diff
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"name": "client",
|
||||
"version": "1.0.0",
|
||||
"main": "index.js",
|
||||
"scripts": {
|
||||
"dev": "node_modules/.bin/vite",
|
||||
"build": "node_modules/.bin/tsc && vite build",
|
||||
"preview": "node_modules/.bin/vite preview"
|
||||
},
|
||||
"keywords": [],
|
||||
"author": "",
|
||||
"license": "ISC",
|
||||
"description": "",
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.14.0",
|
||||
"@types/react": "^19.1.8",
|
||||
"@types/react-dom": "^19.1.6",
|
||||
"@vitejs/plugin-react-swc": "^3.8.1",
|
||||
"typescript": "^5.8.3",
|
||||
"vite": "^6.2.5"
|
||||
},
|
||||
"dependencies": {
|
||||
"@fontsource-variable/geist": "^5.2.6",
|
||||
"@fontsource-variable/geist-mono": "^5.2.6",
|
||||
"@pipecat-ai/client-js": "^1.0.1",
|
||||
"@pipecat-ai/client-react": "^1.0.1",
|
||||
"@pipecat-ai/small-webrtc-transport": "^1.0.0",
|
||||
"@pipecat-ai/daily-transport": "^1.0.0",
|
||||
"@pipecat-ai/voice-ui-kit": "^0.1.0",
|
||||
"react": "^19.1.0",
|
||||
"react-dom": "^19.1.0"
|
||||
}
|
||||
}
|
||||
@@ -1,7 +0,0 @@
|
||||
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M3.3088 5.05615C3.64682 4.92779 4.02833 5.02411 4.26653 5.29797L7.36884 8.86461H16.6312L19.7335 5.29797C19.9717 5.02411 20.3532 4.92779 20.6912 5.05615C21.0292 5.18452 21.253 5.51072 21.253 5.87504V13.75H24V15.5H19.5181V8.19909L17.6762 10.3167C17.5115 10.506 17.2738 10.6146 17.0241 10.6146H6.9759C6.72616 10.6146 6.48854 10.506 6.32383 10.3167L4.48193 8.19909V15.5H0V13.75H2.74699V5.87504C2.74699 5.51072 2.97078 5.18452 3.3088 5.05615Z" fill="black"/>
|
||||
<path d="M19.5181 17.25H24V19H19.5181V17.25Z" fill="black"/>
|
||||
<path d="M0 17.25H4.48193V19H0V17.25Z" fill="black"/>
|
||||
<path d="M9.25301 14.3333C9.25301 14.9777 8.73517 15.5 8.09639 15.5C7.4576 15.5 6.93976 14.9777 6.93976 14.3333C6.93976 13.689 7.4576 13.1667 8.09639 13.1667C8.73517 13.1667 9.25301 13.689 9.25301 14.3333Z" fill="black"/>
|
||||
<path d="M17.0602 14.3333C17.0602 14.9777 16.5424 15.5 15.9036 15.5C15.2648 15.5 14.747 14.9777 14.747 14.3333C14.747 13.689 15.2648 13.1667 15.9036 13.1667C16.5424 13.1667 17.0602 13.689 17.0602 14.3333Z" fill="black"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 1.1 KiB |
@@ -1,7 +0,0 @@
|
||||
<svg width="332" height="192" viewBox="0 0 332 192" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M45.7718 0.769635C50.4477 -0.990844 55.7252 0.330188 59.0204 4.08595L101.936 53H230.064L272.98 4.08595C276.275 0.330188 281.552 -0.990844 286.228 0.769635C290.904 2.53011 294 7.00367 294 12V120H332V144H270V43.8728L244.52 72.9141C242.242 75.5111 238.955 77 235.5 77H96.5C93.0452 77 89.7581 75.5111 87.4796 72.9141L62 43.8728V144H0V120H38V12C38 7.00367 41.0958 2.53011 45.7718 0.769635Z" fill="black"/>
|
||||
<path d="M270 168H332V192H270V168Z" fill="black"/>
|
||||
<path d="M0 168H62V192H0V168Z" fill="black"/>
|
||||
<path d="M128 128C128 136.837 120.837 144 112 144C103.163 144 96 136.837 96 128C96 119.164 103.163 112 112 112C120.837 112 128 119.164 128 128Z" fill="black"/>
|
||||
<path d="M236 128C236 136.837 228.837 144 220 144C211.163 144 204 136.837 204 128C204 119.164 211.163 112 220 112C228.837 112 236 119.164 236 128Z" fill="black"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 937 B |
@@ -1,28 +0,0 @@
|
||||
import {
|
||||
ConsoleTemplate,
|
||||
FullScreenContainer,
|
||||
ThemeProvider,
|
||||
} from '@pipecat-ai/voice-ui-kit';
|
||||
import { StrictMode } from 'react';
|
||||
import { createRoot } from 'react-dom/client';
|
||||
|
||||
//@ts-ignore - fontsource-variable/geist is not typed
|
||||
import '@fontsource-variable/geist';
|
||||
//@ts-ignore - fontsource-variable/geist is not typed
|
||||
import '@fontsource-variable/geist-mono';
|
||||
|
||||
createRoot(document.getElementById('root')!).render(
|
||||
// @ts-ignore
|
||||
<StrictMode>
|
||||
<ThemeProvider>
|
||||
<FullScreenContainer>
|
||||
<ConsoleTemplate
|
||||
transportType="smallwebrtc"
|
||||
connectParams={{
|
||||
connectionUrl: '/api/offer',
|
||||
}}
|
||||
/>
|
||||
</FullScreenContainer>
|
||||
</ThemeProvider>
|
||||
</StrictMode>
|
||||
);
|
||||
@@ -1,7 +0,0 @@
|
||||
@import "@pipecat-ai/voice-ui-kit/styles.css";
|
||||
|
||||
html,
|
||||
body {
|
||||
height: 100%;
|
||||
margin: 0;
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2016",
|
||||
"module": "ESNext",
|
||||
"lib": ["ES2016", "DOM", "DOM.Iterable"],
|
||||
"types": ["node"],
|
||||
"skipLibCheck": true,
|
||||
"jsx": "preserve",
|
||||
|
||||
/* Bundler mode */
|
||||
"moduleResolution": "bundler",
|
||||
"allowImportingTsExtensions": true,
|
||||
"allowJs": true,
|
||||
"noEmit": true,
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"moduleDetection": "force",
|
||||
"esModuleInterop": true,
|
||||
|
||||
/* Linting */
|
||||
"strict": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": false,
|
||||
"noFallthroughCasesInSwitch": true
|
||||
},
|
||||
"include": ["src"]
|
||||
}
|
||||
@@ -1,18 +0,0 @@
|
||||
import { defineConfig } from "vite";
|
||||
import react from "@vitejs/plugin-react-swc";
|
||||
|
||||
export default defineConfig({
|
||||
base: "./", //Use relative paths so it works at any mount path
|
||||
plugins: [react()],
|
||||
publicDir: "public",
|
||||
server: {
|
||||
allowedHosts: true, // Allows external connections like ngrok
|
||||
proxy: {
|
||||
// Proxy /api requests to the backend server
|
||||
"/api": {
|
||||
target: "http://0.0.0.0:7860", // Replace with your backend URL
|
||||
changeOrigin: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
@@ -1,127 +0,0 @@
|
||||
#
|
||||
# Copyright (c) 2024–2025, Daily
|
||||
#
|
||||
# SPDX-License-Identifier: BSD 2-Clause License
|
||||
#
|
||||
|
||||
"""Client-Server Web Example.
|
||||
|
||||
This is the server-side bot implementation for the Pipecat client-server
|
||||
web example. It runs a simple voice AI bot that you can connect to using a
|
||||
web browser and speak with it.
|
||||
|
||||
Required AI services:
|
||||
- Deepgram (Speech-to-Text)
|
||||
- OpenAI (LLM)
|
||||
- Cartesia (Text-to-Speech)
|
||||
|
||||
The example connects between client and server using a P2P WebRTC connection.
|
||||
|
||||
Run the bot using::
|
||||
|
||||
python bot.py
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from loguru import logger
|
||||
|
||||
from pipecat.audio.vad.silero import SileroVADAnalyzer
|
||||
from pipecat.pipeline.pipeline import Pipeline
|
||||
from pipecat.pipeline.runner import PipelineRunner
|
||||
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
||||
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
|
||||
from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIObserver, RTVIProcessor
|
||||
from pipecat.runner.types import RunnerArguments
|
||||
from pipecat.services.cartesia.tts import CartesiaTTSService
|
||||
from pipecat.services.deepgram.stt import DeepgramSTTService
|
||||
from pipecat.services.openai.llm import OpenAILLMService
|
||||
from pipecat.transports.base_transport import BaseTransport, TransportParams
|
||||
from pipecat.transports.network.small_webrtc import SmallWebRTCTransport
|
||||
|
||||
load_dotenv(override=True)
|
||||
|
||||
|
||||
async def run_bot(transport: BaseTransport):
|
||||
logger.info(f"Starting bot")
|
||||
|
||||
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
|
||||
|
||||
tts = CartesiaTTSService(
|
||||
api_key=os.getenv("CARTESIA_API_KEY"),
|
||||
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
|
||||
)
|
||||
|
||||
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"))
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a friendly AI assistant. Respond naturally and keep your answers conversational.",
|
||||
},
|
||||
]
|
||||
|
||||
context = OpenAILLMContext(messages)
|
||||
context_aggregator = llm.create_context_aggregator(context)
|
||||
|
||||
rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
|
||||
|
||||
pipeline = Pipeline(
|
||||
[
|
||||
transport.input(), # Transport user input
|
||||
rtvi, # RTVI processor
|
||||
stt,
|
||||
context_aggregator.user(), # User responses
|
||||
llm, # LLM
|
||||
tts, # TTS
|
||||
transport.output(), # Transport bot output
|
||||
context_aggregator.assistant(), # Assistant spoken responses
|
||||
]
|
||||
)
|
||||
|
||||
task = PipelineTask(
|
||||
pipeline,
|
||||
params=PipelineParams(
|
||||
enable_metrics=True,
|
||||
enable_usage_metrics=True,
|
||||
),
|
||||
observers=[RTVIObserver(rtvi)],
|
||||
)
|
||||
|
||||
@transport.event_handler("on_client_connected")
|
||||
async def on_client_connected(transport, client):
|
||||
logger.info(f"Client connected")
|
||||
# Kick off the conversation.
|
||||
messages.append({"role": "system", "content": "Say hello and briefly introduce yourself."})
|
||||
await task.queue_frames([context_aggregator.user().get_context_frame()])
|
||||
|
||||
@transport.event_handler("on_client_disconnected")
|
||||
async def on_client_disconnected(transport, client):
|
||||
logger.info(f"Client disconnected")
|
||||
await task.cancel()
|
||||
|
||||
runner = PipelineRunner(handle_sigint=False)
|
||||
|
||||
await runner.run(task)
|
||||
|
||||
|
||||
async def bot(runner_args: RunnerArguments):
|
||||
"""Main bot entry point for the bot starter."""
|
||||
|
||||
transport = SmallWebRTCTransport(
|
||||
params=TransportParams(
|
||||
audio_in_enabled=True,
|
||||
audio_out_enabled=True,
|
||||
vad_analyzer=SileroVADAnalyzer(),
|
||||
),
|
||||
webrtc_connection=runner_args.webrtc_connection,
|
||||
)
|
||||
|
||||
await run_bot(transport)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
from pipecat.runner.run import main
|
||||
|
||||
main()
|
||||
@@ -1,3 +0,0 @@
|
||||
DEEPGRAM_API_KEY=your_deepgram_api_key
|
||||
OPENAI_API_KEY=your_openai_api_key
|
||||
CARTESIA_API_KEY=your_cartesia_api_key
|
||||
@@ -1 +0,0 @@
|
||||
pipecat-ai[webrtc,silero,deepgram,openai,cartesia,runner]>=0.0.77
|
||||
164
examples/phone-bot-twilio/.gitignore
vendored
164
examples/phone-bot-twilio/.gitignore
vendored
@@ -1,164 +0,0 @@
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
share/python-wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
|
||||
# PyInstaller
|
||||
# Usually these files are written by a python script from a template
|
||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
||||
*.manifest
|
||||
*.spec
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
.coverage
|
||||
.coverage.*
|
||||
.cache
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
*.cover
|
||||
*.py,cover
|
||||
.hypothesis/
|
||||
.pytest_cache/
|
||||
cover/
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Django stuff:
|
||||
*.log
|
||||
local_settings.py
|
||||
db.sqlite3
|
||||
db.sqlite3-journal
|
||||
|
||||
# Flask stuff:
|
||||
instance/
|
||||
.webassets-cache
|
||||
|
||||
# Scrapy stuff:
|
||||
.scrapy
|
||||
|
||||
# Sphinx documentation
|
||||
docs/_build/
|
||||
|
||||
# PyBuilder
|
||||
.pybuilder/
|
||||
target/
|
||||
|
||||
# Jupyter Notebook
|
||||
.ipynb_checkpoints
|
||||
|
||||
# IPython
|
||||
profile_default/
|
||||
ipython_config.py
|
||||
|
||||
# pyenv
|
||||
# For a library or package, you might want to ignore these files since the code is
|
||||
# intended to run in multiple environments; otherwise, check them in:
|
||||
# .python-version
|
||||
|
||||
# pipenv
|
||||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
||||
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
||||
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
||||
# install all needed dependencies.
|
||||
#Pipfile.lock
|
||||
|
||||
# poetry
|
||||
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
|
||||
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
||||
# commonly ignored for libraries.
|
||||
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
|
||||
#poetry.lock
|
||||
|
||||
# pdm
|
||||
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
|
||||
#pdm.lock
|
||||
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
|
||||
# in version control.
|
||||
# https://pdm.fming.dev/#use-with-ide
|
||||
.pdm.toml
|
||||
|
||||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
|
||||
__pypackages__/
|
||||
|
||||
# Celery stuff
|
||||
celerybeat-schedule
|
||||
celerybeat.pid
|
||||
|
||||
# SageMath parsed files
|
||||
*.sage.py
|
||||
|
||||
# Environments
|
||||
.env
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# Spyder project settings
|
||||
.spyderproject
|
||||
.spyproject
|
||||
|
||||
# Rope project settings
|
||||
.ropeproject
|
||||
|
||||
# mkdocs documentation
|
||||
/site
|
||||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
.dmypy.json
|
||||
dmypy.json
|
||||
|
||||
# Pyre type checker
|
||||
.pyre/
|
||||
|
||||
# pytype static type analyzer
|
||||
.pytype/
|
||||
|
||||
# Cython debug symbols
|
||||
cython_debug/
|
||||
|
||||
# PyCharm
|
||||
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
|
||||
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
|
||||
# and can be added to the global gitignore or merged into this file. For a more nuclear
|
||||
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
||||
#.idea/
|
||||
runpod.toml
|
||||
|
||||
# Examples
|
||||
templates/streams.xml
|
||||
@@ -1,116 +0,0 @@
|
||||
# Phone Bot Twilio
|
||||
|
||||
Learn how to connect your Pipecat bot to a phone number so users can call and have voice conversations. This example shows the complete setup for telephone-based AI interactions using Twilio's telephony services. At the end, you'll be able to talk to your bot on the phone.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+
|
||||
- [ngrok](https://ngrok.com/docs/getting-started/) (for tunneling)
|
||||
- [Twilio Account](https://www.twilio.com/login) and [phone number](https://help.twilio.com/articles/223135247-How-to-Search-for-and-Buy-a-Twilio-Phone-Number-from-Console)
|
||||
- AI Service API keys for: [Deepgram](https://console.deepgram.com/signup), [OpenAI](https://auth.openai.com/create-account), and [Cartesia](https://play.cartesia.ai/sign-up)
|
||||
|
||||
## Setup
|
||||
|
||||
This example requires running both a server and ngrok tunnel in **two separate terminal windows**.
|
||||
|
||||
### Terminal 1: Start ngrok and Configure Twilio
|
||||
|
||||
1. Start ngrok:
|
||||
In a new terminal, start ngrok to tunnel the local server:
|
||||
|
||||
```bash
|
||||
ngrok http 7860
|
||||
```
|
||||
|
||||
> Want a fixed ngrok URL? Use the `--subdomain` flag:
|
||||
> `ngrok http --subdomain=your_ngrok_name 7860`
|
||||
|
||||
2. Update the Twilio Webhook:
|
||||
|
||||
- Go to your Twilio phone number's configuration page
|
||||
- Under "Voice Configuration", in the "A call comes in" section:
|
||||
- Select "Webhook" from the dropdown
|
||||
- Enter your ngrok URL: `https://your-ngrok-url.ngrok.io`
|
||||
- Ensure "HTTP POST" is selected
|
||||
- Click Save at the bottom of the page
|
||||
|
||||
3. Configure streams.xml:
|
||||
- Copy the template file to create your local version:
|
||||
```bash
|
||||
cp templates/streams.xml.template templates/streams.xml
|
||||
```
|
||||
- In `templates/streams.xml`, replace `<your_server_url>` with your ngrok URL (without `https://`)
|
||||
- The final URL should look like: `wss://abc123.ngrok.io/ws`
|
||||
|
||||
### Terminal 2: Server Setup
|
||||
|
||||
1. Set up a virtual environment
|
||||
|
||||
From the `examples/phone-bot-twilio` directory, run:
|
||||
|
||||
```bash
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
||||
```
|
||||
|
||||
> Using `uv`? Create your venv using: `uv venv && source .venv/bin/activate`.
|
||||
|
||||
2. Install dependencies
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
> Using `uv`? Install requirements using: `uv pip install -r requirements.txt`.
|
||||
|
||||
3. Configure environment variables
|
||||
|
||||
Create a `.env` file:
|
||||
|
||||
```bash
|
||||
cp env.example .env
|
||||
```
|
||||
|
||||
Then, add your API keys:
|
||||
|
||||
```
|
||||
DEEPGRAM_API_KEY=your_deepgram_api_key
|
||||
OPENAI_API_KEY=your_openai_api_key
|
||||
CARTESIA_API_KEY=your_cartesia_api_key
|
||||
```
|
||||
|
||||
> Optional: Add your `TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN` to enable auto-hangup.
|
||||
|
||||
4. Run the Application
|
||||
|
||||
```bash
|
||||
python server.py
|
||||
```
|
||||
|
||||
### Test Your Phone Bot
|
||||
|
||||
**Call your Twilio phone number** to start talking with your AI bot! 🚀
|
||||
|
||||
> 💡 **Tip**: Check your server terminal for debug logs showing Pipecat's internal workings.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Call doesn't connect**: Verify your ngrok URL is correctly set in both Twilio webhook and `streams.xml`
|
||||
- **No audio or bot doesn't respond**: Check that all API keys are correctly set in your `.env` file
|
||||
- **Webhook errors**: Ensure your server is running and ngrok tunnel is active before making calls
|
||||
- **ngrok tunnel issues**: Free ngrok URLs change each restart - remember to update both Twilio and `streams.xml`
|
||||
|
||||
## Understanding the Call Flow
|
||||
|
||||
1. **Incoming Call**: User dials your Twilio number
|
||||
2. **Webhook**: Twilio sends call data to your ngrok URL
|
||||
3. **WebSocket**: Your server establishes real-time audio connection via Websocket and exchanges Media Streams with Twilio
|
||||
4. **Processing**: Audio flows through your Pipecat Pipeline
|
||||
5. **Response**: Synthesized speech streams back to caller
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **Deploy to production**: Replace ngrok with a proper server deployment
|
||||
- **Explore other telephony providers**: Try [Telnyx](https://github.com/pipecat-ai/pipecat-examples/tree/main/telnyx-chatbot) or [Plivo](https://github.com/pipecat-ai/pipecat-examples/tree/main/plivo-chatbot) examples
|
||||
- **Advanced telephony features**: Check out [pipecat-examples](https://github.com/pipecat-ai/pipecat-examples) for call recording, transfer, and more
|
||||
- **Join Discord**: Connect with other developers on [Discord](https://discord.gg/pipecat)
|
||||
@@ -1,144 +0,0 @@
|
||||
#
|
||||
# Copyright (c) 2024–2025, Daily
|
||||
#
|
||||
# SPDX-License-Identifier: BSD 2-Clause License
|
||||
#
|
||||
|
||||
"""Pipecat Twilio Phone Example.
|
||||
|
||||
The example runs a simple voice AI bot that you can connect to using a
|
||||
phone via Twilio.
|
||||
|
||||
Required AI services:
|
||||
- Deepgram (Speech-to-Text)
|
||||
- OpenAI (LLM)
|
||||
- Cartesia (Text-to-Speech)
|
||||
|
||||
The example connects between client and server using a Twilio websocket
|
||||
connection.
|
||||
|
||||
Run the bot using::
|
||||
|
||||
python bot.py --transport twilio --proxy your_ngrok.ngrok.io
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from loguru import logger
|
||||
|
||||
from pipecat.audio.vad.silero import SileroVADAnalyzer
|
||||
from pipecat.pipeline.pipeline import Pipeline
|
||||
from pipecat.pipeline.runner import PipelineRunner
|
||||
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
||||
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
|
||||
from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIObserver, RTVIProcessor
|
||||
from pipecat.runner.types import RunnerArguments
|
||||
from pipecat.runner.utils import parse_telephony_websocket
|
||||
from pipecat.serializers.twilio import TwilioFrameSerializer
|
||||
from pipecat.services.cartesia.tts import CartesiaTTSService
|
||||
from pipecat.services.deepgram.stt import DeepgramSTTService
|
||||
from pipecat.services.openai.llm import OpenAILLMService
|
||||
from pipecat.transports.base_transport import BaseTransport
|
||||
from pipecat.transports.network.fastapi_websocket import (
|
||||
FastAPIWebsocketParams,
|
||||
FastAPIWebsocketTransport,
|
||||
)
|
||||
|
||||
load_dotenv(override=True)
|
||||
|
||||
|
||||
async def run_bot(transport: BaseTransport):
|
||||
logger.info(f"Starting bot")
|
||||
|
||||
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
|
||||
|
||||
tts = CartesiaTTSService(
|
||||
api_key=os.getenv("CARTESIA_API_KEY"),
|
||||
voice_id="71a7ad14-091c-4e8e-a314-022ece01c121", # British Reading Lady
|
||||
)
|
||||
|
||||
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"))
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a friendly AI assistant. Respond naturally and keep your answers conversational.",
|
||||
},
|
||||
]
|
||||
|
||||
context = OpenAILLMContext(messages)
|
||||
context_aggregator = llm.create_context_aggregator(context)
|
||||
|
||||
rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
|
||||
|
||||
pipeline = Pipeline(
|
||||
[
|
||||
transport.input(), # Transport user input
|
||||
rtvi, # RTVI processor
|
||||
stt,
|
||||
context_aggregator.user(), # User responses
|
||||
llm, # LLM
|
||||
tts, # TTS
|
||||
transport.output(), # Transport bot output
|
||||
context_aggregator.assistant(), # Assistant spoken responses
|
||||
]
|
||||
)
|
||||
|
||||
task = PipelineTask(
|
||||
pipeline,
|
||||
params=PipelineParams(
|
||||
enable_metrics=True,
|
||||
enable_usage_metrics=True,
|
||||
),
|
||||
observers=[RTVIObserver(rtvi)],
|
||||
)
|
||||
|
||||
@transport.event_handler("on_client_connected")
|
||||
async def on_client_connected(transport, client):
|
||||
logger.info(f"Client connected")
|
||||
# Kick off the conversation.
|
||||
messages.append({"role": "system", "content": "Say hello and briefly introduce yourself."})
|
||||
await task.queue_frames([context_aggregator.user().get_context_frame()])
|
||||
|
||||
@transport.event_handler("on_client_disconnected")
|
||||
async def on_client_disconnected(transport, client):
|
||||
logger.info(f"Client disconnected")
|
||||
await task.cancel()
|
||||
|
||||
runner = PipelineRunner(handle_sigint=False)
|
||||
|
||||
await runner.run(task)
|
||||
|
||||
|
||||
async def bot(runner_args: RunnerArguments):
|
||||
"""Main bot entry point for the bot starter."""
|
||||
|
||||
transport_type, call_data = await parse_telephony_websocket(runner_args.websocket)
|
||||
logger.info(f"Auto-detected transport: {transport_type}")
|
||||
|
||||
serializer = TwilioFrameSerializer(
|
||||
stream_sid=call_data["stream_id"],
|
||||
call_sid=call_data["call_id"],
|
||||
account_sid=os.getenv("TWILIO_ACCOUNT_SID", ""),
|
||||
auth_token=os.getenv("TWILIO_AUTH_TOKEN", ""),
|
||||
)
|
||||
|
||||
transport = FastAPIWebsocketTransport(
|
||||
websocket=runner_args.websocket,
|
||||
params=FastAPIWebsocketParams(
|
||||
audio_in_enabled=True,
|
||||
audio_out_enabled=True,
|
||||
add_wav_header=False,
|
||||
vad_analyzer=SileroVADAnalyzer(),
|
||||
serializer=serializer,
|
||||
),
|
||||
)
|
||||
|
||||
await run_bot(transport)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
from pipecat.runner.run import main
|
||||
|
||||
main()
|
||||
@@ -1,3 +0,0 @@
|
||||
OPENAI_API_KEY=
|
||||
DEEPGRAM_API_KEY=
|
||||
CARTESIA_API_KEY=
|
||||
@@ -1 +0,0 @@
|
||||
pipecat-ai[cartesia,openai,silero,deepgram,websocket,runner]>=0.0.77
|
||||
@@ -1,7 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<Response>
|
||||
<Connect>
|
||||
<Stream url="wss://<your_server_url>/ws"></Stream>
|
||||
</Connect>
|
||||
<Pause length="40"/>
|
||||
</Response>
|
||||
Reference in New Issue
Block a user