diff --git a/examples/README.md b/examples/README.md index dfda87856..5fbba1561 100644 --- a/examples/README.md +++ b/examples/README.md @@ -32,14 +32,15 @@ Next, follow the steps in the README for each demo. ## Projects: -| Project | Description | Services | -| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------- | -| [Simple Chatbot](simple-chatbot) | Basic voice-driven conversational bot. A good starting point for learning the flow of the framework. | Deepgram, OpenAI, Daily, Daily Prebuilt UI | -| [Storytelling Chatbot](storytelling-chatbot) | Stitches together multiple third-party services to create a collaborative storytime experience. | Deepgram, ElevenLabs, Open AI, Fal, Daily, Custom UI | -| [Translation Chatbot](translation-chatbot) | Listens for user speech, then translates that speech to Spanish and speaks the translation back. Demonstrates multi-participant use-cases. | Deepgram, Azure, OpenAI, Daily, Daily Prebuilt UI | -| [Moondream Chatbot](moondream-chatbot) | Demonstrates how to add vision capabilities to GPT4. **Note: works best with a GPU** | Deepgram, OpenAI, Moondream, Daily, Daily Prebuilt UI | -| Function-calling Chatbot (TBC) | A chatbot that can call functions in response to user input. | Deepgram, OpenAI, Fireworks, Daily, Daily Prebuilt UI | -| [Dialin Chatbot](dialin-chatbot) | A chatbot that connects to an incoming phone call from Daily or Twilio. | Deepgram, OpenAI, ElevenLabs, Daily, Twilio | +| Project | Description | Services | +|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------| +| [Simple Chatbot](simple-chatbot) | Basic voice-driven conversational bot. A good starting point for learning the flow of the framework. | Deepgram, ElevenLabs, OpenAI, Daily, Daily Prebuilt UI | +| [Storytelling Chatbot](storytelling-chatbot) | Stitches together multiple third-party services to create a collaborative storytime experience. | Deepgram, ElevenLabs, OpenAI, Fal, Daily, Custom UI | +| [Translation Chatbot](translation-chatbot) | Listens for user speech, then translates that speech to Spanish and speaks the translation back. Demonstrates multi-participant use-cases. | Deepgram, Azure, OpenAI, Daily, Daily Prebuilt UI | +| [Moondream Chatbot](moondream-chatbot) | Demonstrates how to add vision capabilities to GPT4. **Note: works best with a GPU** | Deepgram, ElevenLabs, OpenAI, Moondream, Daily, Daily Prebuilt UI | +| [Patient intake](patient-intake) | A chatbot that can call functions in response to user input. | Deepgram, ElevenLabs, OpenAI, Daily, Daily Prebuilt UI | +| [Dialin Chatbot](dialin-chatbot) | A chatbot that connects to an incoming phone call from Daily or Twilio. | Deepgram, ElevenLabs, OpenAI, Daily, Twilio | +| [Twilio Chatbot](twilio-chatbot) | A chatbot that connects to an incoming phone call from Twilio. | Deepgram, ElevenLabs, OpenAI, Daily, Twilio | > [!IMPORTANT] > These example projects use Daily as a WebRTC transport and can be joined using their hosted Prebuilt UI. diff --git a/examples/twilio-chatbot/README.md b/examples/twilio-chatbot/README.md index 66a626ef7..012724b01 100644 --- a/examples/twilio-chatbot/README.md +++ b/examples/twilio-chatbot/README.md @@ -7,6 +7,7 @@ This project is a FastAPI-based chatbot that integrates with Twilio to handle We - [Features](#features) - [Requirements](#requirements) - [Installation](#installation) +- [Configure Twilio URLs](#configure-twilio-urls) - [Running the Application](#running-the-application) - [Usage](#usage) @@ -38,11 +39,19 @@ This project is a FastAPI-based chatbot that integrates with Twilio to handle We ``` 3. **Create .env**: - create .env based on .env.example + create .env based on env.example 4. **Install ngrok**: Follow the instructions on the [ngrok website](https://ngrok.com/download) to download and install ngrok. +## Configure Twilio URLs + +1. **Update the Twilio Webhook**: + Copy the ngrok URL and update your Twilio phone number webhook URL to `http:///start_call`. + +2. **Update the streams.xml**: + Copy the ngrok URL and update templates/streams.xml with `wss:///ws`. + ## Running the Application ### Using Python @@ -57,13 +66,6 @@ This project is a FastAPI-based chatbot that integrates with Twilio to handle We ```sh ngrok http 8765 ``` - -3. **Update the Twilio Webhook**: - Copy the ngrok URL and update your Twilio phone number webhook URL to `http:///start_call`. - -3. **Update the streams.xml**: - Copy the ngrok URL and update your .xml URL to `wss:///ws`. - ### Using Docker 1. **Build the Docker image**: @@ -73,22 +75,8 @@ This project is a FastAPI-based chatbot that integrates with Twilio to handle We 2. **Run the Docker container**: ```sh - docker build -t twilio-chatbot . docker run -it --rm -p 8765:8765 twilio-chatbot ``` - -3. **Start ngrok**: - In a new terminal, start ngrok to tunnel the local server: - ```sh - ngrok http 8765 - ``` - -4. **Update the Twilio Webhook**: - Copy the ngrok URL and update your Twilio phone number webhook URL to `http:///start_call`. - -5. **Update the streams.xml**: - Copy the ngrok URL and update your .xml URL to `wss:///ws`. - ## Usage To start a call, simply make a call to your Twilio phone number. The webhook URL will direct the call to your FastAPI application, which will handle it accordingly. diff --git a/examples/twilio-chatbot/test_bot.py b/examples/twilio-chatbot/bot.py similarity index 90% rename from examples/twilio-chatbot/test_bot.py rename to examples/twilio-chatbot/bot.py index d49bef0aa..8a368f830 100644 --- a/examples/twilio-chatbot/test_bot.py +++ b/examples/twilio-chatbot/bot.py @@ -2,7 +2,7 @@ import aiohttp import os import sys -from pipecat.frames.frames import LLMMessagesFrame, Frame, AudioRawFrame +from pipecat.frames.frames import EndFrame, LLMMessagesFrame from pipecat.pipeline.pipeline import Pipeline from pipecat.pipeline.runner import PipelineRunner from pipecat.pipeline.task import PipelineParams, PipelineTask @@ -10,8 +10,6 @@ from pipecat.processors.aggregators.llm_response import ( LLMAssistantResponseAggregator, LLMUserResponseAggregator ) -from pipecat.processors.frame_processor import FrameProcessor, FrameDirection - from pipecat.services.openai import OpenAILLMService from pipecat.services.deepgram import DeepgramSTTService from pipecat.services.elevenlabs import ElevenLabsTTSService @@ -32,10 +30,8 @@ async def run_bot(websocket_client): transport = FastAPIWebsocketTransport( websocket=websocket_client, params=FastAPIWebsocketParams( - audio_in_enabled=True, audio_out_enabled=True, add_wav_header=False, - transcription_enabled=False, vad_enabled=True, vad_analyzer=SileroVADAnalyzer(), vad_audio_passthrough=True @@ -57,7 +53,7 @@ async def run_bot(websocket_client): messages = [ { "role": "system", - "content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.", + "content": "You are a helpful LLM in an audio call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.", }, ] @@ -83,6 +79,10 @@ async def run_bot(websocket_client): {"role": "system", "content": "Please introduce yourself to the user."}) await task.queue_frames([LLMMessagesFrame(messages)]) + @transport.event_handler("on_client_disconnected") + async def on_client_disconnected(transport, client): + await task.queue_frames([EndFrame()]) + runner = PipelineRunner(handle_sigint=False) await runner.run(task) diff --git a/examples/twilio-chatbot/.env.example b/examples/twilio-chatbot/env.example similarity index 100% rename from examples/twilio-chatbot/.env.example rename to examples/twilio-chatbot/env.example diff --git a/examples/twilio-chatbot/server.py b/examples/twilio-chatbot/server.py index b671d1fd6..025f9dd17 100644 --- a/examples/twilio-chatbot/server.py +++ b/examples/twilio-chatbot/server.py @@ -1,8 +1,10 @@ -from fastapi import FastAPI, WebSocket, WebSocketDisconnect -from fastapi.middleware.cors import CORSMiddleware import uvicorn + +from fastapi import FastAPI, WebSocket +from fastapi.middleware.cors import CORSMiddleware from starlette.responses import HTMLResponse -from test_bot import run_bot + +from bot import run_bot app = FastAPI()