Call Transfer demo (#1348)
* Updated code to dial out to an operator, keep track of operator conversation while escalated and then return to conversation when finished * Removed unnecessary imports * Updated bot runner code, added call routing file and then updated the call transfer and voicemail detection examples * Updated the bot files * Made prompt one level higher in the body and an array * Updated call transfer examples to work correctly * Updated gemini voicemail detection example to work * Added twilio bot support back to the bot_runner * Moved some state management, participant management and other logic to the helper file. * Updated comments * Updated env and requirements file * Ran the examples and made sure code works. Still need to work on the prompts a bit * Fixed format issue * Add support to disable summary in call transfer * Added support for operator transfer mode * Updated readme file * Updated readme based on feedback, and handling of various properties in the json to be more flexible for future examples * Updated number of endpoints * Updated readme to remove fly deployment text and replaced with Pipecat Cloud * Starting to tweak function calls and prompts * Updated examples to more consistently call the functions and say what they need to say * Updated examples * Updated examples * Updated examples to work correctly * Add simple bot versions of dialin and dialout * Refactored the bot runner file to make adding future examples easier * Based on feedback, removed examples for multiple LLMs and also adjusted voicemail detection code to be simpler * Made sure to only capture the users transcription once * Updated readme with latest changes * Forgot to update the order of examples in one place * Fixed formatting issue * Adjusted based on james feedback * Changed default_mode to default_calltransfer_mode
This commit is contained in:
@@ -1,209 +1,602 @@
|
||||
<!-- @format -->
|
||||
|
||||
<div align="center">
|
||||
<img alt="pipecat" width="300px" height="auto" src="image.png">
|
||||
<img alt="pipecat" width="300px" height="auto" src="image.png">
|
||||
</div>
|
||||
|
||||
# Phone Chatbot
|
||||
# Pipecat Phone Chatbot
|
||||
|
||||
Example project that demonstrates how to add phone funtionality to your Pipecat bots. We include examples for Daily (`bot_daily.py`) dial-in and dial-out, and Twilio (`bot_twilio.py`) dial-in, depending on who you want to use as a phone vendor.
|
||||
This repository contains examples for building intelligent phone chatbots using AI for various use cases including:
|
||||
|
||||
- 🔁 Transport: Daily WebRTC
|
||||
- 💬 Speech-to-Text: Deepgram via Daily transport
|
||||
- 🤖 LLM: GPT4-o / OpenAI
|
||||
- 🔉 Text-to-Speech: ElevenLabs
|
||||
- **Simple dial-in**: Basic incoming call handling
|
||||
- **Simple dial-out**: Basic outgoing call handling
|
||||
- **Voicemail detection**: Bot calls a number, detects if it reaches voicemail or a human, and responds appropriately
|
||||
- **Call transfer**: Bot handles initial customer interaction and transfers to a human operator when needed
|
||||
|
||||
#### Should I use Daily or Twilio as a vendor?
|
||||
## Architecture Overview
|
||||
|
||||
If you're starting from scratch, using Daily to provision phone numbers alongside Daily as a transport offers some convenience (such as automatic call forwarding.)
|
||||
These examples use the following components:
|
||||
|
||||
If you already have Twilio numbers and workflows that you want to connect to your Pipecat bots, there is some additional configuration required (you'll need to create a `on_dialin_ready` and use the Twilio client to trigger the forward.)
|
||||
- 🔁 **Transport**: Daily WebRTC
|
||||
- 💬 **Speech-to-Text**: Deepgram via Daily transport
|
||||
- 🤖 **LLMs**: Each example uses a specific LLM (OpenAI GPT-4o or Google Gemini)
|
||||
- 🔉 **Text-to-Speech**: Cartesia
|
||||
|
||||
You can read more about this, as well as see respective walkthroughs in our docs.
|
||||
## Getting Started
|
||||
|
||||
## Setup
|
||||
### Prerequisites
|
||||
|
||||
1. Create and activate a virtual environment:
|
||||
|
||||
```shell
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
```
|
||||
|
||||
2. Install requirements:
|
||||
|
||||
```shell
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
3. Copy env.example to .env and configure:
|
||||
|
||||
3. Set up your environment variables:
|
||||
|
||||
```shell
|
||||
cp env.example .env
|
||||
```
|
||||
4. Install [ngrok](https://ngrok.com/) so your local server can receive requests from Daily's servers.
|
||||
|
||||
## Using Daily numbers
|
||||
Edit the `.env` file to include your API keys.
|
||||
|
||||
### Running the example
|
||||
4. Install [ngrok](https://ngrok.com/) to make your local server accessible to external services.
|
||||
|
||||
To run either the dial-in or dial-out example, follow these steps to get started:
|
||||
### Phone Number Provider: Daily vs Twilio
|
||||
|
||||
1. Run `bot_runner.py` to handle incoming HTTP requests:
|
||||
If you're starting from scratch, we recommend using Daily to provision phone numbers alongside Daily as a transport for simplicity (this provides automatic call forwarding).
|
||||
|
||||
If you already have Twilio numbers and workflows, you can connect them to your Pipecat bots with some additional configuration (`on_dialin_ready` and using the Twilio client to trigger forwarding).
|
||||
|
||||
Most examples in this repository show how to use Daily for dial-in/dial-out operations.
|
||||
|
||||
## Running the Examples
|
||||
|
||||
### 1. Start the Bot Runner Service
|
||||
|
||||
The bot runner handles incoming requests and manages bot processes:
|
||||
|
||||
```shell
|
||||
python bot_runner.py --host localhost
|
||||
```
|
||||
|
||||
### 2. Create a Public Endpoint with ngrok
|
||||
|
||||
Start ngrok to create a public URL for your local server:
|
||||
|
||||
```shell
|
||||
ngrok http --domain yourdomain.ngrok.app 7860
|
||||
```
|
||||
|
||||
## Example 1: Simple Dial-in
|
||||
|
||||
This example demonstrates basic handling of incoming calls without additional features like call transfer.
|
||||
|
||||
### Testing in Daily Prebuilt (No Actual Phone Calls)
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/start" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"simple_dialin": {
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
This returns a Daily room URL where you can test the bot's basic conversation capabilities.
|
||||
|
||||
## Example 2: Simple Dial-out
|
||||
|
||||
This example demonstrates basic handling of outgoing calls without additional features like voicemail detection.
|
||||
|
||||
### Testing in Daily Prebuilt (No Actual Phone Calls)
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/start" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"simple_dialout": {
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
This returns a Daily room URL where you can test the bot's basic conversation capabilities.
|
||||
|
||||
### Making Actual Phone Calls
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/start" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"dialout_settings": [{
|
||||
"phoneNumber": "+12345678910"
|
||||
}],
|
||||
"simple_dialout": {
|
||||
"testInPrebuilt": false
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Example 3: Voicemail Detection
|
||||
|
||||
This example demonstrates a bot that can dial out to a phone number, detect whether it reached a human or voicemail system, and respond appropriately.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. Bot dials a phone number
|
||||
2. Bot listens to determine if it's connected to a person or voicemail
|
||||
3. If it detects voicemail, it leaves a predefined message and hangs up
|
||||
4. If it detects a human, it engages in conversation
|
||||
|
||||
### Testing in Daily Prebuilt (No Actual Phone Calls)
|
||||
|
||||
To test without making actual phone calls:
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/start" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"voicemail_detection": {
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
This will return a Daily room URL you can use to test the bot in the browser.
|
||||
|
||||
### Making Actual Phone Calls
|
||||
|
||||
To have the bot dial out to a real phone number:
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/start" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"dialout_settings": [{
|
||||
"phoneNumber": "+12345678910"
|
||||
}],
|
||||
"voicemail_detection": {
|
||||
"testInPrebuilt": false
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
> **Note:** To enable dial-out capabilities, you must first:
|
||||
>
|
||||
> 1. Contact [help@daily.co](mailto:help@daily.co) to enable dial-out for your domain
|
||||
> 2. Purchase a phone number to dial out from
|
||||
> 3. Ensure rooms have dial-out enabled (the bot runner handles this)
|
||||
> 4. Use an owner token for the bot (also handled by the bot runner)
|
||||
|
||||
## Example 4: Call Transfer
|
||||
|
||||
This example demonstrates a bot that handles initial customer interaction and can transfer the call to a human operator when requested.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. Customer calls in and speaks with the bot
|
||||
2. When the customer asks for a supervisor/manager, the bot initiates a transfer
|
||||
3. The bot dials out to an appropriate operator
|
||||
4. When the operator joins, the bot summarizes the conversation
|
||||
5. The bot remains silent while operator and customer talk
|
||||
6. When the operator leaves, the bot resumes handling the call
|
||||
|
||||
### Testing in Daily Prebuilt (No Actual Phone Calls)
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/start" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"call_transfer": {
|
||||
"mode": "dialout",
|
||||
"speakSummary": true,
|
||||
"storeSummary": false,
|
||||
"operatorNumber": "+12345678910",
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
This returns a Daily room URL. In the room, the expected flow is:
|
||||
|
||||
1. Join the room and speak with the bot
|
||||
2. Ask to speak with a manager/supervisor
|
||||
3. The bot will add the "operator" to the call
|
||||
4. The bot will summarize the conversation and then go silent
|
||||
5. To simulate the operator, you can mute yourself in Daily Prebuilt and speak as if you're the operator
|
||||
6. When finished, have the "operator" leave the call
|
||||
7. The bot will resume speaking and can recall details from the conversation
|
||||
8. End the call by closing Daily Prebuilt or telling the bot you're done
|
||||
|
||||
### Using with Real Phone Calls
|
||||
|
||||
For incoming calls from customers, Daily will send a webhook to your `/start` endpoint. This webhook contains:
|
||||
|
||||
```json
|
||||
{
|
||||
"From": "+CALLERS_PHONE",
|
||||
"To": "$PURCHASED_PHONE",
|
||||
"callId": "callid-read-only-string",
|
||||
"callDomain": "callDomain-read-only-string"
|
||||
}
|
||||
```
|
||||
|
||||
The system will:
|
||||
|
||||
1. Identify the customer based on their phone number
|
||||
2. Determine the appropriate operator to contact
|
||||
3. Customize the bot's behavior based on transfer settings
|
||||
|
||||
#### Operator Assignment
|
||||
|
||||
The `call_connection_manager.py` file contains mappings for:
|
||||
|
||||
1. `CUSTOMER_MAP`: Links phone numbers to customer names
|
||||
2. `OPERATOR_CONTACT_MAP`: Contains operator contact information
|
||||
3. `CUSTOMER_TO_OPERATOR_MAP`: Defines which operators should handle which customers
|
||||
|
||||
You can customize these mappings or integrate with your existing customer database.
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### Request Body Structure
|
||||
|
||||
When making requests to the `/start` endpoint, the config object can include:
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"prompts": [
|
||||
{
|
||||
"name": "call_transfer_initial_prompt",
|
||||
"text": "Your custom prompt here"
|
||||
},
|
||||
{
|
||||
"name": "call_transfer_prompt",
|
||||
"text": "Your custom prompt here"
|
||||
},
|
||||
{
|
||||
"name": "call_transfer_finished_prompt",
|
||||
"text": "Your custom prompt here"
|
||||
},
|
||||
{
|
||||
"name": "voicemail_detection_prompt",
|
||||
"text": "Your custom prompt here"
|
||||
},
|
||||
{
|
||||
"name": "voicemail_prompt",
|
||||
"text": "Your custom prompt here"
|
||||
},
|
||||
{
|
||||
"name": "human_conversation_prompt",
|
||||
"text": "Your custom prompt here"
|
||||
}
|
||||
],
|
||||
"dialin_settings": {
|
||||
"From": "+CALLERS_PHONE",
|
||||
"To": "$PURCHASED_PHONE",
|
||||
"callId": "callid-read-only-string",
|
||||
"callDomain": "callDomain-read-only-string"
|
||||
},
|
||||
"dialout_settings": [
|
||||
{
|
||||
"phoneNumber": "+12345678910",
|
||||
"callerId": "caller-id-uuid",
|
||||
"sipUri": "sip:maria@example.com"
|
||||
}
|
||||
],
|
||||
"call_transfer": {
|
||||
"mode": "dialout",
|
||||
"speakSummary": true,
|
||||
"storeSummary": false,
|
||||
"operatorNumber": "+12345678910",
|
||||
"testInPrebuilt": false
|
||||
},
|
||||
"voicemail_detection": {
|
||||
"testInPrebuilt": true
|
||||
},
|
||||
"simple_dialin": {
|
||||
"testInPrebuilt": true
|
||||
},
|
||||
"simple_dialout": {
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Configuration Parameters
|
||||
|
||||
- `prompts`: An array of objects containing prompts that you want the examples to use.
|
||||
- `dialin_settings`: Information about incoming calls (typically from webhook)
|
||||
- `dialout_settings`: For outbound calls:
|
||||
- `phoneNumber`: Number to dial
|
||||
- `callerId`: UUID of the number to display (optional)
|
||||
- `sipUri`: SIP URI to connect to (alternative to phoneNumber)
|
||||
- `call_transfer`: For call transfer example:
|
||||
- `mode`: Currently only `"dialout"` is supported
|
||||
- `speakSummary`: Whether the bot should summarize the conversation for the operator
|
||||
- `storeSummary`: For future implementation
|
||||
- `operatorNumber`: Operator phone number
|
||||
- `testInPrebuilt`: Test without actual phone calls
|
||||
- `voicemail_detection`: For voicemail detection example:
|
||||
- `testInPrebuilt`: Test without actual phone calls
|
||||
- `simple_dialin`: For simple dialin example:
|
||||
- `testInPrebuilt`: Test without actual phone calls
|
||||
- `simple_dialout`: For simple dialout example:
|
||||
- `testInPrebuilt`: Test without actual phone calls
|
||||
|
||||
## Feature Compatibility
|
||||
|
||||
The following table shows which feature combinations are supported when making requests to the `/start` endpoint. The table is organized by use case to help you create the correct configuration.
|
||||
|
||||
| Use Case | `call_transfer` | `voicemail_detection` | `simple_dialin` | `simple_dialout` | `dialin_settings` | `dialout_settings` | `operatorNumber` | `testInPrebuilt` | Status |
|
||||
| --------------------------------------------------------------- | --------------- | --------------------- | --------------- | ---------------- | ----------------- | ------------------ | ---------------- | ---------------- | ---------------- |
|
||||
| **Basic incoming call handling (simple_dialin)** | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✅ Supported |
|
||||
| **Test mode: Simple dialin in Daily Prebuilt** | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✅ Supported |
|
||||
| **Basic outgoing call handling (simple_dialout)** | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✅ Supported |
|
||||
| **Test mode: Simple dialout in Daily Prebuilt** | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | ✅ Supported |
|
||||
| **Standard call transfer (incoming call)** | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓/✗ | ✗ | ✅ Supported |
|
||||
| **Standard voicemail detection (outgoing call)** | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✅ Supported |
|
||||
| **Test mode: Call transfer in Daily Prebuilt** | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✓ | ✅ Supported |
|
||||
| **Test mode: Voicemail detection in Daily Prebuilt** | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✅ Supported |
|
||||
| Call transfer requires operatorNumber | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓/✗ | ❌ Not Supported |
|
||||
| Voicemail detection requires dialout_settings or testInPrebuilt | ✗ | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓/✗ | ❌ Not Supported |
|
||||
| Cannot combine different bot types | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓/✗ | ❌ Not Supported |
|
||||
| Call_transfer needs dialin_settings in non-test mode | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ❌ Not Supported |
|
||||
| Voicemail_detection needs dialout_settings in non-test mode | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ❌ Not Supported |
|
||||
| Insufficient configuration | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓/✗ | ❌ Not Supported |
|
||||
|
||||
### Legend:
|
||||
|
||||
- ✓: Required
|
||||
- ✗: Not allowed
|
||||
- ✓/✗: Optional
|
||||
- ✅: Supported
|
||||
- ❌: Not Supported
|
||||
|
||||
### Notes:
|
||||
|
||||
- `dialin_settings` is typically populated automatically from webhook data for incoming calls
|
||||
- `dialout_settings` must be specified manually for outgoing calls
|
||||
- `operatorNumber` is specified within the `call_transfer` object (`"call_transfer": {"operatorNumber": "+1234567890", ...}`)
|
||||
- `testInPrebuilt` is specified within the bot type object (e.g., `"call_transfer": {"testInPrebuilt": true, ...}`)
|
||||
- For call transfers, `operatorNumber` must be provided to specify which operator to dial. If it is not provided, we will base it off of the operator map in call_connection_manager.py
|
||||
- In test mode (`testInPrebuilt: true`), some requirements are relaxed to allow testing in Daily Prebuilt
|
||||
- Multiple customers to dial out to can be specified by providing an array of objects in `dialout_settings`
|
||||
- Bot types are mutually exclusive - you cannot combine multiple bot types in a single configuration
|
||||
|
||||
### Configuration Examples
|
||||
|
||||
#### Standard call transfer (incoming call):
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"dialin_settings": {
|
||||
"from": "+12345678901",
|
||||
"to": "+19876543210",
|
||||
"call_id": "call-id-string",
|
||||
"call_domain": "domain-string"
|
||||
},
|
||||
"call_transfer": {
|
||||
"mode": "dialout",
|
||||
"speakSummary": true,
|
||||
"operatorNumber": "+12345678910"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Test mode: Call transfer in Daily Prebuilt:
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"call_transfer": {
|
||||
"mode": "dialout",
|
||||
"speakSummary": true,
|
||||
"operatorNumber": "+12345678910",
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Test mode: Voicemail detection in Daily Prebuilt:
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"voicemail_detection": {
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Standard voicemail detection:
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"dialout_settings": [
|
||||
{
|
||||
"phoneNumber": "+12345678910"
|
||||
}
|
||||
],
|
||||
"voicemail_detection": {
|
||||
"testInPrebuilt": false
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Simple dialin (incoming call):
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"dialin_settings": {
|
||||
"from": "+12345678901",
|
||||
"to": "+19876543210",
|
||||
"call_id": "call-id-string",
|
||||
"call_domain": "domain-string"
|
||||
},
|
||||
"simple_dialin": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Test mode: Simple dialin in Daily Prebuilt:
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"simple_dialin": {
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Simple dialout (outgoing call):
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"dialout_settings": [
|
||||
{
|
||||
"phoneNumber": "+12345678910"
|
||||
}
|
||||
],
|
||||
"simple_dialout": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Test mode: Simple dialout in Daily Prebuilt:
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"simple_dialout": {
|
||||
"testInPrebuilt": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Using Twilio (Alternative)
|
||||
|
||||
To use Twilio for call handling:
|
||||
|
||||
1. Start the bot runner:
|
||||
|
||||
```shell
|
||||
python bot_runner.py --host localhost
|
||||
```
|
||||
|
||||
2. Start ngrok running in a terminal window:
|
||||
2. Start ngrok:
|
||||
|
||||
```shell
|
||||
ngrok http --domain yourdomain.ngrok.app 8000
|
||||
ngrok http --domain yourdomain.ngrok.app 7860
|
||||
```
|
||||
|
||||
3. In a different terminal window, run the Daily bot file:
|
||||
```shell
|
||||
python bot_daily.py
|
||||
```
|
||||
|
||||
### Dial-in
|
||||
|
||||
To dial-in to the bot, you will need to enable dial-in for your Daily domain. Follow [this guide](https://docs.daily.co/guides/products/dial-in-dial-out/dialin-pinless#provisioning-sip-interconnect-and-pinless-dialin-workflow) to set up your domain.
|
||||
|
||||
Note: For the `room_creation_api` property, point at your ngrok hostname: `"room_creation_api": "https://yourdomain.ngrok.app/daily_start_bot"`.
|
||||
|
||||
Once your domain is configured, receiving a phone call at a number associated with your Daily account will result in a POST to the `/daily_start_bot` endpoint, which will start a bot session.
|
||||
|
||||
### Dial-out
|
||||
|
||||
For the bot to dial out to a number, make a POST request to `/daily_start_bot` and include the dial-out phone number in the body of the request as `dialoutNumber`.
|
||||
|
||||
For example:
|
||||
|
||||
```shell
|
||||
curl -X "POST" "http://localhost:7860/daily_start_bot" \
|
||||
-H 'Content-Type: application/json; charset=utf-8' \
|
||||
-d $'{
|
||||
"dialoutNumber": "+12125551234"
|
||||
}'
|
||||
```
|
||||
|
||||
### Voicemail detection
|
||||
|
||||
To start the bot and test voicemail detection, send a POST request to /daily_start_bot with "detectVoicemail": true in the request body.
|
||||
|
||||
- If you only include `"detectVoicemail": true`, the bot will not dial out. Instead, you can test it in Daily Prebuilt by visiting the URL provided in the response.
|
||||
- If you include both `"detectVoicemail": true` and a phone number under `"dialoutNumber"`, the bot will dial out to that number.
|
||||
|
||||
Example: Testing in Daily Prebuilt:
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/daily_start_bot" \ py pipecat
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"detectVoicemail": true}'
|
||||
```
|
||||
|
||||
Example: Testing with Dial-Out:
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/daily_start_bot" \ py pipecat
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"dialoutNumber": "+18057145330", "detectVoicemail": true}'
|
||||
```
|
||||
|
||||
### New! Using Gemini 2.0 Flash Lite with Daily
|
||||
|
||||
We have introduced support for Google's Gemini 2.0 Flash Lite model in this example. This lightweight model offers faster response times and reduced costs while maintaining good conversational capabilities.
|
||||
|
||||
**Quick Start**
|
||||
To use the Gemini-based bot instead of OpenAI:
|
||||
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/daily_gemini_start_bot" \ py pipecat
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"detectVoicemail": true}'
|
||||
```
|
||||
|
||||
All request body parameters supported by /daily_start_bot (such as detectVoicemail, dialoutNumber, etc.) are also compatible with /daily_gemini_start_bot.
|
||||
|
||||
This example uses context switching to help steer the bot in the right direction. As Flash Lite is a smaller model, breaking the prompt down into smaller piece helps to improve the bot's accuracy.
|
||||
|
||||
For example, instead of giving one large prompt like:
|
||||
|
||||
```python
|
||||
system_instruction="""You are a chatbot that needs to detect if you're talking to a voicemail system or human, then either leave a message or have a conversation. If it's voicemail, say "Hello, this is a message..." and hang up. If it's a human, introduce yourself and be helpful until they say goodbye."""
|
||||
```
|
||||
|
||||
We break it into stages:
|
||||
|
||||
First prompt focuses only on detection: "Determine if this is voicemail or human"
|
||||
After detection, we switch to a new context: either "Leave this specific voicemail message" or "Have a conversation with the human".
|
||||
|
||||
**Implementation Details**
|
||||
The implementation is available in bot_daily_gemini.py and features:
|
||||
|
||||
- Staged prompting approach: Breaking down complex tasks into smaller, more focused prompts to improve the lightweight model's performance
|
||||
- Dynamic context switching: The bot can change its behavior in real-time based on what it detects (voicemail vs. human caller)
|
||||
- Function-based architecture: Uses function calling to trigger context switches and call termination
|
||||
|
||||
### More information
|
||||
|
||||
For more configuration options, please consult [Daily's API documentation](https://docs.daily.co).
|
||||
|
||||
## Using Twilio numbers
|
||||
|
||||
### Running the example
|
||||
|
||||
Follow these steps to get started:
|
||||
|
||||
1. Run `bot_runner.py` to handle incoming HTTP requests:
|
||||
|
||||
```shell
|
||||
python bot_runner.py --host localhost
|
||||
```
|
||||
|
||||
2. Start ngrok running in a terminal window:
|
||||
|
||||
```shell
|
||||
ngrok http --domain yourdomain.ngrok.app 8000
|
||||
```
|
||||
|
||||
3. In a different terminal window, run the Daily bot file:
|
||||
3. In another terminal, run the Twilio bot:
|
||||
```shell
|
||||
python bot_twilio.py
|
||||
```
|
||||
|
||||
As above, but target the following URL:
|
||||
Make requests to `/start_twilio_bot` for Twilio-specific functionality.
|
||||
|
||||
`POST /twilio_start_bot`
|
||||
## Deployment
|
||||
|
||||
For more configuration options, please consult Twilio's API documentation.
|
||||
See Pipecat Cloud deployment docs for how to deploy this example: https://docs.pipecat.daily.co/agents/deploy
|
||||
|
||||
## Deployment example
|
||||
We also have a great, easy to use quickstart guide here: https://docs.pipecat.daily.co/quickstart
|
||||
|
||||
A Dockerfile is included in this demo for convenience. Here is an example of how to build and deploy your bot to [fly.io](https://fly.io).
|
||||
## Using Different LLM Providers
|
||||
|
||||
_Please note: This demo spawns agents as subprocesses for convenience / demonstration purposes. You would likely not want to do this in production as it would limit concurrency to available system resources. For more information on how to deploy your bots using VMs, refer to the Pipecat documentation._
|
||||
Each example in this repository is implemented with a specific LLM provider:
|
||||
|
||||
### Build the docker image
|
||||
- **Simple dial-in**: Uses OpenAI
|
||||
- **Simple dial-out**: Uses OpenAI
|
||||
- **Voicemail detection**: Uses Google Gemini
|
||||
- **Call transfer**: Uses OpenAI
|
||||
|
||||
`docker build -t tag:project .`
|
||||
If you want to implement one of these examples with a different LLM provider than what's provided:
|
||||
|
||||
### Launch the fly project
|
||||
- To implement **call_transfer** with **Gemini**, reference the `voicemail_detection.py` file for how to structure LLM context, function calling, and other Gemini-specific implementations.
|
||||
- To implement **voicemail_detection** with **OpenAI**, reference the `call_transfer.py` file for OpenAI-specific implementation details.
|
||||
|
||||
`mv fly.example.toml fly.toml`
|
||||
The key differences between implementations involve how context is managed, function calling syntax, and message formatting. Looking at both implementations side-by-side provides a good template for adapting any example to your preferred LLM provider.
|
||||
|
||||
`fly launch` (using the included fly.toml)
|
||||
## Customizing Bot Prompts
|
||||
|
||||
### Setup your secrets on Fly
|
||||
All examples include default prompts that work well for standard use cases. However, you can customize how the bot behaves by providing your own prompts in the request body.
|
||||
|
||||
Set the necessary secrets (found in `env.example`)
|
||||
### Available Prompt Types
|
||||
|
||||
`fly secrets set DAILY_API_KEY=... OPENAI_API_KEY=... ELEVENLABS_API_KEY=... ELEVENLABS_VOICE_ID=...`
|
||||
- `call_transfer_initial_prompt`: The initial prompt the bot uses when greeting a customer
|
||||
- `call_transfer_prompt`: Instructions for the bot when summarizing the conversation for an operator
|
||||
- `call_transfer_finished_prompt`: Instructions for when the operator leaves the call
|
||||
- `voicemail_detection_prompt`: Instructions for detecting whether a call connected to voicemail
|
||||
- `voicemail_prompt`: The message to leave when voicemail is detected
|
||||
- `human_conversation_prompt`: Instructions for conversation when a human is detected
|
||||
|
||||
If you're using Twilio as a number vendor:
|
||||
### Customization Example
|
||||
|
||||
`fly secrets set TWILIO_ACCOUNT_SID=... TWILIO_AUTH_TOKEN=...`
|
||||
```shell
|
||||
curl -X POST "http://localhost:7860/start" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"config": {
|
||||
"prompts": [
|
||||
{
|
||||
"name": "voicemail_prompt",
|
||||
"text": "Hello, this is ACME Corporation calling. Please call us back at 555-123-4567 regarding your recent order. Thank you!"
|
||||
}
|
||||
],
|
||||
"dialout_settings": [{
|
||||
"phoneNumber": "+12345678910"
|
||||
}],
|
||||
"voicemail_detection": {
|
||||
"testInPrebuilt": false
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### Deploy!
|
||||
This example would use all default prompts except for the voicemail message, which would be replaced with your custom message.
|
||||
|
||||
`fly deploy`
|
||||
### Template Variables
|
||||
|
||||
## Need to do something more advanced?
|
||||
Some prompts support template variables that are automatically replaced:
|
||||
|
||||
This demo covers the basics of bot telephony. If you want to know more about working with PSTN / SIP, please ping us on [Discord](https://discord.gg/pipecat)!
|
||||
- `{customer_name}`: Will be replaced with the customer's name if available
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
For more advanced phone integration scenarios using PSTN/SIP, please reach out on [Discord](https://discord.gg/pipecat).
|
||||
|
||||
Reference in New Issue
Block a user