Smart Turn Detection Demo

This demo showcases Pipecat's Smart Turn Detection feature - an advanced conversational turn detection system that uses machine learning to identify when a speaker has finished their turn in a conversation. Unlike basic Voice Activity Detection (VAD) which only detects speech vs. silence, Smart Turn detects natural conversational cues like intonation patterns, pacing, and linguistic signals.

This demo uses the pipecat-ai/smart-turn model - an open-source, community-driven conversational turn detection model designed to provide more natural turn-taking in voice interactions. The model is being hosted on Fal's infrastructure for GPU acceleration, offering inference times between 50-60ms.

In the client UI, you can see the transcription messages along with the smart-turn model's prediction results in real-time.

Try the demo

Try the hosted version of the demo here: https://pcc-smart-turn.vercel.app/.

Run the demo locally

Run the Server

Set up and activate your virtual environment:

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Install dependencies:
```
pip install -r requirements.txt
```
Create your .env file and set your env vars:
```
cp env.example .env
```
Keys to provide:
- GOOGLE_API_KEY
- CARTESIA_API_KEY
- DEEPGRAM_API_KEY
- DAILY_API_KEY
- FAL_SMART_TURN_API_KEY
Run the server:
```
LOCAL=1 python server.py
```

Run the client

Open a new terminal and navigate to the client directory:
```
cd client
```
Install dependencies:
```
npm install
```
Create your .env.local file:
```
cp env.local.example .env.local
```
Note: No keys need to be modified. NEXT_PUBLIC_API_BASE_URL is already configured for local use.
Start the development server:
```
npm run dev
```
Open http://localhost:3000 in your browser.

Deploy the app

Deploy the server to Pipecat Cloud

Navigate to server
```
cd server
```
You should already have a .env set up from running locally. If not, do that now.
Update your build and deploy scripts.
- In build.sh, set DOCKER_USERNAME and AGENT_NAME.
- In pcc-deploy.toml, set image, which specifies where your Docker image is stored.
Build your Docker image by running the build script:
```
./build.sh
```
Note: This builds, tags and pushes your docker image and assumes Docker Hub is the container registry.
Make sure you have the Pipecat Cloud CLI installed:
```
pip install pipecatcloud
```
Login via the Pipecat Cloud CLI:
```
pcc auth login
```
Note: If you don't have an account, sign up at https://pipecat.daily.co.

Add a secrets set:

pcc secrets set pcc-smart-turn-secrets --file .env

Deploy your agent:
```
pcc deploy
```
Note: This uses your pcc-deploy.toml settings. Modify as needed.

Deploy the client to Vercel

This project uses TypeScript, React, and Next.js, making it a perfect fit for Vercel.

In your client directory, install Vercel's CLI tool: npm install -g vercel
Verify it's installed using vercel --version
Log in your Vercel account using vercel login
Deploy your client to Vercel using vercel

Follow the vercel prompts to deploy your project.

Test your deployed app

Now with the client and server deployed, you can join the call using your Vercel URL.

See the debug information for the Smart Turn data. It prints a log line for each smart-turn inference:

Smart Turn: COMPLETE, Probability: 95.3%, Model inference: 65.23ms, Server processing: 82.09ms, End-to-end: 245.43ms

3.8 KiB Raw Blame History