Compare commits
101 Commits
hush/nova3
...
hush/realt
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
230d92850a | ||
|
|
96c6aeaada | ||
|
|
6722aae598 | ||
|
|
66564392a6 | ||
|
|
f258f5ab66 | ||
|
|
f8f0578c3d | ||
|
|
aa60a413f3 | ||
|
|
3e66f2378d | ||
|
|
9a50f33e36 | ||
|
|
4bd5e9c0a7 | ||
|
|
12092c8715 | ||
|
|
92cc6d39f2 | ||
|
|
34a50033cb | ||
|
|
e60b65228b | ||
|
|
e74864335b | ||
|
|
27a088a457 | ||
|
|
cfe72143b8 | ||
|
|
36a729cbfe | ||
|
|
d2f006682c | ||
|
|
fb7fe540f5 | ||
|
|
1ec68bd071 | ||
|
|
4536d03e82 | ||
|
|
699704732c | ||
|
|
376d969a77 | ||
|
|
68789dfcf0 | ||
|
|
fe9fc61c4e | ||
|
|
6028f0f23a | ||
|
|
e9a0959e28 | ||
|
|
f66be2cfa7 | ||
|
|
f818bed58f | ||
|
|
07b9be5308 | ||
|
|
40c2452d6e | ||
|
|
30cdd1b71a | ||
|
|
2110b79507 | ||
|
|
fc544fa61c | ||
|
|
976fe95304 | ||
|
|
408270b647 | ||
|
|
1dfb75bc9d | ||
|
|
cefc2a1088 | ||
|
|
3b9b9200ea | ||
|
|
d6f29a0f4b | ||
|
|
5b762d11ef | ||
|
|
2f3e2da6b9 | ||
|
|
45058d4a94 | ||
|
|
5b637bd826 | ||
|
|
2d4fd7e903 | ||
|
|
b5662520aa | ||
|
|
af45c170b5 | ||
|
|
65f548b2ec | ||
|
|
b29ab8c608 | ||
|
|
d6dc37f0b6 | ||
|
|
12bce2e8c0 | ||
|
|
4acf7296e0 | ||
|
|
98706d429c | ||
|
|
41720b1a13 | ||
|
|
3ef4245166 | ||
|
|
3bb0797922 | ||
|
|
7c7b4c52af | ||
|
|
01f083b7fc | ||
|
|
91fcaebe25 | ||
|
|
9c5fe5c85e | ||
|
|
7e5e167a4b | ||
|
|
d04c4b36f3 | ||
|
|
a811e53626 | ||
|
|
df57202a05 | ||
|
|
69e6f3fdb7 | ||
|
|
6809254963 | ||
|
|
81093d3bed | ||
|
|
d9a67164f6 | ||
|
|
98259af54e | ||
|
|
039d144c79 | ||
|
|
d0f67fc189 | ||
|
|
6e3f96aa83 | ||
|
|
293677588d | ||
|
|
77e777b1ce | ||
|
|
7e7926059c | ||
|
|
c948754eff | ||
|
|
83f1a8830d | ||
|
|
80f8e05fcf | ||
|
|
afd1a1e80b | ||
|
|
84ac88cad7 | ||
|
|
211163e5c7 | ||
|
|
1b0bcebef6 | ||
|
|
89736b03c4 | ||
|
|
4edda718ed | ||
|
|
22a62edc9e | ||
|
|
50b6cc8135 | ||
|
|
45cf36925a | ||
|
|
83a71e1fec | ||
|
|
e809c8680e | ||
|
|
c926063d74 | ||
|
|
0334550356 | ||
|
|
90b9dce710 | ||
|
|
a5cdd5f1b8 | ||
|
|
5f937b8479 | ||
|
|
7e3e126730 | ||
|
|
75ca0571bb | ||
|
|
a48e5d0714 | ||
|
|
2b6a992207 | ||
|
|
24cf106ed2 | ||
|
|
95c8346cb5 |
15
.gitignore
vendored
15
.gitignore
vendored
@@ -32,6 +32,21 @@ fly.toml
|
|||||||
|
|
||||||
# Example files
|
# Example files
|
||||||
pipecat/examples/twilio-chatbot/templates/streams.xml
|
pipecat/examples/twilio-chatbot/templates/streams.xml
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/node_modules/
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/.expo/
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/dist/
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/npm-debug.*
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/*.jks
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/*.p8
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/*.p12
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/*.key
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/*.mobileprovision
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/*.orig.*
|
||||||
|
pipecat/examples/bot-ready-signalling/client/react-native/web-build/
|
||||||
|
|
||||||
|
# macOS
|
||||||
|
.DS_Store
|
||||||
|
|
||||||
|
|
||||||
# Documentation
|
# Documentation
|
||||||
docs/api/_build/
|
docs/api/_build/
|
||||||
|
|||||||
119
CHANGELOG.md
119
CHANGELOG.md
@@ -5,6 +5,125 @@ All notable changes to **Pipecat** will be documented in this file.
|
|||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Pipecat version will now be logged on every application startup. This will
|
||||||
|
help us identify what version we are running in case of any issues.
|
||||||
|
|
||||||
|
- Added a new `StopFrame` which can be used to stop a pipeline task while
|
||||||
|
keeping the frame processors running. The frame processors could then be used
|
||||||
|
in a different pipeline. The difference between a `StopFrame` and a
|
||||||
|
`StopTaskFrame` is that, as with `EndFrame` and `EndTaskFrame`, the
|
||||||
|
`StopFrame` is pushed from the task and the `StopTaskFrame` is pushed upstream
|
||||||
|
inside the pipeline by any processor.
|
||||||
|
|
||||||
|
- Added a new `PipelineTask` parameter `observers` that replaces the previous
|
||||||
|
`PipelineParams.observers`.
|
||||||
|
|
||||||
|
- Added a new `PipelineTask` parameter `check_dangling_tasks` to enable or
|
||||||
|
disable checking for frame processors' dangling tasks when the Pipeline
|
||||||
|
finishes running.
|
||||||
|
|
||||||
|
- Added new `on_completion_timeout` event for LLM services (all OpenAI-based
|
||||||
|
services, Anthropic and Google). Note that this event will only get triggered
|
||||||
|
if LLM timeouts are setup and if the timeout was reached. It can be useful to
|
||||||
|
retrigger another completion and see if the timeout was just a blip.
|
||||||
|
|
||||||
|
- Added new log observers `LLMLogObserver` and `TranscriptionLogObserver` that
|
||||||
|
can be useful for debugging your pipelines.
|
||||||
|
|
||||||
|
- Added `room_url` property to `DailyTransport`.
|
||||||
|
|
||||||
|
- Added `addons` argument to `DeepgramSTTService`.
|
||||||
|
|
||||||
|
- Added `exponential_backoff_time()` to `utils.network` module.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- ⚠️ `PipelineTask` now requires keyword arguments (except for the first one for
|
||||||
|
the pipeline).
|
||||||
|
|
||||||
|
- The base `TTSService` class now strips leading newlines before sending text
|
||||||
|
to the TTS provider. This change is to solve issues where some TTS providers,
|
||||||
|
like Azure, would not output text due to newlines.
|
||||||
|
|
||||||
|
- `GrokLLMSService` now uses `grok-2` as the default model.
|
||||||
|
|
||||||
|
- `AnthropicLLMService` now uses `claude-3-7-sonnet-20250219` as the default
|
||||||
|
model.
|
||||||
|
|
||||||
|
- `RimeHttpTTSService` needs an `aiohttp.ClientSession` to be passed to the
|
||||||
|
constructor as all the other HTTP-based services.
|
||||||
|
|
||||||
|
- `RimeHttpTTSService` doesn't use a default voice anymore.
|
||||||
|
|
||||||
|
- `DeepgramSTTService` now uses the new `nova-3` model by default. If you want
|
||||||
|
to use the previous model you can pass `LiveOptions(model="nova-2-general")`.
|
||||||
|
(see https://deepgram.com/learn/introducing-nova-3-speech-to-text-api)
|
||||||
|
|
||||||
|
```python
|
||||||
|
stt = DeepgramSTTService(..., live_options=LiveOptions(model="nova-2-general"))
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deprecated
|
||||||
|
|
||||||
|
- `PipelineParams.observers` is now deprecated, you the new `PipelineTask`
|
||||||
|
parameter `observers`.
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
|
||||||
|
- Remove `TransportParams.audio_out_is_live` since it was not being used at all.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- Fixed an `AudioContextWordTTSService` issue that would cause an `EndFrame` to
|
||||||
|
disconnect from the TTS service before audio from all the contexts was
|
||||||
|
received. This affected services like Cartesia and Rime.
|
||||||
|
|
||||||
|
- Fixed an issue that was not allowing to pass an `OpenAILLMContext` to create
|
||||||
|
`GoogleLLMService`'s context aggregators.
|
||||||
|
|
||||||
|
- Fixed a `ElevenLabsTTSService`, `FishAudioTTSService`, `LMNTTTSService` and
|
||||||
|
`PlayHTTTSService` issue that was resulting in audio requested before an
|
||||||
|
interruption being played after an interruption.
|
||||||
|
|
||||||
|
- Fixed `match_endofsentence` support for ellipses.
|
||||||
|
|
||||||
|
- Fixed an issue that would cause undesired interruptions via
|
||||||
|
`EmulateUserStartedSpeakingFrame` when only interim transcriptions (i.e. no
|
||||||
|
final transcriptions) where received.
|
||||||
|
|
||||||
|
- Fixed an issue where `EndTaskFrame` was not triggering
|
||||||
|
`on_client_disconnected` or closing the WebSocket in FastAPI.
|
||||||
|
|
||||||
|
- Fixed an issue in `DeepgramSTTService` where the `sample_rate` passed to the
|
||||||
|
`LiveOptions` was not being used, causing the service to use the default
|
||||||
|
sample rate of pipeline.
|
||||||
|
|
||||||
|
- Fixed a context aggregator issue that would not append the LLM text response
|
||||||
|
to the context if a function call happened in the same LLM turn.
|
||||||
|
|
||||||
|
- Fixed an issue that was causing HTTP TTS services to push `TTSStoppedFrame`
|
||||||
|
more than once.
|
||||||
|
|
||||||
|
- Fixed a `FishAudioTTSService` issue where `TTSStoppedFrame` was not being
|
||||||
|
pushed.
|
||||||
|
|
||||||
|
- Fixed an issue that `start_callback` was not invoked for some LLM services.
|
||||||
|
|
||||||
|
- Fixed an issue that would cause `DeepgramSTTService` to stop working after an
|
||||||
|
error occurred (e.g. sudden network loss). If the network recovered we would
|
||||||
|
not reconnect.
|
||||||
|
|
||||||
|
- Fixed a `STTMuteFilter` issue that would not mute user audio frames causing
|
||||||
|
transcriptions to be generated by the STT service.
|
||||||
|
|
||||||
|
### Other
|
||||||
|
|
||||||
|
- Added Gemini support to `examples/phone-chatbot`.
|
||||||
|
|
||||||
## [0.0.57] - 2025-02-14
|
## [0.0.57] - 2025-02-14
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
@@ -3,10 +3,10 @@ coverage~=7.6.12
|
|||||||
grpcio-tools~=1.67.1
|
grpcio-tools~=1.67.1
|
||||||
pip-tools~=7.4.1
|
pip-tools~=7.4.1
|
||||||
pre-commit~=4.0.1
|
pre-commit~=4.0.1
|
||||||
pyright~=1.1.393
|
pyright~=1.1.394
|
||||||
pytest~=8.3.4
|
pytest~=8.3.4
|
||||||
pytest-asyncio~=0.25.2
|
pytest-asyncio~=0.25.3
|
||||||
ruff~=0.9.5
|
ruff~=0.9.7
|
||||||
setuptools~=70.0.0
|
setuptools~=70.0.0
|
||||||
setuptools_scm~=8.1.0
|
setuptools_scm~=8.1.0
|
||||||
python-dotenv~=1.0.1
|
python-dotenv~=1.0.1
|
||||||
|
|||||||
@@ -18,6 +18,9 @@ AZURE_DALLE_API_KEY=...
|
|||||||
AZURE_DALLE_ENDPOINT=https://...
|
AZURE_DALLE_ENDPOINT=https://...
|
||||||
AZURE_DALLE_MODEL=...
|
AZURE_DALLE_MODEL=...
|
||||||
|
|
||||||
|
# Cartesia
|
||||||
|
CARTESIA_API_KEY=...
|
||||||
|
|
||||||
# Daily
|
# Daily
|
||||||
DAILY_API_KEY=...
|
DAILY_API_KEY=...
|
||||||
DAILY_SAMPLE_ROOM_URL=https://...
|
DAILY_SAMPLE_ROOM_URL=https://...
|
||||||
|
|||||||
1
examples/bot-ready-signalling/client/react-native/.nvmrc
Normal file
1
examples/bot-ready-signalling/client/react-native/.nvmrc
Normal file
@@ -0,0 +1 @@
|
|||||||
|
22.14
|
||||||
60
examples/bot-ready-signalling/client/react-native/README.md
Normal file
60
examples/bot-ready-signalling/client/react-native/README.md
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
# React Native Implementation
|
||||||
|
|
||||||
|
Basic implementation using the [Pipecat React Native SDK](https://docs.pipecat.ai/client/react-native/introduction).
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Expo requirements
|
||||||
|
|
||||||
|
This project cannot be used with an [Expo Go](https://docs.expo.dev/workflow/expo-go/) app because [it requires custom native code](https://docs.expo.io/workflow/customizing/).
|
||||||
|
|
||||||
|
When a project requires custom native code or a config plugin, we need to transition from using [Expo Go](https://docs.expo.dev/workflow/expo-go/)
|
||||||
|
to a [development build](https://docs.expo.dev/development/introduction/).
|
||||||
|
|
||||||
|
More details about the custom native code used by this demo can be found in [rn-daily-js-expo-config-plugin](https://github.com/daily-co/rn-daily-js-expo-config-plugin).
|
||||||
|
|
||||||
|
### Building remotely
|
||||||
|
|
||||||
|
If you do not have experience with Xcode and Android Studio builds or do not have them installed locally on your computer, you will need to follow [this guide from Expo to use EAS Build](https://docs.expo.dev/development/create-development-builds/#create-and-install-eas-build).
|
||||||
|
|
||||||
|
### Building locally
|
||||||
|
|
||||||
|
You will need to have installed locally on your computer:
|
||||||
|
- [Xcode](https://developer.apple.com/xcode/) to build for iOS;
|
||||||
|
- [Android Studio](https://developer.android.com/studio) to build for Android;
|
||||||
|
|
||||||
|
#### Install the demo dependencies
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Use the version of node specified in .nvmrc
|
||||||
|
nvm i
|
||||||
|
|
||||||
|
# Install dependencies
|
||||||
|
npm i
|
||||||
|
|
||||||
|
# Before a native app can be compiled, the native source code must be generated.
|
||||||
|
npx expo prebuild
|
||||||
|
|
||||||
|
# Configure the environment variable to connect to the local server
|
||||||
|
cp env.example .env
|
||||||
|
# edit .env and add your local ip address, for example: http://192.168.1.16:7860
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Running on Android
|
||||||
|
|
||||||
|
After plugging in an Android device [configured for debugging](https://developer.android.com/studio/debug/dev-options), run the following command:
|
||||||
|
|
||||||
|
```
|
||||||
|
npm run android
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Running on iOS
|
||||||
|
|
||||||
|
Run the following command:
|
||||||
|
|
||||||
|
```
|
||||||
|
npm run ios
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Connect to the server
|
||||||
|
Use the http://localhost:5173 in your app.
|
||||||
75
examples/bot-ready-signalling/client/react-native/app.json
Normal file
75
examples/bot-ready-signalling/client/react-native/app.json
Normal file
@@ -0,0 +1,75 @@
|
|||||||
|
{
|
||||||
|
"expo": {
|
||||||
|
"name": "bot-ready-rn",
|
||||||
|
"slug": "bot-ready-rn",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"orientation": "portrait",
|
||||||
|
"icon": "./assets/icon.png",
|
||||||
|
"userInterfaceStyle": "light",
|
||||||
|
"splash": {
|
||||||
|
"image": "./assets/splash.png",
|
||||||
|
"resizeMode": "contain",
|
||||||
|
"backgroundColor": "#ffffff"
|
||||||
|
},
|
||||||
|
"updates": {
|
||||||
|
"fallbackToCacheTimeout": 0
|
||||||
|
},
|
||||||
|
"assetBundlePatterns": [
|
||||||
|
"**/*"
|
||||||
|
],
|
||||||
|
"ios": {
|
||||||
|
"supportsTablet": true,
|
||||||
|
"bitcode": false,
|
||||||
|
"bundleIdentifier": "co.daily.expo.BotReady",
|
||||||
|
"infoPlist": {
|
||||||
|
"UIBackgroundModes": [
|
||||||
|
"voip"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"appleTeamId": "EEBGKV9N3N"
|
||||||
|
},
|
||||||
|
"android": {
|
||||||
|
"adaptiveIcon": {
|
||||||
|
"foregroundImage": "./assets/adaptive-icon.png",
|
||||||
|
"backgroundColor": "#FFFFFF"
|
||||||
|
},
|
||||||
|
"package": "co.daily.expo.BotReady",
|
||||||
|
"permissions": [
|
||||||
|
"android.permission.ACCESS_NETWORK_STATE",
|
||||||
|
"android.permission.BLUETOOTH",
|
||||||
|
"android.permission.CAMERA",
|
||||||
|
"android.permission.INTERNET",
|
||||||
|
"android.permission.MODIFY_AUDIO_SETTINGS",
|
||||||
|
"android.permission.RECORD_AUDIO",
|
||||||
|
"android.permission.SYSTEM_ALERT_WINDOW",
|
||||||
|
"android.permission.WAKE_LOCK",
|
||||||
|
"android.permission.FOREGROUND_SERVICE",
|
||||||
|
"android.permission.FOREGROUND_SERVICE_CAMERA",
|
||||||
|
"android.permission.FOREGROUND_SERVICE_MICROPHONE",
|
||||||
|
"android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION",
|
||||||
|
"android.permission.POST_NOTIFICATIONS"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"web": {
|
||||||
|
"favicon": "./assets/favicon.png"
|
||||||
|
},
|
||||||
|
"plugins": [
|
||||||
|
"@config-plugins/react-native-webrtc",
|
||||||
|
"@daily-co/config-plugin-rn-daily-js",
|
||||||
|
[
|
||||||
|
"expo-build-properties",
|
||||||
|
{
|
||||||
|
"android": {
|
||||||
|
"minSdkVersion": 24,
|
||||||
|
"compileSdkVersion": 35,
|
||||||
|
"targetSdkVersion": 34,
|
||||||
|
"buildToolsVersion": "35.0.0"
|
||||||
|
},
|
||||||
|
"ios": {
|
||||||
|
"deploymentTarget": "15.1"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 17 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.4 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 46 KiB |
@@ -0,0 +1,7 @@
|
|||||||
|
module.exports = function(api) {
|
||||||
|
api.cache(true);
|
||||||
|
return {
|
||||||
|
presets: ['babel-preset-expo'],
|
||||||
|
plugins: [["module:react-native-dotenv"]],
|
||||||
|
};
|
||||||
|
};
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
API_BASE_URL=http://YOUR_LOCAL_IP:7860
|
||||||
7
examples/bot-ready-signalling/client/react-native/index.js
vendored
Normal file
7
examples/bot-ready-signalling/client/react-native/index.js
vendored
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
import { registerRootComponent } from "expo";
|
||||||
|
|
||||||
|
import App from "./src/App";
|
||||||
|
|
||||||
|
// registerRootComponent calls AppRegistry.registerComponent('main', () => App);
|
||||||
|
// It also ensures that the environment is set up appropriately
|
||||||
|
registerRootComponent(App);
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
// Learn more https://docs.expo.io/guides/customizing-metro
|
||||||
|
const { getDefaultConfig } = require('expo/metro-config');
|
||||||
|
|
||||||
|
module.exports = getDefaultConfig(__dirname);
|
||||||
10983
examples/bot-ready-signalling/client/react-native/package-lock.json
generated
Normal file
10983
examples/bot-ready-signalling/client/react-native/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,31 @@
|
|||||||
|
{
|
||||||
|
"name": "bot-ready-rn",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"scripts": {
|
||||||
|
"start": "expo start --dev-client",
|
||||||
|
"android": "expo run:android --device",
|
||||||
|
"ios": "expo run:ios --device",
|
||||||
|
"web": "expo start --web"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@config-plugins/react-native-webrtc": "^10.0.0",
|
||||||
|
"@daily-co/config-plugin-rn-daily-js": "0.0.7",
|
||||||
|
"@daily-co/react-native-daily-js": "^0.70.0",
|
||||||
|
"@daily-co/react-native-webrtc": "^118.0.3-daily.2",
|
||||||
|
"@react-native-async-storage/async-storage": "1.23.1",
|
||||||
|
"expo": "^52.0.0",
|
||||||
|
"expo-build-properties": "~0.13.1",
|
||||||
|
"expo-dev-client": "~5.0.5",
|
||||||
|
"expo-splash-screen": "~0.29.16",
|
||||||
|
"expo-status-bar": "~2.0.0",
|
||||||
|
"react": "18.3.1",
|
||||||
|
"react-native": "0.76.3",
|
||||||
|
"react-native-background-timer": "^2.4.1",
|
||||||
|
"react-native-dotenv": "^3.4.11",
|
||||||
|
"react-native-get-random-values": "^1.11.0"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@babel/core": "^7.12.9"
|
||||||
|
},
|
||||||
|
"private": true
|
||||||
|
}
|
||||||
121
examples/bot-ready-signalling/client/react-native/src/App.js
vendored
Normal file
121
examples/bot-ready-signalling/client/react-native/src/App.js
vendored
Normal file
@@ -0,0 +1,121 @@
|
|||||||
|
import React, { useState, useEffect } from 'react';
|
||||||
|
import {SafeAreaView, View, Text, Button, StyleSheet, ScrollView} from 'react-native';
|
||||||
|
import Daily from "@daily-co/react-native-daily-js";
|
||||||
|
import { API_BASE_URL } from "@env";
|
||||||
|
|
||||||
|
const CallScreen = () => {
|
||||||
|
const [connectionStatus, setConnectionStatus] = useState('Disconnected');
|
||||||
|
const [isConnected, setIsConnected] = useState(false);
|
||||||
|
const [callObject, setCallObject] = useState(null);
|
||||||
|
const [logs, setLogs] = useState([]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (callObject) {
|
||||||
|
setupTrackListeners(callObject);
|
||||||
|
}
|
||||||
|
}, [callObject]);
|
||||||
|
|
||||||
|
const log = (message) => {
|
||||||
|
setLogs((prevLogs) => [...prevLogs, `${new Date().toISOString()} - ${message}`]);
|
||||||
|
console.log(message);
|
||||||
|
};
|
||||||
|
|
||||||
|
const setupTrackListeners = (callObject) => {
|
||||||
|
callObject.on("joined-meeting", () => {
|
||||||
|
setConnectionStatus('Connected');
|
||||||
|
setIsConnected(true);
|
||||||
|
log('Client connected');
|
||||||
|
});
|
||||||
|
callObject.on("left-meeting", () => {
|
||||||
|
setConnectionStatus('Disconnected');
|
||||||
|
setIsConnected(false);
|
||||||
|
log('Client disconnected');
|
||||||
|
});
|
||||||
|
callObject.on("participant-left", () => {
|
||||||
|
// When the bot leaves, we are also disconnecting from the call
|
||||||
|
disconnect().catch((err) => {
|
||||||
|
log(`Failed to disconnect ${err}`);
|
||||||
|
})
|
||||||
|
});
|
||||||
|
// Trigger so the bot can start sending audio
|
||||||
|
callObject.on("track-started", (evt) => {
|
||||||
|
if (evt.track.kind === "audio" && evt.participant.local === false) {
|
||||||
|
handleEventToConsole(evt)
|
||||||
|
log("Sending the message that will trigger the bot to play the audio.")
|
||||||
|
callObject.sendAppMessage("playable")
|
||||||
|
}
|
||||||
|
});
|
||||||
|
callObject.on("error", (evt) => log(`Error: ${evt.error}`));
|
||||||
|
// Other events just for awareness
|
||||||
|
callObject.on("track-stopped", handleEventToConsole);
|
||||||
|
callObject.on("participant-joined", handleEventToConsole);
|
||||||
|
callObject.on("participant-updated", handleEventToConsole);
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleEventToConsole = (evt) => {
|
||||||
|
log(`Received event: ${evt.action}`);
|
||||||
|
};
|
||||||
|
|
||||||
|
const connect = async () => {
|
||||||
|
try {
|
||||||
|
const callObject = Daily.createCallObject({ subscribeToTracksAutomatically: true });
|
||||||
|
setCallObject(callObject);
|
||||||
|
const connectionUrl = `${API_BASE_URL}/connect`
|
||||||
|
const res = await fetch(connectionUrl, { method: "POST", headers: { "Content-Type": "application/json" } });
|
||||||
|
const roomInfo = await res.json();
|
||||||
|
await callObject.join({ url: roomInfo.room_url });
|
||||||
|
} catch (error) {
|
||||||
|
log(`Error connecting: ${error.message}`);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const disconnect = async () => {
|
||||||
|
if (callObject) {
|
||||||
|
try {
|
||||||
|
await callObject.leave();
|
||||||
|
await callObject.destroy();
|
||||||
|
setCallObject(null);
|
||||||
|
} catch (error) {
|
||||||
|
log(`Error disconnecting: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<SafeAreaView style={styles.safeArea}>
|
||||||
|
<View style={styles.container}>
|
||||||
|
<View style={styles.statusBar}>
|
||||||
|
<Text>Status: <Text style={styles.status}>{connectionStatus}</Text></Text>
|
||||||
|
<View style={styles.controls}>
|
||||||
|
<Button
|
||||||
|
title={isConnected ? "Disconnect" : "Connect"}
|
||||||
|
onPress={isConnected ? disconnect : connect}
|
||||||
|
/>
|
||||||
|
</View>
|
||||||
|
</View>
|
||||||
|
|
||||||
|
<View style={styles.debugPanel}>
|
||||||
|
<Text style={styles.debugTitle}>Debug Info</Text>
|
||||||
|
<ScrollView style={styles.debugLog}>
|
||||||
|
{logs.map((logEntry, index) => (
|
||||||
|
<Text key={index} style={styles.logText}>{logEntry}</Text>
|
||||||
|
))}
|
||||||
|
</ScrollView>
|
||||||
|
</View>
|
||||||
|
</View>
|
||||||
|
</SafeAreaView>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
const styles = StyleSheet.create({
|
||||||
|
safeArea: { flex: 1, backgroundColor: '#f0f0f0', padding: 20 },
|
||||||
|
container: { flex: 1, margin: 20 },
|
||||||
|
statusBar: { flexDirection: 'row', justifyContent: 'space-between', alignItems: 'center', padding: 10, backgroundColor: '#fff', borderRadius: 8, marginBottom: 20 },
|
||||||
|
status: { fontWeight: 'bold' },
|
||||||
|
controls: { flexDirection: 'row', gap: 10 },
|
||||||
|
debugPanel: { height: '80%', backgroundColor: '#fff', borderRadius: 8, padding: 20},
|
||||||
|
debugTitle: { fontSize: 16, fontWeight: 'bold' },
|
||||||
|
debugLog: { height: '100%', overflow: 'scroll', backgroundColor: '#f8f8f8', padding: 10, borderRadius: 4, fontFamily: 'monospace', fontSize: 12, lineHeight: 1.4 },
|
||||||
|
});
|
||||||
|
|
||||||
|
export default CallScreen;
|
||||||
@@ -17,7 +17,7 @@ from runner import configure
|
|||||||
from pipecat.frames.frames import AudioRawFrame, EndFrame, OutputAudioRawFrame, TTSSpeakFrame
|
from pipecat.frames.frames import AudioRawFrame, EndFrame, OutputAudioRawFrame, TTSSpeakFrame
|
||||||
from pipecat.pipeline.pipeline import Pipeline
|
from pipecat.pipeline.pipeline import Pipeline
|
||||||
from pipecat.pipeline.runner import PipelineRunner
|
from pipecat.pipeline.runner import PipelineRunner
|
||||||
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
from pipecat.pipeline.task import PipelineTask
|
||||||
from pipecat.services.cartesia import CartesiaTTSService
|
from pipecat.services.cartesia import CartesiaTTSService
|
||||||
from pipecat.transports.services.daily import DailyParams, DailyTransport
|
from pipecat.transports.services.daily import DailyParams, DailyTransport
|
||||||
|
|
||||||
|
|||||||
@@ -119,7 +119,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -124,7 +124,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@audiobuffer.event_handler("on_audio_data")
|
@audiobuffer.event_handler("on_audio_data")
|
||||||
async def on_audio_data(buffer, audio, sample_rate, num_channels):
|
async def on_audio_data(buffer, audio, sample_rate, num_channels):
|
||||||
|
|||||||
@@ -70,7 +70,7 @@ async def main(room_url: str, token: str):
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -62,7 +62,7 @@ async def main(room_url: str, token: str):
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -18,8 +18,7 @@ from pipecat.pipeline.pipeline import Pipeline
|
|||||||
from pipecat.pipeline.runner import PipelineRunner
|
from pipecat.pipeline.runner import PipelineRunner
|
||||||
from pipecat.pipeline.task import PipelineTask
|
from pipecat.pipeline.task import PipelineTask
|
||||||
from pipecat.services.fal import FalImageGenService
|
from pipecat.services.fal import FalImageGenService
|
||||||
from pipecat.transports.base_transport import TransportParams
|
from pipecat.transports.local.tk import TkLocalTransport, TkTransportParams
|
||||||
from pipecat.transports.local.tk import TkLocalTransport
|
|
||||||
|
|
||||||
load_dotenv(override=True)
|
load_dotenv(override=True)
|
||||||
|
|
||||||
@@ -34,7 +33,9 @@ async def main():
|
|||||||
|
|
||||||
transport = TkLocalTransport(
|
transport = TkLocalTransport(
|
||||||
tk_root,
|
tk_root,
|
||||||
TransportParams(camera_out_enabled=True, camera_out_width=1024, camera_out_height=1024),
|
TkTransportParams(
|
||||||
|
camera_out_enabled=True, camera_out_width=1024, camera_out_height=1024
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
imagegen = FalImageGenService(
|
imagegen = FalImageGenService(
|
||||||
|
|||||||
@@ -44,7 +44,8 @@ async def main():
|
|||||||
runner = PipelineRunner()
|
runner = PipelineRunner()
|
||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
Pipeline([imagegen, transport.output()]), PipelineParams(enable_metrics=True)
|
Pipeline([imagegen, transport.output()]),
|
||||||
|
params=PipelineParams(enable_metrics=True),
|
||||||
)
|
)
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
|
|||||||
@@ -30,8 +30,7 @@ from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
|
|||||||
from pipecat.services.cartesia import CartesiaHttpTTSService
|
from pipecat.services.cartesia import CartesiaHttpTTSService
|
||||||
from pipecat.services.fal import FalImageGenService
|
from pipecat.services.fal import FalImageGenService
|
||||||
from pipecat.services.openai import OpenAILLMService
|
from pipecat.services.openai import OpenAILLMService
|
||||||
from pipecat.transports.base_transport import TransportParams
|
from pipecat.transports.local.tk import TkLocalTransport, TkTransportParams
|
||||||
from pipecat.transports.local.tk import TkLocalTransport, TkOutputTransport
|
|
||||||
|
|
||||||
load_dotenv(override=True)
|
load_dotenv(override=True)
|
||||||
|
|
||||||
@@ -152,7 +151,7 @@ async def main():
|
|||||||
|
|
||||||
transport = TkLocalTransport(
|
transport = TkLocalTransport(
|
||||||
tk_root,
|
tk_root,
|
||||||
TransportParams(
|
TkTransportParams(
|
||||||
audio_out_enabled=True,
|
audio_out_enabled=True,
|
||||||
camera_out_enabled=True,
|
camera_out_enabled=True,
|
||||||
camera_out_width=1024,
|
camera_out_width=1024,
|
||||||
|
|||||||
@@ -105,7 +105,10 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(enable_metrics=True, enable_usage_metrics=True),
|
params=PipelineParams(
|
||||||
|
enable_metrics=True,
|
||||||
|
enable_usage_metrics=True,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
|
|||||||
@@ -127,7 +127,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -79,7 +79,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -103,7 +103,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -75,7 +75,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -77,7 +77,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -83,7 +83,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -75,7 +75,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -80,7 +80,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -71,7 +71,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -88,7 +88,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -79,7 +79,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -80,7 +80,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -251,7 +251,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -78,7 +78,11 @@ async def main():
|
|||||||
runner = PipelineRunner()
|
runner = PipelineRunner()
|
||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline, PipelineParams(audio_in_sample_rate=24000, audio_out_sample_rate=24000)
|
pipeline,
|
||||||
|
params=PipelineParams(
|
||||||
|
audio_in_sample_rate=24000,
|
||||||
|
audio_out_sample_rate=24000,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
await runner.run(task)
|
await runner.run(task)
|
||||||
|
|||||||
@@ -24,8 +24,7 @@ from pipecat.pipeline.pipeline import Pipeline
|
|||||||
from pipecat.pipeline.runner import PipelineRunner
|
from pipecat.pipeline.runner import PipelineRunner
|
||||||
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
||||||
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
|
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
|
||||||
from pipecat.transports.base_transport import TransportParams
|
from pipecat.transports.local.tk import TkLocalTransport, TkTransportParams
|
||||||
from pipecat.transports.local.tk import TkLocalTransport
|
|
||||||
from pipecat.transports.services.daily import DailyParams, DailyTransport
|
from pipecat.transports.services.daily import DailyParams, DailyTransport
|
||||||
|
|
||||||
load_dotenv(override=True)
|
load_dotenv(override=True)
|
||||||
@@ -67,7 +66,7 @@ async def main():
|
|||||||
|
|
||||||
tk_transport = TkLocalTransport(
|
tk_transport = TkLocalTransport(
|
||||||
tk_root,
|
tk_root,
|
||||||
TransportParams(
|
TkTransportParams(
|
||||||
audio_out_enabled=True,
|
audio_out_enabled=True,
|
||||||
camera_out_enabled=True,
|
camera_out_enabled=True,
|
||||||
camera_out_is_live=True,
|
camera_out_is_live=True,
|
||||||
@@ -83,7 +82,11 @@ async def main():
|
|||||||
pipeline = Pipeline([daily_transport.input(), MirrorProcessor(), tk_transport.output()])
|
pipeline = Pipeline([daily_transport.input(), MirrorProcessor(), tk_transport.output()])
|
||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline, PipelineParams(audio_in_sample_rate=24000, audio_out_sample_rate=24000)
|
pipeline,
|
||||||
|
params=PipelineParams(
|
||||||
|
audio_in_sample_rate=24000,
|
||||||
|
audio_out_sample_rate=24000,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
async def run_tk():
|
async def run_tk():
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -112,7 +112,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -99,7 +99,13 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True, enable_metrics=True))
|
task = PipelineTask(
|
||||||
|
pipeline,
|
||||||
|
params=PipelineParams(
|
||||||
|
allow_interruptions=True,
|
||||||
|
enable_metrics=True,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -153,7 +153,13 @@ If you need to use a tool, simply use the tool. Do not tell the user the tool yo
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True, enable_metrics=True))
|
task = PipelineTask(
|
||||||
|
pipeline,
|
||||||
|
params=PipelineParams(
|
||||||
|
allow_interruptions=True,
|
||||||
|
enable_metrics=True,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ from loguru import logger
|
|||||||
from runner import configure
|
from runner import configure
|
||||||
|
|
||||||
from pipecat.audio.vad.silero import SileroVADAnalyzer
|
from pipecat.audio.vad.silero import SileroVADAnalyzer
|
||||||
|
from pipecat.frames.frames import TTSSpeakFrame
|
||||||
from pipecat.pipeline.pipeline import Pipeline
|
from pipecat.pipeline.pipeline import Pipeline
|
||||||
from pipecat.pipeline.runner import PipelineRunner
|
from pipecat.pipeline.runner import PipelineRunner
|
||||||
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
||||||
@@ -30,6 +31,12 @@ logger.add(sys.stderr, level="DEBUG")
|
|||||||
video_participant_id = None
|
video_participant_id = None
|
||||||
|
|
||||||
|
|
||||||
|
async def start_fetch_weather(function_name, llm, context):
|
||||||
|
"""Push a frame to the LLM; this is handy when the LLM response might take a while."""
|
||||||
|
await llm.push_frame(TTSSpeakFrame("Let me check on that."))
|
||||||
|
logger.debug(f"Starting fetch_weather_from_api with function_name: {function_name}")
|
||||||
|
|
||||||
|
|
||||||
async def get_weather(function_name, tool_call_id, arguments, llm, context, result_callback):
|
async def get_weather(function_name, tool_call_id, arguments, llm, context, result_callback):
|
||||||
location = arguments["location"]
|
location = arguments["location"]
|
||||||
await result_callback(f"The weather in {location} is currently 72 degrees and sunny.")
|
await result_callback(f"The weather in {location} is currently 72 degrees and sunny.")
|
||||||
@@ -63,7 +70,7 @@ async def main():
|
|||||||
)
|
)
|
||||||
|
|
||||||
llm = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"), model="gemini-2.0-flash-001")
|
llm = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"), model="gemini-2.0-flash-001")
|
||||||
llm.register_function("get_weather", get_weather)
|
llm.register_function("get_weather", get_weather, start_fetch_weather)
|
||||||
llm.register_function("get_image", get_image)
|
llm.register_function("get_image", get_image)
|
||||||
|
|
||||||
tools = [
|
tools = [
|
||||||
@@ -145,7 +152,7 @@ indicate you should use the get_image tool are:
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -116,7 +116,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -113,7 +113,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -117,7 +117,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -116,7 +116,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -116,7 +116,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -123,7 +123,7 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -123,7 +123,7 @@ Start by asking me for my location. Then, use 'get_weather_current' to give me a
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -117,7 +117,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -83,7 +83,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -133,7 +133,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -126,7 +126,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -85,7 +85,13 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True, enable_metrics=True))
|
task = PipelineTask(
|
||||||
|
pipeline,
|
||||||
|
params=PipelineParams(
|
||||||
|
allow_interruptions=True,
|
||||||
|
enable_metrics=True,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
# When a participant joins, start transcription for that participant so the
|
# When a participant joins, start transcription for that participant so the
|
||||||
# bot can "hear" and respond to them.
|
# bot can "hear" and respond to them.
|
||||||
|
|||||||
@@ -108,7 +108,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
report_only_initial_ttfb=True,
|
report_only_initial_ttfb=True,
|
||||||
|
|||||||
@@ -38,7 +38,6 @@ async def main():
|
|||||||
"GStreamer",
|
"GStreamer",
|
||||||
DailyParams(
|
DailyParams(
|
||||||
audio_out_enabled=True,
|
audio_out_enabled=True,
|
||||||
audio_out_is_live=True,
|
|
||||||
camera_out_enabled=True,
|
camera_out_enabled=True,
|
||||||
camera_out_width=1280,
|
camera_out_width=1280,
|
||||||
camera_out_height=720,
|
camera_out_height=720,
|
||||||
|
|||||||
@@ -16,10 +16,13 @@ from runner import configure
|
|||||||
|
|
||||||
from pipecat.audio.vad.silero import SileroVADAnalyzer
|
from pipecat.audio.vad.silero import SileroVADAnalyzer
|
||||||
from pipecat.audio.vad.vad_analyzer import VADParams
|
from pipecat.audio.vad.vad_analyzer import VADParams
|
||||||
|
from pipecat.frames.frames import TranscriptionMessage, TranscriptionUpdateFrame
|
||||||
from pipecat.pipeline.pipeline import Pipeline
|
from pipecat.pipeline.pipeline import Pipeline
|
||||||
from pipecat.pipeline.runner import PipelineRunner
|
from pipecat.pipeline.runner import PipelineRunner
|
||||||
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
||||||
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
|
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
|
||||||
|
from pipecat.processors.transcript_processor import TranscriptProcessor
|
||||||
|
from pipecat.services.deepgram import DeepgramSTTService
|
||||||
from pipecat.services.openai_realtime_beta import (
|
from pipecat.services.openai_realtime_beta import (
|
||||||
InputAudioTranscription,
|
InputAudioTranscription,
|
||||||
OpenAIRealtimeBetaLLMService,
|
OpenAIRealtimeBetaLLMService,
|
||||||
@@ -140,21 +143,29 @@ Remember, your responses should be short. Just one or two sentences, usually."""
|
|||||||
tools,
|
tools,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
|
||||||
|
|
||||||
|
# Create transcript processor and handler
|
||||||
|
transcript = TranscriptProcessor()
|
||||||
|
|
||||||
context_aggregator = llm.create_context_aggregator(context)
|
context_aggregator = llm.create_context_aggregator(context)
|
||||||
|
|
||||||
pipeline = Pipeline(
|
pipeline = Pipeline(
|
||||||
[
|
[
|
||||||
transport.input(), # Transport user input
|
transport.input(), # Transport user input
|
||||||
|
stt,
|
||||||
|
transcript.user(), # User transcripts
|
||||||
context_aggregator.user(),
|
context_aggregator.user(),
|
||||||
llm, # LLM
|
llm, # LLM
|
||||||
context_aggregator.assistant(),
|
context_aggregator.assistant(),
|
||||||
|
transcript.assistant(), # Assistant transcripts
|
||||||
transport.output(), # Transport bot output
|
transport.output(), # Transport bot output
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
@@ -162,9 +173,16 @@ Remember, your responses should be short. Just one or two sentences, usually."""
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Register event handler for transcript updates
|
||||||
|
@transcript.event_handler("on_transcript_update")
|
||||||
|
async def on_transcript_update(processor, frame):
|
||||||
|
logger.debug(f"Received transcript update with {len(frame.messages)} new messages")
|
||||||
|
for msg in frame.messages:
|
||||||
|
logger.debug(msg)
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
await transport.capture_participant_transcription(participant["id"])
|
# await transport.capture_participant_transcription(participant["id"])
|
||||||
# Kick off the conversation.
|
# Kick off the conversation.
|
||||||
await task.queue_frames([context_aggregator.user().get_context_frame()])
|
await task.queue_frames([context_aggregator.user().get_context_frame()])
|
||||||
|
|
||||||
|
|||||||
@@ -212,7 +212,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -237,7 +237,7 @@ Remember, your responses should be short. Just one or two sentences, usually."""
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -209,7 +209,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -263,7 +263,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -87,7 +87,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
# We just use 16000 because that's what Tavus is expecting and
|
# We just use 16000 because that's what Tavus is expecting and
|
||||||
# we avoid resampling.
|
# we avoid resampling.
|
||||||
audio_in_sample_rate=16000,
|
audio_in_sample_rate=16000,
|
||||||
|
|||||||
@@ -145,7 +145,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -138,6 +138,7 @@ class OutputGate(FrameProcessor):
|
|||||||
self._gate_open = start_open
|
self._gate_open = start_open
|
||||||
self._frames_buffer = []
|
self._frames_buffer = []
|
||||||
self._notifier = notifier
|
self._notifier = notifier
|
||||||
|
self._gate_task = None
|
||||||
|
|
||||||
def close_gate(self):
|
def close_gate(self):
|
||||||
self._gate_open = False
|
self._gate_open = False
|
||||||
@@ -178,10 +179,13 @@ class OutputGate(FrameProcessor):
|
|||||||
|
|
||||||
async def _start(self):
|
async def _start(self):
|
||||||
self._frames_buffer = []
|
self._frames_buffer = []
|
||||||
self._gate_task = self.create_task(self._gate_task_handler())
|
if not self._gate_task:
|
||||||
|
self._gate_task = self.create_task(self._gate_task_handler())
|
||||||
|
|
||||||
async def _stop(self):
|
async def _stop(self):
|
||||||
await self.cancel_task(self._gate_task)
|
if self._gate_task:
|
||||||
|
await self.cancel_task(self._gate_task)
|
||||||
|
self._gate_task = None
|
||||||
|
|
||||||
async def _gate_task_handler(self):
|
async def _gate_task_handler(self):
|
||||||
while True:
|
while True:
|
||||||
@@ -351,7 +355,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -342,6 +342,7 @@ class OutputGate(FrameProcessor):
|
|||||||
self._gate_open = start_open
|
self._gate_open = start_open
|
||||||
self._frames_buffer = []
|
self._frames_buffer = []
|
||||||
self._notifier = notifier
|
self._notifier = notifier
|
||||||
|
self._gate_task = None
|
||||||
|
|
||||||
def close_gate(self):
|
def close_gate(self):
|
||||||
self._gate_open = False
|
self._gate_open = False
|
||||||
@@ -382,10 +383,13 @@ class OutputGate(FrameProcessor):
|
|||||||
|
|
||||||
async def _start(self):
|
async def _start(self):
|
||||||
self._frames_buffer = []
|
self._frames_buffer = []
|
||||||
self._gate_task = self.create_task(self._gate_task_handler())
|
if not self._gate_task:
|
||||||
|
self._gate_task = self.create_task(self._gate_task_handler())
|
||||||
|
|
||||||
async def _stop(self):
|
async def _stop(self):
|
||||||
await self.cancel_task(self._gate_task)
|
if self._gate_task:
|
||||||
|
await self.cancel_task(self._gate_task)
|
||||||
|
self._gate_task = None
|
||||||
|
|
||||||
async def _gate_task_handler(self):
|
async def _gate_task_handler(self):
|
||||||
while True:
|
while True:
|
||||||
@@ -560,7 +564,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -25,10 +25,8 @@ from pipecat.frames.frames import (
|
|||||||
InputAudioRawFrame,
|
InputAudioRawFrame,
|
||||||
LLMFullResponseEndFrame,
|
LLMFullResponseEndFrame,
|
||||||
LLMFullResponseStartFrame,
|
LLMFullResponseStartFrame,
|
||||||
LLMMessagesFrame,
|
|
||||||
StartFrame,
|
StartFrame,
|
||||||
StartInterruptionFrame,
|
StartInterruptionFrame,
|
||||||
StopInterruptionFrame,
|
|
||||||
SystemFrame,
|
SystemFrame,
|
||||||
TextFrame,
|
TextFrame,
|
||||||
TranscriptionFrame,
|
TranscriptionFrame,
|
||||||
@@ -555,6 +553,7 @@ class OutputGate(FrameProcessor):
|
|||||||
self._notifier = notifier
|
self._notifier = notifier
|
||||||
self._context = context
|
self._context = context
|
||||||
self._transcription_buffer = user_transcription_buffer
|
self._transcription_buffer = user_transcription_buffer
|
||||||
|
self._gate_task = None
|
||||||
|
|
||||||
def close_gate(self):
|
def close_gate(self):
|
||||||
self._gate_open = False
|
self._gate_open = False
|
||||||
@@ -602,10 +601,13 @@ class OutputGate(FrameProcessor):
|
|||||||
|
|
||||||
async def _start(self):
|
async def _start(self):
|
||||||
self._frames_buffer = []
|
self._frames_buffer = []
|
||||||
self._gate_task = self.create_task(self._gate_task_handler())
|
if not self._gate_task:
|
||||||
|
self._gate_task = self.create_task(self._gate_task_handler())
|
||||||
|
|
||||||
async def _stop(self):
|
async def _stop(self):
|
||||||
await self.cancel_task(self._gate_task)
|
if self._gate_task:
|
||||||
|
await self.cancel_task(self._gate_task)
|
||||||
|
self._gate_task = None
|
||||||
|
|
||||||
async def _gate_task_handler(self):
|
async def _gate_task_handler(self):
|
||||||
while True:
|
while True:
|
||||||
@@ -740,7 +742,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -87,7 +87,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -122,7 +122,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -354,7 +354,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -63,7 +63,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -89,7 +89,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -120,7 +120,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -79,7 +79,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -106,7 +106,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
#
|
#
|
||||||
# Copyright (c) 2024, Daily
|
# Copyright (c) 2024-2025, Daily
|
||||||
#
|
#
|
||||||
# SPDX-License-Identifier: BSD 2-Clause License
|
# SPDX-License-Identifier: BSD 2-Clause License
|
||||||
#
|
#
|
||||||
@@ -34,7 +34,7 @@ search_tool = {"google_search": {}}
|
|||||||
tools = [search_tool]
|
tools = [search_tool]
|
||||||
|
|
||||||
system_instruction = """
|
system_instruction = """
|
||||||
You are an expert at providing the most recent news from any place. Your responses will be converted to audio, so avoid using special characters or overly complex formatting.
|
You are an expert at providing the most recent news from any place. Your responses will be converted to audio, so avoid using special characters or overly complex formatting.
|
||||||
|
|
||||||
Always use the google search API to retrieve the latest news. You must also use it to check which day is today.
|
Always use the google search API to retrieve the latest news. You must also use it to check which day is today.
|
||||||
|
|
||||||
@@ -93,7 +93,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -83,7 +83,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
),
|
),
|
||||||
|
|||||||
@@ -150,7 +150,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -150,7 +150,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -178,7 +178,7 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
@@ -18,12 +18,10 @@ from pipecat.frames.frames import (
|
|||||||
BotStartedSpeakingFrame,
|
BotStartedSpeakingFrame,
|
||||||
BotStoppedSpeakingFrame,
|
BotStoppedSpeakingFrame,
|
||||||
Frame,
|
Frame,
|
||||||
LLMFullResponseEndFrame,
|
|
||||||
LLMFullResponseStartFrame,
|
|
||||||
LLMTextFrame,
|
|
||||||
StartInterruptionFrame,
|
StartInterruptionFrame,
|
||||||
)
|
)
|
||||||
from pipecat.observers.base_observer import BaseObserver
|
from pipecat.observers.base_observer import BaseObserver
|
||||||
|
from pipecat.observers.loggers.llm_log_observer import LLMLogObserver
|
||||||
from pipecat.pipeline.pipeline import Pipeline
|
from pipecat.pipeline.pipeline import Pipeline
|
||||||
from pipecat.pipeline.runner import PipelineRunner
|
from pipecat.pipeline.runner import PipelineRunner
|
||||||
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
from pipecat.pipeline.task import PipelineParams, PipelineTask
|
||||||
@@ -73,38 +71,6 @@ class DebugObserver(BaseObserver):
|
|||||||
logger.info(f"🤖 BOT STOP SPEAKING: {src} {arrow} {dst} at {time_sec:.2f}s")
|
logger.info(f"🤖 BOT STOP SPEAKING: {src} {arrow} {dst} at {time_sec:.2f}s")
|
||||||
|
|
||||||
|
|
||||||
class LLMLogObserver(BaseObserver):
|
|
||||||
"""Observer to log LLM activity to the console.
|
|
||||||
|
|
||||||
Logs all frame instances of:
|
|
||||||
- LLMFullResponseStartFrame (only from LLM service)
|
|
||||||
- LLMTextFrame
|
|
||||||
- LLMFullResponseEndFrame (only from LLM service)
|
|
||||||
|
|
||||||
This allows you to track when the LLM starts responding, what it generates, and when it finishes.
|
|
||||||
Log format: [LLM EVENT]: [details] at [timestamp]s
|
|
||||||
"""
|
|
||||||
|
|
||||||
async def on_push_frame(
|
|
||||||
self,
|
|
||||||
src: FrameProcessor,
|
|
||||||
dst: FrameProcessor,
|
|
||||||
frame: Frame,
|
|
||||||
direction: FrameDirection,
|
|
||||||
timestamp: int,
|
|
||||||
):
|
|
||||||
time_sec = timestamp / 1_000_000_000
|
|
||||||
|
|
||||||
# Only log start/end frames from OpenAILLMService
|
|
||||||
if isinstance(frame, (LLMFullResponseStartFrame, LLMFullResponseEndFrame)):
|
|
||||||
if isinstance(src, OpenAILLMService):
|
|
||||||
event = "START" if isinstance(frame, LLMFullResponseStartFrame) else "END"
|
|
||||||
logger.info(f"🧠 LLM {event} RESPONSE at {time_sec:.2f}s")
|
|
||||||
# Log all LLMTextFrames
|
|
||||||
elif isinstance(frame, LLMTextFrame):
|
|
||||||
logger.info(f"🧠 LLM GENERATING: {frame.text!r} at {time_sec:.2f}s")
|
|
||||||
|
|
||||||
|
|
||||||
async def main():
|
async def main():
|
||||||
async with aiohttp.ClientSession() as session:
|
async with aiohttp.ClientSession() as session:
|
||||||
(room_url, token) = await configure(session)
|
(room_url, token) = await configure(session)
|
||||||
@@ -151,13 +117,13 @@ async def main():
|
|||||||
|
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
report_only_initial_ttfb=True,
|
report_only_initial_ttfb=True,
|
||||||
observers=[DebugObserver(), LLMLogObserver()],
|
|
||||||
),
|
),
|
||||||
|
observers=[DebugObserver(), LLMLogObserver()],
|
||||||
)
|
)
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ async def main():
|
|||||||
|
|
||||||
pipeline = Pipeline([NullProcessor()])
|
pipeline = Pipeline([NullProcessor()])
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(enable_heartbeats=True))
|
task = PipelineTask(pipeline, params=PipelineParams(enable_heartbeats=True))
|
||||||
|
|
||||||
runner = PipelineRunner()
|
runner = PipelineRunner()
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
#
|
#
|
||||||
# Copyright (c) 2024, Daily
|
# Copyright (c) 2024-2025, Daily
|
||||||
#
|
#
|
||||||
# SPDX-License-Identifier: BSD 2-Clause License
|
# SPDX-License-Identifier: BSD 2-Clause License
|
||||||
#
|
#
|
||||||
@@ -38,7 +38,7 @@ search_tool = {"google_search_retrieval": {}}
|
|||||||
tools = [search_tool]
|
tools = [search_tool]
|
||||||
|
|
||||||
system_instruction = """
|
system_instruction = """
|
||||||
You are an expert at providing the most recent news from any place. Your responses will be converted to audio, so avoid using special characters or overly complex formatting.
|
You are an expert at providing the most recent news from any place. Your responses will be converted to audio, so avoid using special characters or overly complex formatting.
|
||||||
|
|
||||||
Always use the google search API to retrieve the latest news. You must also use it to check which day is today.
|
Always use the google search API to retrieve the latest news. You must also use it to check which day is today.
|
||||||
|
|
||||||
@@ -117,7 +117,7 @@ async def main():
|
|||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
|
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
|
||||||
|
|
||||||
@transport.event_handler("on_first_participant_joined")
|
@transport.event_handler("on_first_participant_joined")
|
||||||
async def on_first_participant_joined(transport, participant):
|
async def on_first_participant_joined(transport, participant):
|
||||||
|
|||||||
@@ -230,7 +230,7 @@ Your response will be turned into speech so use only simple words and punctuatio
|
|||||||
)
|
)
|
||||||
task = PipelineTask(
|
task = PipelineTask(
|
||||||
pipeline,
|
pipeline,
|
||||||
PipelineParams(
|
params=PipelineParams(
|
||||||
allow_interruptions=True,
|
allow_interruptions=True,
|
||||||
enable_metrics=True,
|
enable_metrics=True,
|
||||||
enable_usage_metrics=True,
|
enable_usage_metrics=True,
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user