Compare commits

..

7 Commits

Author SHA1 Message Date
Nikita Gamolsky
0265c1d3ef alllow interrupt 2024-11-02 16:12:29 -07:00
Nikita Gamolsky
ffa0e5a122 working with summary 2024-11-02 15:33:03 -07:00
Nikita Gamolsky
cdeab597b3 more variables 2024-11-02 14:05:19 -07:00
Nikita Gamolsky
abd486025b more updates 2024-11-02 13:46:28 -07:00
Nikita Gamolsky
c4cdb2d809 update to use context global 2024-11-02 13:37:35 -07:00
Nikita Gamolsky
05ba10c969 update 2024-11-02 13:27:08 -07:00
Kwindla Hultman Kramer
2f80683dc4 initial commit of screen capture in 99-anthropic-hackathon.py 2024-11-02 10:42:31 -07:00
2703 changed files with 1672 additions and 181010 deletions

View File

@@ -9,119 +9,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- `GroqLLMService` and `GrokLLMService` for Groq and Grok API integration, with
OpenAI-compatible interface.
- New examples demonstrating function calling with Groq, Grok, Azure OpenAI,
and Fireworks: `14f-function-calling-groq.py`, `14g-function-calling-grok.py`,
`14h-function-calling-azure.py`, and `14i-function-calling-fireworks.py`.
- In order to obtain the audio stored by the `AudioBufferProcessor` you can now
also register an `on_audio_data` event handler. The `on_audio_data` handler
will be called every time `buffer_size` (a new constructor argument) is
reached. If `buffer_size` is 0 (default) you need to manually get the audio as
before using `AudioBufferProcessor.merge_audio_buffers()`.
```
@audiobuffer.event_handler("on_audio_data")
async def on_audio_data(processor, audio, sample_rate, num_channels):
await save_audio(audio, sample_rate, num_channels)
```
- Added a new RTVI message called `disconnect-bot`, which when handled pushes
an `EndFrame` to trigger the pipeline to stop.
### Changed
- All input frames (text, audio, image, etc.) are now system frames. This means
they are processed immediately by all processors instead of being queued
internally.
- Expanded the transcriptions.language module to support a superset of
languages.
- Updated STT and TTS services with language options that match the supported
languages for each service.
- Updated the `AzureLLMService` to use the `OpenAILLMService`. Updated the
`api_version` to `2024-09-01-preview`.
- Updated the `FireworksLLMService` to use the `OpenAILLMService`. Updated the
default model to `accounts/fireworks/models/firefunction-v2`.
### Removed
- Removed `AppFrame`. This was used as a special user custom frame, but there's
actually no use case for that.
### Fixed
- Fixed an issue in `DailyTransport` that would cause some internal callbacks to
not be executed.
- Fixed an issue where other frames were being processed while a `CancelFrame`
was being pushed down the pipeline.
- `AudioBufferProcessor` now handles interruptions properly.
- Fixed a `WebsocketServerTransport` issue that would prevent interruptions with
`TwilioSerializer` from working.
- `DailyTransport.capture_participant_video` now allows capturing user's screen
share by simply passing `video_source="screenVideo"`.
- Fixed Google Gemini message handling to properly convert appended messages to
Gemini's required format.
- Fixed an issue with `FireworksLLMService` where chat completions were failing
by removing the `stream_options` from the chat completion options.
## [0.0.49] - 2024-11-17
### Added
- Added RTVI `on_bot_started` event which is useful in a single turn
interaction.
- Added `DailyTransport` events `dialin-connected`, `dialin-stopped`,
`dialin-error` and `dialin-warning`. Needs daily-python >= 0.13.0.
- Added `RimeHttpTTSService` and the `07q-interruptible-rime.py` foundational
example.
- Added `STTMuteFilter`, a general-purpose processor that combines STT
muting and interruption control. When active, it prevents both transcription
and interruptions during bot speech. The processor supports multiple
strategies: `FIRST_SPEECH` (mute only during bot's first
speech), `ALWAYS` (mute during all bot speech), or `CUSTOM` (using provided
callback).
- Added `STTMuteFrame`, a control frame that enables/disables speech
transcription in STT services.
## [0.0.48] - 2024-11-10 "Antonio release"
### Added
- There's now an input queue in each frame processor. When you call
`FrameProcessor.push_frame()` this will internally call
`FrameProcessor.queue_frame()` on the next processor (upstream or downstream)
and the frame will be internally queued (except system frames). Then, the
queued frames will get processed. With this input queue it is also possible
for FrameProcessors to block processing more frames by calling
`FrameProcessor.pause_processing_frames()`. The way to resume processing
frames is by calling `FrameProcessor.resume_processing_frames()`.
- Added audio filter `NoisereduceFilter`.
- Introduce input transport audio filters (`BaseAudioFilter`). Audio filters can
be used to remove background noises before audio is sent to VAD.
- Introduce output transport audio mixers (`BaseAudioMixer`). Output transport
audio mixers can be used, for example, to add background sounds or any other
audio mixing functionality before the output audio is actually written to the
transport.
- Added `GatedOpenAILLMContextAggregator`. This aggregator keeps the last
received OpenAI LLM context frame and it doesn't let it through until the
notifier is notified.
@@ -144,8 +31,6 @@ async def on_audio_data(processor, audio, sample_rate, num_channels):
grained control of what media subscriptions you want for each participant in a
room.
- Added audio filter `KrispFilter`.
### Changed
- The following `DailyTransport` functions are now `async` which means they need
@@ -157,16 +42,8 @@ async def on_audio_data(processor, audio, sample_rate, num_channels):
output to 24000 and also the default output transport sample rate. This
improves audio quality at the cost of some extra bandwidth.
- `AzureTTSService` now uses Azure websockets instead of HTTP requests.
- The previous `AzureTTSService` HTTP implementation is now
`AzureHttpTTSService`.
### Fixed
- Websocket transports (FastAPI and Websocket) now synchronize with time before
sending data. This allows for interruptions to just work out of the box.
- Improved bot speaking detection for all TTS services by using actual bot
audio.
@@ -178,14 +55,9 @@ async def on_audio_data(processor, audio, sample_rate, num_channels):
- Fixed an issue with PlayHTTTSService, where the TTFB metrics were reporting
very small time values.
- Fixed an issue where AzureTTSService wasn't initializing the specified
language.
### Other
- Add `23-bot-background-sound.py` foundational example.
- Added a new foundational example `22-natural-conversation.py`. This example
- Added a new foundational example 22-natural-conversation.py. This examples
shows how to achieve a more natural conversation detecting when the user ends
statement.

View File

@@ -1,21 +1,14 @@
<h1><div align="center">
<div align="center">
 <img alt="pipecat" width="300px" height="auto" src="https://raw.githubusercontent.com/pipecat-ai/pipecat/main/pipecat.png">
</div></h1>
</div>
# Pipecat
[![PyPI](https://img.shields.io/pypi/v/pipecat-ai)](https://pypi.org/project/pipecat-ai) [![Discord](https://img.shields.io/discord/1239284677165056021)](https://discord.gg/pipecat) <a href="https://app.commanddash.io/agent/github_pipecat-ai_pipecat"><img src="https://img.shields.io/badge/AI-Code%20Agent-EB9FDA"></a>
Pipecat is an open source Python framework for building voice and multimodal conversational agents. It handles the complex orchestration of AI services, network transport, audio processing, and multimodal interactions, letting you focus on creating engaging experiences.
`pipecat` is a framework for building voice (and multimodal) conversational agents. Things like personal coaches, meeting assistants, [story-telling toys for kids](https://storytelling-chatbot.fly.dev/), customer support bots, [intake flows](https://www.youtube.com/watch?v=lDevgsp9vn0), and snarky social companions.
## What you can build
- **Voice Assistants**: [Natural, real-time conversations with AI](https://demo.dailybots.ai/)
- **Interactive Agents**: Personal coaches and meeting assistants
- **Multimodal Apps**: Combine voice, video, images, and text
- **Creative Tools**: [Story-telling experiences](https://storytelling-chatbot.fly.dev/) and social companions
- **Business Solutions**: [Customer intake flows](https://www.youtube.com/watch?v=lDevgsp9vn0) and support bots
- **Complex conversational flows**: [Refer to Pipecat Flows](https://github.com/pipecat-ai/pipecat-flows) to learn more
## See it in action
Take a look at some example apps:
<p float="left">
<a href="https://github.com/pipecat-ai/pipecat/tree/main/examples/simple-chatbot"><img src="https://raw.githubusercontent.com/pipecat-ai/pipecat/main/examples/simple-chatbot/image.png" width="280" /></a>&nbsp;
@@ -25,54 +18,33 @@ Pipecat is an open source Python framework for building voice and multimodal con
<a href="https://github.com/pipecat-ai/pipecat/tree/main/examples/moondream-chatbot"><img src="https://raw.githubusercontent.com/pipecat-ai/pipecat/main/examples/moondream-chatbot/image.png" width="280" /></a>
</p>
## Key features
- **Voice-first Design**: Built-in speech recognition, TTS, and conversation handling
- **Flexible Integration**: Works with popular AI services (OpenAI, ElevenLabs, etc.)
- **Pipeline Architecture**: Build complex apps from simple, reusable components
- **Real-time Processing**: Frame-based pipeline architecture for fluid interactions
- **Production Ready**: Enterprise-grade WebRTC and Websocket support
💡 Looking to build structured conversations? Check out [Pipecat Flows](https://github.com/pipecat-ai/pipecat-flows) for managing complex conversational states and transitions.
## Getting started
## Getting started with voice agents
You can get started with Pipecat running on your local machine, then move your agent processes to the cloud when youre ready. You can also add a 📞 telephone number, 🖼️ image output, 📺 video input, use different LLMs, and more.
```shell
# Install the module
# install the module
pip install pipecat-ai
# Set up your environment
# set up an .env file with API keys
cp dot-env.template .env
```
To keep things lightweight, only the core framework is included by default. If you need support for third-party AI services, you can add the necessary dependencies with:
By default, in order to minimize dependencies, only the basic framework functionality is available. Some third-party AI services require additional dependencies that you can install with:
```shell
pip install "pipecat-ai[option,...]"
```
Available options include:
Your project may or may not need these, so they're made available as optional requirements. Here is a list:
| Category | Services | Install Command Example |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/api-reference/services/stt/assemblyai), [Azure](https://docs.pipecat.ai/api-reference/services/stt/azure), [Deepgram](https://docs.pipecat.ai/api-reference/services/stt/deepgram), [Gladia](https://docs.pipecat.ai/api-reference/services/stt/gladia), [Whisper](https://docs.pipecat.ai/api-reference/services/stt/whisper) | `pip install "pipecat-ai[deepgram]"` |
| LLMs | [Anthropic](https://docs.pipecat.ai/api-reference/services/llm/anthropic), [Azure](https://docs.pipecat.ai/api-reference/services/llm/azure), [Fireworks AI](https://docs.pipecat.ai/api-reference/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/api-reference/services/llm/gemini), [Grok](https://docs.pipecat.ai/api-reference/services/llm/grok), [Groq](https://docs.pipecat.ai/api-reference/services/llm/groq) [Ollama](https://docs.pipecat.ai/api-reference/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/api-reference/services/llm/openai), [Together AI](https://docs.pipecat.ai/api-reference/services/llm/together) | `pip install "pipecat-ai[openai]"` |
| Text-to-Speech | [AWS](https://docs.pipecat.ai/api-reference/services/tts/aws), [Azure](https://docs.pipecat.ai/api-reference/services/tts/azure), [Cartesia](https://docs.pipecat.ai/api-reference/services/tts/cartesia), [Deepgram](https://docs.pipecat.ai/api-reference/services/tts/deepgram), [ElevenLabs](https://docs.pipecat.ai/api-reference/services/tts/elevenlabs), [Google](https://docs.pipecat.ai/api-reference/services/tts/google), [LMNT](https://docs.pipecat.ai/api-reference/services/tts/lmnt), [OpenAI](https://docs.pipecat.ai/api-reference/services/tts/openai), [PlayHT](https://docs.pipecat.ai/api-reference/services/tts/playht), [Rime](https://docs.pipecat.ai/api-reference/services/tts/rime), [XTTS](https://docs.pipecat.ai/api-reference/services/tts/xtts) | `pip install "pipecat-ai[cartesia]"` |
| Speech-to-Speech | [OpenAI Realtime](https://docs.pipecat.ai/api-reference/services/s2s/openai) | `pip install "pipecat-ai[openai]"` |
| Transport | [Daily (WebRTC)](https://docs.pipecat.ai/api-reference/services/transport/daily), WebSocket, Local | `pip install "pipecat-ai[daily]"` |
| Video | [Tavus](https://docs.pipecat.ai/api-reference/services/video/tavus) | `pip install "pipecat-ai[tavus]"` |
| Vision & Image | [Moondream](https://docs.pipecat.ai/api-reference/services/vision/moondream), [fal](https://docs.pipecat.ai/api-reference/services/image-generation/fal) | `pip install "pipecat-ai[moondream]"` |
| Audio Processing | [Silero VAD](https://docs.pipecat.ai/api-reference/utilities/audio/silero-vad-analyzer), [Krisp](https://docs.pipecat.ai/api-reference/utilities/audio/krisp-filter), [Noisereduce](https://docs.pipecat.ai/api-reference/utilities/audio/noisereduce-filter) | `pip install "pipecat-ai[silero]"` |
| Analytics & Metrics | [Canonical AI](https://docs.pipecat.ai/api-reference/services/analytics/canonical), [Sentry](https://docs.pipecat.ai/api-reference/services/analytics/sentry) | `pip install "pipecat-ai[canonical]"` |
📚 [View full services documentation →](https://docs.pipecat.ai/api-reference/services/supported-services)
- **AI services**: `anthropic`, `assemblyai`, `aws`, `azure`, `deepgram`, `gladia`, `google`, `fal`, `lmnt`, `moondream`, `openai`, `openpipe`, `playht`, `silero`, `whisper`, `xtts`
- **Transports**: `local`, `websocket`, `daily`
## Code examples
- [Foundational](https://github.com/pipecat-ai/pipecat/tree/main/examples/foundational) — small snippets that build on each other, introducing one or two concepts at a time
- [Example apps](https://github.com/pipecat-ai/pipecat/tree/main/examples/) — complete applications that you can use as starting points for development
- [foundational](https://github.com/pipecat-ai/pipecat/tree/main/examples/foundational) — small snippets that build on each other, introducing one or two concepts at a time
- [example apps](https://github.com/pipecat-ai/pipecat/tree/main/examples/) — complete applications that you can use as starting points for development
## A simple voice agent running locally
@@ -137,7 +109,7 @@ Run it with:
python app.py
```
Daily provides a prebuilt WebRTC user interface. While the app is running, you can visit at `https://<yourdomain>.daily.co/<room_url>` and listen to the bot say hello!
Daily provides a prebuilt WebRTC user interface. Whilst the app is running, you can visit at `https://<yourdomain>.daily.co/<room_url>` and listen to the bot say hello!
## WebRTC for production use
@@ -147,6 +119,16 @@ One way to get up and running quickly with WebRTC is to sign up for a Daily deve
Sign up [here](https://dashboard.daily.co/u/signup) and [create a room](https://docs.daily.co/reference/rest-api/rooms) in the developer Dashboard.
## What is VAD?
Voice Activity Detection &mdash; very important for knowing when a user has finished speaking to your bot. If you are not using press-to-talk, and want Pipecat to detect when the user has finished talking, VAD is an essential component for a natural feeling conversation.
Pipecat makes use of WebRTC VAD by default when using a WebRTC transport layer. Optionally, you can use Silero VAD for improved accuracy at the cost of higher CPU usage.
```shell
pip install pipecat-ai[silero]
```
## Hacking on the framework itself
_Note that you may need to set up a virtual environment before following the instructions below. For instance, you might need to run the following from the root of the repo:_
@@ -224,23 +206,8 @@ Install the
}
```
## Contributing
We welcome contributions from the community! Whether you're fixing bugs, improving documentation, or adding new features, here's how you can help:
- **Found a bug?** Open an [issue](https://github.com/pipecat-ai/pipecat/issues)
- **Have a feature idea?** Start a [discussion](https://discord.gg/pipecat)
- **Want to contribute code?** Check our [CONTRIBUTING.md](CONTRIBUTING.md) guide
- **Documentation improvements?** [Docs](https://github.com/pipecat-ai/docs) PRs are always welcome
Before submitting a pull request, please check existing issues and PRs to avoid duplicates.
We aim to review all contributions promptly and provide constructive feedback to help get your changes merged.
## Getting help
➡️ [Join our Discord](https://discord.gg/pipecat)
➡️ [Read the docs](https://docs.pipecat.ai)
➡️ [Reach us on X](https://x.com/pipecat_ai)

View File

@@ -1,110 +0,0 @@
# Understanding Different Frame Types in the Pipecat System
In the Pipecat system, frames are used to represent different types of data and control signals that flow through the pipeline. Understanding these frame types is crucial for working with the system effectively. This tutorial will cover the main categories of frames and their specific uses.
## 1. Base Frame Classes
### Frame
The `Frame` class is the base class for all frames. It includes:
- `id`: A unique identifier
- `name`: A descriptive name
- `pts`: Presentation timestamp (optional)
### DataFrame
`DataFrame` is a subclass of `Frame` and serves as a base for most data-carrying frames.
## 2. Audio Frames
### AudioRawFrame
Represents a chunk of audio with properties:
- `audio`: Raw audio data
- `sample_rate`: Audio sample rate
- `num_channels`: Number of audio channels
Subclasses include:
- `InputAudioRawFrame`: For audio from input sources
- `OutputAudioRawFrame`: For audio to be played by output devices
- `TTSAudioRawFrame`: For audio generated by Text-to-Speech services
## 3. Image Frames
### ImageRawFrame
Represents an image with properties:
- `image`: Raw image data
- `size`: Image dimensions
- `format`: Image format (e.g., JPEG, PNG)
Subclasses include:
- `InputImageRawFrame`: For images from input sources
- `OutputImageRawFrame`: For images to be displayed
- `UserImageRawFrame`: For images associated with a specific user
- `VisionImageRawFrame`: For images with associated text for description
- `URLImageRawFrame`: For images with an associated URL
### SpriteFrame
Represents an animated sprite, containing a list of `ImageRawFrame` objects.
## 4. Text and Transcription Frames
### TextFrame
Represents a chunk of text, used for various purposes in the pipeline.
### TranscriptionFrame
A specialized `TextFrame` for speech transcriptions, including:
- `user_id`: ID of the speaking user
- `timestamp`: When the transcription was generated
- `language`: Detected language of the speech
### InterimTranscriptionFrame
Similar to `TranscriptionFrame`, but for interim (not final) transcriptions.
## 5. LLM (Language Model) Frames
### LLMMessagesFrame
Contains a list of messages for an LLM service to process.
### LLMMessagesAppendFrame and LLMMessagesUpdateFrame
Used to modify the current context of LLM messages.
### LLMSetToolsFrame
Specifies tools (functions) available for the LLM to use.
### LLMEnablePromptCachingFrame
Controls prompt caching in certain LLMs.
## 6. System and Control Frames
### SystemFrame
Base class for system-level frames.
Important system frames include:
- `StartFrame`: Initiates a pipeline
- `CancelFrame`: Stops a pipeline immediately
- `ErrorFrame`: Notifies of errors (with `FatalErrorFrame` for unrecoverable errors)
- `EndTaskFrame` and `CancelTaskFrame`: Control pipeline tasks
- `StartInterruptionFrame` and `StopInterruptionFrame`: Indicate user speech for interruptions
### ControlFrame
Base class for control-flow frames.
Notable control frames:
- `EndFrame`: Signals the end of a pipeline
- `LLMFullResponseStartFrame` and `LLMFullResponseEndFrame`: Bracket LLM responses
- `UserStartedSpeakingFrame` and `UserStoppedSpeakingFrame`: Indicate user speech activity
- `BotStartedSpeakingFrame` and `BotStoppedSpeakingFrame`: Indicate bot speech activity
- `TTSStartedFrame` and `TTSStoppedFrame`: Bracket Text-to-Speech responses
## 7. Special Purpose Frames
### MetricsFrame
Contains performance metrics data.
### FunctionCallInProgressFrame and FunctionCallResultFrame
Used for handling LLM function (tool) calls.
### ServiceUpdateSettingsFrame
Base class for updating service settings, with specific subclasses for LLM, TTS, and STT services.
## Conclusion
Understanding these frame types is essential for working with the Pipecat system. Each frame type serves a specific purpose in the pipeline, whether it's carrying data (like audio or images), controlling the flow of the pipeline, or managing system-level operations. By using the appropriate frame types, you can effectively process and transmit various kinds of information through your pipeline.

View File

@@ -52,7 +52,4 @@ OPENPIPE_API_KEY=...
# Tavus
TAVUS_API_KEY=...
TAVUS_REPLICA_ID=...
TAVUS_PERSONA_ID=...
#Krisp
KRISP_MODEL_PATH=...
TAVUS_PERSONA_ID=...

View File

@@ -42,7 +42,6 @@ Next, follow the steps in the README for each demo.
| [Dialin Chatbot](dialin-chatbot) | A chatbot that connects to an incoming phone call from Daily or Twilio. | Deepgram, ElevenLabs, OpenAI, Daily, Twilio |
| [Twilio Chatbot](twilio-chatbot) | A chatbot that connects to an incoming phone call from Twilio. | Deepgram, ElevenLabs, OpenAI, Daily, Twilio |
| [studypal](studypal) | A chatbot to have a conversation about any article on the web | |
| [WebSocket Chatbot Server](websocket-server) | A real-time websocket server that handles audio streaming and bot interactions with speech-to-text and text-to-speech capabilities | `python-websockets`, `openai`, `deepgram`, `silero-tts`, `numpy` |
> [!IMPORTANT]
> These example projects use Daily as a WebRTC transport and can be joined using their hosted Prebuilt UI.

View File

@@ -102,6 +102,7 @@ async def main():
audio_buffer_processor=audio_buffer_processor,
aiohttp_session=session,
api_key=os.getenv("CANONICAL_API_KEY"),
api_url=os.getenv("CANONICAL_API_URL"),
call_id=str(uuid.uuid4()),
assistant="pipecat-chatbot",
assistant_speaks_first=True,

View File

@@ -4,9 +4,7 @@
# SPDX-License-Identifier: BSD 2-Clause License
#
import aiofiles
import asyncio
import io
import os
import sys
@@ -34,17 +32,15 @@ logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def save_audio(audio: bytes, sample_rate: int, num_channels: int):
if len(audio) > 0:
async def save_audio(audiobuffer):
if audiobuffer.has_audio():
merged_audio = audiobuffer.merge_audio_buffers()
filename = f"conversation_recording{datetime.datetime.now().strftime('%Y%m%d_%H%M%S')}.wav"
with io.BytesIO() as buffer:
with wave.open(buffer, "wb") as wf:
wf.setsampwidth(2)
wf.setnchannels(num_channels)
wf.setframerate(sample_rate)
wf.writeframes(audio)
async with aiofiles.open(filename, "wb") as file:
await file.write(buffer.getvalue())
with wave.open(filename, "wb") as wf:
wf.setnchannels(2)
wf.setsampwidth(2)
wf.setframerate(audiobuffer._sample_rate)
wf.writeframes(merged_audio)
print(f"Merged audio saved to {filename}")
else:
print("No audio data to save")
@@ -110,9 +106,7 @@ async def main():
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
# Save audio every 10 seconds.
audiobuffer = AudioBufferProcessor(buffer_size=480000)
audiobuffer = AudioBufferProcessor()
pipeline = Pipeline(
[
transport.input(), # microphone
@@ -127,10 +121,6 @@ async def main():
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
@audiobuffer.event_handler("on_audio_data")
async def on_audio_data(buffer, audio, sample_rate, num_channels):
await save_audio(audio, sample_rate, num_channels)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
@@ -140,6 +130,7 @@ async def main():
async def on_participant_left(transport, participant, reason):
print(f"Participant left: {participant}")
await task.queue_frame(EndFrame())
await save_audio(audiobuffer)
runner = PipelineRunner()

View File

@@ -1,4 +1,3 @@
aiofiles
python-dotenv
fastapi[all]
uvicorn

View File

@@ -1,91 +0,0 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
dist/
*.egg-info/
*.egg
.installed.cfg
.eggs/
downloads/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
MANIFEST
# Virtual Environments
venv/
env/
.env
.venv/
ENV/
env.bak/
venv.bak/
# IDE
.idea/
.vscode/
.spyderproject
.spyproject
.ropeproject
# Testing and Coverage
.coverage
.coverage.*
htmlcov/
.pytest_cache/
.tox/
.nox/
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
cover/
# Logs and Databases
*.log
*.db
db.sqlite3
db.sqlite3-journal
pip-log.txt
# System Files
.DS_Store
Thumbs.db
desktop.ini
*.swp
*.swo
*.bak
*.tmp
*~
# Build and Documentation
docs/_build/
.pybuilder/
target/
instance/
.webassets-cache
.pdm.toml
.pdm-python
.pdm-build/
__pypackages__/
# Other
*.mo
*.pot
*.sage.py
.mypy_cache/
.dmypy.json
dmypy.json
.pyre/
.pytype/
cython_debug/
.ipynb_checkpoints

View File

@@ -1,37 +0,0 @@
# Deploying Pipecat to Modal.com
Barebones deployment example for [modal.com](https://www.modal.com)
1. Install dependencies
```bash
python -m venv venv
source venv/bin/active # or OS equivalent
pip install -r requirements.txt
```
2. Setup .env
```bash
cp env.example .env
```
Alternatively, you can configure your Modal app to use [secrets](https://modal.com/docs/guide/secrets)
3. Test the app locally
```bash
modal serve app.py
```
4. Deploy to production
```bash
modal deploy app.py
```
## Configuration options
This app sets some sensible defaults for reducing cold starts, such as `minkeep_warm=1`, which will keep at least 1 warm instance ready for your bot function.
It has been configured to only allow a concurrency of 1 (`max_inputs=1`) as each user will require their own running function.

View File

@@ -1,75 +0,0 @@
import os
import aiohttp
import modal
from fastapi import HTTPException
from fastapi.responses import JSONResponse
from loguru import logger
from bot import _voice_bot_process
MAX_SESSION_TIME = 15 * 60 # 15 minutes
app = modal.App("pipecat-modal")
image = modal.Image.debian_slim(python_version="3.12").pip_install_from_requirements(
"requirements.txt"
)
@app.function(
image=image,
cpu=1.0,
secrets=[modal.Secret.from_dotenv()],
keep_warm=1,
enable_memory_snapshot=True,
max_inputs=1, # Do not reuse instances across requests
retries=0,
)
def launch_bot_process(room_url: str, token: str):
_voice_bot_process(room_url, token)
@app.function(
image=image,
secrets=[modal.Secret.from_dotenv()],
)
@modal.web_endpoint(method="POST")
async def start():
from pipecat.transports.services.helpers.daily_rest import (
DailyRESTHelper,
DailyRoomParams,
)
logger.info("Request received")
async with aiohttp.ClientSession() as session:
daily_rest_helper = DailyRESTHelper(
daily_api_key=os.getenv("DAILY_API_KEY", ""),
daily_api_url=os.getenv("DAILY_API_URL", "https://api.daily.co/v1"),
aiohttp_session=session,
)
# Create new Daily room
room = await daily_rest_helper.create_room(DailyRoomParams())
if not room.url:
raise HTTPException(
status_code=500,
detail="Unable to create room",
)
logger.info(f"Created room: {room.url}")
# Create bot token for room
token = await daily_rest_helper.get_token(room.url, MAX_SESSION_TIME)
if not token:
raise HTTPException(status_code=500, detail=f"Failed to get token for room: {room.url}")
logger.info(f"Bot token created: {token}")
# Spawn a new bot process
launch_bot_process.spawn(room_url=room.url, token=token)
# Return room URL to the user to join
# Note: in production, you would want to return a token to the user
return JSONResponse(content={"room_url": room.url, token: token})

View File

@@ -1,90 +0,0 @@
import asyncio
import os
import sys
from dotenv import load_dotenv
from loguru import logger
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def main(room_url: str, token: str):
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import EndFrame, LLMMessagesFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.openai import OpenAILLMService
from pipecat.transports.services.daily import DailyParams, DailyTransport
transport = DailyTransport(
room_url,
token,
"bot",
DailyParams(
audio_out_enabled=True,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY", ""), voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22"
)
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(),
context_aggregator.user(),
llm,
tts,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
messages.append({"role": "system", "content": "Please introduce yourself to the user."})
await task.queue_frames([LLMMessagesFrame(messages)])
@transport.event_handler("on_participant_left")
async def on_participant_left(transport, participant, reason):
await task.queue_frame(EndFrame())
runner = PipelineRunner()
await runner.run(task)
def _voice_bot_process(room_url: str, token: str):
asyncio.run(main(room_url, token))

View File

@@ -1,3 +0,0 @@
DAILY_API_KEY=
OPENAI_API_KEY=
CARTESIA_API_KEY=

View File

@@ -1,5 +0,0 @@
python-dotenv==1.0.1
modal==0.65.48
pipecat-ai[daily,silero,cartesia,openai]==0.0.48
fastapi==0.115.4
aiohttp==3.10.10

View File

@@ -9,11 +9,11 @@ import aiohttp
import os
import sys
from pipecat.frames.frames import EndFrame, TTSSpeakFrame
from pipecat.frames.frames import EndFrame, TextFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.task import PipelineTask
from pipecat.pipeline.runner import PipelineRunner
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.cartesia import CartesiaHttpTTSService
from pipecat.transports.services.daily import DailyParams, DailyTransport
from runner import configure
@@ -36,7 +36,7 @@ async def main():
room_url, None, "Say One Thing", DailyParams(audio_out_enabled=True)
)
tts = CartesiaTTSService(
tts = CartesiaHttpTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
@@ -50,9 +50,12 @@ async def main():
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
participant_name = participant.get("info", {}).get("userName", "")
await task.queue_frames(
[TTSSpeakFrame(f"Hello there, {participant_name}!"), EndFrame()]
)
await task.queue_frame(TextFrame(f"Hello there, {participant_name}!"))
# Register an event handler to exit the application when the user leaves.
@transport.event_handler("on_participant_left")
async def on_participant_left(transport, participant, reason):
await task.queue_frame(EndFrame())
await runner.run(task)

View File

@@ -9,7 +9,7 @@ import aiohttp
import os
import sys
from pipecat.frames.frames import EndFrame, TTSSpeakFrame
from pipecat.frames.frames import TextFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
@@ -28,24 +28,25 @@ logger.add(sys.stderr, level="DEBUG")
async def main():
transport = LocalAudioTransport(TransportParams(audio_out_enabled=True))
async with aiohttp.ClientSession() as session:
transport = LocalAudioTransport(TransportParams(audio_out_enabled=True))
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
pipeline = Pipeline([tts, transport.output()])
pipeline = Pipeline([tts, transport.output()])
task = PipelineTask(pipeline)
task = PipelineTask(pipeline)
async def say_something():
await asyncio.sleep(1)
await task.queue_frames([TTSSpeakFrame("Hello there, how is it going!"), EndFrame()])
async def say_something():
await asyncio.sleep(1)
await task.queue_frame(TextFrame("Hello there!"))
runner = PipelineRunner()
runner = PipelineRunner()
await asyncio.gather(runner.run(task), say_something())
await asyncio.gather(runner.run(task), say_something())
if __name__ == "__main__":

View File

@@ -13,7 +13,7 @@ from pipecat.frames.frames import EndFrame, LLMMessagesFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.cartesia import CartesiaHttpTTSService
from pipecat.services.openai import OpenAILLMService
from pipecat.transports.services.daily import DailyParams, DailyTransport
@@ -37,7 +37,7 @@ async def main():
room_url, None, "Say One Thing From an LLM", DailyParams(audio_out_enabled=True)
)
tts = CartesiaTTSService(
tts = CartesiaHttpTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
@@ -57,7 +57,11 @@ async def main():
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await task.queue_frames([LLMMessagesFrame(messages), EndFrame()])
await task.queue_frame(LLMMessagesFrame(messages))
@transport.event_handler("on_participant_left")
async def on_participant_left(transport, participant, reason):
await task.queue_frame(EndFrame())
await runner.run(task)

View File

@@ -12,7 +12,7 @@ import sys
from dataclasses import dataclass
from pipecat.frames.frames import (
DataFrame,
AppFrame,
Frame,
LLMFullResponseStartFrame,
LLMMessagesFrame,
@@ -42,7 +42,7 @@ logger.add(sys.stderr, level="DEBUG")
@dataclass
class MonthFrame(DataFrame):
class MonthFrame(AppFrame):
month: str
def __str__(self):

View File

@@ -31,11 +31,11 @@ logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
(room_url, _) = await configure(session)
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
None,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,

View File

@@ -49,7 +49,7 @@ async def main():
tts = PlayHTTTSService(
user_id=os.getenv("PLAYHT_USER_ID"),
api_key=os.getenv("PLAYHT_API_KEY"),
voice_url="s3://voice-cloning-zero-shot/d9ff78ba-d016-47f6-b0ef-dd630f59414e/female-cs/manifest.json",
voice_url="s3://voice-cloning-zero-shot/801a663f-efd0-4254-98d0-5c175514c3e8/jennifer/manifest.json",
params=PlayHTTTSService.InputParams(language=Language.EN),
)

View File

@@ -32,11 +32,11 @@ logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
(room_url, _) = await configure(session)
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
None,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,

View File

@@ -32,11 +32,11 @@ logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
(room_url, _) = await configure(session)
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
None,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,

View File

@@ -1,278 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import aiohttp
import asyncio
import os
import sys
import google.ai.generativelanguage as glm
from dataclasses import dataclass
from dotenv import load_dotenv
from loguru import logger
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.google import GoogleLLMService
from pipecat.processors.frame_processor import FrameProcessor
from pipecat.transports.services.daily import DailyParams, DailyTransport
from pipecat.frames.frames import (
LLMFullResponseStartFrame,
LLMFullResponseEndFrame,
InputAudioRawFrame,
Frame,
StartInterruptionFrame,
TextFrame,
TranscriptionFrame,
UserStartedSpeakingFrame,
UserStoppedSpeakingFrame,
)
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
marker = "|----|"
system_message = f"""
You are a helpful LLM in a WebRTC call. Your goals are to be helpful and brief in your responses.
You are expert at transcribing audio to text. You will receive a mixture of audio and text input. When
asked to transcribe what the user said, output an exact, word-for-word transcription.
Your output will be converted to audio so don't include special characters in your answers.
Each time you answer, you should respond in three parts.
1. Transcribe exactly what the user said.
2. Output the separator field '{marker}'.
3. Respond to the user's input in a helpful, creative way using only simple text and punctuation.
Example:
User: How many ounces are in a pound?
You: How many ounces are in a pound?
{marker}
There are 16 ounces in a pound.
"""
@dataclass
class MagicDemoTranscriptionFrame(Frame):
text: str
class UserAudioCollector(FrameProcessor):
def __init__(self, context, user_context_aggregator):
super().__init__()
self._context = context
self._user_context_aggregator = user_context_aggregator
self._audio_frames = []
self._start_secs = 0.2 # this should match VAD start_secs (hardcoding for now)
self._user_speaking = False
async def process_frame(self, frame, direction):
await super().process_frame(frame, direction)
if isinstance(frame, TranscriptionFrame):
# We could gracefully handle both audio input and text/transcription input ...
# but let's leave that as an exercise to the reader. :-)
return
if isinstance(frame, UserStartedSpeakingFrame):
self._user_speaking = True
elif isinstance(frame, UserStoppedSpeakingFrame):
self._user_speaking = False
self._context.add_audio_frames_message(audio_frames=self._audio_frames)
await self._user_context_aggregator.push_frame(
self._user_context_aggregator.get_context_frame()
)
elif isinstance(frame, InputAudioRawFrame):
if self._user_speaking:
self._audio_frames.append(frame)
else:
# Append the audio frame to our buffer. Treat the buffer as a ring buffer, dropping the oldest
# frames as necessary. Assume all audio frames have the same duration.
self._audio_frames.append(frame)
frame_duration = len(frame.audio) / 16 * frame.num_channels / frame.sample_rate
buffer_duration = frame_duration * len(self._audio_frames)
while buffer_duration > self._start_secs:
self._audio_frames.pop(0)
buffer_duration -= frame_duration
await self.push_frame(frame, direction)
class TranscriptExtractor(FrameProcessor):
def __init__(self, context):
super().__init__()
self._context = context
self._accumulator = ""
self._processing_llm_response = False
self._accumulating_transcript = False
def reset(self):
self._accumulator = ""
self._processing_llm_response = False
self._accumulating_transcript = False
async def process_frame(self, frame, direction):
await super().process_frame(frame, direction)
if isinstance(frame, LLMFullResponseStartFrame):
self._processing_llm_response = True
self._accumulating_transcript = True
elif isinstance(frame, TextFrame) and self._processing_llm_response:
if self._accumulating_transcript:
text = frame.text
split_index = text.find(marker)
if split_index < 0:
self._accumulator += frame.text
# do not push this frame
return
else:
self._accumulating_transcript = False
self._accumulator += text[:split_index]
frame.text = text[split_index + len(marker) :]
await self.push_frame(frame)
return
elif isinstance(frame, LLMFullResponseEndFrame):
await self.push_frame(MagicDemoTranscriptionFrame(text=self._accumulator.strip()))
self.reset()
await self.push_frame(frame, direction)
class TanscriptionContextFixup(FrameProcessor):
def __init__(self, context):
super().__init__()
self._context = context
self._transcript = "THIS IS A TRANSCRIPT"
def swap_user_audio(self):
if not self._transcript:
return
message = self._context.messages[-2]
last_part = message.parts[-1]
if (
message.role == "user"
and last_part.inline_data
and last_part.inline_data.mime_type == "audio/wav"
):
self._context.messages[-2] = glm.Content(
role="user", parts=[glm.Part(text=self._transcript)]
)
def add_transcript_back_to_inference_output(self):
if not self._transcript:
return
message = self._context.messages[-1]
last_part = message.parts[-1]
if message.role == "model" and last_part.text:
self._context.messages[-1].parts[-1].text += f"\n\n{marker}\n{self._transcript}\n"
async def process_frame(self, frame, direction):
await super().process_frame(frame, direction)
if isinstance(frame, MagicDemoTranscriptionFrame):
self._transcript = frame.text
elif isinstance(frame, LLMFullResponseEndFrame) or isinstance(
frame, StartInterruptionFrame
):
self.swap_user_audio()
self.add_transcript_back_to_inference_output()
self._transcript = ""
await self.push_frame(frame, direction)
async def main():
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
# No transcription at all. just audio input to Gemini!
# transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
vad_audio_passthrough=True,
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = GoogleLLMService(
model="gemini-1.5-flash-latest",
# model="gemini-exp-1114",
api_key=os.getenv("GOOGLE_API_KEY"),
)
messages = [
{
"role": "system",
"content": system_message,
},
{
"role": "user",
"content": "Start by saying hello.",
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
audio_collector = UserAudioCollector(context, context_aggregator.user())
pull_transcript_out_of_llm_output = TranscriptExtractor(context)
fixup_context_messages = TanscriptionContextFixup(context)
pipeline = Pipeline(
[
transport.input(), # Transport user input
audio_collector,
context_aggregator.user(), # User responses
llm, # LLM
pull_transcript_out_of_llm_output,
tts, # TTS
transport.output(), # Transport bot output
context_aggregator.assistant(), # Assistant spoken responses
fixup_context_messages,
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,95 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from runner import configure
from pipecat.frames.frames import LLMMessagesFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.llm_response import (
LLMAssistantResponseAggregator,
LLMUserResponseAggregator,
)
from pipecat.services.deepgram import DeepgramSTTService, DeepgramTTSService
from pipecat.services.openai import OpenAILLMService
from pipecat.transports.services.daily import DailyParams, DailyTransport
from pipecat.vad.silero import SileroVADAnalyzer
from pipecat.audio.filters.krisp_filter import KrispFilter
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
vad_audio_passthrough=True,
audio_in_filter=KrispFilter(),
),
)
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
tts = DeepgramTTSService(api_key=os.getenv("DEEPGRAM_API_KEY"), voice="aura-helios-en")
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
tma_in = LLMUserResponseAggregator(messages)
tma_out = LLMAssistantResponseAggregator(messages)
pipeline = Pipeline(
[
transport.input(), # Transport user input
stt, # STT
tma_in, # User responses
llm, # LLM
tts, # TTS
transport.output(), # Transport bot output
tma_out, # Assistant spoken responses
]
)
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
# Kick off the conversation.
messages.append({"role": "system", "content": "Please introduce yourself to the user."})
await task.queue_frames([LLMMessagesFrame(messages)])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,100 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import LLMMessagesFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.openai import OpenAILLMService
from pipecat.services.rime import RimeHttpTTSService
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = RimeHttpTTSService(
api_key=os.getenv("RIME_API_KEY", ""),
voice_id="rex",
params=RimeHttpTTSService.InputParams(reduce_latency=True),
)
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(), # Transport user input
context_aggregator.user(), # User responses
llm, # LLM
tts, # TTS
transport.output(), # Transport bot output
context_aggregator.assistant(), # Assistant spoken responses
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
messages.append({"role": "system", "content": "Please introduce yourself to the user."})
await task.queue_frames([LLMMessagesFrame(messages)])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -67,8 +67,7 @@ async def main():
llm = AnthropicLLMService(
api_key=os.getenv("ANTHROPIC_API_KEY"),
# model="claude-3-5-sonnet-20240620",
model="claude-3-5-sonnet-latest",
model="claude-3-5-sonnet-20240620",
enable_prompt_caching_beta=True,
)
llm.register_function("get_weather", get_weather)

View File

@@ -5,15 +5,10 @@
#
import asyncio
import aiohttp
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from openai.types.chat import ChatCompletionToolParam
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
@@ -23,6 +18,14 @@ from pipecat.services.openai import OpenAILLMContext
from pipecat.services.together import TogetherLLMService
from pipecat.transports.services.daily import DailyParams, DailyTransport
from openai.types.chat import ChatCompletionToolParam
from runner import configure
from loguru import logger
from dotenv import load_dotenv
load_dotenv(override=True)
logger.remove(0)
@@ -122,7 +125,7 @@ async def main():
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
# await tts.say("Hi! Ask me about the weather in San Francisco.")
runner = PipelineRunner()

View File

@@ -64,11 +64,7 @@ async def main():
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = GoogleLLMService(
model="gemini-1.5-flash-latest",
# model="gemini-exp-1114",
api_key=os.getenv("GOOGLE_API_KEY"),
)
llm = GoogleLLMService(model="gemini-1.5-flash-latest", api_key=os.getenv("GOOGLE_API_KEY"))
llm.register_function("get_weather", get_weather)
llm.register_function("get_image", get_image)
@@ -155,6 +151,7 @@ indicate you should use the get_image tool are:
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)

View File

@@ -1,139 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from openai.types.chat import ChatCompletionToolParam
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.groq import GroqLLMService
from pipecat.services.openai import OpenAILLMContext
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def start_fetch_weather(function_name, llm, context):
# note: we can't push a frame to the LLM here. the bot
# can interrupt itself and/or cause audio overlapping glitches.
# possible question for Aleix and Chad about what the right way
# to trigger speech is, now, with the new queues/async/sync refactors.
# await llm.push_frame(TextFrame("Let me check on that."))
logger.debug(f"Starting fetch_weather_from_api with function_name: {function_name}")
async def fetch_weather_from_api(function_name, tool_call_id, args, llm, context, result_callback):
await result_callback({"conditions": "nice", "temperature": "75"})
async def main():
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = GroqLLMService(
api_key=os.getenv("GROQ_API_KEY"), model="llama3-groq-70b-8192-tool-use-preview"
)
# Register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function(None, fetch_weather_from_api, start_callback=start_fetch_weather)
tools = [
ChatCompletionToolParam(
type="function",
function={
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location"],
},
},
)
]
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages, tools)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(),
context_aggregator.user(),
llm,
tts,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,137 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from openai.types.chat import ChatCompletionToolParam
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.grok import GrokLLMService
from pipecat.services.openai import OpenAILLMContext
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def start_fetch_weather(function_name, llm, context):
# note: we can't push a frame to the LLM here. the bot
# can interrupt itself and/or cause audio overlapping glitches.
# possible question for Aleix and Chad about what the right way
# to trigger speech is, now, with the new queues/async/sync refactors.
# await llm.push_frame(TextFrame("Let me check on that."))
logger.debug(f"Starting fetch_weather_from_api with function_name: {function_name}")
async def fetch_weather_from_api(function_name, tool_call_id, args, llm, context, result_callback):
await result_callback({"conditions": "nice", "temperature": "75"})
async def main():
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = GrokLLMService(api_key=os.getenv("GROK_API_KEY"))
# Register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function(None, fetch_weather_from_api, start_callback=start_fetch_weather)
tools = [
ChatCompletionToolParam(
type="function",
function={
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
},
)
]
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages, tools)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(),
context_aggregator.user(),
llm,
tts,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,141 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from openai.types.chat import ChatCompletionToolParam
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.services.azure import AzureLLMService
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.openai import OpenAILLMContext
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def start_fetch_weather(function_name, llm, context):
# note: we can't push a frame to the LLM here. the bot
# can interrupt itself and/or cause audio overlapping glitches.
# possible question for Aleix and Chad about what the right way
# to trigger speech is, now, with the new queues/async/sync refactors.
# await llm.push_frame(TextFrame("Let me check on that."))
logger.debug(f"Starting fetch_weather_from_api with function_name: {function_name}")
async def fetch_weather_from_api(function_name, tool_call_id, args, llm, context, result_callback):
await result_callback({"conditions": "nice", "temperature": "75"})
async def main():
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = AzureLLMService(
api_key=os.getenv("AZURE_CHATGPT_API_KEY"),
endpoint=os.getenv("AZURE_CHATGPT_ENDPOINT"),
model=os.getenv("AZURE_CHATGPT_MODEL"),
)
# Register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function(None, fetch_weather_from_api, start_callback=start_fetch_weather)
tools = [
ChatCompletionToolParam(
type="function",
function={
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
},
)
]
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages, tools)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(),
context_aggregator.user(),
llm,
tts,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,140 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from openai.types.chat import ChatCompletionToolParam
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.fireworks import FireworksLLMService
from pipecat.services.openai import OpenAILLMContext
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def start_fetch_weather(function_name, llm, context):
# note: we can't push a frame to the LLM here. the bot
# can interrupt itself and/or cause audio overlapping glitches.
# possible question for Aleix and Chad about what the right way
# to trigger speech is, now, with the new queues/async/sync refactors.
# await llm.push_frame(TextFrame("Let me check on that."))
logger.debug(f"Starting fetch_weather_from_api with function_name: {function_name}")
async def fetch_weather_from_api(function_name, tool_call_id, args, llm, context, result_callback):
await result_callback({"conditions": "nice", "temperature": "75"})
async def main():
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = FireworksLLMService(
api_key=os.getenv("FIREWORKS_API_KEY"),
model="accounts/fireworks/models/firefunction-v2",
)
# Register a function_name of None to get all functions
# sent to the same callback with an additional function_name parameter.
llm.register_function(None, fetch_weather_from_api, start_callback=start_fetch_weather)
tools = [
ChatCompletionToolParam(
type="function",
function={
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
},
)
]
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages, tools)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(),
context_aggregator.user(),
llm,
tts,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -98,13 +98,12 @@ async def load_conversation(function_name, tool_call_id, args, llm, context, res
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a succinct, creative and helpful way. Prefer responses that are one sentence long unless you are asked for a longer or more detailed response.",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
{"role": "user", "content": "Start the call by saying the word 'hello'. Say only that word."},
# {"role": "user", "content": ""},
# {"role": "assistant", "content": []},
# {"role": "user", "content": "Tell me"},
# {"role": "user", "content": "a joke"},
{"role": "user", "content": ""},
{"role": "assistant", "content": []},
{"role": "user", "content": "Tell me"},
{"role": "user", "content": "a joke"},
]
tools = [
{
@@ -184,7 +183,7 @@ async def main():
)
llm = AnthropicLLMService(
api_key=os.getenv("ANTHROPIC_API_KEY"), model="claude-3-5-sonnet-latest"
api_key=os.getenv("ANTHROPIC_API_KEY"), model="claude-3-5-sonnet-20240620"
)
# you can either register a single function for all function calls, or specific functions

View File

@@ -1,339 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import aiohttp
import asyncio
import os
import sys
import time
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import LLMMessagesFrame, TextFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.parallel_pipeline import ParallelPipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import (
OpenAILLMContext,
)
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.deepgram import DeepgramSTTService
from pipecat.services.openai import OpenAILLMService
from pipecat.sync.event_notifier import EventNotifier
from pipecat.transports.services.daily import DailyParams, DailyTransport
from pipecat.processors.frame_processor import FrameProcessor, FrameDirection
from pipecat.frames.frames import (
CancelFrame,
EndFrame,
Frame,
StartFrame,
StartInterruptionFrame,
StopInterruptionFrame,
SystemFrame,
TranscriptionFrame,
UserStartedSpeakingFrame,
UserStoppedSpeakingFrame,
)
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContextFrame
from pipecat.sync.base_notifier import BaseNotifier
from pipecat.processors.filters.function_filter import FunctionFilter
from pipecat.processors.user_idle_processor import UserIdleProcessor
from runner import configure
from loguru import logger
from dotenv import load_dotenv
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
classifier_statement = "Determine if the user's statement ends with a complete thought and you should respond. The user text is transcribed speech. It may contain multiple fragments concatentated together. You are trying to determine only the completeness of the last user statement. The previous assistant statement is provided only for context. Categorize the text as either complete with the user now expecting a response, or incomplete. Return 'YES' if text is likely complete and the user is expecting a response. Return 'NO' if the text seems to be a partial expression or unfinished thought."
class StatementJudgeContextFilter(FrameProcessor):
def __init__(self, notifier: BaseNotifier, **kwargs):
super().__init__(**kwargs)
self._notifier = notifier
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
# We must not block system frames.
if isinstance(frame, SystemFrame):
await self.push_frame(frame, direction)
return
# Just treat an LLMMessagesFrame as complete, no matter what.
if isinstance(frame, LLMMessagesFrame):
await self._notifier.notify()
return
# Otherwise, we only want to handle OpenAILLMContextFrames, and only want to push a simple
# messages frame that contains a system prompt and the most recent user messages,
# concatenated.
if isinstance(frame, OpenAILLMContextFrame):
logger.debug(f"Context Frame: {frame}")
# Take text content from the most recent user messages.
messages = frame.context.messages
user_text_messages = []
last_assistant_message = None
for message in reversed(messages):
if message["role"] != "user":
if message["role"] == "assistant":
last_assistant_message = message
break
if isinstance(message["content"], str):
user_text_messages.append(message["content"])
elif isinstance(message["content"], list):
for content in message["content"]:
if content["type"] == "text":
user_text_messages.insert(0, content["text"])
# If we have any user text content, push an LLMMessagesFrame
if user_text_messages:
logger.debug(f"User text messages: {user_text_messages}")
user_message = " ".join(reversed(user_text_messages))
logger.debug(f"User message: {user_message}")
messages = [
{
"role": "system",
"content": classifier_statement,
}
]
if last_assistant_message:
messages.append(last_assistant_message)
messages.append({"role": "user", "content": user_message})
await self.push_frame(LLMMessagesFrame(messages))
class CompletenessCheck(FrameProcessor):
def __init__(self, notifier: BaseNotifier):
super().__init__()
self._notifier = notifier
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TextFrame) and frame.text == "YES":
logger.debug("Completeness check YES")
await self.push_frame(UserStoppedSpeakingFrame())
await self._notifier.notify()
elif isinstance(frame, TextFrame) and frame.text == "NO":
logger.debug("Completeness check NO")
class OutputGate(FrameProcessor):
def __init__(self, notifier: BaseNotifier, **kwargs):
super().__init__(**kwargs)
self._gate_open = False
self._frames_buffer = []
self._notifier = notifier
def close_gate(self):
self._gate_open = False
def open_gate(self):
self._gate_open = True
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
# We must not block system frames.
if isinstance(frame, SystemFrame):
if isinstance(frame, StartFrame):
await self._start()
if isinstance(frame, (EndFrame, CancelFrame)):
await self._stop()
if isinstance(frame, StartInterruptionFrame):
self._frames_buffer = []
self.close_gate()
await self.push_frame(frame, direction)
return
# Ignore frames that are not following the direction of this gate.
if direction != FrameDirection.DOWNSTREAM:
await self.push_frame(frame, direction)
return
if self._gate_open:
await self.push_frame(frame, direction)
return
self._frames_buffer.append((frame, direction))
async def _start(self):
self._frames_buffer = []
self._gate_task = self.get_event_loop().create_task(self._gate_task_handler())
async def _stop(self):
self._gate_task.cancel()
await self._gate_task
async def _gate_task_handler(self):
while True:
try:
await self._notifier.wait()
self.open_gate()
for frame, direction in self._frames_buffer:
await self.push_frame(frame, direction)
self._frames_buffer = []
except asyncio.CancelledError:
break
async def main():
async with aiohttp.ClientSession() as session:
(room_url, _) = await configure(session)
transport = DailyTransport(
room_url,
None,
"Respond bot",
DailyParams(
audio_out_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
vad_audio_passthrough=True,
),
)
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
# This is the LLM that will be used to detect if the user has finished a
# statement. This doesn't really need to be an LLM, we could use NLP
# libraries for that, but we have the machinery to use an LLM, so we might as well!
statement_llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
# This is the regular LLM.
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
# We have instructed the LLM to return 'YES' if it thinks the user
# completed a sentence. So, if it's 'YES' we will return true in this
# predicate which will wake up the notifier.
async def wake_check_filter(frame):
logger.debug(f"Completeness check frame: {frame}")
return frame.text == "YES"
# This is a notifier that we use to synchronize the two LLMs.
notifier = EventNotifier()
# This turns the LLM context into an inference request to classify the user's speech
# as complete or incomplete.
statement_judge_context_filter = StatementJudgeContextFilter(notifier=notifier)
# This sends a UserStoppedSpeakingFrame and triggers the notifier event
completeness_check = CompletenessCheck(notifier=notifier)
# # Notify if the user hasn't said anything.
async def user_idle_notifier(frame):
await notifier.notify()
# Sometimes the LLM will fail detecting if a user has completed a
# sentence, this will wake up the notifier if that happens.
user_idle = UserIdleProcessor(callback=user_idle_notifier, timeout=5.0)
bot_output_gate = OutputGate(notifier=notifier)
async def block_user_stopped_speaking(frame):
return not isinstance(frame, UserStoppedSpeakingFrame)
async def pass_only_llm_trigger_frames(frame):
return (
isinstance(frame, OpenAILLMContextFrame)
or isinstance(frame, LLMMessagesFrame)
or isinstance(frame, StartInterruptionFrame)
or isinstance(frame, StopInterruptionFrame)
)
pipeline = Pipeline(
[
transport.input(),
stt,
context_aggregator.user(),
ParallelPipeline(
[
# Pass everything except UserStoppedSpeaking to the elements after
# this ParallelPipeline
FunctionFilter(filter=block_user_stopped_speaking),
],
[
# Ignore everything except an OpenAILLMContextFrame. Pass a specially constructed
# LLMMessagesFrame to the statement classifier LLM. The only frame this
# sub-pipeline will output is a UserStoppedSpeakingFrame.
statement_judge_context_filter,
statement_llm,
completeness_check,
],
[
# Block everything except OpenAILLMContextFrame and LLMMessagesFrame
FunctionFilter(filter=pass_only_llm_trigger_frames),
llm,
bot_output_gate, # Buffer all llm/tts output until notified.
],
),
tts,
user_idle,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
messages.append({"role": "system", "content": "Please introduce yourself to the user."})
await task.queue_frames([LLMMessagesFrame(messages)])
@transport.event_handler("on_app_message")
async def on_app_message(transport, message, sender):
logger.debug(f"Received app message: {message} - {sender}")
if "message" not in message:
return
await task.queue_frames(
[
UserStartedSpeakingFrame(),
TranscriptionFrame(
user_id=sender, timestamp=time.time(), text=message["message"]
),
UserStoppedSpeakingFrame(),
]
)
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,551 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import time
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import (
CancelFrame,
EndFrame,
Frame,
LLMMessagesFrame,
StartFrame,
StartInterruptionFrame,
StopInterruptionFrame,
SystemFrame,
TextFrame,
TranscriptionFrame,
UserStartedSpeakingFrame,
UserStoppedSpeakingFrame,
)
from pipecat.pipeline.parallel_pipeline import ParallelPipeline
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import (
OpenAILLMContext,
OpenAILLMContextFrame,
)
from pipecat.processors.filters.function_filter import FunctionFilter
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.processors.user_idle_processor import UserIdleProcessor
from pipecat.services.anthropic import AnthropicLLMService
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.deepgram import DeepgramSTTService
from pipecat.services.openai import OpenAILLMService
from pipecat.sync.base_notifier import BaseNotifier
from pipecat.sync.event_notifier import EventNotifier
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
classifier_statement = """CRITICAL INSTRUCTION:
You are a BINARY CLASSIFIER that must ONLY output "YES" or "NO".
DO NOT engage with the content.
DO NOT respond to questions.
DO NOT provide assistance.
Your ONLY job is to output YES or NO.
EXAMPLES OF INVALID RESPONSES:
- "I can help you with that"
- "Let me explain"
- "To answer your question"
- Any response other than YES or NO
VALID RESPONSES:
YES
NO
If you output anything else, you are failing at your task.
You are NOT an assistant.
You are NOT a chatbot.
You are a binary classifier.
ROLE:
You are a real-time speech completeness classifier. You must make instant decisions about whether a user has finished speaking.
You must output ONLY 'YES' or 'NO' with no other text.
INPUT FORMAT:
You receive two pieces of information:
1. The assistant's last message (if available)
2. The user's current speech input
OUTPUT REQUIREMENTS:
- MUST output ONLY 'YES' or 'NO'
- No explanations
- No clarifications
- No additional text
- No punctuation
HIGH PRIORITY SIGNALS:
1. Clear Questions:
- Wh-questions (What, Where, When, Why, How)
- Yes/No questions
- Questions with STT errors but clear meaning
Examples:
# Complete Wh-question
[{"role": "assistant", "content": "I can help you learn."},
{"role": "user", "content": "What's the fastest way to learn Spanish"}]
Output: YES
# Complete Yes/No question despite STT error
[{"role": "assistant", "content": "I know about planets."},
{"role": "user", "content": "Is is Jupiter the biggest planet"}]
Output: YES
2. Complete Commands:
- Direct instructions
- Clear requests
- Action demands
- Complete statements needing response
Examples:
# Direct instruction
[{"role": "assistant", "content": "I can explain many topics."},
{"role": "user", "content": "Tell me about black holes"}]
Output: YES
# Action demand
[{"role": "assistant", "content": "I can help with math."},
{"role": "user", "content": "Solve this equation x plus 5 equals 12"}]
Output: YES
3. Direct Responses:
- Answers to specific questions
- Option selections
- Clear acknowledgments with completion
Examples:
# Specific answer
[{"role": "assistant", "content": "What's your favorite color?"},
{"role": "user", "content": "I really like blue"}]
Output: YES
# Option selection
[{"role": "assistant", "content": "Would you prefer morning or evening?"},
{"role": "user", "content": "Morning"}]
Output: YES
MEDIUM PRIORITY SIGNALS:
1. Speech Pattern Completions:
- Self-corrections reaching completion
- False starts with clear ending
- Topic changes with complete thought
- Mid-sentence completions
Examples:
# Self-correction reaching completion
[{"role": "assistant", "content": "What would you like to know?"},
{"role": "user", "content": "Tell me about... no wait, explain how rainbows form"}]
Output: YES
# Topic change with complete thought
[{"role": "assistant", "content": "The weather is nice today."},
{"role": "user", "content": "Actually can you tell me who invented the telephone"}]
Output: YES
# Mid-sentence completion
[{"role": "assistant", "content": "Hello I'm ready."},
{"role": "user", "content": "What's the capital of? France"}]
Output: YES
2. Context-Dependent Brief Responses:
- Acknowledgments (okay, sure, alright)
- Agreements (yes, yeah)
- Disagreements (no, nah)
- Confirmations (correct, exactly)
Examples:
# Acknowledgment
[{"role": "assistant", "content": "Should we talk about history?"},
{"role": "user", "content": "Sure"}]
Output: YES
# Disagreement with completion
[{"role": "assistant", "content": "Is that what you meant?"},
{"role": "user", "content": "No not really"}]
Output: YES
LOW PRIORITY SIGNALS:
1. STT Artifacts (Consider but don't over-weight):
- Repeated words
- Unusual punctuation
- Capitalization errors
- Word insertions/deletions
Examples:
# Word repetition but complete
[{"role": "assistant", "content": "I can help with that."},
{"role": "user", "content": "What what is the time right now"}]
Output: YES
# Missing punctuation but complete
[{"role": "assistant", "content": "I can explain that."},
{"role": "user", "content": "Please tell me how computers work"}]
Output: YES
2. Speech Features:
- Filler words (um, uh, like)
- Thinking pauses
- Word repetitions
- Brief hesitations
Examples:
# Filler words but complete
[{"role": "assistant", "content": "What would you like to know?"},
{"role": "user", "content": "Um uh how do airplanes fly"}]
Output: YES
# Thinking pause but incomplete
[{"role": "assistant", "content": "I can explain anything."},
{"role": "user", "content": "Well um I want to know about the"}]
Output: NO
DECISION RULES:
1. Return YES if:
- ANY high priority signal shows clear completion
- Medium priority signals combine to show completion
- Meaning is clear despite low priority artifacts
2. Return NO if:
- No high priority signals present
- Thought clearly trails off
- Multiple incomplete indicators
- User appears mid-formulation
3. When uncertain:
- If you can understand the intent → YES
- If meaning is unclear → NO
- Always make a binary decision
- Never request clarification
Examples:
# Incomplete despite corrections
[{"role": "assistant", "content": "What would you like to know about?"},
{"role": "user", "content": "Can you tell me about"}]
Output: NO
# Complete despite multiple artifacts
[{"role": "assistant", "content": "I can help you learn."},
{"role": "user", "content": "How do you I mean what's the best way to learn programming"}]
Output: YES
# Trailing off incomplete
[{"role": "assistant", "content": "I can explain anything."},
{"role": "user", "content": "I was wondering if you could tell me why"}]
Output: NO
"""
conversational_system_message = """You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.
Please be very concise in your responses. Unless you are explicitly asked to do otherwise, give me the shortest complete answer possible without unnecessary elaboration. Generally you should answer with a single sentence.
"""
class StatementJudgeContextFilter(FrameProcessor):
def __init__(self, notifier: BaseNotifier, **kwargs):
super().__init__(**kwargs)
self._notifier = notifier
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
# We must not block system frames.
if isinstance(frame, SystemFrame):
await self.push_frame(frame, direction)
return
# Just treat an LLMMessagesFrame as complete, no matter what.
if isinstance(frame, LLMMessagesFrame):
await self._notifier.notify()
return
# Otherwise, we only want to handle OpenAILLMContextFrames, and only want to push a simple
# messages frame that contains a system prompt and the most recent user messages,
# concatenated.
if isinstance(frame, OpenAILLMContextFrame):
# Take text content from the most recent user messages.
messages = frame.context.messages
user_text_messages = []
last_assistant_message = None
for message in reversed(messages):
if message["role"] != "user":
if message["role"] == "assistant":
last_assistant_message = message
break
if isinstance(message["content"], str):
user_text_messages.append(message["content"])
elif isinstance(message["content"], list):
for content in message["content"]:
if content["type"] == "text":
user_text_messages.insert(0, content["text"])
# If we have any user text content, push an LLMMessagesFrame
if user_text_messages:
user_message = " ".join(reversed(user_text_messages))
logger.debug(f"!!! {user_message}")
messages = [
{
"role": "system",
"content": classifier_statement,
}
]
if last_assistant_message:
messages.append(last_assistant_message)
messages.append({"role": "user", "content": user_message})
await self.push_frame(LLMMessagesFrame(messages))
class CompletenessCheck(FrameProcessor):
def __init__(self, notifier: BaseNotifier):
super().__init__()
self._notifier = notifier
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TextFrame) and frame.text == "YES":
logger.debug("!!! Completeness check YES")
await self.push_frame(UserStoppedSpeakingFrame())
await self._notifier.notify()
elif isinstance(frame, TextFrame) and frame.text == "NO":
logger.debug("!!! Completeness check NO")
class OutputGate(FrameProcessor):
def __init__(self, notifier: BaseNotifier, **kwargs):
super().__init__(**kwargs)
self._gate_open = False
self._frames_buffer = []
self._notifier = notifier
def close_gate(self):
self._gate_open = False
def open_gate(self):
self._gate_open = True
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
# We must not block system frames.
if isinstance(frame, SystemFrame):
if isinstance(frame, StartFrame):
await self._start()
if isinstance(frame, (EndFrame, CancelFrame)):
await self._stop()
if isinstance(frame, StartInterruptionFrame):
self._frames_buffer = []
self.close_gate()
await self.push_frame(frame, direction)
return
# Ignore frames that are not following the direction of this gate.
if direction != FrameDirection.DOWNSTREAM:
await self.push_frame(frame, direction)
return
if self._gate_open:
await self.push_frame(frame, direction)
return
self._frames_buffer.append((frame, direction))
async def _start(self):
self._frames_buffer = []
self._gate_task = self.get_event_loop().create_task(self._gate_task_handler())
async def _stop(self):
self._gate_task.cancel()
await self._gate_task
async def _gate_task_handler(self):
while True:
try:
await self._notifier.wait()
self.open_gate()
for frame, direction in self._frames_buffer:
await self.push_frame(frame, direction)
self._frames_buffer = []
except asyncio.CancelledError:
break
async def main():
async with aiohttp.ClientSession() as session:
(room_url, _) = await configure(session)
transport = DailyTransport(
room_url,
None,
"Respond bot",
DailyParams(
audio_out_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
vad_audio_passthrough=True,
),
)
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
# This is the LLM that will be used to detect if the user has finished a
# statement. This doesn't really need to be an LLM, we could use NLP
# libraries for that, but we have the machinery to use an LLM, so we might as well!
statement_llm = AnthropicLLMService(
api_key=os.getenv("ANTHROPIC_API_KEY"),
model="claude-3-5-sonnet-20241022",
)
# This is the regular LLM.
llm = OpenAILLMService(
api_key=os.getenv("OPENAI_API_KEY"),
model="gpt-4o",
)
messages = [
{
"role": "system",
"content": conversational_system_message,
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
# We have instructed the LLM to return 'YES' if it thinks the user
# completed a sentence. So, if it's 'YES' we will return true in this
# predicate which will wake up the notifier.
async def wake_check_filter(frame):
return frame.text == "YES"
# This is a notifier that we use to synchronize the two LLMs.
notifier = EventNotifier()
# This turns the LLM context into an inference request to classify the user's speech
# as complete or incomplete.
statement_judge_context_filter = StatementJudgeContextFilter(notifier=notifier)
# This sends a UserStoppedSpeakingFrame and triggers the notifier event
completeness_check = CompletenessCheck(notifier=notifier)
# # Notify if the user hasn't said anything.
async def user_idle_notifier(frame):
await notifier.notify()
# Sometimes the LLM will fail detecting if a user has completed a
# sentence, this will wake up the notifier if that happens.
user_idle = UserIdleProcessor(callback=user_idle_notifier, timeout=5.0)
bot_output_gate = OutputGate(notifier=notifier)
async def block_user_stopped_speaking(frame):
return not isinstance(frame, UserStoppedSpeakingFrame)
async def pass_only_llm_trigger_frames(frame):
return (
isinstance(frame, OpenAILLMContextFrame)
or isinstance(frame, LLMMessagesFrame)
or isinstance(frame, StartInterruptionFrame)
or isinstance(frame, StopInterruptionFrame)
)
pipeline = Pipeline(
[
transport.input(),
stt,
context_aggregator.user(),
ParallelPipeline(
[
# Pass everything except UserStoppedSpeaking to the elements after
# this ParallelPipeline
FunctionFilter(filter=block_user_stopped_speaking),
],
[
# Ignore everything except an OpenAILLMContextFrame. Pass a specially constructed
# LLMMessagesFrame to the statement classifier LLM. The only frame this
# sub-pipeline will output is a UserStoppedSpeakingFrame.
statement_judge_context_filter,
statement_llm,
completeness_check,
],
[
# Block everything except OpenAILLMContextFrame and LLMMessagesFrame
FunctionFilter(filter=pass_only_llm_trigger_frames),
llm,
bot_output_gate, # Buffer all llm/tts output until notified.
],
),
tts,
user_idle,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
messages.append(
{
"role": "user",
"content": "Start by just saying \"Hello I'm ready.\" Don't say anything else.",
}
)
await task.queue_frames([LLMMessagesFrame(messages)])
@transport.event_handler("on_app_message")
async def on_app_message(transport, message, sender):
logger.debug(f"Received app message: {message} - {sender}")
if "message" not in message:
return
await task.queue_frames(
[
UserStartedSpeakingFrame(),
TranscriptionFrame(
user_id=sender, timestamp=time.time(), text=message["message"]
),
UserStoppedSpeakingFrame(),
]
)
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,355 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import aiohttp
import asyncio
import os
import sys
import time
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import LLMMessagesFrame, TextFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.parallel_pipeline import ParallelPipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.services.deepgram import DeepgramSTTService
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import (
OpenAILLMContext,
)
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.google import GoogleLLMService, GoogleLLMContext
from pipecat.sync.event_notifier import EventNotifier
from pipecat.transports.services.daily import DailyParams, DailyTransport
from pipecat.processors.frame_processor import FrameProcessor, FrameDirection
from pipecat.frames.frames import (
CancelFrame,
EndFrame,
Frame,
InputAudioRawFrame,
StartFrame,
StartInterruptionFrame,
StopInterruptionFrame,
SystemFrame,
TranscriptionFrame,
UserStartedSpeakingFrame,
UserStoppedSpeakingFrame,
)
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContextFrame
from pipecat.sync.base_notifier import BaseNotifier
from pipecat.processors.filters.function_filter import FunctionFilter
from pipecat.processors.user_idle_processor import UserIdleProcessor
from runner import configure
from loguru import logger
from dotenv import load_dotenv
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
classifier_statement = """You are an audio language classifier model. You are receiving audio from a user in a WebRTC call. Your job is to decide whether the user has finished speaking or not.
Categorize the input you receive as either:
1. a complete thought, statement, or question, or
2. an incomplete thought, statement, or question
Output 'YES' if the input is likely to be a completed thought, statement, or question.
Output 'NO' if the input indicates that the user is still speaking and does not yet expect a response yet.
If you are unsure, output 'YES'.
"""
conversational_system_message = """You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.
Please be very concise in your responses. Unless you are explicitly asked to do otherwise, give me the shortest complete answer possible without unnecessary elaboration. Generally you should answer with a single sentence.
"""
class StatementJudgeAudioContextAccumulator(FrameProcessor):
def __init__(self, *, notifier: BaseNotifier, **kwargs):
super().__init__(**kwargs)
self._notifier = notifier
self._audio_frames = []
self._audio_frames = []
self._start_secs = 0.2 # this should match VAD start_secs (hardcoding for now)
self._user_speaking = False
async def reset(self):
self._audio_frames = []
self._user_speaking = False
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
# ignore context frame
if isinstance(frame, OpenAILLMContextFrame):
return
if isinstance(frame, TranscriptionFrame):
# We could gracefully handle both audio input and text/transcription input ...
# but let's leave that as an exercise to the reader. :-)
return
if isinstance(frame, UserStartedSpeakingFrame):
self._user_speaking = True
elif isinstance(frame, UserStoppedSpeakingFrame):
self._user_speaking = False
context = GoogleLLMContext()
context.set_messages([{"role": "system", "content": classifier_statement}])
context.add_audio_frames_message(audio_frames=self._audio_frames)
await self.push_frame(OpenAILLMContextFrame(context=context))
elif isinstance(frame, InputAudioRawFrame):
if self._user_speaking:
self._audio_frames.append(frame)
else:
# Append the audio frame to our buffer. Treat the buffer as a ring buffer, dropping the oldest
# frames as necessary. Assume all audio frames have the same duration.
self._audio_frames.append(frame)
frame_duration = len(frame.audio) / 16 * frame.num_channels / frame.sample_rate
buffer_duration = frame_duration * len(self._audio_frames)
while buffer_duration > self._start_secs:
self._audio_frames.pop(0)
buffer_duration -= frame_duration
await self.push_frame(frame, direction)
class CompletenessCheck(FrameProcessor):
def __init__(
self, notifier: BaseNotifier, audio_accumulator: StatementJudgeAudioContextAccumulator
):
super().__init__()
self._notifier = notifier
self._audio_accumulator = audio_accumulator
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TextFrame) and frame.text.startswith("YES"):
logger.debug("Completeness check YES")
await self.push_frame(UserStoppedSpeakingFrame())
await self._audio_accumulator.reset()
await self._notifier.notify()
elif isinstance(frame, TextFrame):
if frame.text.strip():
logger.debug(f"Completeness check NO - '{frame.text}'")
class OutputGate(FrameProcessor):
def __init__(self, notifier: BaseNotifier, **kwargs):
super().__init__(**kwargs)
self._gate_open = False
self._frames_buffer = []
self._notifier = notifier
def close_gate(self):
self._gate_open = False
def open_gate(self):
self._gate_open = True
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
# We must not block system frames.
if isinstance(frame, SystemFrame):
if isinstance(frame, StartFrame):
await self._start()
if isinstance(frame, (EndFrame, CancelFrame)):
await self._stop()
if isinstance(frame, StartInterruptionFrame):
self._frames_buffer = []
self.close_gate()
await self.push_frame(frame, direction)
return
# Ignore frames that are not following the direction of this gate.
if direction != FrameDirection.DOWNSTREAM:
await self.push_frame(frame, direction)
return
if self._gate_open:
await self.push_frame(frame, direction)
return
self._frames_buffer.append((frame, direction))
async def _start(self):
self._frames_buffer = []
self._gate_task = self.get_event_loop().create_task(self._gate_task_handler())
async def _stop(self):
self._gate_task.cancel()
await self._gate_task
async def _gate_task_handler(self):
while True:
try:
await self._notifier.wait()
self.open_gate()
for frame, direction in self._frames_buffer:
await self.push_frame(frame, direction)
self._frames_buffer = []
except asyncio.CancelledError:
break
async def main():
async with aiohttp.ClientSession() as session:
(room_url, _) = await configure(session)
transport = DailyTransport(
room_url,
None,
"Respond bot",
DailyParams(
audio_out_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
vad_audio_passthrough=True,
),
)
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
# This is the LLM that will be used to detect if the user has finished a
# statement. This doesn't really need to be an LLM, we could use NLP
# libraries for that, but we have the machinery to use an LLM, so we might as well!
statement_llm = GoogleLLMService(
model="gemini-1.5-flash-latest", api_key=os.getenv("GOOGLE_API_KEY")
)
# This is the regular LLM.
llm = GoogleLLMService(model="gemini-1.5-flash-latest", api_key=os.getenv("GOOGLE_API_KEY"))
messages = [
{
"role": "system",
"content": conversational_system_message,
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
# We have instructed the LLM to return 'YES' if it thinks the user
# completed a sentence. So, if it's 'YES' we will return true in this
# predicate which will wake up the notifier.
async def wake_check_filter(frame):
return frame.text == "YES"
# This is a notifier that we use to synchronize the two LLMs.
notifier = EventNotifier()
# This turns the LLM context into an inference request to classify the user's speech
# as complete or incomplete.
statement_judge_context_filter = StatementJudgeAudioContextAccumulator(notifier=notifier)
# This sends a UserStoppedSpeakingFrame and triggers the notifier event
completeness_check = CompletenessCheck(
notifier=notifier, audio_accumulator=statement_judge_context_filter
)
# # Notify if the user hasn't said anything.
async def user_idle_notifier(frame):
await notifier.notify()
# Sometimes the LLM will fail detecting if a user has completed a
# sentence, this will wake up the notifier if that happens.
user_idle = UserIdleProcessor(callback=user_idle_notifier, timeout=5.0)
bot_output_gate = OutputGate(notifier=notifier)
async def block_user_stopped_speaking(frame):
return not isinstance(frame, UserStoppedSpeakingFrame)
async def pass_only_llm_trigger_frames(frame):
return (
isinstance(frame, OpenAILLMContextFrame)
or isinstance(frame, LLMMessagesFrame)
or isinstance(frame, StartInterruptionFrame)
or isinstance(frame, StopInterruptionFrame)
)
pipeline = Pipeline(
[
transport.input(),
ParallelPipeline(
[
# Pass everything except UserStoppedSpeaking to the elements after
# this ParallelPipeline
FunctionFilter(filter=block_user_stopped_speaking),
],
[
statement_judge_context_filter,
statement_llm,
completeness_check,
],
[
stt,
context_aggregator.user(),
# Block everything except OpenAILLMContextFrame and LLMMessagesFrame
FunctionFilter(filter=pass_only_llm_trigger_frames),
llm,
bot_output_gate, # Buffer all llm/tts output until notified.
],
),
tts,
user_idle,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
@transport.event_handler("on_app_message")
async def on_app_message(transport, message, sender):
logger.debug(f"Received app message: {message} - {sender}")
if "message" not in message:
return
await task.queue_frames(
[
UserStartedSpeakingFrame(),
TranscriptionFrame(
user_id=sender, timestamp=time.time(), text=message["message"]
),
UserStoppedSpeakingFrame(),
]
)
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,121 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import argparse
import asyncio
import aiohttp
import os
import sys
from pipecat.audio.mixers.soundfile_mixer import SoundfileMixer
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import LLMMessagesFrame, MixerUpdateSettingsFrame, MixerEnableFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.services.openai import OpenAILLMService
from pipecat.transports.services.daily import DailyParams, DailyTransport
from runner import configure_with_args
from loguru import logger
from dotenv import load_dotenv
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
parser = argparse.ArgumentParser(description="Bot Background Sound")
parser.add_argument("-i", "--input", type=str, required=True, help="Input audio file")
(room_url, token, args) = await configure_with_args(session, parser)
soundfile_mixer = SoundfileMixer(
sound_files={"office": args.input},
default_sound="office",
volume=2.0,
)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
audio_out_mixer=soundfile_mixer,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(), # Transport user input
context_aggregator.user(), # User responses
llm, # LLM
tts, # TTS
transport.output(), # Transport bot output
context_aggregator.assistant(), # Assistant spoken responses
]
)
task = PipelineTask(
pipeline,
PipelineParams(
allow_interruptions=True,
enable_metrics=True,
enable_usage_metrics=True,
report_only_initial_ttfb=True,
),
)
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
await transport.capture_participant_transcription(participant["id"])
# Show how to use mixer control frames.
await asyncio.sleep(10.0)
await task.queue_frame(MixerUpdateSettingsFrame({"volume": 0.5}))
await asyncio.sleep(5.0)
await task.queue_frame(MixerEnableFrame(False))
await asyncio.sleep(5.0)
await task.queue_frame(MixerEnableFrame(True))
await asyncio.sleep(5.0)
# Kick off the conversation.
messages.append({"role": "system", "content": "Please introduce yourself to the user."})
await task.queue_frames([LLMMessagesFrame(messages)])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -1,98 +0,0 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import os
import sys
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import (
LLMMessagesFrame,
)
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.processors.filters.stt_mute_filter import STTMuteConfig, STTMuteFilter, STTMuteStrategy
from pipecat.services.deepgram import DeepgramSTTService, DeepgramTTSService
from pipecat.services.openai import OpenAILLMService
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
async def main():
async with aiohttp.ClientSession() as session:
(room_url, _) = await configure(session)
transport = DailyTransport(
room_url,
None,
"Respond bot",
DailyParams(
audio_out_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
vad_audio_passthrough=True,
),
)
stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
# Configure the mute processor to mute only during first speech
stt_mute_processor = STTMuteFilter(
stt_service=stt, config=STTMuteConfig(strategy=STTMuteStrategy.FIRST_SPEECH)
)
tts = DeepgramTTSService(api_key=os.getenv("DEEPGRAM_API_KEY"), voice="aura-helios-en")
llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
messages = [
{
"role": "system",
"content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
},
]
context = OpenAILLMContext(messages)
context_aggregator = llm.create_context_aggregator(context)
pipeline = Pipeline(
[
transport.input(), # Transport user input
stt_mute_processor, # Add the mute processor before STT
stt, # STT
context_aggregator.user(), # User responses
llm, # LLM
tts, # TTS
transport.output(), # Transport bot output
context_aggregator.assistant(), # Assistant spoken responses
]
)
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True))
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
# Kick off the conversation.
messages.append({"role": "system", "content": "Please introduce yourself to the user."})
await task.queue_frames([LLMMessagesFrame(messages)])
runner = PipelineRunner()
await runner.run(task)
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -0,0 +1,298 @@
#
# Copyright (c) 2024, Daily
#
# SPDX-License-Identifier: BSD 2-Clause License
#
import asyncio
import base64
import io
import os
import sys
from collections import deque
import aiohttp
from dotenv import load_dotenv
from loguru import logger
from PIL import Image
from runner import configure
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import (
BotInterruptionFrame,
Frame,
ImageRawFrame,
LLMFullResponseEndFrame,
LLMMessagesFrame,
TextFrame,
TranscriptionFrame,
)
from pipecat.pipeline.parallel_pipeline import ParallelPipeline
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import (
OpenAILLMContext,
OpenAILLMContextFrame,
)
from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
from pipecat.processors.frameworks.rtvi import (
RTVIBotTranscriptionProcessor,
RTVIUserTranscriptionProcessor,
)
from pipecat.services.anthropic import AnthropicLLMContext, AnthropicLLMService
from pipecat.services.cartesia import CartesiaTTSService
from pipecat.transports.services.daily import DailyParams, DailyTransport
load_dotenv(override=True)
logger.remove(0)
logger.add(sys.stderr, level="DEBUG")
MAX_FRAMES = 5
FRAMES_PER_SECOND = 0.2
video_participant_id = None
anthropic_context = None
recent_image_frames = deque(maxlen=MAX_FRAMES)
most_recent_image_summary = ""
class ImageFrameCatcher(FrameProcessor):
async def process_frame(self, frame: Frame, direction: FrameDirection):
global recent_image_frames
await super().process_frame(frame, direction)
if isinstance(frame, ImageRawFrame):
recent_image_frames.append(frame)
else:
await self.push_frame(frame, direction)
class TranscriptFrameCatcher(FrameProcessor):
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, TranscriptionFrame):
logger.debug(
f"TranscriptLogger: {frame}, num frames: {len(recent_image_frames)}, anthropic context: {anthropic_context}"
)
if anthropic_context:
add_message_with_images(
anthropic_context, frame.text, frames=list(recent_image_frames)
)
await self.push_frame(frame, direction)
class MessageFrameCatcher(FrameProcessor):
async def process_frame(self, frame: Frame, direction: FrameDirection):
await super().process_frame(frame, direction)
if isinstance(frame, OpenAILLMContextFrame):
last_message = frame.context.messages[-1]
system_message = """
Give me a concise summary of the images supplied.
"""
frame = LLMMessagesFrame(
messages=[
{
"role": "system",
"content": system_message,
},
last_message,
],
)
await self.push_frame(frame, direction)
return
class MessageFrameCatcher2(FrameProcessor):
def __init__(self):
super().__init__()
self.text_blob = ""
async def process_frame(self, frame: Frame, direction: FrameDirection):
global most_recent_image_summary
await super().process_frame(frame, direction)
if isinstance(frame, TextFrame):
self.text_blob += f" {frame.text}"
if isinstance(frame, LLMFullResponseEndFrame):
logger.debug(f"MessageFrameCatcher2: {self.text_blob}")
most_recent_image_summary = self.text_blob
self.text_blob = ""
await self.push_frame(frame, direction)
async def main():
global llm
global anthropic_context
async with aiohttp.ClientSession() as session:
(room_url, token) = await configure(session)
transport = DailyTransport(
room_url,
token,
"Respond bot",
DailyParams(
audio_out_enabled=True,
transcription_enabled=True,
vad_enabled=True,
vad_analyzer=SileroVADAnalyzer(),
),
)
tts = CartesiaTTSService(
api_key=os.getenv("CARTESIA_API_KEY"),
voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
)
llm = AnthropicLLMService(
api_key=os.getenv("ANTHROPIC_API_KEY"),
model="claude-3-5-sonnet-20240620",
enable_prompt_caching_beta=True,
)
vision_llm = AnthropicLLMService(
api_key=os.getenv("ANTHROPIC_API_KEY"),
model="claude-3-5-sonnet-20240620",
enable_prompt_caching_beta=True,
)
# todo: test with very short initial user message
system_prompt = """\
You are a helpful assistant who converses with a user and answers questions. Respond concisely to general questions. Keep
your answers brief unless explicitly asked for more information.
Your response will be turned into speech so use only simple words and punctuation.
"""
messages = [
{
"role": "system",
"content": [
{
"type": "text",
"text": system_prompt,
}
],
},
{"role": "user", "content": "Start the conversation by saying 'hello'."},
]
context = OpenAILLMContext(messages)
anthropic_context = AnthropicLLMContext.upgrade_to_anthropic(context)
context_aggregator = llm.create_context_aggregator(context)
rtvi_user_transcription = RTVIUserTranscriptionProcessor()
rtvi_bot_transcription = RTVIBotTranscriptionProcessor()
pipeline = Pipeline(
[
transport.input(), # Transport user input
ImageFrameCatcher(),
TranscriptFrameCatcher(),
rtvi_user_transcription,
context_aggregator.user(), # User speech to text
ParallelPipeline(
[
llm, # LLM
rtvi_bot_transcription,
tts, # TTS
transport.output(), # Transport bot output
context_aggregator.assistant(), # Assistant spoken responses and tool context
],
[MessageFrameCatcher(), vision_llm, MessageFrameCatcher2()],
),
],
)
task = PipelineTask(pipeline, PipelineParams(allow_interruptions=True, enable_metrics=True))
@transport.event_handler("on_first_participant_joined")
async def on_first_participant_joined(transport, participant):
global video_participant_id
video_participant_id = participant["id"]
await transport.capture_participant_transcription(video_participant_id)
await transport.capture_participant_video(
video_participant_id, framerate=FRAMES_PER_SECOND, video_source="screenVideo"
)
# Kick off the conversation.
await task.queue_frames([context_aggregator.user().get_context_frame()])
@transport.event_handler("on_app_message")
async def on_app_message(transport, message, sender):
logger.debug(f"Received app message: {message} - {context}")
if not recent_image_frames:
logger.debug("No image frames to send")
return
add_message_with_images(
anthropic_context, message["message"], frames=list(recent_image_frames)
)
interrupt_message = "STOP"
if interrupt_message == message["message"]:
logger.debug("Interrupting")
await task.queue_frames([BotInterruptionFrame()])
else:
await task.queue_frames([context_aggregator.user().get_context_frame()])
runner = PipelineRunner()
await runner.run(task)
def add_message_with_images(c, message, frames=None):
if frames is None:
frames = list(recent_image_frames)
if not frames:
logger.debug("No image frames to send")
return
# Create content list starting with all images
content = []
for frame in frames:
buffer = io.BytesIO()
Image.frombytes(frame.format, frame.size, frame.image).save(buffer, format="JPEG")
encoded_image = base64.b64encode(buffer.getvalue()).decode("utf-8")
content.append(
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/jpeg",
"data": encoded_image,
},
}
)
# Add text message at the end if provided
if message:
content.append({"type": "text", "text": message})
# Go through all messages and replace user messages containing images
if c.messages:
for i, msg in enumerate(c.messages):
if (
msg["role"] == "user"
and isinstance(msg["content"], list)
and len(msg["content"]) > 0
):
if msg["content"][0].get("type") == "image":
logger.debug(
f"Replacing user message {i} containing images with summary: {most_recent_image_summary}"
)
c.messages[i] = {"role": "user", "content": most_recent_image_summary}
c.add_message({"role": "user", "content": content})
if __name__ == "__main__":
asyncio.run(main())

View File

@@ -62,11 +62,3 @@ Then, visit `http://localhost:7860/` in your browser to start a chatbot session.
docker build -t chatbot .
docker run --env-file .env -p 7860:7860 chatbot
```
## Cartesia best practices
Since this example is using Cartesia, checkout the best practices given in Cartesia's docs. LLM prompts should be modified accordingly.
<https://docs.cartesia.ai/build-with-sonic/formatting-text-for-sonic/best-practices>
<https://docs.cartesia.ai/build-with-sonic/formatting-text-for-sonic/inserting-breaks-pauses>
<https://docs.cartesia.ai/build-with-sonic/formatting-text-for-sonic/spelling-out-input-text>

View File

@@ -182,7 +182,7 @@ class IntakeProcessor:
}
)
print(f"!!! about to await llm process frame in start prescrpitions")
await llm.queue_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
await llm.process_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
print(f"!!! past await process frame in start prescriptions")
async def start_allergies(self, function_name, llm, context):
@@ -222,7 +222,7 @@ class IntakeProcessor:
"content": "Now ask the user if they have any medical conditions the doctor should know about. Once they've answered the question, call the list_conditions function.",
}
)
await llm.queue_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
await llm.process_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
async def start_conditions(self, function_name, llm, context):
print("!!! doing start conditions")
@@ -261,7 +261,7 @@ class IntakeProcessor:
"content": "Finally, ask the user the reason for their doctor visit today. Once they answer, call the list_visit_reasons function.",
}
)
await llm.queue_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
await llm.process_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
async def start_visit_reasons(self, function_name, llm, context):
print("!!! doing start visit reasons")
@@ -270,7 +270,7 @@ class IntakeProcessor:
context.add_message(
{"role": "system", "content": "Now, thank the user and end the conversation."}
)
await llm.queue_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
await llm.process_frame(OpenAILLMContextFrame(context), FrameDirection.DOWNSTREAM)
async def save_data(self, function_name, tool_call_id, args, llm, context, result_callback):
logger.info(f"!!! Saving data: {args}")

View File

@@ -1,60 +0,0 @@
# Simple Chatbot Full Stack
A full-stack implementation of an AI chatbot with real-time audio/video interaction.
## Structure
- `server/` - Python-based bot server using FastAPI
- `client/` - JavaScript client using RTVI and Daily.co for WebRTC
## Setup
### Server Setup
1. Navigate to the server directory:
```bash
cd server
```
2. Create and activate a virtual environment:
```bash
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
```
3. Install requirements:
```bash
pip install -r requirements.txt
```
4. Copy env.example to .env and add your credentials
5. Start the server:
```bash
python server.py
```
### Client Setup
1. Navigate to the client directory:
```bash
cd client
```
2. Install dependencies:
```bash
npm install
```
3. Start the development server:
```bash
npm run dev
```
4. Open the URL shown in the terminal (usually http://localhost:5173)
## Usage
1. Start the server (it will run on port 7860)
2. Start the client server (it will run on port 5173)
3. Open http://localhost:5173 in your browser
4. Click "Connect" to start a session with the bot
## Requirements
- Python 3.10+
- Node.js 14+
- Modern web browser with WebRTC support

View File

@@ -1,40 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>AI Chatbot</title>
</head>
<body>
<div class="container">
<div class="status-bar">
<div class="status">
Status: <span id="connection-status">Disconnected</span>
</div>
<div class="controls">
<button id="connect-btn">Connect</button>
<button id="disconnect-btn" disabled>Disconnect</button>
</div>
</div>
<div class="main-content">
<div class="bot-container">
<div id="bot-video-container">
</div>
<audio id="bot-audio" autoplay></audio>
</div>
</div>
<div class="debug-panel">
<h3>Debug Info</h3>
<div id="debug-log"></div>
</div>
</div>
<script type="module" src="/src/app.js"></script>
<link rel="stylesheet" href="/src/style.css">
</body>
</html>

View File

@@ -1 +0,0 @@
../esbuild/bin/esbuild

View File

@@ -1 +0,0 @@
../nanoid/bin/nanoid.cjs

View File

@@ -1 +0,0 @@
../rollup/dist/bin/rollup

View File

@@ -1 +0,0 @@
../uuid/dist/bin/uuid

View File

@@ -1 +0,0 @@
../vite/bin/vite.js

View File

@@ -1,593 +0,0 @@
{
"name": "client",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"node_modules/@babel/runtime": {
"version": "7.26.0",
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.26.0.tgz",
"integrity": "sha512-FDSOghenHTiToteC/QRlv2q3DhPZ/oOXTBoirfWNx1Cx3TMVcGWQtMMmQcSvb/JjpNeGzx8Pq/b4fKEJuWm1sw==",
"license": "MIT",
"dependencies": {
"regenerator-runtime": "^0.14.0"
},
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@daily-co/daily-js": {
"version": "0.72.2",
"resolved": "https://registry.npmjs.org/@daily-co/daily-js/-/daily-js-0.72.2.tgz",
"integrity": "sha512-beUN/V4S4++ZYIUAfRnRt/rUjc2jkCrc2YxghMEyUPxjZy1n73OCtbty68RDMpSYkIs89ailJaUNRLcPhIuMaw==",
"license": "BSD-2-Clause",
"dependencies": {
"@babel/runtime": "^7.12.5",
"@sentry/browser": "^7.60.1",
"bowser": "^2.8.1",
"dequal": "^2.0.3",
"events": "^3.1.0"
},
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/@daily-co/realtime-ai-daily": {
"version": "0.2.1",
"resolved": "https://registry.npmjs.org/@daily-co/realtime-ai-daily/-/realtime-ai-daily-0.2.1.tgz",
"integrity": "sha512-F3S0+bpWx7ALx9kNCSNUkTUAflsDv1DyGW2XLKDG8YsYhaT8WXXBJw6kTKUvV2BF9lzJrI0gg911ATbZMgJyRA==",
"license": "BSD-2-Clause",
"dependencies": {
"@daily-co/daily-js": "^0.72.1",
"realtime-ai": "0.2.1"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.24.0",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.0.tgz",
"integrity": "sha512-CKyDpRbK1hXwv79soeTJNHb5EiG6ct3efd/FTPdzOWdbZZfGhpbcqIpiD0+vwmpu0wTIL97ZRPZu8vUt46nBSw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@rollup/rollup-darwin-arm64": {
"version": "4.28.0",
"resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.28.0.tgz",
"integrity": "sha512-lmKx9yHsppblnLQZOGxdO66gT77bvdBtr/0P+TPOseowE7D9AJoBw8ZDULRasXRWf1Z86/gcOdpBrV6VDUY36Q==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@sentry-internal/feedback": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry-internal/feedback/-/feedback-7.120.0.tgz",
"integrity": "sha512-+nU2PXMAyrYyK64PlfxXyRZ+LIl6IWAcdnBeX916WqOJy2WWmtdOrAX8muVwLVIXHzp1EMG1nEZgtpL/Vr2XKQ==",
"license": "MIT",
"dependencies": {
"@sentry/core": "7.120.0",
"@sentry/types": "7.120.0",
"@sentry/utils": "7.120.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/@sentry-internal/replay-canvas": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry-internal/replay-canvas/-/replay-canvas-7.120.0.tgz",
"integrity": "sha512-ZEFZBP+Jxmy/8IY7IZDZVPqAJ6pPxAFo1lNTd8xfpbno3WAtHw0FLewLfjrFt0zfIgCk8EXj4PW355zRP3C2NQ==",
"license": "MIT",
"dependencies": {
"@sentry/core": "7.120.0",
"@sentry/replay": "7.120.0",
"@sentry/types": "7.120.0",
"@sentry/utils": "7.120.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/@sentry-internal/tracing": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry-internal/tracing/-/tracing-7.120.0.tgz",
"integrity": "sha512-VymJoIGMV0PcTJyshka9uJ1sKpR7bHooqW5jTEr6g0dYAwB723fPXHjVW+7SETF7i5+yr2KMprYKreqRidKyKA==",
"license": "MIT",
"dependencies": {
"@sentry/core": "7.120.0",
"@sentry/types": "7.120.0",
"@sentry/utils": "7.120.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@sentry/browser": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry/browser/-/browser-7.120.0.tgz",
"integrity": "sha512-2hRE3QPLBBX+qqZEHY2IbJv4YvfXY7m/bWmNjN15phyNK3oBcm2Pa8ZiKUYrk8u/4DCEGzNUlhOmFgaxwSfpNw==",
"license": "MIT",
"dependencies": {
"@sentry-internal/feedback": "7.120.0",
"@sentry-internal/replay-canvas": "7.120.0",
"@sentry-internal/tracing": "7.120.0",
"@sentry/core": "7.120.0",
"@sentry/integrations": "7.120.0",
"@sentry/replay": "7.120.0",
"@sentry/types": "7.120.0",
"@sentry/utils": "7.120.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@sentry/core": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry/core/-/core-7.120.0.tgz",
"integrity": "sha512-uTc2sUQ0heZrMI31oFOHGxjKgw16MbV3C2mcT7qcrb6UmSGR9WqPOXZhnVVuzPWCnQ8B5IPPVdynK//J+9/m6g==",
"license": "MIT",
"dependencies": {
"@sentry/types": "7.120.0",
"@sentry/utils": "7.120.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@sentry/integrations": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry/integrations/-/integrations-7.120.0.tgz",
"integrity": "sha512-/Hs9MgSmG4JFNyeQkJ+MWh/fxO/U38Pz0VSH3hDrfyCjI8vH9Vz9inGEQXgB9Ke4eH8XnhsQ7xPnM27lWJts6g==",
"license": "MIT",
"dependencies": {
"@sentry/core": "7.120.0",
"@sentry/types": "7.120.0",
"@sentry/utils": "7.120.0",
"localforage": "^1.8.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@sentry/replay": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry/replay/-/replay-7.120.0.tgz",
"integrity": "sha512-wV9fIYwNtMvFOHQB5eSm+kCorRXsX5+v1DxyTC8Lee1hfzcUQ2Wvqh75VktpXuM9TeZE8h7aQ4Wo4qCgTUdtvA==",
"license": "MIT",
"dependencies": {
"@sentry-internal/tracing": "7.120.0",
"@sentry/core": "7.120.0",
"@sentry/types": "7.120.0",
"@sentry/utils": "7.120.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/@sentry/types": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry/types/-/types-7.120.0.tgz",
"integrity": "sha512-3mvELhBQBo6EljcRrJzfpGJYHKIZuBXmqh0y8prh03SWE62pwRL614GIYtd4YOC6OP1gfPn8S8h9w3dD5bF5HA==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/@sentry/utils": {
"version": "7.120.0",
"resolved": "https://registry.npmjs.org/@sentry/utils/-/utils-7.120.0.tgz",
"integrity": "sha512-XZsPcBHoYu4+HYn14IOnhabUZgCF99Xn4IdWn8Hjs/c+VPtuAVDhRTsfPyPrpY3OcN8DgO5fZX4qcv/6kNbX1A==",
"license": "MIT",
"dependencies": {
"@sentry/types": "7.120.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/@types/estree": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.6.tgz",
"integrity": "sha512-AYnb1nQyY49te+VRAVgmzfcgjYS91mY5P0TKUDCLEM+gNnA+3T6rWITXRLYCpahpqSQbN5cE+gHpnPyXjHWxcw==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/events": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@types/events/-/events-3.0.3.tgz",
"integrity": "sha512-trOc4AAUThEz9hapPtSd7wf5tiQKvTtu5b371UxXdTuqzIh0ArcRspRP0i0Viu+LXstIQ1z96t1nsPxT9ol01g==",
"license": "MIT"
},
"node_modules/bowser": {
"version": "2.11.0",
"resolved": "https://registry.npmjs.org/bowser/-/bowser-2.11.0.tgz",
"integrity": "sha512-AlcaJBi/pqqJBIQ8U9Mcpc9i8Aqxn88Skv5d+xBX006BY5u8N3mGLHa5Lgppa7L/HfwgwLgZ6NYs+Ag6uUmJRA==",
"license": "MIT"
},
"node_modules/clone-deep": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/clone-deep/-/clone-deep-4.0.1.tgz",
"integrity": "sha512-neHB9xuzh/wk0dIHweyAXv2aPGZIVk3pLMe+/RNzINf17fe0OG96QroktYAUm7SM1PBnzTabaLboqqxDyMU+SQ==",
"license": "MIT",
"dependencies": {
"is-plain-object": "^2.0.4",
"kind-of": "^6.0.2",
"shallow-clone": "^3.0.0"
},
"engines": {
"node": ">=6"
}
},
"node_modules/dequal": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/dequal/-/dequal-2.0.3.tgz",
"integrity": "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==",
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/esbuild": {
"version": "0.24.0",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.0.tgz",
"integrity": "sha512-FuLPevChGDshgSicjisSooU0cemp/sGXR841D5LHMB7mTVOmsEHcAxaH3irL53+8YDIeVNQEySh4DaYU/iuPqQ==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.24.0",
"@esbuild/android-arm": "0.24.0",
"@esbuild/android-arm64": "0.24.0",
"@esbuild/android-x64": "0.24.0",
"@esbuild/darwin-arm64": "0.24.0",
"@esbuild/darwin-x64": "0.24.0",
"@esbuild/freebsd-arm64": "0.24.0",
"@esbuild/freebsd-x64": "0.24.0",
"@esbuild/linux-arm": "0.24.0",
"@esbuild/linux-arm64": "0.24.0",
"@esbuild/linux-ia32": "0.24.0",
"@esbuild/linux-loong64": "0.24.0",
"@esbuild/linux-mips64el": "0.24.0",
"@esbuild/linux-ppc64": "0.24.0",
"@esbuild/linux-riscv64": "0.24.0",
"@esbuild/linux-s390x": "0.24.0",
"@esbuild/linux-x64": "0.24.0",
"@esbuild/netbsd-x64": "0.24.0",
"@esbuild/openbsd-arm64": "0.24.0",
"@esbuild/openbsd-x64": "0.24.0",
"@esbuild/sunos-x64": "0.24.0",
"@esbuild/win32-arm64": "0.24.0",
"@esbuild/win32-ia32": "0.24.0",
"@esbuild/win32-x64": "0.24.0"
}
},
"node_modules/events": {
"version": "3.3.0",
"resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz",
"integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==",
"license": "MIT",
"engines": {
"node": ">=0.8.x"
}
},
"node_modules/immediate": {
"version": "3.0.6",
"resolved": "https://registry.npmjs.org/immediate/-/immediate-3.0.6.tgz",
"integrity": "sha512-XXOFtyqDjNDAQxVfYxuF7g9Il/IbWmmlQg2MYKOH8ExIT1qg6xc4zyS3HaEEATgs1btfzxq15ciUiY7gjSXRGQ==",
"license": "MIT"
},
"node_modules/is-plain-object": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/is-plain-object/-/is-plain-object-2.0.4.tgz",
"integrity": "sha512-h5PpgXkWitc38BBMYawTYMWJHFZJVnBquFE57xFpjB8pJFiF6gZ+bU+WyI/yqXiFR5mdLsgYNaPe8uao6Uv9Og==",
"license": "MIT",
"dependencies": {
"isobject": "^3.0.1"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/isobject": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/isobject/-/isobject-3.0.1.tgz",
"integrity": "sha512-WhB9zCku7EGTj/HQQRz5aUQEUeoQZH2bWcltRErOpymJ4boYE6wL9Tbr23krRPSZ+C5zqNSrSw+Cc7sZZ4b7vg==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/kind-of": {
"version": "6.0.3",
"resolved": "https://registry.npmjs.org/kind-of/-/kind-of-6.0.3.tgz",
"integrity": "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/lie": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/lie/-/lie-3.1.1.tgz",
"integrity": "sha512-RiNhHysUjhrDQntfYSfY4MU24coXXdEOgw9WGcKHNeEwffDYbF//u87M1EWaMGzuFoSbqW0C9C6lEEhDOAswfw==",
"license": "MIT",
"dependencies": {
"immediate": "~3.0.5"
}
},
"node_modules/localforage": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/localforage/-/localforage-1.10.0.tgz",
"integrity": "sha512-14/H1aX7hzBBmmh7sGPd+AOMkkIrHM3Z1PAyGgZigA1H1p5O5ANnMyWzvpAETtG68/dC4pC0ncy3+PPGzXZHPg==",
"license": "Apache-2.0",
"dependencies": {
"lie": "3.1.1"
}
},
"node_modules/nanoid": {
"version": "3.3.8",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.8.tgz",
"integrity": "sha512-WNLf5Sd8oZxOm+TzppcYk8gVOgP+l58xNy58D0nbUnOxOWRWvlcCV4kUF7ltmI6PsrLl/BgKEyS4mqsGChFN0w==",
"dev": true,
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/ai"
}
],
"license": "MIT",
"bin": {
"nanoid": "bin/nanoid.cjs"
},
"engines": {
"node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1"
}
},
"node_modules/picocolors": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
"integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==",
"dev": true,
"license": "ISC"
},
"node_modules/postcss": {
"version": "8.4.49",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.49.tgz",
"integrity": "sha512-OCVPnIObs4N29kxTjzLfUryOkvZEq+pf8jTF0lg8E7uETuWHA+v7j3c/xJmiqpX450191LlmZfUKkXxkTry7nA==",
"dev": true,
"funding": [
{
"type": "opencollective",
"url": "https://opencollective.com/postcss/"
},
{
"type": "tidelift",
"url": "https://tidelift.com/funding/github/npm/postcss"
},
{
"type": "github",
"url": "https://github.com/sponsors/ai"
}
],
"license": "MIT",
"dependencies": {
"nanoid": "^3.3.7",
"picocolors": "^1.1.1",
"source-map-js": "^1.2.1"
},
"engines": {
"node": "^10 || ^12 || >=14"
}
},
"node_modules/realtime-ai": {
"version": "0.2.1",
"resolved": "https://registry.npmjs.org/realtime-ai/-/realtime-ai-0.2.1.tgz",
"integrity": "sha512-2zhCO9V9zdoBwusjq6FkiEF3yrwyJryLUo+OMYPU0rkXYh4SVcIP1dx06qbEMRTuaB9U2wEWvqxPaEQnXNzovw==",
"license": "BSD-2-Clause",
"dependencies": {
"@types/events": "^3.0.3",
"clone-deep": "^4.0.1",
"events": "^3.3.0",
"typed-emitter": "^2.1.0",
"uuid": "^10.0.0"
}
},
"node_modules/regenerator-runtime": {
"version": "0.14.1",
"resolved": "https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.14.1.tgz",
"integrity": "sha512-dYnhHh0nJoMfnkZs6GmmhFknAGRrLznOu5nc9ML+EJxGvrx6H7teuevqVqCuPcPK//3eDrrjQhehXVx9cnkGdw==",
"license": "MIT"
},
"node_modules/rollup": {
"version": "4.28.0",
"resolved": "https://registry.npmjs.org/rollup/-/rollup-4.28.0.tgz",
"integrity": "sha512-G9GOrmgWHBma4YfCcX8PjH0qhXSdH8B4HDE2o4/jaxj93S4DPCIDoLcXz99eWMji4hB29UFCEd7B2gwGJDR9cQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/estree": "1.0.6"
},
"bin": {
"rollup": "dist/bin/rollup"
},
"engines": {
"node": ">=18.0.0",
"npm": ">=8.0.0"
},
"optionalDependencies": {
"@rollup/rollup-android-arm-eabi": "4.28.0",
"@rollup/rollup-android-arm64": "4.28.0",
"@rollup/rollup-darwin-arm64": "4.28.0",
"@rollup/rollup-darwin-x64": "4.28.0",
"@rollup/rollup-freebsd-arm64": "4.28.0",
"@rollup/rollup-freebsd-x64": "4.28.0",
"@rollup/rollup-linux-arm-gnueabihf": "4.28.0",
"@rollup/rollup-linux-arm-musleabihf": "4.28.0",
"@rollup/rollup-linux-arm64-gnu": "4.28.0",
"@rollup/rollup-linux-arm64-musl": "4.28.0",
"@rollup/rollup-linux-powerpc64le-gnu": "4.28.0",
"@rollup/rollup-linux-riscv64-gnu": "4.28.0",
"@rollup/rollup-linux-s390x-gnu": "4.28.0",
"@rollup/rollup-linux-x64-gnu": "4.28.0",
"@rollup/rollup-linux-x64-musl": "4.28.0",
"@rollup/rollup-win32-arm64-msvc": "4.28.0",
"@rollup/rollup-win32-ia32-msvc": "4.28.0",
"@rollup/rollup-win32-x64-msvc": "4.28.0",
"fsevents": "~2.3.2"
}
},
"node_modules/rxjs": {
"version": "7.8.1",
"resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.1.tgz",
"integrity": "sha512-AA3TVj+0A2iuIoQkWEK/tqFjBq2j+6PO6Y0zJcvzLAFhEFIO3HL0vls9hWLncZbAAbK0mar7oZ4V079I/qPMxg==",
"license": "Apache-2.0",
"optional": true,
"dependencies": {
"tslib": "^2.1.0"
}
},
"node_modules/shallow-clone": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/shallow-clone/-/shallow-clone-3.0.1.tgz",
"integrity": "sha512-/6KqX+GVUdqPuPPd2LxDDxzX6CAbjJehAAOKlNpqqUpAqPM6HeL8f+o3a+JsyGjn2lv0WY8UsTgUJjU9Ok55NA==",
"license": "MIT",
"dependencies": {
"kind-of": "^6.0.2"
},
"engines": {
"node": ">=8"
}
},
"node_modules/source-map-js": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
"integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==",
"dev": true,
"license": "BSD-3-Clause",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/tslib": {
"version": "2.8.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
"license": "0BSD",
"optional": true
},
"node_modules/typed-emitter": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/typed-emitter/-/typed-emitter-2.1.0.tgz",
"integrity": "sha512-g/KzbYKbH5C2vPkaXGu8DJlHrGKHLsM25Zg9WuC9pMGfuvT+X25tZQWo5fK1BjBm8+UrVE9LDCvaY0CQk+fXDA==",
"license": "MIT",
"optionalDependencies": {
"rxjs": "*"
}
},
"node_modules/uuid": {
"version": "10.0.0",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-10.0.0.tgz",
"integrity": "sha512-8XkAphELsDnEGrDxUOHB3RGvXz6TeuYSGEZBOjtTtPm2lwhGBjLgOzLHB63IUWfBpNucQjND6d3AOudO+H3RWQ==",
"funding": [
"https://github.com/sponsors/broofa",
"https://github.com/sponsors/ctavan"
],
"license": "MIT",
"bin": {
"uuid": "dist/bin/uuid"
}
},
"node_modules/vite": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/vite/-/vite-6.0.2.tgz",
"integrity": "sha512-XdQ+VsY2tJpBsKGs0wf3U/+azx8BBpYRHFAyKm5VeEZNOJZRB63q7Sc8Iup3k0TrN3KO6QgyzFf+opSbfY1y0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"esbuild": "^0.24.0",
"postcss": "^8.4.49",
"rollup": "^4.23.0"
},
"bin": {
"vite": "bin/vite.js"
},
"engines": {
"node": "^18.0.0 || ^20.0.0 || >=22.0.0"
},
"funding": {
"url": "https://github.com/vitejs/vite?sponsor=1"
},
"optionalDependencies": {
"fsevents": "~2.3.3"
},
"peerDependencies": {
"@types/node": "^18.0.0 || ^20.0.0 || >=22.0.0",
"jiti": ">=1.21.0",
"less": "*",
"lightningcss": "^1.21.0",
"sass": "*",
"sass-embedded": "*",
"stylus": "*",
"sugarss": "*",
"terser": "^5.16.0",
"tsx": "^4.8.1",
"yaml": "^2.4.2"
},
"peerDependenciesMeta": {
"@types/node": {
"optional": true
},
"jiti": {
"optional": true
},
"less": {
"optional": true
},
"lightningcss": {
"optional": true
},
"sass": {
"optional": true
},
"sass-embedded": {
"optional": true
},
"stylus": {
"optional": true
},
"sugarss": {
"optional": true
},
"terser": {
"optional": true
},
"tsx": {
"optional": true
},
"yaml": {
"optional": true
}
}
}
}
}

File diff suppressed because one or more lines are too long

View File

@@ -1,25 +0,0 @@
{
"hash": "5b0fc1aa",
"configHash": "b261e656",
"lockfileHash": "49cc3cc9",
"browserHash": "0db32c31",
"optimized": {
"@daily-co/realtime-ai-daily": {
"src": "../../@daily-co/realtime-ai-daily/dist/index.module.js",
"file": "@daily-co_realtime-ai-daily.js",
"fileHash": "1c7897e6",
"needsInterop": false
},
"realtime-ai": {
"src": "../../realtime-ai/dist/index.module.js",
"file": "realtime-ai.js",
"fileHash": "ba7c0239",
"needsInterop": false
}
},
"chunks": {
"chunk-MC2NFNB2": {
"file": "chunk-MC2NFNB2.js"
}
}
}

File diff suppressed because one or more lines are too long

View File

@@ -1,3 +0,0 @@
{
"type": "module"
}

View File

@@ -1,46 +0,0 @@
import {
$08bedc6ef0d1c66c$export$4eda4fd287fbbca5,
$08bedc6ef0d1c66c$export$59b4786f333aac02,
$08bedc6ef0d1c66c$export$885fb96b850e8fbb,
$08bedc6ef0d1c66c$export$975d7330b0c579b7,
$08bedc6ef0d1c66c$export$c67992fa684a81a6,
$08bedc6ef0d1c66c$export$e0624a511a2c4e9,
$08bedc6ef0d1c66c$export$e7544ab812238a61,
$2665d8e6d1596258$export$86495b081fef8e52,
$4a333e41af7a850f$export$3cf39a62d076dd5c,
$4a333e41af7a850f$export$441bcd2e10762760,
$505461462111ea0b$export$23bc637255b2a471,
$74c1449bc91bda44$export$fa42a01c1d60f4a1,
$bff4129f8f902365$export$28ad8d0d400d3e2d,
$bff4129f8f902365$export$3336fb47fe34a146,
$bff4129f8f902365$export$378529d7a8bead8b,
$bff4129f8f902365$export$38b3db05cbf0e240,
$bff4129f8f902365$export$69aa9ab0334b212,
$bff4129f8f902365$export$882b13c7fda338f5,
$bff4129f8f902365$export$e9a960646cc432aa,
$cbe8e0de0049ed6e$export$6b4624d233c61fcb,
$d881613f2029ce0c$export$8728b60ea57bf43e
} from "./chunk-MC2NFNB2.js";
export {
$08bedc6ef0d1c66c$export$885fb96b850e8fbb as BotNotReadyError,
$08bedc6ef0d1c66c$export$4eda4fd287fbbca5 as ConfigUpdateError,
$08bedc6ef0d1c66c$export$c67992fa684a81a6 as ConnectionTimeoutError,
$4a333e41af7a850f$export$3cf39a62d076dd5c as LLMHelper,
$4a333e41af7a850f$export$441bcd2e10762760 as LLMMessageType,
$bff4129f8f902365$export$e9a960646cc432aa as MessageDispatcher,
$bff4129f8f902365$export$378529d7a8bead8b as RTVIActionRequest,
$74c1449bc91bda44$export$fa42a01c1d60f4a1 as RTVIClient,
$505461462111ea0b$export$23bc637255b2a471 as RTVIClientHelper,
$08bedc6ef0d1c66c$export$59b4786f333aac02 as RTVIError,
$cbe8e0de0049ed6e$export$6b4624d233c61fcb as RTVIEvent,
$bff4129f8f902365$export$69aa9ab0334b212 as RTVIMessage,
$bff4129f8f902365$export$38b3db05cbf0e240 as RTVIMessageType,
$bff4129f8f902365$export$28ad8d0d400d3e2d as RTVI_ACTION_TYPE,
$bff4129f8f902365$export$882b13c7fda338f5 as RTVI_MESSAGE_LABEL,
$08bedc6ef0d1c66c$export$e7544ab812238a61 as StartBotError,
$2665d8e6d1596258$export$86495b081fef8e52 as Transport,
$08bedc6ef0d1c66c$export$e0624a511a2c4e9 as TransportStartError,
$08bedc6ef0d1c66c$export$975d7330b0c579b7 as VoiceError,
$bff4129f8f902365$export$3336fb47fe34a146 as VoiceMessage,
$d881613f2029ce0c$export$8728b60ea57bf43e as httpActionGenerator
};

View File

@@ -1,7 +0,0 @@
{
"version": 3,
"sources": [],
"sourcesContent": [],
"mappings": "",
"names": []
}

View File

@@ -1,22 +0,0 @@
MIT License
Copyright (c) 2014-present Sebastian McKenzie and other contributors
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -1,19 +0,0 @@
# @babel/runtime
> babel's modular runtime helpers
See our website [@babel/runtime](https://babeljs.io/docs/babel-runtime) for more information.
## Install
Using npm:
```sh
npm install --save @babel/runtime
```
or using yarn:
```sh
yarn add @babel/runtime
```

View File

@@ -1,4 +0,0 @@
function _AwaitValue(t) {
this.wrapped = t;
}
module.exports = _AwaitValue, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,4 +0,0 @@
function _OverloadYield(e, d) {
this.v = e, this.k = d;
}
module.exports = _OverloadYield, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,9 +0,0 @@
function _applyDecoratedDescriptor(i, e, r, n, l) {
var a = {};
return Object.keys(n).forEach(function (i) {
a[i] = n[i];
}), a.enumerable = !!a.enumerable, a.configurable = !!a.configurable, ("value" in a || a.initializer) && (a.writable = !0), a = r.slice().reverse().reduce(function (r, n) {
return n(i, e, r) || r;
}, a), l && void 0 !== a.initializer && (a.value = a.initializer ? a.initializer.call(l) : void 0, a.initializer = void 0), void 0 === a.initializer ? (Object.defineProperty(i, e, a), null) : a;
}
module.exports = _applyDecoratedDescriptor, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,236 +0,0 @@
var _typeof = require("./typeof.js")["default"];
var setFunctionName = require("./setFunctionName.js");
var toPropertyKey = require("./toPropertyKey.js");
function old_createMetadataMethodsForProperty(e, t, a, r) {
return {
getMetadata: function getMetadata(o) {
old_assertNotFinished(r, "getMetadata"), old_assertMetadataKey(o);
var i = e[o];
if (void 0 !== i) if (1 === t) {
var n = i["public"];
if (void 0 !== n) return n[a];
} else if (2 === t) {
var l = i["private"];
if (void 0 !== l) return l.get(a);
} else if (Object.hasOwnProperty.call(i, "constructor")) return i.constructor;
},
setMetadata: function setMetadata(o, i) {
old_assertNotFinished(r, "setMetadata"), old_assertMetadataKey(o);
var n = e[o];
if (void 0 === n && (n = e[o] = {}), 1 === t) {
var l = n["public"];
void 0 === l && (l = n["public"] = {}), l[a] = i;
} else if (2 === t) {
var s = n.priv;
void 0 === s && (s = n["private"] = new Map()), s.set(a, i);
} else n.constructor = i;
}
};
}
function old_convertMetadataMapToFinal(e, t) {
var a = e[Symbol.metadata || Symbol["for"]("Symbol.metadata")],
r = Object.getOwnPropertySymbols(t);
if (0 !== r.length) {
for (var o = 0; o < r.length; o++) {
var i = r[o],
n = t[i],
l = a ? a[i] : null,
s = n["public"],
c = l ? l["public"] : null;
s && c && Object.setPrototypeOf(s, c);
var d = n["private"];
if (d) {
var u = Array.from(d.values()),
f = l ? l["private"] : null;
f && (u = u.concat(f)), n["private"] = u;
}
l && Object.setPrototypeOf(n, l);
}
a && Object.setPrototypeOf(t, a), e[Symbol.metadata || Symbol["for"]("Symbol.metadata")] = t;
}
}
function old_createAddInitializerMethod(e, t) {
return function (a) {
old_assertNotFinished(t, "addInitializer"), old_assertCallable(a, "An initializer"), e.push(a);
};
}
function old_memberDec(e, t, a, r, o, i, n, l, s) {
var c;
switch (i) {
case 1:
c = "accessor";
break;
case 2:
c = "method";
break;
case 3:
c = "getter";
break;
case 4:
c = "setter";
break;
default:
c = "field";
}
var d,
u,
f = {
kind: c,
name: l ? "#" + t : toPropertyKey(t),
isStatic: n,
isPrivate: l
},
p = {
v: !1
};
if (0 !== i && (f.addInitializer = old_createAddInitializerMethod(o, p)), l) {
d = 2, u = Symbol(t);
var v = {};
0 === i ? (v.get = a.get, v.set = a.set) : 2 === i ? v.get = function () {
return a.value;
} : (1 !== i && 3 !== i || (v.get = function () {
return a.get.call(this);
}), 1 !== i && 4 !== i || (v.set = function (e) {
a.set.call(this, e);
})), f.access = v;
} else d = 1, u = t;
try {
return e(s, Object.assign(f, old_createMetadataMethodsForProperty(r, d, u, p)));
} finally {
p.v = !0;
}
}
function old_assertNotFinished(e, t) {
if (e.v) throw Error("attempted to call " + t + " after decoration was finished");
}
function old_assertMetadataKey(e) {
if ("symbol" != _typeof(e)) throw new TypeError("Metadata keys must be symbols, received: " + e);
}
function old_assertCallable(e, t) {
if ("function" != typeof e) throw new TypeError(t + " must be a function");
}
function old_assertValidReturnValue(e, t) {
var a = _typeof(t);
if (1 === e) {
if ("object" !== a || null === t) throw new TypeError("accessor decorators must return an object with get, set, or init properties or void 0");
void 0 !== t.get && old_assertCallable(t.get, "accessor.get"), void 0 !== t.set && old_assertCallable(t.set, "accessor.set"), void 0 !== t.init && old_assertCallable(t.init, "accessor.init"), void 0 !== t.initializer && old_assertCallable(t.initializer, "accessor.initializer");
} else if ("function" !== a) throw new TypeError((0 === e ? "field" : 10 === e ? "class" : "method") + " decorators must return a function or void 0");
}
function old_getInit(e) {
var t;
return null == (t = e.init) && (t = e.initializer) && void 0 !== console && console.warn(".initializer has been renamed to .init as of March 2022"), t;
}
function old_applyMemberDec(e, t, a, r, o, i, n, l, s) {
var c,
d,
u,
f,
p,
v,
y,
h = a[0];
if (n ? (0 === o || 1 === o ? (c = {
get: a[3],
set: a[4]
}, u = "get") : 3 === o ? (c = {
get: a[3]
}, u = "get") : 4 === o ? (c = {
set: a[3]
}, u = "set") : c = {
value: a[3]
}, 0 !== o && (1 === o && setFunctionName(a[4], "#" + r, "set"), setFunctionName(a[3], "#" + r, u))) : 0 !== o && (c = Object.getOwnPropertyDescriptor(t, r)), 1 === o ? f = {
get: c.get,
set: c.set
} : 2 === o ? f = c.value : 3 === o ? f = c.get : 4 === o && (f = c.set), "function" == typeof h) void 0 !== (p = old_memberDec(h, r, c, l, s, o, i, n, f)) && (old_assertValidReturnValue(o, p), 0 === o ? d = p : 1 === o ? (d = old_getInit(p), v = p.get || f.get, y = p.set || f.set, f = {
get: v,
set: y
}) : f = p);else for (var m = h.length - 1; m >= 0; m--) {
var b;
void 0 !== (p = old_memberDec(h[m], r, c, l, s, o, i, n, f)) && (old_assertValidReturnValue(o, p), 0 === o ? b = p : 1 === o ? (b = old_getInit(p), v = p.get || f.get, y = p.set || f.set, f = {
get: v,
set: y
}) : f = p, void 0 !== b && (void 0 === d ? d = b : "function" == typeof d ? d = [d, b] : d.push(b)));
}
if (0 === o || 1 === o) {
if (void 0 === d) d = function d(e, t) {
return t;
};else if ("function" != typeof d) {
var g = d;
d = function d(e, t) {
for (var a = t, r = 0; r < g.length; r++) a = g[r].call(e, a);
return a;
};
} else {
var _ = d;
d = function d(e, t) {
return _.call(e, t);
};
}
e.push(d);
}
0 !== o && (1 === o ? (c.get = f.get, c.set = f.set) : 2 === o ? c.value = f : 3 === o ? c.get = f : 4 === o && (c.set = f), n ? 1 === o ? (e.push(function (e, t) {
return f.get.call(e, t);
}), e.push(function (e, t) {
return f.set.call(e, t);
})) : 2 === o ? e.push(f) : e.push(function (e, t) {
return f.call(e, t);
}) : Object.defineProperty(t, r, c));
}
function old_applyMemberDecs(e, t, a, r, o) {
for (var i, n, l = new Map(), s = new Map(), c = 0; c < o.length; c++) {
var d = o[c];
if (Array.isArray(d)) {
var u,
f,
p,
v = d[1],
y = d[2],
h = d.length > 3,
m = v >= 5;
if (m ? (u = t, f = r, 0 != (v -= 5) && (p = n = n || [])) : (u = t.prototype, f = a, 0 !== v && (p = i = i || [])), 0 !== v && !h) {
var b = m ? s : l,
g = b.get(y) || 0;
if (!0 === g || 3 === g && 4 !== v || 4 === g && 3 !== v) throw Error("Attempted to decorate a public method/accessor that has the same name as a previously decorated public method/accessor. This is not currently supported by the decorators plugin. Property name was: " + y);
!g && v > 2 ? b.set(y, v) : b.set(y, !0);
}
old_applyMemberDec(e, u, d, y, v, m, h, f, p);
}
}
old_pushInitializers(e, i), old_pushInitializers(e, n);
}
function old_pushInitializers(e, t) {
t && e.push(function (e) {
for (var a = 0; a < t.length; a++) t[a].call(e);
return e;
});
}
function old_applyClassDecs(e, t, a, r) {
if (r.length > 0) {
for (var o = [], i = t, n = t.name, l = r.length - 1; l >= 0; l--) {
var s = {
v: !1
};
try {
var c = Object.assign({
kind: "class",
name: n,
addInitializer: old_createAddInitializerMethod(o, s)
}, old_createMetadataMethodsForProperty(a, 0, n, s)),
d = r[l](i, c);
} finally {
s.v = !0;
}
void 0 !== d && (old_assertValidReturnValue(10, d), i = d);
}
e.push(i, function () {
for (var e = 0; e < o.length; e++) o[e].call(i);
});
}
}
function applyDecs(e, t, a) {
var r = [],
o = {},
i = {};
return old_applyMemberDecs(r, e, i, o, t), old_convertMetadataMapToFinal(e.prototype, i), old_applyClassDecs(r, e, o, a), old_convertMetadataMapToFinal(e, o), r;
}
module.exports = applyDecs, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,184 +0,0 @@
var _typeof = require("./typeof.js")["default"];
function applyDecs2203Factory() {
function createAddInitializerMethod(e, t) {
return function (r) {
!function (e, t) {
if (e.v) throw Error("attempted to call addInitializer after decoration was finished");
}(t), assertCallable(r, "An initializer"), e.push(r);
};
}
function memberDec(e, t, r, a, n, i, s, o) {
var c;
switch (n) {
case 1:
c = "accessor";
break;
case 2:
c = "method";
break;
case 3:
c = "getter";
break;
case 4:
c = "setter";
break;
default:
c = "field";
}
var l,
u,
f = {
kind: c,
name: s ? "#" + t : t,
"static": i,
"private": s
},
p = {
v: !1
};
0 !== n && (f.addInitializer = createAddInitializerMethod(a, p)), 0 === n ? s ? (l = r.get, u = r.set) : (l = function l() {
return this[t];
}, u = function u(e) {
this[t] = e;
}) : 2 === n ? l = function l() {
return r.value;
} : (1 !== n && 3 !== n || (l = function l() {
return r.get.call(this);
}), 1 !== n && 4 !== n || (u = function u(e) {
r.set.call(this, e);
})), f.access = l && u ? {
get: l,
set: u
} : l ? {
get: l
} : {
set: u
};
try {
return e(o, f);
} finally {
p.v = !0;
}
}
function assertCallable(e, t) {
if ("function" != typeof e) throw new TypeError(t + " must be a function");
}
function assertValidReturnValue(e, t) {
var r = _typeof(t);
if (1 === e) {
if ("object" !== r || null === t) throw new TypeError("accessor decorators must return an object with get, set, or init properties or void 0");
void 0 !== t.get && assertCallable(t.get, "accessor.get"), void 0 !== t.set && assertCallable(t.set, "accessor.set"), void 0 !== t.init && assertCallable(t.init, "accessor.init");
} else if ("function" !== r) throw new TypeError((0 === e ? "field" : 10 === e ? "class" : "method") + " decorators must return a function or void 0");
}
function applyMemberDec(e, t, r, a, n, i, s, o) {
var c,
l,
u,
f,
p,
d,
h = r[0];
if (s ? c = 0 === n || 1 === n ? {
get: r[3],
set: r[4]
} : 3 === n ? {
get: r[3]
} : 4 === n ? {
set: r[3]
} : {
value: r[3]
} : 0 !== n && (c = Object.getOwnPropertyDescriptor(t, a)), 1 === n ? u = {
get: c.get,
set: c.set
} : 2 === n ? u = c.value : 3 === n ? u = c.get : 4 === n && (u = c.set), "function" == typeof h) void 0 !== (f = memberDec(h, a, c, o, n, i, s, u)) && (assertValidReturnValue(n, f), 0 === n ? l = f : 1 === n ? (l = f.init, p = f.get || u.get, d = f.set || u.set, u = {
get: p,
set: d
}) : u = f);else for (var v = h.length - 1; v >= 0; v--) {
var g;
void 0 !== (f = memberDec(h[v], a, c, o, n, i, s, u)) && (assertValidReturnValue(n, f), 0 === n ? g = f : 1 === n ? (g = f.init, p = f.get || u.get, d = f.set || u.set, u = {
get: p,
set: d
}) : u = f, void 0 !== g && (void 0 === l ? l = g : "function" == typeof l ? l = [l, g] : l.push(g)));
}
if (0 === n || 1 === n) {
if (void 0 === l) l = function l(e, t) {
return t;
};else if ("function" != typeof l) {
var y = l;
l = function l(e, t) {
for (var r = t, a = 0; a < y.length; a++) r = y[a].call(e, r);
return r;
};
} else {
var m = l;
l = function l(e, t) {
return m.call(e, t);
};
}
e.push(l);
}
0 !== n && (1 === n ? (c.get = u.get, c.set = u.set) : 2 === n ? c.value = u : 3 === n ? c.get = u : 4 === n && (c.set = u), s ? 1 === n ? (e.push(function (e, t) {
return u.get.call(e, t);
}), e.push(function (e, t) {
return u.set.call(e, t);
})) : 2 === n ? e.push(u) : e.push(function (e, t) {
return u.call(e, t);
}) : Object.defineProperty(t, a, c));
}
function pushInitializers(e, t) {
t && e.push(function (e) {
for (var r = 0; r < t.length; r++) t[r].call(e);
return e;
});
}
return function (e, t, r) {
var a = [];
return function (e, t, r) {
for (var a, n, i = new Map(), s = new Map(), o = 0; o < r.length; o++) {
var c = r[o];
if (Array.isArray(c)) {
var l,
u,
f = c[1],
p = c[2],
d = c.length > 3,
h = f >= 5;
if (h ? (l = t, 0 != (f -= 5) && (u = n = n || [])) : (l = t.prototype, 0 !== f && (u = a = a || [])), 0 !== f && !d) {
var v = h ? s : i,
g = v.get(p) || 0;
if (!0 === g || 3 === g && 4 !== f || 4 === g && 3 !== f) throw Error("Attempted to decorate a public method/accessor that has the same name as a previously decorated public method/accessor. This is not currently supported by the decorators plugin. Property name was: " + p);
!g && f > 2 ? v.set(p, f) : v.set(p, !0);
}
applyMemberDec(e, l, c, p, f, h, d, u);
}
}
pushInitializers(e, a), pushInitializers(e, n);
}(a, e, t), function (e, t, r) {
if (r.length > 0) {
for (var a = [], n = t, i = t.name, s = r.length - 1; s >= 0; s--) {
var o = {
v: !1
};
try {
var c = r[s](n, {
kind: "class",
name: i,
addInitializer: createAddInitializerMethod(a, o)
});
} finally {
o.v = !0;
}
void 0 !== c && (assertValidReturnValue(10, c), n = c);
}
e.push(n, function () {
for (var e = 0; e < a.length; e++) a[e].call(n);
});
}
}(a, e, r), a;
};
}
var applyDecs2203Impl;
function applyDecs2203(e, t, r) {
return (applyDecs2203Impl = applyDecs2203Impl || applyDecs2203Factory())(e, t, r);
}
module.exports = applyDecs2203, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,191 +0,0 @@
var _typeof = require("./typeof.js")["default"];
var setFunctionName = require("./setFunctionName.js");
var toPropertyKey = require("./toPropertyKey.js");
function applyDecs2203RFactory() {
function createAddInitializerMethod(e, t) {
return function (r) {
!function (e, t) {
if (e.v) throw Error("attempted to call addInitializer after decoration was finished");
}(t), assertCallable(r, "An initializer"), e.push(r);
};
}
function memberDec(e, t, r, n, a, i, o, s) {
var c;
switch (a) {
case 1:
c = "accessor";
break;
case 2:
c = "method";
break;
case 3:
c = "getter";
break;
case 4:
c = "setter";
break;
default:
c = "field";
}
var l,
u,
f = {
kind: c,
name: o ? "#" + t : toPropertyKey(t),
"static": i,
"private": o
},
p = {
v: !1
};
0 !== a && (f.addInitializer = createAddInitializerMethod(n, p)), 0 === a ? o ? (l = r.get, u = r.set) : (l = function l() {
return this[t];
}, u = function u(e) {
this[t] = e;
}) : 2 === a ? l = function l() {
return r.value;
} : (1 !== a && 3 !== a || (l = function l() {
return r.get.call(this);
}), 1 !== a && 4 !== a || (u = function u(e) {
r.set.call(this, e);
})), f.access = l && u ? {
get: l,
set: u
} : l ? {
get: l
} : {
set: u
};
try {
return e(s, f);
} finally {
p.v = !0;
}
}
function assertCallable(e, t) {
if ("function" != typeof e) throw new TypeError(t + " must be a function");
}
function assertValidReturnValue(e, t) {
var r = _typeof(t);
if (1 === e) {
if ("object" !== r || null === t) throw new TypeError("accessor decorators must return an object with get, set, or init properties or void 0");
void 0 !== t.get && assertCallable(t.get, "accessor.get"), void 0 !== t.set && assertCallable(t.set, "accessor.set"), void 0 !== t.init && assertCallable(t.init, "accessor.init");
} else if ("function" !== r) throw new TypeError((0 === e ? "field" : 10 === e ? "class" : "method") + " decorators must return a function or void 0");
}
function applyMemberDec(e, t, r, n, a, i, o, s) {
var c,
l,
u,
f,
p,
d,
h,
v = r[0];
if (o ? (0 === a || 1 === a ? (c = {
get: r[3],
set: r[4]
}, u = "get") : 3 === a ? (c = {
get: r[3]
}, u = "get") : 4 === a ? (c = {
set: r[3]
}, u = "set") : c = {
value: r[3]
}, 0 !== a && (1 === a && setFunctionName(r[4], "#" + n, "set"), setFunctionName(r[3], "#" + n, u))) : 0 !== a && (c = Object.getOwnPropertyDescriptor(t, n)), 1 === a ? f = {
get: c.get,
set: c.set
} : 2 === a ? f = c.value : 3 === a ? f = c.get : 4 === a && (f = c.set), "function" == typeof v) void 0 !== (p = memberDec(v, n, c, s, a, i, o, f)) && (assertValidReturnValue(a, p), 0 === a ? l = p : 1 === a ? (l = p.init, d = p.get || f.get, h = p.set || f.set, f = {
get: d,
set: h
}) : f = p);else for (var g = v.length - 1; g >= 0; g--) {
var y;
void 0 !== (p = memberDec(v[g], n, c, s, a, i, o, f)) && (assertValidReturnValue(a, p), 0 === a ? y = p : 1 === a ? (y = p.init, d = p.get || f.get, h = p.set || f.set, f = {
get: d,
set: h
}) : f = p, void 0 !== y && (void 0 === l ? l = y : "function" == typeof l ? l = [l, y] : l.push(y)));
}
if (0 === a || 1 === a) {
if (void 0 === l) l = function l(e, t) {
return t;
};else if ("function" != typeof l) {
var m = l;
l = function l(e, t) {
for (var r = t, n = 0; n < m.length; n++) r = m[n].call(e, r);
return r;
};
} else {
var b = l;
l = function l(e, t) {
return b.call(e, t);
};
}
e.push(l);
}
0 !== a && (1 === a ? (c.get = f.get, c.set = f.set) : 2 === a ? c.value = f : 3 === a ? c.get = f : 4 === a && (c.set = f), o ? 1 === a ? (e.push(function (e, t) {
return f.get.call(e, t);
}), e.push(function (e, t) {
return f.set.call(e, t);
})) : 2 === a ? e.push(f) : e.push(function (e, t) {
return f.call(e, t);
}) : Object.defineProperty(t, n, c));
}
function applyMemberDecs(e, t) {
for (var r, n, a = [], i = new Map(), o = new Map(), s = 0; s < t.length; s++) {
var c = t[s];
if (Array.isArray(c)) {
var l,
u,
f = c[1],
p = c[2],
d = c.length > 3,
h = f >= 5;
if (h ? (l = e, 0 != (f -= 5) && (u = n = n || [])) : (l = e.prototype, 0 !== f && (u = r = r || [])), 0 !== f && !d) {
var v = h ? o : i,
g = v.get(p) || 0;
if (!0 === g || 3 === g && 4 !== f || 4 === g && 3 !== f) throw Error("Attempted to decorate a public method/accessor that has the same name as a previously decorated public method/accessor. This is not currently supported by the decorators plugin. Property name was: " + p);
!g && f > 2 ? v.set(p, f) : v.set(p, !0);
}
applyMemberDec(a, l, c, p, f, h, d, u);
}
}
return pushInitializers(a, r), pushInitializers(a, n), a;
}
function pushInitializers(e, t) {
t && e.push(function (e) {
for (var r = 0; r < t.length; r++) t[r].call(e);
return e;
});
}
return function (e, t, r) {
return {
e: applyMemberDecs(e, t),
get c() {
return function (e, t) {
if (t.length > 0) {
for (var r = [], n = e, a = e.name, i = t.length - 1; i >= 0; i--) {
var o = {
v: !1
};
try {
var s = t[i](n, {
kind: "class",
name: a,
addInitializer: createAddInitializerMethod(r, o)
});
} finally {
o.v = !0;
}
void 0 !== s && (assertValidReturnValue(10, s), n = s);
}
return [n, function () {
for (var e = 0; e < r.length; e++) r[e].call(n);
}];
}
}(e, r);
}
};
};
}
function applyDecs2203R(e, t, r) {
return (module.exports = applyDecs2203R = applyDecs2203RFactory(), module.exports.__esModule = true, module.exports["default"] = module.exports)(e, t, r);
}
module.exports = applyDecs2203R, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,222 +0,0 @@
var _typeof = require("./typeof.js")["default"];
var checkInRHS = require("./checkInRHS.js");
var setFunctionName = require("./setFunctionName.js");
var toPropertyKey = require("./toPropertyKey.js");
function applyDecs2301Factory() {
function createAddInitializerMethod(e, t) {
return function (r) {
!function (e, t) {
if (e.v) throw Error("attempted to call addInitializer after decoration was finished");
}(t), assertCallable(r, "An initializer"), e.push(r);
};
}
function assertInstanceIfPrivate(e, t) {
if (!e(t)) throw new TypeError("Attempted to access private element on non-instance");
}
function memberDec(e, t, r, n, a, i, s, o, c) {
var u;
switch (a) {
case 1:
u = "accessor";
break;
case 2:
u = "method";
break;
case 3:
u = "getter";
break;
case 4:
u = "setter";
break;
default:
u = "field";
}
var l,
f,
p = {
kind: u,
name: s ? "#" + t : toPropertyKey(t),
"static": i,
"private": s
},
d = {
v: !1
};
if (0 !== a && (p.addInitializer = createAddInitializerMethod(n, d)), s || 0 !== a && 2 !== a) {
if (2 === a) l = function l(e) {
return assertInstanceIfPrivate(c, e), r.value;
};else {
var h = 0 === a || 1 === a;
(h || 3 === a) && (l = s ? function (e) {
return assertInstanceIfPrivate(c, e), r.get.call(e);
} : function (e) {
return r.get.call(e);
}), (h || 4 === a) && (f = s ? function (e, t) {
assertInstanceIfPrivate(c, e), r.set.call(e, t);
} : function (e, t) {
r.set.call(e, t);
});
}
} else l = function l(e) {
return e[t];
}, 0 === a && (f = function f(e, r) {
e[t] = r;
});
var v = s ? c.bind() : function (e) {
return t in e;
};
p.access = l && f ? {
get: l,
set: f,
has: v
} : l ? {
get: l,
has: v
} : {
set: f,
has: v
};
try {
return e(o, p);
} finally {
d.v = !0;
}
}
function assertCallable(e, t) {
if ("function" != typeof e) throw new TypeError(t + " must be a function");
}
function assertValidReturnValue(e, t) {
var r = _typeof(t);
if (1 === e) {
if ("object" !== r || null === t) throw new TypeError("accessor decorators must return an object with get, set, or init properties or void 0");
void 0 !== t.get && assertCallable(t.get, "accessor.get"), void 0 !== t.set && assertCallable(t.set, "accessor.set"), void 0 !== t.init && assertCallable(t.init, "accessor.init");
} else if ("function" !== r) throw new TypeError((0 === e ? "field" : 10 === e ? "class" : "method") + " decorators must return a function or void 0");
}
function curryThis2(e) {
return function (t) {
e(this, t);
};
}
function applyMemberDec(e, t, r, n, a, i, s, o, c) {
var u,
l,
f,
p,
d,
h,
v,
y,
g = r[0];
if (s ? (0 === a || 1 === a ? (u = {
get: (d = r[3], function () {
return d(this);
}),
set: curryThis2(r[4])
}, f = "get") : 3 === a ? (u = {
get: r[3]
}, f = "get") : 4 === a ? (u = {
set: r[3]
}, f = "set") : u = {
value: r[3]
}, 0 !== a && (1 === a && setFunctionName(u.set, "#" + n, "set"), setFunctionName(u[f || "value"], "#" + n, f))) : 0 !== a && (u = Object.getOwnPropertyDescriptor(t, n)), 1 === a ? p = {
get: u.get,
set: u.set
} : 2 === a ? p = u.value : 3 === a ? p = u.get : 4 === a && (p = u.set), "function" == typeof g) void 0 !== (h = memberDec(g, n, u, o, a, i, s, p, c)) && (assertValidReturnValue(a, h), 0 === a ? l = h : 1 === a ? (l = h.init, v = h.get || p.get, y = h.set || p.set, p = {
get: v,
set: y
}) : p = h);else for (var m = g.length - 1; m >= 0; m--) {
var b;
void 0 !== (h = memberDec(g[m], n, u, o, a, i, s, p, c)) && (assertValidReturnValue(a, h), 0 === a ? b = h : 1 === a ? (b = h.init, v = h.get || p.get, y = h.set || p.set, p = {
get: v,
set: y
}) : p = h, void 0 !== b && (void 0 === l ? l = b : "function" == typeof l ? l = [l, b] : l.push(b)));
}
if (0 === a || 1 === a) {
if (void 0 === l) l = function l(e, t) {
return t;
};else if ("function" != typeof l) {
var I = l;
l = function l(e, t) {
for (var r = t, n = 0; n < I.length; n++) r = I[n].call(e, r);
return r;
};
} else {
var w = l;
l = function l(e, t) {
return w.call(e, t);
};
}
e.push(l);
}
0 !== a && (1 === a ? (u.get = p.get, u.set = p.set) : 2 === a ? u.value = p : 3 === a ? u.get = p : 4 === a && (u.set = p), s ? 1 === a ? (e.push(function (e, t) {
return p.get.call(e, t);
}), e.push(function (e, t) {
return p.set.call(e, t);
})) : 2 === a ? e.push(p) : e.push(function (e, t) {
return p.call(e, t);
}) : Object.defineProperty(t, n, u));
}
function applyMemberDecs(e, t, r) {
for (var n, a, i, s = [], o = new Map(), c = new Map(), u = 0; u < t.length; u++) {
var l = t[u];
if (Array.isArray(l)) {
var f,
p,
d = l[1],
h = l[2],
v = l.length > 3,
y = d >= 5,
g = r;
if (y ? (f = e, 0 != (d -= 5) && (p = a = a || []), v && !i && (i = function i(t) {
return checkInRHS(t) === e;
}), g = i) : (f = e.prototype, 0 !== d && (p = n = n || [])), 0 !== d && !v) {
var m = y ? c : o,
b = m.get(h) || 0;
if (!0 === b || 3 === b && 4 !== d || 4 === b && 3 !== d) throw Error("Attempted to decorate a public method/accessor that has the same name as a previously decorated public method/accessor. This is not currently supported by the decorators plugin. Property name was: " + h);
!b && d > 2 ? m.set(h, d) : m.set(h, !0);
}
applyMemberDec(s, f, l, h, d, y, v, p, g);
}
}
return pushInitializers(s, n), pushInitializers(s, a), s;
}
function pushInitializers(e, t) {
t && e.push(function (e) {
for (var r = 0; r < t.length; r++) t[r].call(e);
return e;
});
}
return function (e, t, r, n) {
return {
e: applyMemberDecs(e, t, n),
get c() {
return function (e, t) {
if (t.length > 0) {
for (var r = [], n = e, a = e.name, i = t.length - 1; i >= 0; i--) {
var s = {
v: !1
};
try {
var o = t[i](n, {
kind: "class",
name: a,
addInitializer: createAddInitializerMethod(r, s)
});
} finally {
s.v = !0;
}
void 0 !== o && (assertValidReturnValue(10, o), n = o);
}
return [n, function () {
for (var e = 0; e < r.length; e++) r[e].call(n);
}];
}
}(e, r);
}
};
};
}
function applyDecs2301(e, t, r, n) {
return (module.exports = applyDecs2301 = applyDecs2301Factory(), module.exports.__esModule = true, module.exports["default"] = module.exports)(e, t, r, n);
}
module.exports = applyDecs2301, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,133 +0,0 @@
var _typeof = require("./typeof.js")["default"];
var checkInRHS = require("./checkInRHS.js");
var setFunctionName = require("./setFunctionName.js");
var toPropertyKey = require("./toPropertyKey.js");
function applyDecs2305(e, t, r, n, o, a) {
function i(e, t, r) {
return function (n, o) {
return r && r(n), e[t].call(n, o);
};
}
function c(e, t) {
for (var r = 0; r < e.length; r++) e[r].call(t);
return t;
}
function s(e, t, r, n) {
if ("function" != typeof e && (n || void 0 !== e)) throw new TypeError(t + " must " + (r || "be") + " a function" + (n ? "" : " or undefined"));
return e;
}
function applyDec(e, t, r, n, o, a, c, u, l, f, p, d, h) {
function m(e) {
if (!h(e)) throw new TypeError("Attempted to access private element on non-instance");
}
var y,
v = t[0],
g = t[3],
b = !u;
if (!b) {
r || Array.isArray(v) || (v = [v]);
var w = {},
S = [],
A = 3 === o ? "get" : 4 === o || d ? "set" : "value";
f ? (p || d ? w = {
get: setFunctionName(function () {
return g(this);
}, n, "get"),
set: function set(e) {
t[4](this, e);
}
} : w[A] = g, p || setFunctionName(w[A], n, 2 === o ? "" : A)) : p || (w = Object.getOwnPropertyDescriptor(e, n));
}
for (var P = e, j = v.length - 1; j >= 0; j -= r ? 2 : 1) {
var D = v[j],
E = r ? v[j - 1] : void 0,
I = {},
O = {
kind: ["field", "accessor", "method", "getter", "setter", "class"][o],
name: n,
metadata: a,
addInitializer: function (e, t) {
if (e.v) throw Error("attempted to call addInitializer after decoration was finished");
s(t, "An initializer", "be", !0), c.push(t);
}.bind(null, I)
};
try {
if (b) (y = s(D.call(E, P, O), "class decorators", "return")) && (P = y);else {
var k, F;
O["static"] = l, O["private"] = f, f ? 2 === o ? k = function k(e) {
return m(e), w.value;
} : (o < 4 && (k = i(w, "get", m)), 3 !== o && (F = i(w, "set", m))) : (k = function k(e) {
return e[n];
}, (o < 2 || 4 === o) && (F = function F(e, t) {
e[n] = t;
}));
var N = O.access = {
has: f ? h.bind() : function (e) {
return n in e;
}
};
if (k && (N.get = k), F && (N.set = F), P = D.call(E, d ? {
get: w.get,
set: w.set
} : w[A], O), d) {
if ("object" == _typeof(P) && P) (y = s(P.get, "accessor.get")) && (w.get = y), (y = s(P.set, "accessor.set")) && (w.set = y), (y = s(P.init, "accessor.init")) && S.push(y);else if (void 0 !== P) throw new TypeError("accessor decorators must return an object with get, set, or init properties or void 0");
} else s(P, (p ? "field" : "method") + " decorators", "return") && (p ? S.push(P) : w[A] = P);
}
} finally {
I.v = !0;
}
}
return (p || d) && u.push(function (e, t) {
for (var r = S.length - 1; r >= 0; r--) t = S[r].call(e, t);
return t;
}), p || b || (f ? d ? u.push(i(w, "get"), i(w, "set")) : u.push(2 === o ? w[A] : i.call.bind(w[A])) : Object.defineProperty(e, n, w)), P;
}
function u(e, t) {
return Object.defineProperty(e, Symbol.metadata || Symbol["for"]("Symbol.metadata"), {
configurable: !0,
enumerable: !0,
value: t
});
}
if (arguments.length >= 6) var l = a[Symbol.metadata || Symbol["for"]("Symbol.metadata")];
var f = Object.create(null == l ? null : l),
p = function (e, t, r, n) {
var o,
a,
i = [],
s = function s(t) {
return checkInRHS(t) === e;
},
u = new Map();
function l(e) {
e && i.push(c.bind(null, e));
}
for (var f = 0; f < t.length; f++) {
var p = t[f];
if (Array.isArray(p)) {
var d = p[1],
h = p[2],
m = p.length > 3,
y = 16 & d,
v = !!(8 & d),
g = 0 == (d &= 7),
b = h + "/" + v;
if (!g && !m) {
var w = u.get(b);
if (!0 === w || 3 === w && 4 !== d || 4 === w && 3 !== d) throw Error("Attempted to decorate a public method/accessor that has the same name as a previously decorated public method/accessor. This is not currently supported by the decorators plugin. Property name was: " + h);
u.set(b, !(d > 2) || d);
}
applyDec(v ? e : e.prototype, p, y, m ? "#" + h : toPropertyKey(h), d, n, v ? a = a || [] : o = o || [], i, v, m, g, 1 === d, v && m ? s : r);
}
}
return l(o), l(a), i;
}(e, t, o, f);
return r.length || u(e, f), {
e: p,
get c() {
var t = [];
return r.length && [u(applyDec(e, [r], n, e.name, 5, f, t), f), c.bind(null, t, e)];
}
};
}
module.exports = applyDecs2305, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,124 +0,0 @@
var _typeof = require("./typeof.js")["default"];
var checkInRHS = require("./checkInRHS.js");
var setFunctionName = require("./setFunctionName.js");
var toPropertyKey = require("./toPropertyKey.js");
function applyDecs2311(e, t, n, r, o, i) {
var a,
c,
u,
s,
f,
l,
p,
d = Symbol.metadata || Symbol["for"]("Symbol.metadata"),
m = Object.defineProperty,
h = Object.create,
y = [h(null), h(null)],
v = t.length;
function g(t, n, r) {
return function (o, i) {
n && (i = o, o = e);
for (var a = 0; a < t.length; a++) i = t[a].apply(o, r ? [i] : []);
return r ? i : o;
};
}
function b(e, t, n, r) {
if ("function" != typeof e && (r || void 0 !== e)) throw new TypeError(t + " must " + (n || "be") + " a function" + (r ? "" : " or undefined"));
return e;
}
function applyDec(e, t, n, r, o, i, u, s, f, l, p) {
function d(e) {
if (!p(e)) throw new TypeError("Attempted to access private element on non-instance");
}
var h = [].concat(t[0]),
v = t[3],
w = !u,
D = 1 === o,
S = 3 === o,
j = 4 === o,
E = 2 === o;
function I(t, n, r) {
return function (o, i) {
return n && (i = o, o = e), r && r(o), P[t].call(o, i);
};
}
if (!w) {
var P = {},
k = [],
F = S ? "get" : j || D ? "set" : "value";
if (f ? (l || D ? P = {
get: setFunctionName(function () {
return v(this);
}, r, "get"),
set: function set(e) {
t[4](this, e);
}
} : P[F] = v, l || setFunctionName(P[F], r, E ? "" : F)) : l || (P = Object.getOwnPropertyDescriptor(e, r)), !l && !f) {
if ((c = y[+s][r]) && 7 != (c ^ o)) throw Error("Decorating two elements with the same name (" + P[F].name + ") is not supported yet");
y[+s][r] = o < 3 ? 1 : o;
}
}
for (var N = e, O = h.length - 1; O >= 0; O -= n ? 2 : 1) {
var T = b(h[O], "A decorator", "be", !0),
z = n ? h[O - 1] : void 0,
A = {},
H = {
kind: ["field", "accessor", "method", "getter", "setter", "class"][o],
name: r,
metadata: a,
addInitializer: function (e, t) {
if (e.v) throw new TypeError("attempted to call addInitializer after decoration was finished");
b(t, "An initializer", "be", !0), i.push(t);
}.bind(null, A)
};
if (w) c = T.call(z, N, H), A.v = 1, b(c, "class decorators", "return") && (N = c);else if (H["static"] = s, H["private"] = f, c = H.access = {
has: f ? p.bind() : function (e) {
return r in e;
}
}, j || (c.get = f ? E ? function (e) {
return d(e), P.value;
} : I("get", 0, d) : function (e) {
return e[r];
}), E || S || (c.set = f ? I("set", 0, d) : function (e, t) {
e[r] = t;
}), N = T.call(z, D ? {
get: P.get,
set: P.set
} : P[F], H), A.v = 1, D) {
if ("object" == _typeof(N) && N) (c = b(N.get, "accessor.get")) && (P.get = c), (c = b(N.set, "accessor.set")) && (P.set = c), (c = b(N.init, "accessor.init")) && k.unshift(c);else if (void 0 !== N) throw new TypeError("accessor decorators must return an object with get, set, or init properties or undefined");
} else b(N, (l ? "field" : "method") + " decorators", "return") && (l ? k.unshift(N) : P[F] = N);
}
return o < 2 && u.push(g(k, s, 1), g(i, s, 0)), l || w || (f ? D ? u.splice(-1, 0, I("get", s), I("set", s)) : u.push(E ? P[F] : b.call.bind(P[F])) : m(e, r, P)), N;
}
function w(e) {
return m(e, d, {
configurable: !0,
enumerable: !0,
value: a
});
}
return void 0 !== i && (a = i[d]), a = h(null == a ? null : a), f = [], l = function l(e) {
e && f.push(g(e));
}, p = function p(t, r) {
for (var i = 0; i < n.length; i++) {
var a = n[i],
c = a[1],
l = 7 & c;
if ((8 & c) == t && !l == r) {
var p = a[2],
d = !!a[3],
m = 16 & c;
applyDec(t ? e : e.prototype, a, m, d ? "#" + p : toPropertyKey(p), l, l < 2 ? [] : t ? s = s || [] : u = u || [], f, !!t, d, r, t && d ? function (t) {
return checkInRHS(t) === e;
} : o);
}
}
}, p(8, 0), p(0, 0), p(8, 1), p(0, 1), l(u), l(s), c = f, v || w(e), {
e: c,
get c() {
var n = [];
return v && [w(e = applyDec(e, [t], r, e.name, 5, n)), g(n, 1)];
}
};
}
module.exports = applyDecs2311, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,6 +0,0 @@
function _arrayLikeToArray(r, a) {
(null == a || a > r.length) && (a = r.length);
for (var e = 0, n = Array(a); e < a; e++) n[e] = r[e];
return n;
}
module.exports = _arrayLikeToArray, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,4 +0,0 @@
function _arrayWithHoles(r) {
if (Array.isArray(r)) return r;
}
module.exports = _arrayWithHoles, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var arrayLikeToArray = require("./arrayLikeToArray.js");
function _arrayWithoutHoles(r) {
if (Array.isArray(r)) return arrayLikeToArray(r);
}
module.exports = _arrayWithoutHoles, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
function _assertClassBrand(e, t, n) {
if ("function" == typeof e ? e === t : e.has(t)) return arguments.length < 3 ? t : n;
throw new TypeError("Private element is not present on this object");
}
module.exports = _assertClassBrand, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
function _assertThisInitialized(e) {
if (void 0 === e) throw new ReferenceError("this hasn't been initialised - super() hasn't been called");
return e;
}
module.exports = _assertThisInitialized, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,24 +0,0 @@
var OverloadYield = require("./OverloadYield.js");
function _asyncGeneratorDelegate(t) {
var e = {},
n = !1;
function pump(e, r) {
return n = !0, r = new Promise(function (n) {
n(t[e](r));
}), {
done: !1,
value: new OverloadYield(r, 1)
};
}
return e["undefined" != typeof Symbol && Symbol.iterator || "@@iterator"] = function () {
return this;
}, e.next = function (t) {
return n ? (n = !1, t) : pump("next", t);
}, "function" == typeof t["throw"] && (e["throw"] = function (t) {
if (n) throw n = !1, t;
return pump("throw", t);
}), "function" == typeof t["return"] && (e["return"] = function (t) {
return n ? (n = !1, t) : pump("return", t);
}), e;
}
module.exports = _asyncGeneratorDelegate, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,45 +0,0 @@
function _asyncIterator(r) {
var n,
t,
o,
e = 2;
for ("undefined" != typeof Symbol && (t = Symbol.asyncIterator, o = Symbol.iterator); e--;) {
if (t && null != (n = r[t])) return n.call(r);
if (o && null != (n = r[o])) return new AsyncFromSyncIterator(n.call(r));
t = "@@asyncIterator", o = "@@iterator";
}
throw new TypeError("Object is not async iterable");
}
function AsyncFromSyncIterator(r) {
function AsyncFromSyncIteratorContinuation(r) {
if (Object(r) !== r) return Promise.reject(new TypeError(r + " is not an object."));
var n = r.done;
return Promise.resolve(r.value).then(function (r) {
return {
value: r,
done: n
};
});
}
return AsyncFromSyncIterator = function AsyncFromSyncIterator(r) {
this.s = r, this.n = r.next;
}, AsyncFromSyncIterator.prototype = {
s: null,
n: null,
next: function next() {
return AsyncFromSyncIteratorContinuation(this.n.apply(this.s, arguments));
},
"return": function _return(r) {
var n = this.s["return"];
return void 0 === n ? Promise.resolve({
value: r,
done: !0
}) : AsyncFromSyncIteratorContinuation(n.apply(this.s, arguments));
},
"throw": function _throw(r) {
var n = this.s["return"];
return void 0 === n ? Promise.reject(r) : AsyncFromSyncIteratorContinuation(n.apply(this.s, arguments));
}
}, new AsyncFromSyncIterator(r);
}
module.exports = _asyncIterator, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,26 +0,0 @@
function asyncGeneratorStep(n, t, e, r, o, a, c) {
try {
var i = n[a](c),
u = i.value;
} catch (n) {
return void e(n);
}
i.done ? t(u) : Promise.resolve(u).then(r, o);
}
function _asyncToGenerator(n) {
return function () {
var t = this,
e = arguments;
return new Promise(function (r, o) {
var a = n.apply(t, e);
function _next(n) {
asyncGeneratorStep(a, r, o, _next, _throw, "next", n);
}
function _throw(n) {
asyncGeneratorStep(a, r, o, _next, _throw, "throw", n);
}
_next(void 0);
});
};
}
module.exports = _asyncToGenerator, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var OverloadYield = require("./OverloadYield.js");
function _awaitAsyncGenerator(e) {
return new OverloadYield(e, 0);
}
module.exports = _awaitAsyncGenerator, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,7 +0,0 @@
var getPrototypeOf = require("./getPrototypeOf.js");
var isNativeReflectConstruct = require("./isNativeReflectConstruct.js");
var possibleConstructorReturn = require("./possibleConstructorReturn.js");
function _callSuper(t, o, e) {
return o = getPrototypeOf(o), possibleConstructorReturn(t, isNativeReflectConstruct() ? Reflect.construct(o, e || [], getPrototypeOf(t).constructor) : o.apply(t, e));
}
module.exports = _callSuper, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,6 +0,0 @@
var _typeof = require("./typeof.js")["default"];
function _checkInRHS(e) {
if (Object(e) !== e) throw TypeError("right-hand side of 'in' should be an object, got " + (null !== e ? _typeof(e) : "null"));
return e;
}
module.exports = _checkInRHS, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,4 +0,0 @@
function _checkPrivateRedeclaration(e, t) {
if (t.has(e)) throw new TypeError("Cannot initialize the same private elements twice on an object");
}
module.exports = _checkPrivateRedeclaration, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,10 +0,0 @@
function _classApplyDescriptorDestructureSet(e, t) {
if (t.set) return "__destrObj" in t || (t.__destrObj = {
set value(r) {
t.set.call(e, r);
}
}), t.__destrObj;
if (!t.writable) throw new TypeError("attempted to set read only private field");
return t;
}
module.exports = _classApplyDescriptorDestructureSet, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,4 +0,0 @@
function _classApplyDescriptorGet(e, t) {
return t.get ? t.get.call(e) : t.value;
}
module.exports = _classApplyDescriptorGet, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,7 +0,0 @@
function _classApplyDescriptorSet(e, t, l) {
if (t.set) t.set.call(e, l);else {
if (!t.writable) throw new TypeError("attempted to set read only private field");
t.value = l;
}
}
module.exports = _classApplyDescriptorSet, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,4 +0,0 @@
function _classCallCheck(a, n) {
if (!(a instanceof n)) throw new TypeError("Cannot call a class as a function");
}
module.exports = _classCallCheck, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var assertClassBrand = require("./assertClassBrand.js");
function _classCheckPrivateStaticAccess(s, a, r) {
return assertClassBrand(a, s, r);
}
module.exports = _classCheckPrivateStaticAccess, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,4 +0,0 @@
function _classCheckPrivateStaticFieldDescriptor(t, e) {
if (void 0 === t) throw new TypeError("attempted to " + e + " private static field before its declaration");
}
module.exports = _classCheckPrivateStaticFieldDescriptor, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var classPrivateFieldGet2 = require("./classPrivateFieldGet2.js");
function _classExtractFieldDescriptor(e, t) {
return classPrivateFieldGet2(t, e);
}
module.exports = _classExtractFieldDescriptor, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,4 +0,0 @@
function _classNameTDZError(e) {
throw new ReferenceError('Class "' + e + '" cannot be referenced in computed property keys.');
}
module.exports = _classNameTDZError, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,7 +0,0 @@
var classApplyDescriptorDestructureSet = require("./classApplyDescriptorDestructureSet.js");
var classPrivateFieldGet2 = require("./classPrivateFieldGet2.js");
function _classPrivateFieldDestructureSet(e, t) {
var r = classPrivateFieldGet2(t, e);
return classApplyDescriptorDestructureSet(e, r);
}
module.exports = _classPrivateFieldDestructureSet, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,7 +0,0 @@
var classApplyDescriptorGet = require("./classApplyDescriptorGet.js");
var classPrivateFieldGet2 = require("./classPrivateFieldGet2.js");
function _classPrivateFieldGet(e, t) {
var r = classPrivateFieldGet2(t, e);
return classApplyDescriptorGet(e, r);
}
module.exports = _classPrivateFieldGet, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var assertClassBrand = require("./assertClassBrand.js");
function _classPrivateFieldGet2(s, a) {
return s.get(assertClassBrand(s, a));
}
module.exports = _classPrivateFieldGet2, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var checkPrivateRedeclaration = require("./checkPrivateRedeclaration.js");
function _classPrivateFieldInitSpec(e, t, a) {
checkPrivateRedeclaration(e, t), t.set(e, a);
}
module.exports = _classPrivateFieldInitSpec, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
function _classPrivateFieldBase(e, t) {
if (!{}.hasOwnProperty.call(e, t)) throw new TypeError("attempted to use private field on non-instance");
return e;
}
module.exports = _classPrivateFieldBase, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var id = 0;
function _classPrivateFieldKey(e) {
return "__private_" + id++ + "_" + e;
}
module.exports = _classPrivateFieldKey, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,7 +0,0 @@
var classApplyDescriptorSet = require("./classApplyDescriptorSet.js");
var classPrivateFieldGet2 = require("./classPrivateFieldGet2.js");
function _classPrivateFieldSet(e, t, r) {
var s = classPrivateFieldGet2(t, e);
return classApplyDescriptorSet(e, s, r), r;
}
module.exports = _classPrivateFieldSet, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var assertClassBrand = require("./assertClassBrand.js");
function _classPrivateFieldSet2(s, a, r) {
return s.set(assertClassBrand(s, a), r), r;
}
module.exports = _classPrivateFieldSet2, module.exports.__esModule = true, module.exports["default"] = module.exports;

View File

@@ -1,5 +0,0 @@
var assertClassBrand = require("./assertClassBrand.js");
function _classPrivateGetter(s, r, a) {
return a(assertClassBrand(s, r));
}
module.exports = _classPrivateGetter, module.exports.__esModule = true, module.exports["default"] = module.exports;

Some files were not shown because too many files have changed in this diff Show More