feat: nova-3 example

update CHANGELOG for 0.0.57

.github/workflows/coverage.yaml

@@ -0,0 +1,54 @@
+name: coverage
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - "**"
+    paths-ignore:
+      - "docs/**"
+
+jobs:
+  coverage:
+    name: "Coverage"
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v4
+      - name: Set up Python
+        id: setup_python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+      - name: Cache virtual environment
+        uses: actions/cache@v3
+        with:
+          # We are hashing dev-requirements.txt and test-requirements.txt which
+          # contain all dependencies needed to run the tests.
+          key: venv-${{ runner.os }}-${{ steps.setup_python.outputs.python-version}}-${{ hashFiles('dev-requirements.txt') }}-${{ hashFiles('test-requirements.txt') }}
+          path: .venv
+      - name: Install system packages
+        id: install_system_packages
+        run: |
+          sudo apt-get install -y portaudio19-dev
+      - name: Setup virtual environment
+        run: |
+          python -m venv .venv
+      - name: Install basic Python dependencies
+        run: |
+          source .venv/bin/activate
+          python -m pip install --upgrade pip
+          pip install -r dev-requirements.txt -r test-requirements.txt
+      - name: Run tests with coverage
+        run: |
+          source .venv/bin/activate
+          coverage run
+          coverage xml
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          slug: pipecat-ai/pipecat

CHANGELOG.md

@@ -5,10 +5,41 @@ All notable changes to **Pipecat** will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.0.57] - 2025-02-14
 
 ### Added
 
+- Added new `AudioContextWordTTSService`. This is a TTS base class for TTS
+  services that handling multiple separate audio requests.
+
+- Added new frames `EmulateUserStartedSpeakingFrame` and
+  `EmulateUserStoppedSpeakingFrame` which can be used to emulated VAD behavior
+  without VAD being present or not being triggered.
+
+- Added a new `audio_in_stream_on_start` field to `TransportParams`.
+
+- Added a new method `start_audio_in_streaming` in the `BaseInputTransport`.
+
+  - This method should be used to start receiving the input audio in case the
+    field `audio_in_stream_on_start` is set to `false`.
+
+- Added support for the `RTVIProcessor` to handle buffered audio in `base64`
+  format, converting it into InputAudioRawFrame for transport.
+
+- Added support for the `RTVIProcessor` to trigger `start_audio_in_streaming`
+  only after the `client-ready` message.
+
+- Added new `MUTE_UNTIL_FIRST_BOT_COMPLETE` strategy to `STTMuteStrategy`. This
+  strategy starts muted and remains muted until the first bot speech completes,
+  ensuring the bot's first response cannot be interrupted. This complements the
+  existing `FIRST_SPEECH` strategy which only mutes during the first detected
+  bot speech.
+
+- Added support for Google Cloud Speech-to-Text V2 through `GoogleSTTService`.
+
+- Added `RimeTTSService`, a new `WordTTSService`. Updated the foundational
+  example `07q-interruptible-rime.py` to use `RimeTTSService`.
+
 - Added support for Groq's Whisper API through the new `GroqSTTService` and
   OpenAI's Whisper API through the new `OpenAISTTService`. Introduced a new
   base class `BaseWhisperSTTService` to handle common Whisper API
@@ -18,8 +49,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   OpenAI-compatible interface. Also, added foundational example
   `14n-function-calling-perplexity.py`.
 
+- Added `DailyTransport.update_remote_participants()`. This allows you to update
+  remote participant's settings, like their permissions or which of their
+  devices are enabled. Requires that the local participant have participant
+  admin permission.
+
 ### Changed
 
+- We don't consider a colon `:` and end of sentence any more.
+
+- Updated `DailyTransport` to respect the `audio_in_stream_on_start` field,
+  ensuring it only starts receiving the audio input if it is enabled.
+
+- Updated `FastAPIWebsocketOutputTransport` to send `TransportMessageFrame` and
+  `TransportMessageUrgentFrame` to the serializer.
+
+- Updated `WebsocketServerOutputTransport` to send `TransportMessageFrame` and
+  `TransportMessageUrgentFrame` to the serializer.
+
+- Enhanced `STTMuteConfig` to validate strategy combinations, preventing
+  `MUTE_UNTIL_FIRST_BOT_COMPLETE` and `FIRST_SPEECH` from being used together
+  as they handle first bot speech differently.
+
+- Updated foundational example `07n-interruptible-google.py` to use all Google
+  services.
+
+- `RimeHttpTTSService` now uses the `mistv2` model by default.
+
 - Improved error handling in `AzureTTSService` to properly detect and log
   synthesis cancellation errors.
 
@@ -33,7 +89,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   model.
 
 - `RTVIObserver` doesn't handle `LLMSearchResponseFrame` frames anymore. For
-  now, to handle those frames you need to create a `GoogleRTVIObserver` instead.
+  now, to handle those frames you need to create a `GoogleRTVIObserver`
+  instead.
 
 ### Deprecated
 
@@ -50,14 +107,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Fixed a `FalImageGenService` issue that was causing the event loop to be
+  blocked while loading the downloadded image.
+
+- Fixed a `CartesiaTTSService` service issue that would cause audio overlapping
+  in some cases.
+
+- Fixed a websocket-based service issue (e.g. `CartesiaTTSService`) that was
+  preventing a reconnection after the server disconnected cleanly, which was
+  causing an inifite loop instead.
+
+- Fixed a `BaseOutputTransport` issue that was causing upstream frames to no be
+  pushed upstream.
+
+- Fixed multiple issue where user transcriptions where not being handled
+  properly. It was possible for short utterances to not trigger VAD which would
+  cause user transcriptions to be ignored. It was also possible for one or more
+  transcriptions to be generated after VAD in which case they would also be
+  ignored.
+
+- Fixed an issue that was causing `BotStoppedSpeakingFrame` to be generated too
+  late. This could then cause issues unblocking `STTMuteFilter` later than
+  desired.
+
 - Fixed an issue that was causing `AudioBufferProcessor` to not record
   synchronized audio.
 
 - Fixed an `RTVI` issue that was causing `bot-tts-text` messages to be sent
   before being processed by the output transport.
 
-- Fixed an issue[#1192] in 11labs where we are trying to reconnect/disconnect the
-  websocket connection even when the connection is already closed.
+- Fixed an issue[#1192] in 11labs where we are trying to reconnect/disconnect
+  the websocket connection even when the connection is already closed.
+
+- Fixed an issue where `has_regular_messages` condition was always true in
+  `GoogleLLMContext` due to `Part` having `function_call` & `function_response`
+  with `None` values.
+
+### Other
+
+- Added new `instant-voice` example. This example showcases how to enable
+  instant voice communication as soon as a user connects.
+
+- Added new `local-input-select-stt` example. This examples allows you to play
+  with local audio inputs by slecting them through a nice text interface.
 
 ## [0.0.56] - 2025-02-06
 
@@ -78,7 +170,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   logging setup.
 
 - Fixed a `SentryMetrics` issue that was preventing any metrics to be sent to
-  Sentry and also was preventing from metrics frames to be pushed to the pipeline.
+  Sentry and also was preventing from metrics frames to be pushed to the
+  pipeline.
 
 - Fixed an issue in `BaseOutputTransport` where incoming audio would not be
   resampled to the desired output sample rate.
@@ -237,7 +330,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - Added `enable_recording` and `geo` parameters to `DailyRoomProperties`.
 
-- Added `RecordingsBucketConfig` to `DailyRoomProperties` to upload recordings to a custom AWS bucket.
+- Added `RecordingsBucketConfig` to `DailyRoomProperties` to upload recordings
+  to a custom AWS bucket.
 
 ### Changed
 

README.md

@@ -2,7 +2,7 @@
  <img alt="pipecat" width="300px" height="auto" src="https://raw.githubusercontent.com/pipecat-ai/pipecat/main/pipecat.png">
 </div></h1>
 
-[![PyPI](https://img.shields.io/pypi/v/pipecat-ai)](https://pypi.org/project/pipecat-ai) ![Tests](https://github.com/pipecat-ai/pipecat/actions/workflows/tests.yaml/badge.svg) [![Docs](https://img.shields.io/badge/Documentation-blue)](https://docs.pipecat.ai) [![Discord](https://img.shields.io/discord/1239284677165056021)](https://discord.gg/pipecat)
+[![PyPI](https://img.shields.io/pypi/v/pipecat-ai)](https://pypi.org/project/pipecat-ai) ![Tests](https://github.com/pipecat-ai/pipecat/actions/workflows/tests.yaml/badge.svg) [![codecov](https://codecov.io/gh/pipecat-ai/pipecat/graph/badge.svg?token=LNVUIVO4Y9)](https://codecov.io/gh/pipecat-ai/pipecat) [![Docs](https://img.shields.io/badge/Documentation-blue)](https://docs.pipecat.ai) [![Discord](https://img.shields.io/discord/1239284677165056021)](https://discord.gg/pipecat)
 
 Pipecat is an open source Python framework for building voice and multimodal conversational agents. It handles the complex orchestration of AI services, network transport, audio processing, and multimodal interactions, letting you focus on creating engaging experiences.
 
@@ -57,7 +57,7 @@ pip install "pipecat-ai[option,...]"
 
 | Category            | Services                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | Install Command Example                 |
 | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- |
-| Speech-to-Text      | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper)                                                                                                                                                                                                                                                                                                                                                                                                                                                            | `pip install "pipecat-ai[deepgram]"`    |
+| Speech-to-Text      | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper)                                                                                                                                                                                                                                                                                                                                                                                              | `pip install "pipecat-ai[deepgram]"`    |
 | LLMs                | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Together AI](https://docs.pipecat.ai/server/services/llm/together) | `pip install "pipecat-ai[openai]"`      |
 | Text-to-Speech      | [AWS](https://docs.pipecat.ai/server/services/tts/aws), [Azure](https://docs.pipecat.ai/server/services/tts/azure), [Cartesia](https://docs.pipecat.ai/server/services/tts/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/tts/deepgram), [ElevenLabs](https://docs.pipecat.ai/server/services/tts/elevenlabs), [Fish](https://docs.pipecat.ai/server/services/tts/fish), [Google](https://docs.pipecat.ai/server/services/tts/google), [LMNT](https://docs.pipecat.ai/server/services/tts/lmnt), [OpenAI](https://docs.pipecat.ai/server/services/tts/openai), [PlayHT](https://docs.pipecat.ai/server/services/tts/playht), [Rime](https://docs.pipecat.ai/server/services/tts/rime), [XTTS](https://docs.pipecat.ai/server/services/tts/xtts)                                                                                                                                                                          | `pip install "pipecat-ai[cartesia]"`    |
 | Speech-to-Speech    | [Gemini Multimodal Live](https://docs.pipecat.ai/server/services/s2s/gemini), [OpenAI Realtime](https://docs.pipecat.ai/server/services/s2s/openai)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | `pip install "pipecat-ai[google]"`      |

codecov.yml

@@ -0,0 +1,11 @@
+coverage:
+  range: 50..90 # coverage lower than 50 is red, higher than 90 green, between color code
+
+  status:
+    project:
+      default:
+        target: auto # auto % coverage target
+        threshold: 5%  # allow for 5% reduction of coverage without failing
+
+    # do not run coverage on patch nor changes
+    patch: false

dev-requirements.txt

@@ -1,4 +1,5 @@
 build~=1.2.2
+coverage~=7.6.12
 grpcio-tools~=1.67.1
 pip-tools~=7.4.1
 pre-commit~=4.0.1

examples/bot-ready-signalling/client/javascript/package-lock.json

@@ -12,7 +12,7 @@
         "@daily-co/daily-js": "0.74.0"
       },
       "devDependencies": {
-        "vite": "^6.0.2"
+        "vite": "^6.0.9"
       }
     },
     "node_modules/@babel/runtime": {
@@ -1007,15 +1007,14 @@
       }
     },
     "node_modules/vite": {
-      "version": "6.0.7",
-      "resolved": "https://registry.npmjs.org/vite/-/vite-6.0.7.tgz",
-      "integrity": "sha512-RDt8r/7qx9940f8FcOIAH9PTViRrghKaK2K1jY3RaAURrEUbm9Du1mJ72G+jlhtG3WwodnfzY8ORQZbBavZEAQ==",
+      "version": "6.1.0",
+      "resolved": "https://registry.npmjs.org/vite/-/vite-6.1.0.tgz",
+      "integrity": "sha512-RjjMipCKVoR4hVfPY6GQTgveinjNuyLw+qruksLDvA5ktI1150VmcMBKmQaEWJhg/j6Uaf6dNCNA0AfdzUb/hQ==",
       "dev": true,
-      "license": "MIT",
       "dependencies": {
         "esbuild": "^0.24.2",
-        "postcss": "^8.4.49",
-        "rollup": "^4.23.0"
+        "postcss": "^8.5.1",
+        "rollup": "^4.30.1"
       },
       "bin": {
         "vite": "bin/vite.js"

examples/bot-ready-signalling/client/javascript/package.json

@@ -12,7 +12,7 @@
   "license": "ISC",
   "description": "",
   "devDependencies": {
-    "vite": "^6.0.2"
+    "vite": "^6.0.9"
   },
   "dependencies": {
     "@daily-co/daily-js": "0.74.0"

examples/foundational/01a-local-audio.py

@@ -16,8 +16,7 @@ from pipecat.pipeline.pipeline import Pipeline
 from pipecat.pipeline.runner import PipelineRunner
 from pipecat.pipeline.task import PipelineTask
 from pipecat.services.cartesia import CartesiaTTSService
-from pipecat.transports.base_transport import TransportParams
-from pipecat.transports.local.audio import LocalAudioTransport
+from pipecat.transports.local.audio import LocalAudioTransport, LocalAudioTransportParams
 
 load_dotenv(override=True)
 
@@ -26,7 +25,7 @@ logger.add(sys.stderr, level="DEBUG")
 
 
 async def main():
-    transport = LocalAudioTransport(TransportParams(audio_out_enabled=True))
+    transport = LocalAudioTransport(LocalAudioTransportParams(audio_out_enabled=True))
 
     tts = CartesiaTTSService(
         api_key=os.getenv("CARTESIA_API_KEY"),
@@ -41,7 +40,7 @@ async def main():
         await asyncio.sleep(1)
         await task.queue_frames([TTSSpeakFrame("Hello there, how is it going!"), EndFrame()])
 
-    runner = PipelineRunner()
+    runner = PipelineRunner(handle_sigint=False if sys.platform == "win32" else True)
 
     await asyncio.gather(runner.run(task), say_something())
 

examples/foundational/06a-image-sync.py

@@ -27,7 +27,7 @@ from pipecat.pipeline.runner import PipelineRunner
 from pipecat.pipeline.task import PipelineParams, PipelineTask
 from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
 from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
-from pipecat.services.cartesia import CartesiaHttpTTSService
+from pipecat.services.cartesia import CartesiaTTSService
 from pipecat.services.openai import OpenAILLMService
 from pipecat.transports.services.daily import DailyParams, DailyTransport
 
@@ -91,7 +91,7 @@ async def main():
             ),
         )
 
-        tts = CartesiaHttpTTSService(
+        tts = CartesiaTTSService(
             api_key=os.getenv("CARTESIA_API_KEY"),
             voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22",  # British Lady
         )

examples/foundational/07c-interruptible-deepgram.py

@@ -9,6 +9,7 @@ import os
 import sys
 
 import aiohttp
+from deepgram import LiveOptions
 from dotenv import load_dotenv
 from loguru import logger
 from runner import configure
@@ -44,7 +45,23 @@ async def main():
             ),
         )
 
-        stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
+        # stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
+        stt = DeepgramSTTService(
+            api_key=os.getenv("DEEPGRAM_API_KEY"),
+            # url=deepgram_url,
+            live_options=LiveOptions(
+                encoding="linear16",
+                language="en-US",
+                model="nova-3",
+                channels=1,
+                interim_results=True,
+                # smart_format=smart_format,
+                # endpointing=endpointing,
+                vad_events=True,
+                diarize=True,
+                filler_words=True,
+            ),
+        )
 
         tts = DeepgramTTSService(api_key=os.getenv("DEEPGRAM_API_KEY"), voice="aura-helios-en")
 

examples/foundational/07n-interruptible-google.py

@@ -18,9 +18,7 @@ from pipecat.pipeline.pipeline import Pipeline
 from pipecat.pipeline.runner import PipelineRunner
 from pipecat.pipeline.task import PipelineParams, PipelineTask
 from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
-from pipecat.services.deepgram import DeepgramSTTService
-from pipecat.services.google import GoogleTTSService
-from pipecat.services.openai import OpenAILLMService
+from pipecat.services.google import GoogleLLMService, GoogleSTTService, GoogleTTSService
 from pipecat.transcriptions.language import Language
 from pipecat.transports.services.daily import DailyParams, DailyTransport
 
@@ -46,14 +44,16 @@ async def main():
             ),
         )
 
-        stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
+        stt = GoogleSTTService(
+            params=GoogleSTTService.InputParams(languages=Language.EN_US),
+        )
 
         tts = GoogleTTSService(
             voice_id="en-US-Journey-F",
             params=GoogleTTSService.InputParams(language=Language.EN_US),
         )
 
-        llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")
+        llm = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"))
 
         messages = [
             {

examples/foundational/07q-interruptible-rime.py

@@ -19,7 +19,7 @@ from pipecat.pipeline.runner import PipelineRunner
 from pipecat.pipeline.task import PipelineParams, PipelineTask
 from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
 from pipecat.services.openai import OpenAILLMService
-from pipecat.services.rime import RimeHttpTTSService
+from pipecat.services.rime import RimeTTSService
 from pipecat.transports.services.daily import DailyParams, DailyTransport
 
 load_dotenv(override=True)
@@ -44,10 +44,9 @@ async def main():
             ),
         )
 
-        tts = RimeHttpTTSService(
+        tts = RimeTTSService(
             api_key=os.getenv("RIME_API_KEY", ""),
             voice_id="rex",
-            params=RimeHttpTTSService.InputParams(reduce_latency=True),
         )
 
         llm = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"), model="gpt-4o")

examples/foundational/13a-whisper-local.py

@@ -16,8 +16,7 @@ from pipecat.pipeline.runner import PipelineRunner
 from pipecat.pipeline.task import PipelineTask
 from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
 from pipecat.services.whisper import WhisperSTTService
-from pipecat.transports.base_transport import TransportParams
-from pipecat.transports.local.audio import LocalAudioTransport
+from pipecat.transports.local.audio import LocalAudioTransport, LocalAudioTransportParams
 
 load_dotenv(override=True)
 
@@ -34,7 +33,7 @@ class TranscriptionLogger(FrameProcessor):
 
 
 async def main():
-    transport = LocalAudioTransport(TransportParams(audio_in_enabled=True))
+    transport = LocalAudioTransport(LocalAudioTransportParams(audio_in_enabled=True))
 
     stt = WhisperSTTService()
 
@@ -44,7 +43,7 @@ async def main():
 
     task = PipelineTask(pipeline)
 
-    runner = PipelineRunner()
+    runner = PipelineRunner(handle_sigint=False if sys.platform == "win32" else True)
 
     await runner.run(task)
 

examples/foundational/22d-natural-conversation-gemini-audio.py

@@ -497,7 +497,7 @@ class UserAggregatorBuffer(LLMResponseAggregator):
         if isinstance(frame, UserStartedSpeakingFrame):
             self._transcription = ""
 
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if self._aggregation:
             self._transcription = self._aggregation
             self._aggregation = ""

examples/foundational/24-stt-mute-filter.py

@@ -62,7 +62,10 @@ async def main():
         # Configure the mute processor with both strategies
         stt_mute_processor = STTMuteFilter(
             config=STTMuteConfig(
-                strategies={STTMuteStrategy.FIRST_SPEECH, STTMuteStrategy.FUNCTION_CALL}
+                strategies={
+                    STTMuteStrategy.MUTE_UNTIL_FIRST_BOT_COMPLETE,
+                    STTMuteStrategy.FUNCTION_CALL,
+                }
             ),
         )
 

examples/foundational/25-google-audio-in.py

@@ -143,7 +143,14 @@ class InputTranscriptionContextFilter(FrameProcessor):
             return
 
         try:
-            message = frame.context.messages[-1]
+            # Make sure we're working with a GoogleLLMContext
+            context = GoogleLLMContext.upgrade_to_google(frame.context)
+            message = context.messages[-1]
+
+            if not isinstance(message, glm.Content):
+                logger.error(f"Expected glm.Content, got {type(message)}")
+                return
+
             last_part = message.parts[-1]
             if not (
                 message.role == "user"

examples/instant-voice/.gitignore

@@ -0,0 +1,7 @@
+venv
+.idea
+server/.env
+client/javascript/node_modules
+client/javascript/.vite
+bkp
+

examples/instant-voice/README.md

@@ -0,0 +1,55 @@
+# Instant Voice
+
+This demo showcases how to enable instant voice communication as soon as a user connects.
+By leveraging optimizations on both the server and client sides, users can start speaking immediately after pressing the connect button.
+
+## How It Works
+
+### Server-Side Improvements:
+- A **pool of Daily rooms** is managed to ensure quick connections.
+- When a user connects, an existing room from the pool is assigned.
+- A new room is created asynchronously to maintain the predefined pool size.
+
+### Client-Side Improvements:
+- Using the **DailyTransport** property `bufferLocalAudioUntilBotReady` set to enabled, users can start speaking immediately
+  upon receiving the `AUDIO_BUFFERING_STARTED` event (typically within ~1s).
+- This allows users to speak even before the bot is fully ready or the WebRTC connection is fully established.
+
+## Quick Start
+
+### 1. Start the Bot Server
+
+1. Navigate to the server directory:
+   ```bash
+   cd server
+   ```
+2. Create and activate a virtual environment:
+   ```bash
+   python3 -m venv venv
+   source venv/bin/activate  # On Windows: venv\Scripts\activate
+   ```
+3. Install dependencies:
+   ```bash
+   pip install -r requirements.txt
+   ```
+4. Copy the `.env.example` file to `.env` and configure it:
+    - Add your API keys.
+5. Start the server:
+   ```bash
+   python src/server.py
+   ```
+
+### 2. Connect Using the Client App
+
+For client-side setup, refer to the [JavaScript Guide](client/javascript/README.md).
+
+## Important Notes
+- The bot server **must** be running before using the client implementation.
+- Ensure your environment variables are correctly set up.
+
+## Requirements
+- **Python 3.10+**
+- **Node.js 16+** (for JavaScript/React client)
+- **Daily API key**
+- **Google API key**
+- **Modern web browser with WebRTC support**

examples/instant-voice/client/javascript/README.md

@@ -0,0 +1,27 @@
+# JavaScript Implementation
+
+Basic implementation using the [Pipecat JavaScript SDK](https://docs.pipecat.ai/client/js/introduction).
+
+## Setup
+
+1. Run the bot server. See the [server README](../../README).
+
+2. Navigate to the `client/javascript` directory:
+
+```bash
+cd client/javascript
+```
+
+3. Install dependencies:
+
+```bash
+npm install
+```
+
+4. Run the client app:
+
+```
+npm run dev
+```
+
+5. Visit http://localhost:5173 in your browser.

examples/instant-voice/client/javascript/index.html

@@ -0,0 +1,37 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>AI Chatbot</title>
+</head>
+
+<body>
+<div class="container">
+  <div class="status-bar">
+    <div class="status">
+      Buffering audio: <span id="buffering-status">No</span>
+    </div>
+    <div class="status">
+      Transport: <span id="connection-status">Disconnected</span>
+    </div>
+    <div class="controls">
+      <button id="connect-btn">Connect</button>
+      <button id="disconnect-btn" disabled>Disconnect</button>
+    </div>
+  </div>
+
+  <audio id="bot-audio" autoplay></audio>
+
+  <div class="debug-panel">
+    <h3>Debug Info</h3>
+    <div id="debug-log"></div>
+  </div>
+</div>
+
+<script type="module" src="/src/app.ts"></script>
+<link rel="stylesheet" href="/src/style.css">
+</body>
+
+</html>

examples/instant-voice/client/javascript/package.json

@@ -0,0 +1,24 @@
+{
+  "name": "client",
+  "version": "1.0.0",
+  "main": "index.js",
+  "scripts": {
+    "dev": "vite",
+    "build": "tsc && vite build",
+    "preview": "vite preview"
+  },
+  "keywords": [],
+  "author": "",
+  "license": "ISC",
+  "description": "",
+  "devDependencies": {
+    "@types/node": "^22.13.1",
+    "@vitejs/plugin-react-swc": "^3.7.2",
+    "typescript": "^5.7.3",
+    "vite": "^6.0.2"
+  },
+  "dependencies": {
+    "@pipecat-ai/client-js": "^0.3.2",
+    "@pipecat-ai/daily-transport": "^0.3.5"
+  }
+}

examples/instant-voice/client/javascript/src/app.ts

@@ -0,0 +1,268 @@
+/**
+ * Copyright (c) 2024–2025, Daily
+ *
+ * SPDX-License-Identifier: BSD 2-Clause License
+ */
+
+/**
+ * RTVI Client Implementation
+ *
+ * This client connects to an RTVI-compatible bot server using WebRTC (via Daily).
+ * It handles audio/video streaming and manages the connection lifecycle.
+ *
+ * Requirements:
+ * - A running RTVI bot server (defaults to http://localhost:7860)
+ * - The server must implement the /connect endpoint that returns Daily.co room credentials
+ * - Browser with WebRTC support
+ */
+
+import {
+  Participant,
+  RTVIClient,
+  RTVIClientOptions,
+  RTVIEvent,
+} from '@pipecat-ai/client-js';
+import { DailyTransport } from '@pipecat-ai/daily-transport';
+import SoundUtils from "./util/soundUtils";
+import { InstantVoiceHelper } from "./util/instantVoiceHelper";
+
+/**
+ * InstantVoiceClient handles the connection and media management for a real-time
+ * voice and video interaction with an AI bot.
+ */
+class InstantVoiceClient {
+
+  private declare rtviClient: RTVIClient;
+  private connectBtn: HTMLButtonElement | null = null;
+  private disconnectBtn: HTMLButtonElement | null = null;
+  private statusSpan: HTMLElement | null = null;
+  private bufferingAudioSpan: HTMLElement | null = null;
+  private debugLog: HTMLElement | null = null;
+  private botAudio: HTMLAudioElement;
+  private declare startTime: number;
+
+  constructor() {
+    this.botAudio = document.createElement('audio');
+    this.botAudio.autoplay = true;
+    document.body.appendChild(this.botAudio);
+    this.setupDOMElements();
+    this.setupEventListeners();
+    this.initializeRTVIClient();
+  }
+
+  /**
+   * Set up references to DOM elements and create necessary media elements
+   */
+  private setupDOMElements(): void {
+    this.connectBtn = document.getElementById('connect-btn') as HTMLButtonElement;
+    this.disconnectBtn = document.getElementById('disconnect-btn') as HTMLButtonElement;
+    this.statusSpan = document.getElementById('connection-status');
+    this.bufferingAudioSpan = document.getElementById('buffering-status');
+    this.debugLog = document.getElementById('debug-log');
+  }
+
+  /**
+   * Set up event listeners for connect/disconnect buttons
+   */
+  private setupEventListeners(): void {
+    this.connectBtn?.addEventListener('click', () => this.connect());
+    this.disconnectBtn?.addEventListener('click', () => this.disconnect());
+  }
+
+  private initializeRTVIClient(): void {
+    const transport = new DailyTransport({
+      bufferLocalAudioUntilBotReady: true
+    });
+    const RTVIConfig: RTVIClientOptions = {
+      transport,
+      params: {
+        // The baseURL and endpoint of your bot server that the client will connect to
+        baseUrl: 'http://localhost:7860',
+        endpoints: { connect: '/connect' },
+      },
+      enableMic: true,
+      enableCam: false,
+      callbacks: {
+        onConnected: () => {
+          this.updateStatus('Connected');
+          if (this.connectBtn) this.connectBtn.disabled = true;
+          if (this.disconnectBtn) this.disconnectBtn.disabled = false;
+        },
+        onDisconnected: () => {
+          this.updateStatus('Disconnected');
+          this.updateBufferingStatus('No');
+          if (this.connectBtn) this.connectBtn.disabled = false;
+          if (this.disconnectBtn) this.disconnectBtn.disabled = true;
+          this.log('Client disconnected');
+        },
+        onBotConnected: (participant: Participant)  => {
+          this.log(`onBotConnected, timeTaken: ${Date.now() - this.startTime}`);
+        },
+        onBotReady: (data) => {
+          this.log(`onBotReady, timeTaken: ${Date.now() - this.startTime}`);
+          this.log(`Bot ready: ${JSON.stringify(data)}`);
+          this.setupMediaTracks();
+        },
+        onUserTranscript: (data) => {
+          if (data.final) {
+            this.log(`User: ${data.text}`);
+          }
+        },
+        onBotTranscript: (data) => this.log(`Bot: ${data.text}`),
+        onMessageError: (error) => console.error('Message error:', error),
+        onError: (error) => console.error('Error:', error),
+      },
+    }
+
+    this.rtviClient = new RTVIClient(RTVIConfig);
+    this.rtviClient.registerHelper("transport", new InstantVoiceHelper({
+          callbacks: {
+            onAudioBufferingStarted: () => {
+              SoundUtils.beep()
+              this.updateBufferingStatus('Yes');
+              this.log(`onMicCaptureStarted, timeTaken: ${Date.now() - this.startTime}`);
+            },
+            onAudioBufferingStopped: () => {
+              this.updateBufferingStatus('No');
+              this.log(`onMicCaptureStopped, timeTaken: ${Date.now() - this.startTime}`);
+            }
+          }
+        }
+    ));
+    this.setupTrackListeners();
+  }
+
+  /**
+   * Add a timestamped message to the debug log
+   */
+  private log(message: string): void {
+    if (!this.debugLog) return;
+    const entry = document.createElement('div');
+    entry.textContent = `${new Date().toISOString()} - ${message}`;
+    if (message.startsWith('User: ')) {
+      entry.style.color = '#2196F3';
+    } else if (message.startsWith('Bot: ')) {
+      entry.style.color = '#4CAF50';
+    }
+    this.debugLog.appendChild(entry);
+    this.debugLog.scrollTop = this.debugLog.scrollHeight;
+    console.log(message);
+  }
+
+  /**
+   * Update the connection status display
+   */
+  private updateStatus(status: string): void {
+    if (this.statusSpan) {
+      this.statusSpan.textContent = status;
+    }
+    this.log(`Status: ${status}`);
+  }
+
+  /**
+   * Update the connection status display
+   */
+  private updateBufferingStatus(status: string): void {
+    if (this.bufferingAudioSpan) {
+      this.bufferingAudioSpan.textContent = status;
+    }
+    this.log(`BufferingStatus: ${status}`);
+  }
+
+  /**
+   * Check for available media tracks and set them up if present
+   * This is called when the bot is ready or when the transport state changes to ready
+   */
+  setupMediaTracks() {
+    if (!this.rtviClient) return;
+    const tracks = this.rtviClient.tracks();
+    if (tracks.bot?.audio) {
+      this.setupAudioTrack(tracks.bot.audio);
+    }
+  }
+
+  /**
+   * Set up listeners for track events (start/stop)
+   * This handles new tracks being added during the session
+   */
+  setupTrackListeners() {
+    if (!this.rtviClient) return;
+
+    // Listen for new tracks starting
+    this.rtviClient.on(RTVIEvent.TrackStarted, (track, participant) => {
+      // Only handle non-local (bot) tracks
+      if (!participant?.local && track.kind === 'audio') {
+        this.setupAudioTrack(track);
+      }
+    });
+
+    // Listen for tracks stopping
+    this.rtviClient.on(RTVIEvent.TrackStopped, (track, participant) => {
+      this.log(`Track stopped: ${track.kind} from ${participant?.name || 'unknown'}`);
+    });
+  }
+
+  /**
+   * Set up an audio track for playback
+   * Handles both initial setup and track updates
+   */
+  private setupAudioTrack(track: MediaStreamTrack): void {
+    this.log('Setting up audio track');
+    if (this.botAudio.srcObject && "getAudioTracks" in this.botAudio.srcObject) {
+      const oldTrack = this.botAudio.srcObject.getAudioTracks()[0];
+      if (oldTrack?.id === track.id) return;
+    }
+    this.botAudio.srcObject = new MediaStream([track]);
+  }
+
+  /**
+   * Initialize and connect to the bot
+   * This sets up the RTVI client, initializes devices, and establishes the connection
+   */
+  public async connect(): Promise<void> {
+    try {
+      this.startTime = Date.now();
+      this.log('Connecting to bot...');
+      await this.rtviClient.connect();
+    } catch (error) {
+      this.log(`Error connecting: ${(error as Error).message}`);
+      this.updateStatus('Error');
+      this.updateBufferingStatus('No');
+
+      // Clean up if there's an error
+      if (this.rtviClient) {
+        try {
+          await this.rtviClient.disconnect();
+        } catch (disconnectError) {
+          this.log(`Error during disconnect: ${disconnectError}`);
+        }
+      }
+    }
+  }
+
+  /**
+   * Disconnect from the bot and clean up media resources
+   */
+  public async disconnect(): Promise<void> {
+    try {
+      await this.rtviClient.disconnect();
+      if (this.botAudio.srcObject && "getAudioTracks" in this.botAudio.srcObject) {
+        this.botAudio.srcObject.getAudioTracks().forEach((track) => track.stop());
+        this.botAudio.srcObject = null;
+      }
+    } catch (error) {
+      this.log(`Error disconnecting: ${(error as Error).message}`);
+    }
+  }
+}
+
+declare global {
+  interface Window {
+    InstantVoiceClient: typeof InstantVoiceClient;
+  }
+}
+
+window.addEventListener('DOMContentLoaded', () => {
+  window.InstantVoiceClient = InstantVoiceClient;
+  new InstantVoiceClient();
+});

examples/instant-voice/client/javascript/src/style.css

@@ -0,0 +1,98 @@
+body {
+    margin: 0;
+    padding: 20px;
+    font-family: Arial, sans-serif;
+    background-color: #f0f0f0;
+}
+
+.container {
+    max-width: 1200px;
+    margin: 0 auto;
+}
+
+.status-bar {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    padding: 10px;
+    background-color: #fff;
+    border-radius: 8px;
+    margin-bottom: 20px;
+}
+
+.controls button {
+    padding: 8px 16px;
+    margin-left: 10px;
+    border: none;
+    border-radius: 4px;
+    cursor: pointer;
+}
+
+#connect-btn {
+    background-color: #4caf50;
+    color: white;
+}
+
+#disconnect-btn {
+    background-color: #f44336;
+    color: white;
+}
+
+button:disabled {
+    opacity: 0.5;
+    cursor: not-allowed;
+}
+
+.main-content {
+    background-color: #fff;
+    border-radius: 8px;
+    padding: 20px;
+    margin-bottom: 20px;
+}
+
+.bot-container {
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+}
+
+#bot-video-container {
+    width: 640px;
+    height: 360px;
+    background-color: #e0e0e0;
+    border-radius: 8px;
+    margin: 20px auto;
+    overflow: hidden;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+}
+
+#bot-video-container video {
+    width: 100%;
+    height: 100%;
+    object-fit: cover;
+}
+
+.debug-panel {
+    background-color: #fff;
+    border-radius: 8px;
+    padding: 20px;
+}
+
+.debug-panel h3 {
+    margin: 0 0 10px 0;
+    font-size: 16px;
+    font-weight: bold;
+}
+
+#debug-log {
+    height: 500px;
+    overflow-y: auto;
+    background-color: #f8f8f8;
+    padding: 10px;
+    border-radius: 4px;
+    font-family: monospace;
+    font-size: 12px;
+    line-height: 1.4;
+}

examples/instant-voice/client/javascript/src/util/instantVoiceHelper.ts

@@ -0,0 +1,39 @@
+import {RTVIClientHelper, RTVIClientHelperOptions, RTVIMessage} from "@pipecat-ai/client-js";
+import {DailyRTVIMessageType} from '@pipecat-ai/daily-transport';
+
+export type InstantVoiceHelperCallbacks = Partial<{
+    onAudioBufferingStarted: () => void;
+    onAudioBufferingStopped: () => void;
+}>;
+
+// --- Interface and class
+export interface InstantVoiceHelperOptions extends RTVIClientHelperOptions {
+    callbacks?: InstantVoiceHelperCallbacks;
+}
+export class InstantVoiceHelper extends RTVIClientHelper {
+
+    protected declare _options: InstantVoiceHelperOptions;
+
+    constructor(options: InstantVoiceHelperOptions) {
+        super(options);
+    }
+
+    handleMessage(rtviMessage: RTVIMessage): void {
+        switch (rtviMessage.type) {
+            case DailyRTVIMessageType.AUDIO_BUFFERING_STARTED:
+                if (this._options.callbacks?.onAudioBufferingStarted) {
+                    this._options.callbacks?.onAudioBufferingStarted()
+                }
+                break;
+            case DailyRTVIMessageType.AUDIO_BUFFERING_STOPPED:
+                if (this._options.callbacks?.onAudioBufferingStopped) {
+                    this._options.callbacks?.onAudioBufferingStopped()
+                }
+                break;
+        }
+    }
+
+    getMessageTypes(): string[] {
+        return [DailyRTVIMessageType.AUDIO_BUFFERING_STARTED, DailyRTVIMessageType.AUDIO_BUFFERING_STOPPED];
+    }
+}

examples/instant-voice/client/javascript/src/util/soundUtils.ts

@@ -0,0 +1,8 @@
+class SoundUtils {
+    static beep() {
+        const snd = new Audio("data:audio/wav;base64,//uQRAAAAWMSLwUIYAAsYkXgoQwAEaYLWfkWgAI0wWs/ItAAAGDgYtAgAyN+QWaAAihwMWm4G8QQRDiMcCBcH3Cc+CDv/7xA4Tvh9Rz/y8QADBwMWgQAZG/ILNAARQ4GLTcDeIIIhxGOBAuD7hOfBB3/94gcJ3w+o5/5eIAIAAAVwWgQAVQ2ORaIQwEMAJiDg95G4nQL7mQVWI6GwRcfsZAcsKkJvxgxEjzFUgfHoSQ9Qq7KNwqHwuB13MA4a1q/DmBrHgPcmjiGoh//EwC5nGPEmS4RcfkVKOhJf+WOgoxJclFz3kgn//dBA+ya1GhurNn8zb//9NNutNuhz31f////9vt///z+IdAEAAAK4LQIAKobHItEIYCGAExBwe8jcToF9zIKrEdDYIuP2MgOWFSE34wYiR5iqQPj0JIeoVdlG4VD4XA67mAcNa1fhzA1jwHuTRxDUQ//iYBczjHiTJcIuPyKlHQkv/LHQUYkuSi57yQT//uggfZNajQ3Vmz+Zt//+mm3Wm3Q576v////+32///5/EOgAAADVghQAAAAA//uQZAUAB1WI0PZugAAAAAoQwAAAEk3nRd2qAAAAACiDgAAAAAAABCqEEQRLCgwpBGMlJkIz8jKhGvj4k6jzRnqasNKIeoh5gI7BJaC1A1AoNBjJgbyApVS4IDlZgDU5WUAxEKDNmmALHzZp0Fkz1FMTmGFl1FMEyodIavcCAUHDWrKAIA4aa2oCgILEBupZgHvAhEBcZ6joQBxS76AgccrFlczBvKLC0QI2cBoCFvfTDAo7eoOQInqDPBtvrDEZBNYN5xwNwxQRfw8ZQ5wQVLvO8OYU+mHvFLlDh05Mdg7BT6YrRPpCBznMB2r//xKJjyyOh+cImr2/4doscwD6neZjuZR4AgAABYAAAABy1xcdQtxYBYYZdifkUDgzzXaXn98Z0oi9ILU5mBjFANmRwlVJ3/6jYDAmxaiDG3/6xjQQCCKkRb/6kg/wW+kSJ5//rLobkLSiKmqP/0ikJuDaSaSf/6JiLYLEYnW/+kXg1WRVJL/9EmQ1YZIsv/6Qzwy5qk7/+tEU0nkls3/zIUMPKNX/6yZLf+kFgAfgGyLFAUwY//uQZAUABcd5UiNPVXAAAApAAAAAE0VZQKw9ISAAACgAAAAAVQIygIElVrFkBS+Jhi+EAuu+lKAkYUEIsmEAEoMeDmCETMvfSHTGkF5RWH7kz/ESHWPAq/kcCRhqBtMdokPdM7vil7RG98A2sc7zO6ZvTdM7pmOUAZTnJW+NXxqmd41dqJ6mLTXxrPpnV8avaIf5SvL7pndPvPpndJR9Kuu8fePvuiuhorgWjp7Mf/PRjxcFCPDkW31srioCExivv9lcwKEaHsf/7ow2Fl1T/9RkXgEhYElAoCLFtMArxwivDJJ+bR1HTKJdlEoTELCIqgEwVGSQ+hIm0NbK8WXcTEI0UPoa2NbG4y2K00JEWbZavJXkYaqo9CRHS55FcZTjKEk3NKoCYUnSQ0rWxrZbFKbKIhOKPZe1cJKzZSaQrIyULHDZmV5K4xySsDRKWOruanGtjLJXFEmwaIbDLX0hIPBUQPVFVkQkDoUNfSoDgQGKPekoxeGzA4DUvnn4bxzcZrtJyipKfPNy5w+9lnXwgqsiyHNeSVpemw4bWb9psYeq//uQZBoABQt4yMVxYAIAAAkQoAAAHvYpL5m6AAgAACXDAAAAD59jblTirQe9upFsmZbpMudy7Lz1X1DYsxOOSWpfPqNX2WqktK0DMvuGwlbNj44TleLPQ+Gsfb+GOWOKJoIrWb3cIMeeON6lz2umTqMXV8Mj30yWPpjoSa9ujK8SyeJP5y5mOW1D6hvLepeveEAEDo0mgCRClOEgANv3B9a6fikgUSu/DmAMATrGx7nng5p5iimPNZsfQLYB2sDLIkzRKZOHGAaUyDcpFBSLG9MCQALgAIgQs2YunOszLSAyQYPVC2YdGGeHD2dTdJk1pAHGAWDjnkcLKFymS3RQZTInzySoBwMG0QueC3gMsCEYxUqlrcxK6k1LQQcsmyYeQPdC2YfuGPASCBkcVMQQqpVJshui1tkXQJQV0OXGAZMXSOEEBRirXbVRQW7ugq7IM7rPWSZyDlM3IuNEkxzCOJ0ny2ThNkyRai1b6ev//3dzNGzNb//4uAvHT5sURcZCFcuKLhOFs8mLAAEAt4UWAAIABAAAAAB4qbHo0tIjVkUU//uQZAwABfSFz3ZqQAAAAAngwAAAE1HjMp2qAAAAACZDgAAAD5UkTE1UgZEUExqYynN1qZvqIOREEFmBcJQkwdxiFtw0qEOkGYfRDifBui9MQg4QAHAqWtAWHoCxu1Yf4VfWLPIM2mHDFsbQEVGwyqQoQcwnfHeIkNt9YnkiaS1oizycqJrx4KOQjahZxWbcZgztj2c49nKmkId44S71j0c8eV9yDK6uPRzx5X18eDvjvQ6yKo9ZSS6l//8elePK/Lf//IInrOF/FvDoADYAGBMGb7FtErm5MXMlmPAJQVgWta7Zx2go+8xJ0UiCb8LHHdftWyLJE0QIAIsI+UbXu67dZMjmgDGCGl1H+vpF4NSDckSIkk7Vd+sxEhBQMRU8j/12UIRhzSaUdQ+rQU5kGeFxm+hb1oh6pWWmv3uvmReDl0UnvtapVaIzo1jZbf/pD6ElLqSX+rUmOQNpJFa/r+sa4e/pBlAABoAAAAA3CUgShLdGIxsY7AUABPRrgCABdDuQ5GC7DqPQCgbbJUAoRSUj+NIEig0YfyWUho1VBBBA//uQZB4ABZx5zfMakeAAAAmwAAAAF5F3P0w9GtAAACfAAAAAwLhMDmAYWMgVEG1U0FIGCBgXBXAtfMH10000EEEEEECUBYln03TTTdNBDZopopYvrTTdNa325mImNg3TTPV9q3pmY0xoO6bv3r00y+IDGid/9aaaZTGMuj9mpu9Mpio1dXrr5HERTZSmqU36A3CumzN/9Robv/Xx4v9ijkSRSNLQhAWumap82WRSBUqXStV/YcS+XVLnSS+WLDroqArFkMEsAS+eWmrUzrO0oEmE40RlMZ5+ODIkAyKAGUwZ3mVKmcamcJnMW26MRPgUw6j+LkhyHGVGYjSUUKNpuJUQoOIAyDvEyG8S5yfK6dhZc0Tx1KI/gviKL6qvvFs1+bWtaz58uUNnryq6kt5RzOCkPWlVqVX2a/EEBUdU1KrXLf40GoiiFXK///qpoiDXrOgqDR38JB0bw7SoL+ZB9o1RCkQjQ2CBYZKd/+VJxZRRZlqSkKiws0WFxUyCwsKiMy7hUVFhIaCrNQsKkTIsLivwKKigsj8XYlwt/WKi2N4d//uQRCSAAjURNIHpMZBGYiaQPSYyAAABLAAAAAAAACWAAAAApUF/Mg+0aohSIRobBAsMlO//Kk4soosy1JSFRYWaLC4qZBYWFRGZdwqKiwkNBVmoWFSJkWFxX4FFRQWR+LsS4W/rFRb/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////VEFHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAU291bmRib3kuZGUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMjAwNGh0dHA6Ly93d3cuc291bmRib3kuZGUAAAAAAAAAACU=");
+        void snd.play();
+    }
+}
+
+export default SoundUtils;

examples/instant-voice/client/javascript/tsconfig.json

@@ -0,0 +1,111 @@
+{
+  "compilerOptions": {
+    /* Visit https://aka.ms/tsconfig to read more about this file */
+
+    /* Projects */
+    // "incremental": true,                              /* Save .tsbuildinfo files to allow for incremental compilation of projects. */
+    // "composite": true,                                /* Enable constraints that allow a TypeScript project to be used with project references. */
+    // "tsBuildInfoFile": "./.tsbuildinfo",              /* Specify the path to .tsbuildinfo incremental compilation file. */
+    // "disableSourceOfProjectReferenceRedirect": true,  /* Disable preferring source files instead of declaration files when referencing composite projects. */
+    // "disableSolutionSearching": true,                 /* Opt a project out of multi-project reference checking when editing. */
+    // "disableReferencedProjectLoad": true,             /* Reduce the number of projects loaded automatically by TypeScript. */
+
+    /* Language and Environment */
+    "target": "es2016",                                  /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */
+    // "lib": [],                                        /* Specify a set of bundled library declaration files that describe the target runtime environment. */
+    // "jsx": "preserve",                                /* Specify what JSX code is generated. */
+    // "experimentalDecorators": true,                   /* Enable experimental support for legacy experimental decorators. */
+    // "emitDecoratorMetadata": true,                    /* Emit design-type metadata for decorated declarations in source files. */
+    // "jsxFactory": "",                                 /* Specify the JSX factory function used when targeting React JSX emit, e.g. 'React.createElement' or 'h'. */
+    // "jsxFragmentFactory": "",                         /* Specify the JSX Fragment reference used for fragments when targeting React JSX emit e.g. 'React.Fragment' or 'Fragment'. */
+    // "jsxImportSource": "",                            /* Specify module specifier used to import the JSX factory functions when using 'jsx: react-jsx*'. */
+    // "reactNamespace": "",                             /* Specify the object invoked for 'createElement'. This only applies when targeting 'react' JSX emit. */
+    // "noLib": true,                                    /* Disable including any library files, including the default lib.d.ts. */
+    // "useDefineForClassFields": true,                  /* Emit ECMAScript-standard-compliant class fields. */
+    // "moduleDetection": "auto",                        /* Control what method is used to detect module-format JS files. */
+
+    /* Modules */
+    "module": "commonjs",                                /* Specify what module code is generated. */
+    // "rootDir": "./",                                  /* Specify the root folder within your source files. */
+    // "moduleResolution": "node10",                     /* Specify how TypeScript looks up a file from a given module specifier. */
+    // "baseUrl": "./",                                  /* Specify the base directory to resolve non-relative module names. */
+    // "paths": {},                                      /* Specify a set of entries that re-map imports to additional lookup locations. */
+    // "rootDirs": [],                                   /* Allow multiple folders to be treated as one when resolving modules. */
+    // "typeRoots": [],                                  /* Specify multiple folders that act like './node_modules/@types'. */
+    // "types": [],                                      /* Specify type package names to be included without being referenced in a source file. */
+    // "allowUmdGlobalAccess": true,                     /* Allow accessing UMD globals from modules. */
+    // "moduleSuffixes": [],                             /* List of file name suffixes to search when resolving a module. */
+    // "allowImportingTsExtensions": true,               /* Allow imports to include TypeScript file extensions. Requires '--moduleResolution bundler' and either '--noEmit' or '--emitDeclarationOnly' to be set. */
+    // "rewriteRelativeImportExtensions": true,          /* Rewrite '.ts', '.tsx', '.mts', and '.cts' file extensions in relative import paths to their JavaScript equivalent in output files. */
+    // "resolvePackageJsonExports": true,                /* Use the package.json 'exports' field when resolving package imports. */
+    // "resolvePackageJsonImports": true,                /* Use the package.json 'imports' field when resolving imports. */
+    // "customConditions": [],                           /* Conditions to set in addition to the resolver-specific defaults when resolving imports. */
+    // "noUncheckedSideEffectImports": true,             /* Check side effect imports. */
+    // "resolveJsonModule": true,                        /* Enable importing .json files. */
+    // "allowArbitraryExtensions": true,                 /* Enable importing files with any extension, provided a declaration file is present. */
+    // "noResolve": true,                                /* Disallow 'import's, 'require's or '<reference>'s from expanding the number of files TypeScript should add to a project. */
+
+    /* JavaScript Support */
+    // "allowJs": true,                                  /* Allow JavaScript files to be a part of your program. Use the 'checkJS' option to get errors from these files. */
+    // "checkJs": true,                                  /* Enable error reporting in type-checked JavaScript files. */
+    // "maxNodeModuleJsDepth": 1,                        /* Specify the maximum folder depth used for checking JavaScript files from 'node_modules'. Only applicable with 'allowJs'. */
+
+    /* Emit */
+    // "declaration": true,                              /* Generate .d.ts files from TypeScript and JavaScript files in your project. */
+    // "declarationMap": true,                           /* Create sourcemaps for d.ts files. */
+    // "emitDeclarationOnly": true,                      /* Only output d.ts files and not JavaScript files. */
+    // "sourceMap": true,                                /* Create source map files for emitted JavaScript files. */
+    // "inlineSourceMap": true,                          /* Include sourcemap files inside the emitted JavaScript. */
+    // "noEmit": true,                                   /* Disable emitting files from a compilation. */
+    // "outFile": "./",                                  /* Specify a file that bundles all outputs into one JavaScript file. If 'declaration' is true, also designates a file that bundles all .d.ts output. */
+    // "outDir": "./",                                   /* Specify an output folder for all emitted files. */
+    // "removeComments": true,                           /* Disable emitting comments. */
+    // "importHelpers": true,                            /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
+    // "downlevelIteration": true,                       /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
+    // "sourceRoot": "",                                 /* Specify the root path for debuggers to find the reference source code. */
+    // "mapRoot": "",                                    /* Specify the location where debugger should locate map files instead of generated locations. */
+    // "inlineSources": true,                            /* Include source code in the sourcemaps inside the emitted JavaScript. */
+    // "emitBOM": true,                                  /* Emit a UTF-8 Byte Order Mark (BOM) in the beginning of output files. */
+    // "newLine": "crlf",                                /* Set the newline character for emitting files. */
+    // "stripInternal": true,                            /* Disable emitting declarations that have '@internal' in their JSDoc comments. */
+    // "noEmitHelpers": true,                            /* Disable generating custom helper functions like '__extends' in compiled output. */
+    // "noEmitOnError": true,                            /* Disable emitting files if any type checking errors are reported. */
+    // "preserveConstEnums": true,                       /* Disable erasing 'const enum' declarations in generated code. */
+    // "declarationDir": "./",                           /* Specify the output directory for generated declaration files. */
+
+    /* Interop Constraints */
+    // "isolatedModules": true,                          /* Ensure that each file can be safely transpiled without relying on other imports. */
+    // "verbatimModuleSyntax": true,                     /* Do not transform or elide any imports or exports not marked as type-only, ensuring they are written in the output file's format based on the 'module' setting. */
+    // "isolatedDeclarations": true,                     /* Require sufficient annotation on exports so other tools can trivially generate declaration files. */
+    // "allowSyntheticDefaultImports": true,             /* Allow 'import x from y' when a module doesn't have a default export. */
+    "esModuleInterop": true,                             /* Emit additional JavaScript to ease support for importing CommonJS modules. This enables 'allowSyntheticDefaultImports' for type compatibility. */
+    // "preserveSymlinks": true,                         /* Disable resolving symlinks to their realpath. This correlates to the same flag in node. */
+    "forceConsistentCasingInFileNames": true,            /* Ensure that casing is correct in imports. */
+
+    /* Type Checking */
+    "strict": true,                                      /* Enable all strict type-checking options. */
+    // "noImplicitAny": true,                            /* Enable error reporting for expressions and declarations with an implied 'any' type. */
+    // "strictNullChecks": true,                         /* When type checking, take into account 'null' and 'undefined'. */
+    // "strictFunctionTypes": true,                      /* When assigning functions, check to ensure parameters and the return values are subtype-compatible. */
+    // "strictBindCallApply": true,                      /* Check that the arguments for 'bind', 'call', and 'apply' methods match the original function. */
+    // "strictPropertyInitialization": true,             /* Check for class properties that are declared but not set in the constructor. */
+    // "strictBuiltinIteratorReturn": true,              /* Built-in iterators are instantiated with a 'TReturn' type of 'undefined' instead of 'any'. */
+    // "noImplicitThis": true,                           /* Enable error reporting when 'this' is given the type 'any'. */
+    // "useUnknownInCatchVariables": true,               /* Default catch clause variables as 'unknown' instead of 'any'. */
+    // "alwaysStrict": true,                             /* Ensure 'use strict' is always emitted. */
+    // "noUnusedLocals": true,                           /* Enable error reporting when local variables aren't read. */
+    // "noUnusedParameters": true,                       /* Raise an error when a function parameter isn't read. */
+    // "exactOptionalPropertyTypes": true,               /* Interpret optional property types as written, rather than adding 'undefined'. */
+    // "noImplicitReturns": true,                        /* Enable error reporting for codepaths that do not explicitly return in a function. */
+    // "noFallthroughCasesInSwitch": true,               /* Enable error reporting for fallthrough cases in switch statements. */
+    // "noUncheckedIndexedAccess": true,                 /* Add 'undefined' to a type when accessed using an index. */
+    // "noImplicitOverride": true,                       /* Ensure overriding members in derived classes are marked with an override modifier. */
+    // "noPropertyAccessFromIndexSignature": true,       /* Enforces using indexed accessors for keys declared using an indexed type. */
+    // "allowUnusedLabels": true,                        /* Disable error reporting for unused labels. */
+    // "allowUnreachableCode": true,                     /* Disable error reporting for unreachable code. */
+
+    /* Completeness */
+    // "skipDefaultLibCheck": true,                      /* Skip type checking .d.ts files that are included with TypeScript. */
+    "skipLibCheck": true                                 /* Skip type checking all .d.ts files. */
+  }
+}

examples/instant-voice/client/javascript/vite.config.js

@@ -0,0 +1,15 @@
+import { defineConfig } from 'vite';
+import react from '@vitejs/plugin-react-swc';
+
+export default defineConfig({
+    plugins: [react()],
+    server: {
+        proxy: {
+            // Proxy /api requests to the backend server
+            '/connect': {
+                target: 'http://0.0.0.0:7860', // Replace with your backend URL
+                changeOrigin: true,
+            },
+        },
+    },
+});

examples/instant-voice/client/javascript/yarn.lock

@@ -0,0 +1,339 @@
+# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
+# yarn lockfile v1
+
+
+"@babel/runtime@^7.12.5":
+  version "7.26.0"
+  resolved "https://registry.npmjs.org/@babel/runtime/-/runtime-7.26.0.tgz"
+  integrity sha512-FDSOghenHTiToteC/QRlv2q3DhPZ/oOXTBoirfWNx1Cx3TMVcGWQtMMmQcSvb/JjpNeGzx8Pq/b4fKEJuWm1sw==
+  dependencies:
+    regenerator-runtime "^0.14.0"
+
+"@daily-co/daily-js@^0.73.0":
+  version "0.73.0"
+  resolved "https://registry.npmjs.org/@daily-co/daily-js/-/daily-js-0.73.0.tgz"
+  integrity sha512-Wz8c60hgmkx8fcEeDAi4L4J0rbafiihWKyXFyhYoFYPsw2OdChHpA4RYwIB+1enRws5IK+/HdmzFDYLQsB4A6w==
+  dependencies:
+    "@babel/runtime" "^7.12.5"
+    "@sentry/browser" "^8.33.1"
+    bowser "^2.8.1"
+    dequal "^2.0.3"
+    events "^3.1.0"
+
+"@esbuild/darwin-arm64@0.24.0":
+  version "0.24.0"
+  resolved "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.0.tgz"
+  integrity sha512-CKyDpRbK1hXwv79soeTJNHb5EiG6ct3efd/FTPdzOWdbZZfGhpbcqIpiD0+vwmpu0wTIL97ZRPZu8vUt46nBSw==
+
+"@pipecat-ai/client-js@^0.3.2", "@pipecat-ai/client-js@~0.3.2":
+  version "0.3.2"
+  resolved "https://registry.npmjs.org/@pipecat-ai/client-js/-/client-js-0.3.2.tgz"
+  integrity sha512-psunOVrJjPka2SWlq53vxVWCA0Vt8pSXsXtn8pOLC0YTKFsUx+b7Z6quYUJcDZjCe1aAg9cKETek3Xal3Co8Tg==
+  dependencies:
+    "@types/events" "^3.0.3"
+    clone-deep "^4.0.1"
+    events "^3.3.0"
+    typed-emitter "^2.1.0"
+    uuid "^10.0.0"
+
+"@pipecat-ai/daily-transport@^0.3.5":
+  version "0.3.5"
+  resolved "https://registry.npmjs.org/@pipecat-ai/daily-transport/-/daily-transport-0.3.5.tgz"
+  integrity sha512-nJ0TvWPCqXPmU81U8cXOqk5mUEEvEuI06Mis+N0jN8KZUrNy1pP08iWbs07ObmIXdnQcoL+kQmHOerT4q/bF0w==
+  dependencies:
+    "@daily-co/daily-js" "^0.73.0"
+
+"@rollup/rollup-darwin-arm64@4.28.0":
+  version "4.28.0"
+  resolved "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.28.0.tgz"
+  integrity sha512-lmKx9yHsppblnLQZOGxdO66gT77bvdBtr/0P+TPOseowE7D9AJoBw8ZDULRasXRWf1Z86/gcOdpBrV6VDUY36Q==
+
+"@sentry-internal/browser-utils@8.49.0":
+  version "8.49.0"
+  resolved "https://registry.npmjs.org/@sentry-internal/browser-utils/-/browser-utils-8.49.0.tgz"
+  integrity sha512-XkPHHdFqsN7EPaB+QGUOEmpFqXiqP67t2rRZ1HG1UwJoe0PhJEKNy7b4+WRwmT7ODSt+PvFk1gNBlJBpThwH7Q==
+  dependencies:
+    "@sentry/core" "8.49.0"
+
+"@sentry-internal/feedback@8.49.0":
+  version "8.49.0"
+  resolved "https://registry.npmjs.org/@sentry-internal/feedback/-/feedback-8.49.0.tgz"
+  integrity sha512-v/wf7WvPxEvZUB7xrCnecI3fhevVo84hw8WlxgZIz6mLUHXEIX8xYWc9H8Yet/KKJ2uEB8GQ8aDsY6S1hVEIUA==
+  dependencies:
+    "@sentry/core" "8.49.0"
+
+"@sentry-internal/replay-canvas@8.49.0":
+  version "8.49.0"
+  resolved "https://registry.npmjs.org/@sentry-internal/replay-canvas/-/replay-canvas-8.49.0.tgz"
+  integrity sha512-/yXxI7f+Wu24FIYoRE7A0AidNxORuhAyPzb5ey1wFqMXP72nG8dXhOpcl0w+bi554FkqkLjdeUDhSOBWYZXH9g==
+  dependencies:
+    "@sentry-internal/replay" "8.49.0"
+    "@sentry/core" "8.49.0"
+
+"@sentry-internal/replay@8.49.0":
+  version "8.49.0"
+  resolved "https://registry.npmjs.org/@sentry-internal/replay/-/replay-8.49.0.tgz"
+  integrity sha512-BDiiCBxskkktTd6FNplBc9V8l14R4T/AwRIZj2itX4xnuHewTTDjVbeyvGol4roA4r+V0Mzoi31hLEGI6yFQ5Q==
+  dependencies:
+    "@sentry-internal/browser-utils" "8.49.0"
+    "@sentry/core" "8.49.0"
+
+"@sentry/browser@^8.33.1":
+  version "8.49.0"
+  resolved "https://registry.npmjs.org/@sentry/browser/-/browser-8.49.0.tgz"
+  integrity sha512-dS4Sw2h8EixHeXOIR++XEVMTen6xCGcIQ/XhJbsjqvddXeIijW0WkxSeTfPkfs0dsqFHSisWmlmo0xhHbXvEsQ==
+  dependencies:
+    "@sentry-internal/browser-utils" "8.49.0"
+    "@sentry-internal/feedback" "8.49.0"
+    "@sentry-internal/replay" "8.49.0"
+    "@sentry-internal/replay-canvas" "8.49.0"
+    "@sentry/core" "8.49.0"
+
+"@sentry/core@8.49.0":
+  version "8.49.0"
+  resolved "https://registry.npmjs.org/@sentry/core/-/core-8.49.0.tgz"
+  integrity sha512-/OAm6LdHhh8TvfDAucWfSJV7M03IOHrJm5LVjrrKr4gwQ1HKd4CDbARsBbPwHIzSRAle0IgG3sbJxEvv52JUIw==
+
+"@swc/core-darwin-arm64@1.10.14":
+  version "1.10.14"
+  resolved "https://registry.npmjs.org/@swc/core-darwin-arm64/-/core-darwin-arm64-1.10.14.tgz"
+  integrity sha512-Dh4VyrhDDb05tdRmqJ/MucOPMTnrB4pRJol18HVyLlqu1HOT5EzonUniNTCdQbUXjgdv5UVJSTE1lYTzrp+myA==
+
+"@swc/core@^1.7.26":
+  version "1.10.14"
+  resolved "https://registry.npmjs.org/@swc/core/-/core-1.10.14.tgz"
+  integrity sha512-WSrnE6JRnH20ZYjOOgSS4aOaPv9gxlkI2KRkN24kagbZnPZMnN8bZZyzw1rrLvwgpuRGv17Uz+hflosbR+SP6w==
+  dependencies:
+    "@swc/counter" "^0.1.3"
+    "@swc/types" "^0.1.17"
+  optionalDependencies:
+    "@swc/core-darwin-arm64" "1.10.14"
+    "@swc/core-darwin-x64" "1.10.14"
+    "@swc/core-linux-arm-gnueabihf" "1.10.14"
+    "@swc/core-linux-arm64-gnu" "1.10.14"
+    "@swc/core-linux-arm64-musl" "1.10.14"
+    "@swc/core-linux-x64-gnu" "1.10.14"
+    "@swc/core-linux-x64-musl" "1.10.14"
+    "@swc/core-win32-arm64-msvc" "1.10.14"
+    "@swc/core-win32-ia32-msvc" "1.10.14"
+    "@swc/core-win32-x64-msvc" "1.10.14"
+
+"@swc/counter@^0.1.3":
+  version "0.1.3"
+  resolved "https://registry.npmjs.org/@swc/counter/-/counter-0.1.3.tgz"
+  integrity sha512-e2BR4lsJkkRlKZ/qCHPw9ZaSxc0MVUd7gtbtaB7aMvHeJVYe8sOB8DBZkP2DtISHGSku9sCK6T6cnY0CtXrOCQ==
+
+"@swc/types@^0.1.17":
+  version "0.1.17"
+  resolved "https://registry.npmjs.org/@swc/types/-/types-0.1.17.tgz"
+  integrity sha512-V5gRru+aD8YVyCOMAjMpWR1Ui577DD5KSJsHP8RAxopAH22jFz6GZd/qxqjO6MJHQhcsjvjOFXyDhyLQUnMveQ==
+  dependencies:
+    "@swc/counter" "^0.1.3"
+
+"@types/estree@1.0.6":
+  version "1.0.6"
+  resolved "https://registry.npmjs.org/@types/estree/-/estree-1.0.6.tgz"
+  integrity sha512-AYnb1nQyY49te+VRAVgmzfcgjYS91mY5P0TKUDCLEM+gNnA+3T6rWITXRLYCpahpqSQbN5cE+gHpnPyXjHWxcw==
+
+"@types/events@^3.0.3":
+  version "3.0.3"
+  resolved "https://registry.npmjs.org/@types/events/-/events-3.0.3.tgz"
+  integrity sha512-trOc4AAUThEz9hapPtSd7wf5tiQKvTtu5b371UxXdTuqzIh0ArcRspRP0i0Viu+LXstIQ1z96t1nsPxT9ol01g==
+
+"@types/node@^18.0.0 || ^20.0.0 || >=22.0.0", "@types/node@^22.13.1":
+  version "22.13.1"
+  resolved "https://registry.npmjs.org/@types/node/-/node-22.13.1.tgz"
+  integrity sha512-jK8uzQlrvXqEU91UxiK5J7pKHyzgnI1Qnl0QDHIgVGuolJhRb9EEl28Cj9b3rGR8B2lhFCtvIm5os8lFnO/1Ew==
+  dependencies:
+    undici-types "~6.20.0"
+
+"@vitejs/plugin-react-swc@^3.7.2":
+  version "3.7.2"
+  resolved "https://registry.npmjs.org/@vitejs/plugin-react-swc/-/plugin-react-swc-3.7.2.tgz"
+  integrity sha512-y0byko2b2tSVVf5Gpng1eEhX1OvPC7x8yns1Fx8jDzlJp4LS6CMkCPfLw47cjyoMrshQDoQw4qcgjsU9VvlCew==
+  dependencies:
+    "@swc/core" "^1.7.26"
+
+bowser@^2.8.1:
+  version "2.11.0"
+  resolved "https://registry.npmjs.org/bowser/-/bowser-2.11.0.tgz"
+  integrity sha512-AlcaJBi/pqqJBIQ8U9Mcpc9i8Aqxn88Skv5d+xBX006BY5u8N3mGLHa5Lgppa7L/HfwgwLgZ6NYs+Ag6uUmJRA==
+
+clone-deep@^4.0.1:
+  version "4.0.1"
+  resolved "https://registry.npmjs.org/clone-deep/-/clone-deep-4.0.1.tgz"
+  integrity sha512-neHB9xuzh/wk0dIHweyAXv2aPGZIVk3pLMe+/RNzINf17fe0OG96QroktYAUm7SM1PBnzTabaLboqqxDyMU+SQ==
+  dependencies:
+    is-plain-object "^2.0.4"
+    kind-of "^6.0.2"
+    shallow-clone "^3.0.0"
+
+dequal@^2.0.3:
+  version "2.0.3"
+  resolved "https://registry.npmjs.org/dequal/-/dequal-2.0.3.tgz"
+  integrity sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==
+
+esbuild@^0.24.0:
+  version "0.24.0"
+  resolved "https://registry.npmjs.org/esbuild/-/esbuild-0.24.0.tgz"
+  integrity sha512-FuLPevChGDshgSicjisSooU0cemp/sGXR841D5LHMB7mTVOmsEHcAxaH3irL53+8YDIeVNQEySh4DaYU/iuPqQ==
+  optionalDependencies:
+    "@esbuild/aix-ppc64" "0.24.0"
+    "@esbuild/android-arm" "0.24.0"
+    "@esbuild/android-arm64" "0.24.0"
+    "@esbuild/android-x64" "0.24.0"
+    "@esbuild/darwin-arm64" "0.24.0"
+    "@esbuild/darwin-x64" "0.24.0"
+    "@esbuild/freebsd-arm64" "0.24.0"
+    "@esbuild/freebsd-x64" "0.24.0"
+    "@esbuild/linux-arm" "0.24.0"
+    "@esbuild/linux-arm64" "0.24.0"
+    "@esbuild/linux-ia32" "0.24.0"
+    "@esbuild/linux-loong64" "0.24.0"
+    "@esbuild/linux-mips64el" "0.24.0"
+    "@esbuild/linux-ppc64" "0.24.0"
+    "@esbuild/linux-riscv64" "0.24.0"
+    "@esbuild/linux-s390x" "0.24.0"
+    "@esbuild/linux-x64" "0.24.0"
+    "@esbuild/netbsd-x64" "0.24.0"
+    "@esbuild/openbsd-arm64" "0.24.0"
+    "@esbuild/openbsd-x64" "0.24.0"
+    "@esbuild/sunos-x64" "0.24.0"
+    "@esbuild/win32-arm64" "0.24.0"
+    "@esbuild/win32-ia32" "0.24.0"
+    "@esbuild/win32-x64" "0.24.0"
+
+events@^3.1.0, events@^3.3.0:
+  version "3.3.0"
+  resolved "https://registry.npmjs.org/events/-/events-3.3.0.tgz"
+  integrity sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==
+
+fsevents@~2.3.2, fsevents@~2.3.3:
+  version "2.3.3"
+  resolved "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz"
+  integrity sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==
+
+is-plain-object@^2.0.4:
+  version "2.0.4"
+  resolved "https://registry.npmjs.org/is-plain-object/-/is-plain-object-2.0.4.tgz"
+  integrity sha512-h5PpgXkWitc38BBMYawTYMWJHFZJVnBquFE57xFpjB8pJFiF6gZ+bU+WyI/yqXiFR5mdLsgYNaPe8uao6Uv9Og==
+  dependencies:
+    isobject "^3.0.1"
+
+isobject@^3.0.1:
+  version "3.0.1"
+  resolved "https://registry.npmjs.org/isobject/-/isobject-3.0.1.tgz"
+  integrity sha512-WhB9zCku7EGTj/HQQRz5aUQEUeoQZH2bWcltRErOpymJ4boYE6wL9Tbr23krRPSZ+C5zqNSrSw+Cc7sZZ4b7vg==
+
+kind-of@^6.0.2:
+  version "6.0.3"
+  resolved "https://registry.npmjs.org/kind-of/-/kind-of-6.0.3.tgz"
+  integrity sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==
+
+nanoid@^3.3.7:
+  version "3.3.8"
+  resolved "https://registry.npmjs.org/nanoid/-/nanoid-3.3.8.tgz"
+  integrity sha512-WNLf5Sd8oZxOm+TzppcYk8gVOgP+l58xNy58D0nbUnOxOWRWvlcCV4kUF7ltmI6PsrLl/BgKEyS4mqsGChFN0w==
+
+picocolors@^1.1.1:
+  version "1.1.1"
+  resolved "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz"
+  integrity sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==
+
+postcss@^8.4.49:
+  version "8.4.49"
+  resolved "https://registry.npmjs.org/postcss/-/postcss-8.4.49.tgz"
+  integrity sha512-OCVPnIObs4N29kxTjzLfUryOkvZEq+pf8jTF0lg8E7uETuWHA+v7j3c/xJmiqpX450191LlmZfUKkXxkTry7nA==
+  dependencies:
+    nanoid "^3.3.7"
+    picocolors "^1.1.1"
+    source-map-js "^1.2.1"
+
+regenerator-runtime@^0.14.0:
+  version "0.14.1"
+  resolved "https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.14.1.tgz"
+  integrity sha512-dYnhHh0nJoMfnkZs6GmmhFknAGRrLznOu5nc9ML+EJxGvrx6H7teuevqVqCuPcPK//3eDrrjQhehXVx9cnkGdw==
+
+rollup@^4.23.0:
+  version "4.28.0"
+  resolved "https://registry.npmjs.org/rollup/-/rollup-4.28.0.tgz"
+  integrity sha512-G9GOrmgWHBma4YfCcX8PjH0qhXSdH8B4HDE2o4/jaxj93S4DPCIDoLcXz99eWMji4hB29UFCEd7B2gwGJDR9cQ==
+  dependencies:
+    "@types/estree" "1.0.6"
+  optionalDependencies:
+    "@rollup/rollup-android-arm-eabi" "4.28.0"
+    "@rollup/rollup-android-arm64" "4.28.0"
+    "@rollup/rollup-darwin-arm64" "4.28.0"
+    "@rollup/rollup-darwin-x64" "4.28.0"
+    "@rollup/rollup-freebsd-arm64" "4.28.0"
+    "@rollup/rollup-freebsd-x64" "4.28.0"
+    "@rollup/rollup-linux-arm-gnueabihf" "4.28.0"
+    "@rollup/rollup-linux-arm-musleabihf" "4.28.0"
+    "@rollup/rollup-linux-arm64-gnu" "4.28.0"
+    "@rollup/rollup-linux-arm64-musl" "4.28.0"
+    "@rollup/rollup-linux-powerpc64le-gnu" "4.28.0"
+    "@rollup/rollup-linux-riscv64-gnu" "4.28.0"
+    "@rollup/rollup-linux-s390x-gnu" "4.28.0"
+    "@rollup/rollup-linux-x64-gnu" "4.28.0"
+    "@rollup/rollup-linux-x64-musl" "4.28.0"
+    "@rollup/rollup-win32-arm64-msvc" "4.28.0"
+    "@rollup/rollup-win32-ia32-msvc" "4.28.0"
+    "@rollup/rollup-win32-x64-msvc" "4.28.0"
+    fsevents "~2.3.2"
+
+rxjs@*:
+  version "7.8.1"
+  resolved "https://registry.npmjs.org/rxjs/-/rxjs-7.8.1.tgz"
+  integrity sha512-AA3TVj+0A2iuIoQkWEK/tqFjBq2j+6PO6Y0zJcvzLAFhEFIO3HL0vls9hWLncZbAAbK0mar7oZ4V079I/qPMxg==
+  dependencies:
+    tslib "^2.1.0"
+
+shallow-clone@^3.0.0:
+  version "3.0.1"
+  resolved "https://registry.npmjs.org/shallow-clone/-/shallow-clone-3.0.1.tgz"
+  integrity sha512-/6KqX+GVUdqPuPPd2LxDDxzX6CAbjJehAAOKlNpqqUpAqPM6HeL8f+o3a+JsyGjn2lv0WY8UsTgUJjU9Ok55NA==
+  dependencies:
+    kind-of "^6.0.2"
+
+source-map-js@^1.2.1:
+  version "1.2.1"
+  resolved "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz"
+  integrity sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==
+
+tslib@^2.1.0:
+  version "2.8.1"
+  resolved "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz"
+  integrity sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==
+
+typed-emitter@^2.1.0:
+  version "2.1.0"
+  resolved "https://registry.npmjs.org/typed-emitter/-/typed-emitter-2.1.0.tgz"
+  integrity sha512-g/KzbYKbH5C2vPkaXGu8DJlHrGKHLsM25Zg9WuC9pMGfuvT+X25tZQWo5fK1BjBm8+UrVE9LDCvaY0CQk+fXDA==
+  optionalDependencies:
+    rxjs "*"
+
+typescript@^5.7.3:
+  version "5.7.3"
+  resolved "https://registry.npmjs.org/typescript/-/typescript-5.7.3.tgz"
+  integrity sha512-84MVSjMEHP+FQRPy3pX9sTVV/INIex71s9TL2Gm5FG/WG1SqXeKyZ0k7/blY/4FdOzI12CBy1vGc4og/eus0fw==
+
+undici-types@~6.20.0:
+  version "6.20.0"
+  resolved "https://registry.npmjs.org/undici-types/-/undici-types-6.20.0.tgz"
+  integrity sha512-Ny6QZ2Nju20vw1SRHe3d9jVu6gJ+4e3+MMpqu7pqE5HT6WsTSlce++GQmK5UXS8mzV8DSYHrQH+Xrf2jVcuKNg==
+
+uuid@^10.0.0:
+  version "10.0.0"
+  resolved "https://registry.npmjs.org/uuid/-/uuid-10.0.0.tgz"
+  integrity sha512-8XkAphELsDnEGrDxUOHB3RGvXz6TeuYSGEZBOjtTtPm2lwhGBjLgOzLHB63IUWfBpNucQjND6d3AOudO+H3RWQ==
+
+"vite@^4 || ^5 || ^6", vite@^6.0.2:
+  version "6.0.2"
+  resolved "https://registry.npmjs.org/vite/-/vite-6.0.2.tgz"
+  integrity sha512-XdQ+VsY2tJpBsKGs0wf3U/+azx8BBpYRHFAyKm5VeEZNOJZRB63q7Sc8Iup3k0TrN3KO6QgyzFf+opSbfY1y0g==
+  dependencies:
+    esbuild "^0.24.0"
+    postcss "^8.4.49"
+    rollup "^4.23.0"
+  optionalDependencies:
+    fsevents "~2.3.3"

examples/instant-voice/server/env.example

@@ -0,0 +1,3 @@
+DAILY_API_KEY=
+GOOGLE_API_KEY=
+DAILY_SAMPLE_ROOM_URL=https://yourdomain.daily.co/yourroom # (for joining the bot to the same room repeatedly for local dev)

examples/instant-voice/server/pyproject.toml

@@ -0,0 +1,11 @@
+[tool.ruff]
+exclude = [".git", "*_pb2.py"]
+line-length = 100
+
+[tool.ruff.lint]
+select = [
+    "I", # Import rules
+]
+
+[tool.ruff.lint.pydocstyle]
+convention = "google"

examples/instant-voice/server/requirements.txt

@@ -0,0 +1,4 @@
+python-dotenv
+fastapi[all]
+uvicorn
+pipecat-ai[openai,silero,websocket,google,daily]

examples/instant-voice/server/src/server.py

@@ -0,0 +1,200 @@
+#
+# Copyright (c) 2025, Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+import asyncio
+import os
+from contextlib import asynccontextmanager
+from typing import Any, Dict, List
+
+import aiohttp
+import uvicorn
+from dotenv import load_dotenv
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.middleware.cors import CORSMiddleware
+from pipecat.transports.services.helpers.daily_rest import DailyRESTHelper, DailyRoomParams
+
+# Load environment variables
+load_dotenv(override=True)
+
+NUMBER_OF_ROOMS = 1
+
+
+class RoomPool:
+    """Manages a pool of pre-created rooms for quick allocation."""
+
+    def __init__(self, daily_rest_helper: DailyRESTHelper):
+        self.daily_rest_helper = daily_rest_helper
+        self.pool: List[Dict[str, str]] = []
+        self.lock = asyncio.Lock()
+
+    async def fill_pool(self, count: int):
+        """Fills the pool with `count` new rooms."""
+        for _ in range(count):
+            await self.add_room()
+
+    async def add_room(self):
+        """Creates a new room and adds it to the pool."""
+        try:
+            room = await self.daily_rest_helper.create_room(DailyRoomParams())
+            if not room.url:
+                raise HTTPException(status_code=500, detail="Failed to create room")
+
+            user_token = await self.daily_rest_helper.get_token(room.url)
+            if not user_token:
+                raise HTTPException(status_code=500, detail="Failed to get user token")
+
+            bot_token = await self.daily_rest_helper.get_token(room.url)
+            if not bot_token:
+                raise HTTPException(status_code=500, detail="Failed to get bot token")
+
+            async with self.lock:
+                self.pool.append(
+                    {"room_url": room.url, "user_token": user_token, "bot_token": bot_token}
+                )
+
+        except Exception as e:
+            print(f"Error adding room to pool: {e}")
+
+    async def get_room(self) -> Dict[str, str]:
+        """Retrieves a room from the pool and requests a new one to maintain the size."""
+        async with self.lock:
+            if not self.pool:
+                raise HTTPException(status_code=503, detail="No available rooms")
+
+            room = self.pool.pop(0)  # Get first available room
+
+        # Start a background task to replenish the pool
+        asyncio.create_task(self.add_room())
+
+        return room
+
+    async def delete_room(self, room_url: str):
+        """Deletes a room when it is not needed anymore"""
+        await self.daily_rest_helper.delete_room_by_url(room_url)
+
+    async def cleanup(self):
+        for rooms in self.pool:
+            room_url = rooms["room_url"]
+            await self.delete_room(room_url)
+
+
+class BotManager:
+    """Manages bot subprocesses asynchronously."""
+
+    def __init__(self):
+        self.bot_procs: Dict[int, asyncio.subprocess.Process] = {}
+        self.room_mappings: Dict[int, str] = {}  # Maps process ID to room URL
+
+    async def start_bot(self, room_url: str, token: str) -> int:
+        bot_file = "single_bot"
+        command = f"python3 -m {bot_file} -u {room_url} -t {token}"
+
+        try:
+            proc = await asyncio.create_subprocess_shell(
+                command,
+                cwd=os.path.dirname(os.path.abspath(__file__)),
+            )
+            if proc.pid is None:
+                raise HTTPException(status_code=500, detail="Failed to get subprocess PID")
+
+            self.bot_procs[proc.pid] = proc
+            self.room_mappings[proc.pid] = room_url
+            # Monitor the process and delete the room when it exits
+            asyncio.create_task(self._monitor_process(proc.pid))
+
+            return proc.pid
+        except Exception as e:
+            raise HTTPException(status_code=500, detail=f"Failed to start subprocess: {e}")
+
+    async def _monitor_process(self, pid: int):
+        """Monitors a bot process and deletes the associated room when it exits."""
+        proc = self.bot_procs.get(pid)
+        if proc:
+            await proc.wait()  # Wait for the process to exit
+            room_url = self.room_mappings.pop(pid, None)
+
+            if room_url:
+                await room_pool.delete_room(room_url)
+                print(f"Deleted room: {room_url}")
+
+            del self.bot_procs[pid]
+
+    async def cleanup(self):
+        """Terminates all running bot processes and deletes associated rooms."""
+        for pid, proc in list(self.bot_procs.items()):
+            try:
+                proc.terminate()
+                await asyncio.wait_for(proc.wait(), timeout=5)
+
+                room_url = self.room_mappings.pop(pid, None)
+                if room_url:
+                    await room_pool.delete_room(room_url)  # Delete room when process terminates
+                    print(f"Deleted room: {room_url}")
+
+            except asyncio.TimeoutError:
+                print(f"Process {pid} did not terminate in time.")
+            except Exception as e:
+                print(f"Error terminating process {pid}: {e}")
+
+        # Clear remaining mappings
+        self.bot_procs.clear()
+        self.room_mappings.clear()
+
+
+# Global instances
+bot_manager = BotManager()
+room_pool: RoomPool  # Will be initialized in lifespan
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    """Handles FastAPI startup and shutdown."""
+    global room_pool
+    aiohttp_session = aiohttp.ClientSession()
+    daily_rest_helper = DailyRESTHelper(
+        daily_api_key=os.getenv("DAILY_API_KEY", ""),
+        daily_api_url=os.getenv("DAILY_API_URL", "https://api.daily.co/v1"),
+        aiohttp_session=aiohttp_session,
+    )
+
+    room_pool = RoomPool(daily_rest_helper)
+    await room_pool.fill_pool(NUMBER_OF_ROOMS)  # Fill pool on startup
+
+    yield  # Run app
+
+    await bot_manager.cleanup()
+    await room_pool.cleanup()
+    await aiohttp_session.close()
+
+
+# Initialize FastAPI app with lifespan manager
+app = FastAPI(lifespan=lifespan)
+
+# Configure CORS to allow requests from any origin
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+
+@app.post("/connect")
+async def bot_connect(request: Request) -> Dict[Any, Any]:
+    try:
+        room = await room_pool.get_room()
+        await bot_manager.start_bot(room["room_url"], room["bot_token"])
+    except HTTPException as e:
+        return {"error": str(e)}
+
+    return {
+        "room_url": room["room_url"],
+        "token": room["user_token"],
+    }
+
+
+if __name__ == "__main__":
+    uvicorn.run(app, host="0.0.0.0", port=7860)

examples/instant-voice/server/src/single_bot.py

@@ -0,0 +1,120 @@
+#
+# Copyright (c) 2025, Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+import argparse
+import asyncio
+import os
+import sys
+
+from dotenv import load_dotenv
+from loguru import logger
+from pipecat.audio.vad.silero import SileroVADAnalyzer
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat.pipeline.runner import PipelineRunner
+from pipecat.pipeline.task import PipelineParams, PipelineTask
+from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
+from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIProcessor
+from pipecat.services.gemini_multimodal_live import GeminiMultimodalLiveLLMService
+from pipecat.transports.services.daily import DailyParams, DailyTransport
+
+load_dotenv(override=True)
+
+logger.remove(0)
+logger.add(sys.stderr, level="DEBUG")
+
+SYSTEM_INSTRUCTION = f"""
+"You are Gemini Chatbot, a friendly, helpful robot.
+
+Your goal is to demonstrate your capabilities in a succinct way.
+
+Your output will be converted to audio so don't include special characters in your answers.
+
+Respond to what the user said in a creative and helpful way. Keep your responses brief. One or two sentences at most.
+"""
+
+
+def extract_arguments():
+    parser = argparse.ArgumentParser(description="Instant Voice Example")
+    parser.add_argument(
+        "-u", "--url", type=str, required=True, help="URL of the Daily room to join"
+    )
+    parser.add_argument(
+        "-t", "--token", type=str, required=False, help="Token of the Daily room to join"
+    )
+    args, unknown = parser.parse_known_args()
+    url = args.url or os.getenv("DAILY_SAMPLE_ROOM_URL")
+    token = args.token
+    return url, token
+
+
+async def main():
+    room_url, token = extract_arguments()
+    print(f"room_url: {room_url}")
+
+    daily_transport = DailyTransport(
+        room_url,
+        token,
+        "Instant voice Chatbot",
+        DailyParams(
+            audio_out_enabled=True,
+            vad_enabled=True,
+            vad_analyzer=SileroVADAnalyzer(),
+            vad_audio_passthrough=True,
+        ),
+    )
+
+    llm = GeminiMultimodalLiveLLMService(
+        api_key=os.getenv("GOOGLE_API_KEY"),
+        voice_id="Puck",  # Aoede, Charon, Fenrir, Kore, Puck
+        transcribe_user_audio=True,
+        transcribe_model_audio=True,
+        system_instruction=SYSTEM_INSTRUCTION,
+    )
+
+    context = OpenAILLMContext()
+    context_aggregator = llm.create_context_aggregator(context)
+
+    # RTVI events for Pipecat client UI
+    rtvi = RTVIProcessor(config=RTVIConfig(config=[]), transport=daily_transport)
+
+    pipeline = Pipeline(
+        [
+            daily_transport.input(),
+            context_aggregator.user(),
+            rtvi,
+            llm,  # LLM
+            daily_transport.output(),
+            context_aggregator.assistant(),
+        ]
+    )
+
+    task = PipelineTask(
+        pipeline,
+        params=PipelineParams(
+            allow_interruptions=True,
+            observers=[rtvi.observer()],
+        ),
+    )
+
+    @rtvi.event_handler("on_client_ready")
+    async def on_client_ready(rtvi):
+        await rtvi.set_bot_ready()
+
+    @daily_transport.event_handler("on_first_participant_joined")
+    async def on_first_participant_joined(transport, participant):
+        await task.queue_frames([context_aggregator.user().get_context_frame()])
+
+    @daily_transport.event_handler("on_participant_left")
+    async def on_participant_left(transport, participant, reason):
+        print(f"Participant left: {participant}")
+        await task.cancel()
+
+    runner = PipelineRunner(handle_sigint=False)
+
+    await runner.run(task)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

examples/local-input-select-stt/README.md

@@ -0,0 +1,88 @@
+
+# Pipecat Audio Transcription Example 🚀🎙️
+
+Welcome to the **Pipecat Audio Transcription Example**! 
+
+This project showcases how to integrate the awesome [pipecat](https://github.com/pipecat-ai/pipecat) library with a neat textual interface (powered by [Textual](https://github.com/Textualize/textual)) to select audio devices, perform real-time speech-to-text (STT) transcription using [Whisper](https://github.com/openai/whisper).
+
+> **Note:** Although the script allows you to select both input and output audio devices, this example only utilizes the audio **input** for transcription.
+
+---
+
+## 🎉 Features
+
+- **Interactive Audio Device Selection:**  
+  Choose your preferred audio input device using a cool, textual UI.
+  
+- **State-of-the-Art Transcription:**  
+  Leverage Whisper's large model (running on CUDA) for high-quality, real-time STT.
+  
+- **Live Transcription Logging:**  
+  Watch your spoken words transform into text on your console instantly.
+  
+- **Easy Setup:**  
+  Everything you need is in the [`requirements.txt`](./requirements.txt).
+
+---
+
+## 🎥 Demo
+
+Get a quick glimpse of the app in action!  
+*(Don't worry – I'll be adding a GIF demo here soon!)*
+
+![Demo GIF](demo.gif)
+
+---
+
+## 🔧 Installation
+
+
+Install Dependencies:
+
+```bash
+pip install -r requirements.txt
+```
+
+
+---
+
+## 🚀 Usage
+
+Run the main script:
+
+```bash
+python bot.py
+```
+
+When the app launches, you'll see a textual interface that lets you select your audio input device. Once selected, the app will begin capturing audio, transcribing it using Whisper.
+
+---
+
+## ⚙️ How It Works
+
+
+1. **LocalAudioTransport:**  
+   Captures audio from your chosen input device.
+   
+2. **WhisperSTTService:**  
+   Processes the audio stream using Whisper's large model for speech-to-text conversion.
+   
+3. **TranscriptionLogger:**  
+   Logs the transcribed text to the console as soon as it's processed.
+
+
+---
+
+## 📦 Dependencies
+
+The project relies on:
+
+- [pipecat](https://github.com/yourusername/pipecat) – For building the audio processing pipeline.
+- [Textual](https://github.com/Textualize/textual) – For the interactive terminal UI.
+- [Whisper](https://github.com/openai/whisper) – For state-of-the-art STT transcription.
+
+---
+
+## Example improvements:
+
+I plan to improve this example with local LLM calls and audio output.

examples/local-input-select-stt/bot.py

@@ -0,0 +1,65 @@
+#
+# Copyright (c) 2024–2025, Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+import asyncio
+import sys
+from typing import Tuple
+
+from dotenv import load_dotenv
+from loguru import logger
+from select_audio_device import AudioDevice, run_device_selector
+
+from pipecat.frames.frames import Frame, TranscriptionFrame
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat.pipeline.runner import PipelineRunner
+from pipecat.pipeline.task import PipelineTask
+from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
+from pipecat.services.whisper import Model, WhisperSTTService
+from pipecat.transports.local.audio import LocalAudioTransport, LocalAudioTransportParams
+
+load_dotenv(override=True)
+
+logger.remove(0)
+logger.add(sys.stderr, level="DEBUG")
+
+
+class TranscriptionLogger(FrameProcessor):
+    async def process_frame(self, frame: Frame, direction: FrameDirection):
+        await super().process_frame(frame, direction)
+
+        if isinstance(frame, TranscriptionFrame):
+            print(f"Transcription: {frame.text}")
+
+
+async def main(input_device: int, output_device: int):
+    transport = LocalAudioTransport(
+        LocalAudioTransportParams(
+            audio_in_enabled=True,
+            audio_out_enabled=False,
+            input_device_index=input_device,
+            output_device_index=output_device,
+        )
+    )
+
+    stt = WhisperSTTService(device="cuda", model=Model.LARGE, no_speech_prob=0.3)
+
+    tl = TranscriptionLogger()
+
+    pipeline = Pipeline([transport.input(), stt, tl])
+
+    task = PipelineTask(pipeline)
+
+    runner = PipelineRunner(handle_sigint=False if sys.platform == "win32" else True)
+
+    await asyncio.gather(runner.run(task))
+
+
+if __name__ == "__main__":
+    res: Tuple[AudioDevice, AudioDevice, int] = asyncio.run(
+        run_device_selector()  # runs the textual app that allows to select input device
+    )
+
+    asyncio.run(main(res[0].index, res[1].index))

examples/local-input-select-stt/requirements.txt

@@ -0,0 +1,8 @@
+--extra-index-url https://download.pytorch.org/whl/cu124
+torch==2.5.0+cu124
+torchvision 
+torchaudio
+pipecat[whisper, openai]
+textual==1.0.0
+pydantic-settings==2.7.1
+pyaudio==0.2.14

examples/local-input-select-stt/select_audio_device.py

@@ -0,0 +1,247 @@
+from typing import List, Optional, Tuple
+
+import pyaudio
+from pydantic import BaseModel, ConfigDict, Field
+from pydantic_settings import BaseSettings
+from textual.app import App, ComposeResult
+from textual.containers import Container
+from textual.widgets import Footer, Header, Label, ListItem, ListView, Select
+from textual.widgets.option_list import Option
+
+# ─── DATA MODELS ───────────────────────────────────────────────────────────────
+
+
+class HostApi(BaseModel):
+    index: int
+    struct_version: int = Field(..., alias="structVersion")
+    type: int
+    name: str
+    device_count: int = Field(..., alias="deviceCount")
+    default_input_device: int = Field(..., alias="defaultInputDevice")
+    default_output_device: int = Field(..., alias="defaultOutputDevice")
+
+
+class AudioDevice(BaseModel):
+    model_config = ConfigDict(populate_by_name=True)
+    index: int
+    struct_version: int = Field(..., alias="structVersion")
+    name: str
+    host_api: int = Field(..., alias="hostApi")
+    max_input_channels: int = Field(..., alias="maxInputChannels")
+    max_output_channels: int = Field(..., alias="maxOutputChannels")
+    default_low_input_latency: float = Field(..., alias="defaultLowInputLatency")
+    default_low_output_latency: float = Field(..., alias="defaultLowOutputLatency")
+    default_high_input_latency: float = Field(..., alias="defaultHighInputLatency")
+    default_high_output_latency: float = Field(..., alias="defaultHighOutputLatency")
+    default_sample_rate: float = Field(..., alias="defaultSampleRate")
+
+
+# ─── SETTINGS MODEL ───────────────────────────────────────────────────────────
+
+
+class AudioSettings(BaseSettings):  # to save settings to a file
+    host_api: Optional[int] = None
+    input_device: Optional[AudioDevice] = None
+    output_device: Optional[AudioDevice] = None
+
+    class Config:
+        env_file = "settings.env"  # or adjust as needed
+
+    def save_to_json(self, filepath: str) -> None:
+        with open(filepath, "w") as f:
+            f.write(self.model_dump_json(indent=2))
+
+
+# ─── TEXTUAL APP ──────────────────────────────────────────────────────────────
+
+
+class AudioDeviceSelectorApp(App):
+    CSS = """
+    Screen {
+        align: center middle;
+    }
+    #container {
+        width: 80%;
+        border: round green;
+        padding: 1 2;
+    }
+    """
+
+    def __init__(
+        self,
+        default_host_api: Optional[int] = None,
+        default_input_device: Optional[AudioDevice] = None,
+        default_output_device: Optional[AudioDevice] = None,
+        **kwargs,
+    ) -> None:
+        super().__init__(**kwargs)
+        # Save defaults passed from settings.
+        self.default_host_api: Optional[int] = default_host_api
+        self.default_input_device: Optional[AudioDevice] = default_input_device
+        self.default_output_device: Optional[AudioDevice] = default_output_device
+
+        self.pyaudio_instance = pyaudio.PyAudio()
+
+        # Static datastructures: host APIs and devices as well‐typed models.
+        self.host_apis: List[HostApi] = []
+        self.current_host_api: Optional[int] = None
+
+        self.all_input_devices: List[AudioDevice] = []
+        self.all_output_devices: List[AudioDevice] = []
+        self.input_devices: List[AudioDevice] = []
+        self.output_devices: List[AudioDevice] = []
+
+        # Stage management: first select input, then output.
+        self.stage: str = "input"
+        self.selected_input_device: Optional[AudioDevice] = None
+        self.selected_output_device: Optional[AudioDevice] = None
+        host_api_count: int = self.pyaudio_instance.get_host_api_count()
+        for i in range(host_api_count):
+            raw_api = self.pyaudio_instance.get_host_api_info_by_index(i)
+            # Inject the index (if not already present)
+            raw_api["index"] = i
+            try:
+                api = HostApi.parse_obj(raw_api)
+                self.host_apis.append(api)
+            except Exception as e:
+                # Skip APIs that don't conform.
+                continue
+
+    def compose(self) -> ComposeResult:
+        options: List[Tuple[str, Option]] = [
+            (
+                api.name,
+                Option(
+                    prompt=str(api.name) if api.name else f"Host API {api.index}",
+                    id=str(api.index),
+                ),
+            )
+            for api in self.host_apis
+        ]
+
+        yield Header()
+
+        yield Footer()
+        with Container(id="container"):
+            yield Label("Select Host API:", id="host-api-label")
+            # Create the Select widget with no options initially.
+            self.host_api_select: Select[HostApi] = Select(options=options, id="host-api-select")
+            yield self.host_api_select
+            self.prompt = Label("Select Input Audio Device:", id="prompt")
+            yield self.prompt
+            self.list_view = ListView(id="device-list")
+            yield self.list_view
+
+    def on_mount(self) -> None:
+        # Populate host APIs from PyAudio.
+
+        # Build the dropdown options.
+
+        self.host_api_select.refresh()  # Force a redraw
+
+        # Determine the default host API.
+        if self.default_host_api is not None:
+            self.current_host_api = self.default_host_api
+        else:
+            default_api_info = self.pyaudio_instance.get_default_host_api_info()
+            self.current_host_api = default_api_info["index"]
+
+        # Delay setting the dropdown's value until the widget is fully initialized.
+        self.set_timer(
+            0,
+            lambda: setattr(self.host_api_select, "value", str(self.current_host_api)),
+        )
+
+        # Load all devices and parse them into AudioDevice objects.
+        device_count: int = self.pyaudio_instance.get_device_count()
+        for i in range(device_count):
+            raw_device = self.pyaudio_instance.get_device_info_by_index(i)
+            raw_device["index"] = i
+            try:
+                device = AudioDevice.parse_obj(raw_device)
+            except Exception as e:
+                # Skip devices missing required fields.
+                continue
+            if device.max_input_channels > 0:
+                self.all_input_devices.append(device)
+            if device.max_output_channels > 0:
+                self.all_output_devices.append(device)
+
+        self.filter_devices()
+        self.populate_list(self.input_devices)
+        if self.default_input_device:
+            self._select_default_in_list(self.default_input_device)
+
+    def filter_devices(self) -> None:
+        """Filter devices based on the selected host API."""
+        self.input_devices = [
+            d for d in self.all_input_devices if d.host_api == self.current_host_api
+        ]
+        self.output_devices = [
+            d for d in self.all_output_devices if d.host_api == self.current_host_api
+        ]
+
+    def populate_list(self, devices: List[AudioDevice]) -> None:
+        """Populate the ListView with a list of AudioDevice objects."""
+        self.list_view.clear()
+        for dev in devices:
+            item_text: str = f"{dev.name} (Index: {dev.index})"
+            item = ListItem(Label(item_text))
+            # Attach the AudioDevice instance to the widget.
+            item.device_info = dev  # type: ignore
+            self.list_view.append(item)
+
+    def _select_default_in_list(self, default_device: AudioDevice) -> None:
+        """Pre-select the default device if present in the current list."""
+        for idx, item in enumerate(self.list_view.children):
+            if hasattr(item, "device_info") and item.device_info.index == default_device.index:
+                self.list_view.index = idx
+                break
+
+    async def on_select_changed(self, event: Select.Changed) -> None:
+        """Handle changes in the host API dropdown."""
+        if event.select.id == "host-api-select":
+            self.current_host_api = int(event.value.id)
+            self.filter_devices()
+            if self.stage == "input":
+                self.populate_list(self.input_devices)
+                if self.default_input_device:
+                    self._select_default_in_list(self.default_input_device)
+            elif self.stage == "output":
+                self.populate_list(self.output_devices)
+                if self.default_output_device:
+                    self._select_default_in_list(self.default_output_device)
+
+    async def on_list_view_selected(self, message: ListView.Selected) -> None:
+        """Record device selection and switch stages."""
+        selected_item = message.item
+        device_info: AudioDevice = selected_item.device_info  # type: ignore
+        if self.stage == "input":
+            self.selected_input_device = device_info
+            self.stage = "output"
+            self.prompt.update("Select Output Audio Device:")
+            self.populate_list(self.output_devices)
+            if self.default_output_device:
+                self._select_default_in_list(self.default_output_device)
+        elif self.stage == "output":
+            self.selected_output_device = device_info
+            await self.action_quit()
+
+
+# ─── HELPER FUNCTIONS ─────────────────────────────────────────────────────────
+
+
+async def run_device_selector(
+    default_host_api: Optional[int] = None,
+    default_input_device: Optional[AudioDevice] = None,
+    default_output_device: Optional[AudioDevice] = None,
+) -> Tuple[AudioDevice, AudioDevice, int]:
+    app = AudioDeviceSelectorApp(
+        default_host_api=default_host_api,
+        default_input_device=default_input_device,
+        default_output_device=default_output_device,
+    )
+    await app.run_async()
+
+    # The current_host_api is guaranteed to be set.
+    return app.selected_input_device, app.selected_output_device, app.current_host_api  # type: ignore

examples/news-chatbot/client/javascript/package-lock.json

@@ -13,7 +13,7 @@
         "@pipecat-ai/daily-transport": "^0.3.4"
       },
       "devDependencies": {
-        "vite": "^6.0.2"
+        "vite": "^6.0.9"
       }
     },
     "node_modules/@babel/runtime": {
@@ -45,14 +45,13 @@
       }
     },
     "node_modules/@esbuild/aix-ppc64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.24.0.tgz",
-      "integrity": "sha512-WtKdFM7ls47zkKHFVzMz8opM7LkcsIp9amDUBIAWirg70RM71WRSjdILPsY5Uv1D42ZpUfaPILDlfactHgsRkw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.24.2.tgz",
+      "integrity": "sha512-thpVCb/rhxE/BnMLQ7GReQLLN8q9qbHmI55F4489/ByVg2aQaQ6kbcLb6FHkocZzQhxc4gx0sCk0tJkKBFzDhA==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "aix"
@@ -62,14 +61,13 @@
       }
     },
     "node_modules/@esbuild/android-arm": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.24.0.tgz",
-      "integrity": "sha512-arAtTPo76fJ/ICkXWetLCc9EwEHKaeya4vMrReVlEIUCAUncH7M4bhMQ+M9Vf+FFOZJdTNMXNBrWwW+OXWpSew==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.24.2.tgz",
+      "integrity": "sha512-tmwl4hJkCfNHwFB3nBa8z1Uy3ypZpxqxfTQOcHX+xRByyYgunVbZ9MzUUfb0RxaHIMnbHagwAxuTL+tnNM+1/Q==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -79,14 +77,13 @@
       }
     },
     "node_modules/@esbuild/android-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.24.0.tgz",
-      "integrity": "sha512-Vsm497xFM7tTIPYK9bNTYJyF/lsP590Qc1WxJdlB6ljCbdZKU9SY8i7+Iin4kyhV/KV5J2rOKsBQbB77Ab7L/w==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.24.2.tgz",
+      "integrity": "sha512-cNLgeqCqV8WxfcTIOeL4OAtSmL8JjcN6m09XIgro1Wi7cF4t/THaWEa7eL5CMoMBdjoHOTh/vwTO/o2TRXIyzg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -96,14 +93,13 @@
       }
     },
     "node_modules/@esbuild/android-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.24.0.tgz",
-      "integrity": "sha512-t8GrvnFkiIY7pa7mMgJd7p8p8qqYIz1NYiAoKc75Zyv73L3DZW++oYMSHPRarcotTKuSs6m3hTOa5CKHaS02TQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.24.2.tgz",
+      "integrity": "sha512-B6Q0YQDqMx9D7rvIcsXfmJfvUYLoP722bgfBlO5cGvNVb5V/+Y7nhBE3mHV9OpxBf4eAS2S68KZztiPaWq4XYw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -113,14 +109,13 @@
       }
     },
     "node_modules/@esbuild/darwin-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.0.tgz",
-      "integrity": "sha512-CKyDpRbK1hXwv79soeTJNHb5EiG6ct3efd/FTPdzOWdbZZfGhpbcqIpiD0+vwmpu0wTIL97ZRPZu8vUt46nBSw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.2.tgz",
+      "integrity": "sha512-kj3AnYWc+CekmZnS5IPu9D+HWtUI49hbnyqk0FLEJDbzCIQt7hg7ucF1SQAilhtYpIujfaHr6O0UHlzzSPdOeA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -130,14 +125,13 @@
       }
     },
     "node_modules/@esbuild/darwin-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.24.0.tgz",
-      "integrity": "sha512-rgtz6flkVkh58od4PwTRqxbKH9cOjaXCMZgWD905JOzjFKW+7EiUObfd/Kav+A6Gyud6WZk9w+xu6QLytdi2OA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.24.2.tgz",
+      "integrity": "sha512-WeSrmwwHaPkNR5H3yYfowhZcbriGqooyu3zI/3GGpF8AyUdsrrP0X6KumITGA9WOyiJavnGZUwPGvxvwfWPHIA==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -147,14 +141,13 @@
       }
     },
     "node_modules/@esbuild/freebsd-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.24.0.tgz",
-      "integrity": "sha512-6Mtdq5nHggwfDNLAHkPlyLBpE5L6hwsuXZX8XNmHno9JuL2+bg2BX5tRkwjyfn6sKbxZTq68suOjgWqCicvPXA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-UN8HXjtJ0k/Mj6a9+5u6+2eZ2ERD7Edt1Q9IZiB5UZAIdPnVKDoG7mdTVGhHJIeEml60JteamR3qhsr1r8gXvg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
@@ -164,14 +157,13 @@
       }
     },
     "node_modules/@esbuild/freebsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.24.0.tgz",
-      "integrity": "sha512-D3H+xh3/zphoX8ck4S2RxKR6gHlHDXXzOf6f/9dbFt/NRBDIE33+cVa49Kil4WUjxMGW0ZIYBYtaGCa2+OsQwQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.24.2.tgz",
+      "integrity": "sha512-TvW7wE/89PYW+IevEJXZ5sF6gJRDY/14hyIGFXdIucxCsbRmLUcjseQu1SyTko+2idmCw94TgyaEZi9HUSOe3Q==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
@@ -181,14 +173,13 @@
       }
     },
     "node_modules/@esbuild/linux-arm": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.24.0.tgz",
-      "integrity": "sha512-gJKIi2IjRo5G6Glxb8d3DzYXlxdEj2NlkixPsqePSZMhLudqPhtZ4BUrpIuTjJYXxvF9njql+vRjB2oaC9XpBw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.24.2.tgz",
+      "integrity": "sha512-n0WRM/gWIdU29J57hJyUdIsk0WarGd6To0s+Y+LwvlC55wt+GT/OgkwoXCXvIue1i1sSNWblHEig00GBWiJgfA==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -198,14 +189,13 @@
       }
     },
     "node_modules/@esbuild/linux-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.24.0.tgz",
-      "integrity": "sha512-TDijPXTOeE3eaMkRYpcy3LarIg13dS9wWHRdwYRnzlwlA370rNdZqbcp0WTyyV/k2zSxfko52+C7jU5F9Tfj1g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.24.2.tgz",
+      "integrity": "sha512-7HnAD6074BW43YvvUmE/35Id9/NB7BeX5EoNkK9obndmZBUk8xmJJeU7DwmUeN7tkysslb2eSl6CTrYz6oEMQg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -215,14 +205,13 @@
       }
     },
     "node_modules/@esbuild/linux-ia32": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.24.0.tgz",
-      "integrity": "sha512-K40ip1LAcA0byL05TbCQ4yJ4swvnbzHscRmUilrmP9Am7//0UjPreh4lpYzvThT2Quw66MhjG//20mrufm40mA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.24.2.tgz",
+      "integrity": "sha512-sfv0tGPQhcZOgTKO3oBE9xpHuUqguHvSo4jl+wjnKwFpapx+vUDcawbwPNuBIAYdRAvIDBfZVvXprIj3HA+Ugw==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -232,14 +221,13 @@
       }
     },
     "node_modules/@esbuild/linux-loong64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.24.0.tgz",
-      "integrity": "sha512-0mswrYP/9ai+CU0BzBfPMZ8RVm3RGAN/lmOMgW4aFUSOQBjA31UP8Mr6DDhWSuMwj7jaWOT0p0WoZ6jeHhrD7g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.24.2.tgz",
+      "integrity": "sha512-CN9AZr8kEndGooS35ntToZLTQLHEjtVB5n7dl8ZcTZMonJ7CCfStrYhrzF97eAecqVbVJ7APOEe18RPI4KLhwQ==",
       "cpu": [
         "loong64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -249,14 +237,13 @@
       }
     },
     "node_modules/@esbuild/linux-mips64el": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.24.0.tgz",
-      "integrity": "sha512-hIKvXm0/3w/5+RDtCJeXqMZGkI2s4oMUGj3/jM0QzhgIASWrGO5/RlzAzm5nNh/awHE0A19h/CvHQe6FaBNrRA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.24.2.tgz",
+      "integrity": "sha512-iMkk7qr/wl3exJATwkISxI7kTcmHKE+BlymIAbHO8xanq/TjHaaVThFF6ipWzPHryoFsesNQJPE/3wFJw4+huw==",
       "cpu": [
         "mips64el"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -266,14 +253,13 @@
       }
     },
     "node_modules/@esbuild/linux-ppc64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.24.0.tgz",
-      "integrity": "sha512-HcZh5BNq0aC52UoocJxaKORfFODWXZxtBaaZNuN3PUX3MoDsChsZqopzi5UupRhPHSEHotoiptqikjN/B77mYQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.24.2.tgz",
+      "integrity": "sha512-shsVrgCZ57Vr2L8mm39kO5PPIb+843FStGt7sGGoqiiWYconSxwTiuswC1VJZLCjNiMLAMh34jg4VSEQb+iEbw==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -283,14 +269,13 @@
       }
     },
     "node_modules/@esbuild/linux-riscv64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.24.0.tgz",
-      "integrity": "sha512-bEh7dMn/h3QxeR2KTy1DUszQjUrIHPZKyO6aN1X4BCnhfYhuQqedHaa5MxSQA/06j3GpiIlFGSsy1c7Gf9padw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.24.2.tgz",
+      "integrity": "sha512-4eSFWnU9Hhd68fW16GD0TINewo1L6dRrB+oLNNbYyMUAeOD2yCK5KXGK1GH4qD/kT+bTEXjsyTCiJGHPZ3eM9Q==",
       "cpu": [
         "riscv64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -300,14 +285,13 @@
       }
     },
     "node_modules/@esbuild/linux-s390x": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.24.0.tgz",
-      "integrity": "sha512-ZcQ6+qRkw1UcZGPyrCiHHkmBaj9SiCD8Oqd556HldP+QlpUIe2Wgn3ehQGVoPOvZvtHm8HPx+bH20c9pvbkX3g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.24.2.tgz",
+      "integrity": "sha512-S0Bh0A53b0YHL2XEXC20bHLuGMOhFDO6GN4b3YjRLK//Ep3ql3erpNcPlEFed93hsQAjAQDNsvcK+hV90FubSw==",
       "cpu": [
         "s390x"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -317,14 +301,13 @@
       }
     },
     "node_modules/@esbuild/linux-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.24.0.tgz",
-      "integrity": "sha512-vbutsFqQ+foy3wSSbmjBXXIJ6PL3scghJoM8zCL142cGaZKAdCZHyf+Bpu/MmX9zT9Q0zFBVKb36Ma5Fzfa8xA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.24.2.tgz",
+      "integrity": "sha512-8Qi4nQcCTbLnK9WoMjdC9NiTG6/E38RNICU6sUNqK0QFxCYgoARqVqxdFmWkdonVsvGqWhmm7MO0jyTqLqwj0Q==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -333,15 +316,30 @@
         "node": ">=18"
       }
     },
+    "node_modules/@esbuild/netbsd-arm64": {
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-wuLK/VztRRpMt9zyHSazyCVdCXlpHkKm34WUyinD2lzK07FAHTq0KQvZZlXikNWkDGoT6x3TD51jKQ7gMVpopw==",
+      "cpu": [
+        "arm64"
+      ],
+      "dev": true,
+      "optional": true,
+      "os": [
+        "netbsd"
+      ],
+      "engines": {
+        "node": ">=18"
+      }
+    },
     "node_modules/@esbuild/netbsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.24.0.tgz",
-      "integrity": "sha512-hjQ0R/ulkO8fCYFsG0FZoH+pWgTTDreqpqY7UnQntnaKv95uP5iW3+dChxnx7C3trQQU40S+OgWhUVwCjVFLvg==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.24.2.tgz",
+      "integrity": "sha512-VefFaQUc4FMmJuAxmIHgUmfNiLXY438XrL4GDNV1Y1H/RW3qow68xTwjZKfj/+Plp9NANmzbH5R40Meudu8mmw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "netbsd"
@@ -351,14 +349,13 @@
       }
     },
     "node_modules/@esbuild/openbsd-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.24.0.tgz",
-      "integrity": "sha512-MD9uzzkPQbYehwcN583yx3Tu5M8EIoTD+tUgKF982WYL9Pf5rKy9ltgD0eUgs8pvKnmizxjXZyLt0z6DC3rRXg==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-YQbi46SBct6iKnszhSvdluqDmxCJA+Pu280Av9WICNwQmMxV7nLRHZfjQzwbPs3jeWnuAhE9Jy0NrnJ12Oz+0A==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "openbsd"
@@ -368,14 +365,13 @@
       }
     },
     "node_modules/@esbuild/openbsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.24.0.tgz",
-      "integrity": "sha512-4ir0aY1NGUhIC1hdoCzr1+5b43mw99uNwVzhIq1OY3QcEwPDO3B7WNXBzaKY5Nsf1+N11i1eOfFcq+D/gOS15Q==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.24.2.tgz",
+      "integrity": "sha512-+iDS6zpNM6EnJyWv0bMGLWSWeXGN/HTaF/LXHXHwejGsVi+ooqDfMCCTerNFxEkM3wYVcExkeGXNqshc9iMaOA==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "openbsd"
@@ -385,14 +381,13 @@
       }
     },
     "node_modules/@esbuild/sunos-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.24.0.tgz",
-      "integrity": "sha512-jVzdzsbM5xrotH+W5f1s+JtUy1UWgjU0Cf4wMvffTB8m6wP5/kx0KiaLHlbJO+dMgtxKV8RQ/JvtlFcdZ1zCPA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.24.2.tgz",
+      "integrity": "sha512-hTdsW27jcktEvpwNHJU4ZwWFGkz2zRJUz8pvddmXPtXDzVKTTINmlmga3ZzwcuMpUvLw7JkLy9QLKyGpD2Yxig==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "sunos"
@@ -402,14 +397,13 @@
       }
     },
     "node_modules/@esbuild/win32-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.24.0.tgz",
-      "integrity": "sha512-iKc8GAslzRpBytO2/aN3d2yb2z8XTVfNV0PjGlCxKo5SgWmNXx82I/Q3aG1tFfS+A2igVCY97TJ8tnYwpUWLCA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.24.2.tgz",
+      "integrity": "sha512-LihEQ2BBKVFLOC9ZItT9iFprsE9tqjDjnbulhHoFxYQtQfai7qfluVODIYxt1PgdoyQkz23+01rzwNwYfutxUQ==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -419,14 +413,13 @@
       }
     },
     "node_modules/@esbuild/win32-ia32": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.24.0.tgz",
-      "integrity": "sha512-vQW36KZolfIudCcTnaTpmLQ24Ha1RjygBo39/aLkM2kmjkWmZGEJ5Gn9l5/7tzXA42QGIoWbICfg6KLLkIw6yw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.24.2.tgz",
+      "integrity": "sha512-q+iGUwfs8tncmFC9pcnD5IvRHAzmbwQ3GPS5/ceCyHdjXubwQWI12MKWSNSMYLJMq23/IUCvJMS76PDqXe1fxA==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -436,14 +429,13 @@
       }
     },
     "node_modules/@esbuild/win32-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.24.0.tgz",
-      "integrity": "sha512-7IAFPrjSQIJrGsK6flwg7NFmwBoSTyF3rl7If0hNUFQU4ilTsEPL6GuMuU9BfIWVVGuRnuIidkSMC+c0Otu8IA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.24.2.tgz",
+      "integrity": "sha512-7VTgWzgMGvup6aSqDPLiW5zHaxYJGTO4OokMjIlrCtf+VpEL+cXKtCvg723iguPYI5oaUNdS+/V7OU2gvXVWEg==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -478,252 +470,247 @@
       }
     },
     "node_modules/@rollup/rollup-android-arm-eabi": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.28.0.tgz",
-      "integrity": "sha512-wLJuPLT6grGZsy34g4N1yRfYeouklTgPhH1gWXCYspenKYD0s3cR99ZevOGw5BexMNywkbV3UkjADisozBmpPQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.34.6.tgz",
+      "integrity": "sha512-+GcCXtOQoWuC7hhX1P00LqjjIiS/iOouHXhMdiDSnq/1DGTox4SpUvO52Xm+div6+106r+TcvOeo/cxvyEyTgg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
       ]
     },
     "node_modules/@rollup/rollup-android-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.28.0.tgz",
-      "integrity": "sha512-eiNkznlo0dLmVG/6wf+Ifi/v78G4d4QxRhuUl+s8EWZpDewgk7PX3ZyECUXU0Zq/Ca+8nU8cQpNC4Xgn2gFNDA==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.34.6.tgz",
+      "integrity": "sha512-E8+2qCIjciYUnCa1AiVF1BkRgqIGW9KzJeesQqVfyRITGQN+dFuoivO0hnro1DjT74wXLRZ7QF8MIbz+luGaJA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
       ]
     },
     "node_modules/@rollup/rollup-darwin-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.28.0.tgz",
-      "integrity": "sha512-lmKx9yHsppblnLQZOGxdO66gT77bvdBtr/0P+TPOseowE7D9AJoBw8ZDULRasXRWf1Z86/gcOdpBrV6VDUY36Q==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.34.6.tgz",
+      "integrity": "sha512-z9Ib+OzqN3DZEjX7PDQMHEhtF+t6Mi2z/ueChQPLS/qUMKY7Ybn5A2ggFoKRNRh1q1T03YTQfBTQCJZiepESAg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
       ]
     },
     "node_modules/@rollup/rollup-darwin-x64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.28.0.tgz",
-      "integrity": "sha512-8hxgfReVs7k9Js1uAIhS6zq3I+wKQETInnWQtgzt8JfGx51R1N6DRVy3F4o0lQwumbErRz52YqwjfvuwRxGv1w==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.34.6.tgz",
+      "integrity": "sha512-PShKVY4u0FDAR7jskyFIYVyHEPCPnIQY8s5OcXkdU8mz3Y7eXDJPdyM/ZWjkYdR2m0izD9HHWA8sGcXn+Qrsyg==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
       ]
     },
     "node_modules/@rollup/rollup-freebsd-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.28.0.tgz",
-      "integrity": "sha512-lA1zZB3bFx5oxu9fYud4+g1mt+lYXCoch0M0V/xhqLoGatbzVse0wlSQ1UYOWKpuSu3gyN4qEc0Dxf/DII1bhQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.34.6.tgz",
+      "integrity": "sha512-YSwyOqlDAdKqs0iKuqvRHLN4SrD2TiswfoLfvYXseKbL47ht1grQpq46MSiQAx6rQEN8o8URtpXARCpqabqxGQ==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
       ]
     },
     "node_modules/@rollup/rollup-freebsd-x64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.28.0.tgz",
-      "integrity": "sha512-aI2plavbUDjCQB/sRbeUZWX9qp12GfYkYSJOrdYTL/C5D53bsE2/nBPuoiJKoWp5SN78v2Vr8ZPnB+/VbQ2pFA==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.34.6.tgz",
+      "integrity": "sha512-HEP4CgPAY1RxXwwL5sPFv6BBM3tVeLnshF03HMhJYCNc6kvSqBgTMmsEjb72RkZBAWIqiPUyF1JpEBv5XT9wKQ==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm-gnueabihf": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.28.0.tgz",
-      "integrity": "sha512-WXveUPKtfqtaNvpf0iOb0M6xC64GzUX/OowbqfiCSXTdi/jLlOmH0Ba94/OkiY2yTGTwteo4/dsHRfh5bDCZ+w==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.34.6.tgz",
+      "integrity": "sha512-88fSzjC5xeH9S2Vg3rPgXJULkHcLYMkh8faix8DX4h4TIAL65ekwuQMA/g2CXq8W+NJC43V6fUpYZNjaX3+IIg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm-musleabihf": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.28.0.tgz",
-      "integrity": "sha512-yLc3O2NtOQR67lI79zsSc7lk31xjwcaocvdD1twL64PK1yNaIqCeWI9L5B4MFPAVGEVjH5k1oWSGuYX1Wutxpg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.34.6.tgz",
+      "integrity": "sha512-wM4ztnutBqYFyvNeR7Av+reWI/enK9tDOTKNF+6Kk2Q96k9bwhDDOlnCUNRPvromlVXo04riSliMBs/Z7RteEg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.28.0.tgz",
-      "integrity": "sha512-+P9G9hjEpHucHRXqesY+3X9hD2wh0iNnJXX/QhS/J5vTdG6VhNYMxJ2rJkQOxRUd17u5mbMLHM7yWGZdAASfcg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.34.6.tgz",
+      "integrity": "sha512-9RyprECbRa9zEjXLtvvshhw4CMrRa3K+0wcp3KME0zmBe1ILmvcVHnypZ/aIDXpRyfhSYSuN4EPdCCj5Du8FIA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm64-musl": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.28.0.tgz",
-      "integrity": "sha512-1xsm2rCKSTpKzi5/ypT5wfc+4bOGa/9yI/eaOLW0oMs7qpC542APWhl4A37AENGZ6St6GBMWhCCMM6tXgTIplw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.34.6.tgz",
+      "integrity": "sha512-qTmklhCTyaJSB05S+iSovfo++EwnIEZxHkzv5dep4qoszUMX5Ca4WM4zAVUMbfdviLgCSQOu5oU8YoGk1s6M9Q==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
+      "optional": true,
+      "os": [
+        "linux"
+      ]
+    },
+    "node_modules/@rollup/rollup-linux-loongarch64-gnu": {
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loongarch64-gnu/-/rollup-linux-loongarch64-gnu-4.34.6.tgz",
+      "integrity": "sha512-4Qmkaps9yqmpjY5pvpkfOerYgKNUGzQpFxV6rnS7c/JfYbDSU0y6WpbbredB5cCpLFGJEqYX40WUmxMkwhWCjw==",
+      "cpu": [
+        "loong64"
+      ],
+      "dev": true,
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-powerpc64le-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-powerpc64le-gnu/-/rollup-linux-powerpc64le-gnu-4.28.0.tgz",
-      "integrity": "sha512-zgWxMq8neVQeXL+ouSf6S7DoNeo6EPgi1eeqHXVKQxqPy1B2NvTbaOUWPn/7CfMKL7xvhV0/+fq/Z/J69g1WAQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-powerpc64le-gnu/-/rollup-linux-powerpc64le-gnu-4.34.6.tgz",
+      "integrity": "sha512-Zsrtux3PuaxuBTX/zHdLaFmcofWGzaWW1scwLU3ZbW/X+hSsFbz9wDIp6XvnT7pzYRl9MezWqEqKy7ssmDEnuQ==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-riscv64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.28.0.tgz",
-      "integrity": "sha512-VEdVYacLniRxbRJLNtzwGt5vwS0ycYshofI7cWAfj7Vg5asqj+pt+Q6x4n+AONSZW/kVm+5nklde0qs2EUwU2g==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.34.6.tgz",
+      "integrity": "sha512-aK+Zp+CRM55iPrlyKiU3/zyhgzWBxLVrw2mwiQSYJRobCURb781+XstzvA8Gkjg/hbdQFuDw44aUOxVQFycrAg==",
       "cpu": [
         "riscv64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-s390x-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.28.0.tgz",
-      "integrity": "sha512-LQlP5t2hcDJh8HV8RELD9/xlYtEzJkm/aWGsauvdO2ulfl3QYRjqrKW+mGAIWP5kdNCBheqqqYIGElSRCaXfpw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.34.6.tgz",
+      "integrity": "sha512-WoKLVrY9ogmaYPXwTH326+ErlCIgMmsoRSx6bO+l68YgJnlOXhygDYSZe/qbUJCSiCiZAQ+tKm88NcWuUXqOzw==",
       "cpu": [
         "s390x"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-x64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.28.0.tgz",
-      "integrity": "sha512-Nl4KIzteVEKE9BdAvYoTkW19pa7LR/RBrT6F1dJCV/3pbjwDcaOq+edkP0LXuJ9kflW/xOK414X78r+K84+msw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.34.6.tgz",
+      "integrity": "sha512-Sht4aFvmA4ToHd2vFzwMFaQCiYm2lDFho5rPcvPBT5pCdC+GwHG6CMch4GQfmWTQ1SwRKS0dhDYb54khSrjDWw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-x64-musl": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.28.0.tgz",
-      "integrity": "sha512-eKpJr4vBDOi4goT75MvW+0dXcNUqisK4jvibY9vDdlgLx+yekxSm55StsHbxUsRxSTt3JEQvlr3cGDkzcSP8bw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.34.6.tgz",
+      "integrity": "sha512-zmmpOQh8vXc2QITsnCiODCDGXFC8LMi64+/oPpPx5qz3pqv0s6x46ps4xoycfUiVZps5PFn1gksZzo4RGTKT+A==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-win32-arm64-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.28.0.tgz",
-      "integrity": "sha512-Vi+WR62xWGsE/Oj+mD0FNAPY2MEox3cfyG0zLpotZdehPFXwz6lypkGs5y38Jd/NVSbOD02aVad6q6QYF7i8Bg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.34.6.tgz",
+      "integrity": "sha512-3/q1qUsO/tLqGBaD4uXsB6coVGB3usxw3qyeVb59aArCgedSF66MPdgRStUd7vbZOsko/CgVaY5fo2vkvPLWiA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
       ]
     },
     "node_modules/@rollup/rollup-win32-ia32-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.28.0.tgz",
-      "integrity": "sha512-kN/Vpip8emMLn/eOza+4JwqDZBL6MPNpkdaEsgUtW1NYN3DZvZqSQrbKzJcTL6hd8YNmFTn7XGWMwccOcJBL0A==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.34.6.tgz",
+      "integrity": "sha512-oLHxuyywc6efdKVTxvc0135zPrRdtYVjtVD5GUm55I3ODxhU/PwkQFD97z16Xzxa1Fz0AEe4W/2hzRtd+IfpOA==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
       ]
     },
     "node_modules/@rollup/rollup-win32-x64-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.28.0.tgz",
-      "integrity": "sha512-Bvno2/aZT6usSa7lRDL2+hMjVAGjuqaymF1ApZm31JXzniR/hvr14jpU+/z4X6Gt5BPlzosscyJZGUvguXIqeQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.34.6.tgz",
+      "integrity": "sha512-0PVwmgzZ8+TZ9oGBmdZoQVXflbvuwzN/HRclujpl4N/q3i+y0lqLw8n1bXA8ru3sApDjlmONaNAuYr38y1Kr9w==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -808,8 +795,7 @@
       "version": "1.0.6",
       "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.6.tgz",
       "integrity": "sha512-AYnb1nQyY49te+VRAVgmzfcgjYS91mY5P0TKUDCLEM+gNnA+3T6rWITXRLYCpahpqSQbN5cE+gHpnPyXjHWxcw==",
-      "dev": true,
-      "license": "MIT"
+      "dev": true
     },
     "node_modules/@types/events": {
       "version": "3.0.3",
@@ -847,12 +833,11 @@
       }
     },
     "node_modules/esbuild": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.0.tgz",
-      "integrity": "sha512-FuLPevChGDshgSicjisSooU0cemp/sGXR841D5LHMB7mTVOmsEHcAxaH3irL53+8YDIeVNQEySh4DaYU/iuPqQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.2.tgz",
+      "integrity": "sha512-+9egpBW8I3CD5XPe0n6BfT5fxLzxrlDzqydF3aviG+9ni1lDC/OvMHcxqEFV0+LANZG5R1bFMWfUrjVsdwxJvA==",
       "dev": true,
       "hasInstallScript": true,
-      "license": "MIT",
       "bin": {
         "esbuild": "bin/esbuild"
       },
@@ -860,30 +845,31 @@
         "node": ">=18"
       },
       "optionalDependencies": {
-        "@esbuild/aix-ppc64": "0.24.0",
-        "@esbuild/android-arm": "0.24.0",
-        "@esbuild/android-arm64": "0.24.0",
-        "@esbuild/android-x64": "0.24.0",
-        "@esbuild/darwin-arm64": "0.24.0",
-        "@esbuild/darwin-x64": "0.24.0",
-        "@esbuild/freebsd-arm64": "0.24.0",
-        "@esbuild/freebsd-x64": "0.24.0",
-        "@esbuild/linux-arm": "0.24.0",
-        "@esbuild/linux-arm64": "0.24.0",
-        "@esbuild/linux-ia32": "0.24.0",
-        "@esbuild/linux-loong64": "0.24.0",
-        "@esbuild/linux-mips64el": "0.24.0",
-        "@esbuild/linux-ppc64": "0.24.0",
-        "@esbuild/linux-riscv64": "0.24.0",
-        "@esbuild/linux-s390x": "0.24.0",
-        "@esbuild/linux-x64": "0.24.0",
-        "@esbuild/netbsd-x64": "0.24.0",
-        "@esbuild/openbsd-arm64": "0.24.0",
-        "@esbuild/openbsd-x64": "0.24.0",
-        "@esbuild/sunos-x64": "0.24.0",
-        "@esbuild/win32-arm64": "0.24.0",
-        "@esbuild/win32-ia32": "0.24.0",
-        "@esbuild/win32-x64": "0.24.0"
+        "@esbuild/aix-ppc64": "0.24.2",
+        "@esbuild/android-arm": "0.24.2",
+        "@esbuild/android-arm64": "0.24.2",
+        "@esbuild/android-x64": "0.24.2",
+        "@esbuild/darwin-arm64": "0.24.2",
+        "@esbuild/darwin-x64": "0.24.2",
+        "@esbuild/freebsd-arm64": "0.24.2",
+        "@esbuild/freebsd-x64": "0.24.2",
+        "@esbuild/linux-arm": "0.24.2",
+        "@esbuild/linux-arm64": "0.24.2",
+        "@esbuild/linux-ia32": "0.24.2",
+        "@esbuild/linux-loong64": "0.24.2",
+        "@esbuild/linux-mips64el": "0.24.2",
+        "@esbuild/linux-ppc64": "0.24.2",
+        "@esbuild/linux-riscv64": "0.24.2",
+        "@esbuild/linux-s390x": "0.24.2",
+        "@esbuild/linux-x64": "0.24.2",
+        "@esbuild/netbsd-arm64": "0.24.2",
+        "@esbuild/netbsd-x64": "0.24.2",
+        "@esbuild/openbsd-arm64": "0.24.2",
+        "@esbuild/openbsd-x64": "0.24.2",
+        "@esbuild/sunos-x64": "0.24.2",
+        "@esbuild/win32-arm64": "0.24.2",
+        "@esbuild/win32-ia32": "0.24.2",
+        "@esbuild/win32-x64": "0.24.2"
       }
     },
     "node_modules/events": {
@@ -901,7 +887,6 @@
       "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
       "dev": true,
       "hasInstallScript": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -951,7 +936,6 @@
           "url": "https://github.com/sponsors/ai"
         }
       ],
-      "license": "MIT",
       "bin": {
         "nanoid": "bin/nanoid.cjs"
       },
@@ -963,13 +947,12 @@
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
       "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==",
-      "dev": true,
-      "license": "ISC"
+      "dev": true
     },
     "node_modules/postcss": {
-      "version": "8.4.49",
-      "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.49.tgz",
-      "integrity": "sha512-OCVPnIObs4N29kxTjzLfUryOkvZEq+pf8jTF0lg8E7uETuWHA+v7j3c/xJmiqpX450191LlmZfUKkXxkTry7nA==",
+      "version": "8.5.2",
+      "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.2.tgz",
+      "integrity": "sha512-MjOadfU3Ys9KYoX0AdkBlFEF1Vx37uCCeN4ZHnmwm9FfpbsGWMZeBLMmmpY+6Ocqod7mkdZ0DT31OlbsFrLlkA==",
       "dev": true,
       "funding": [
         {
@@ -985,9 +968,8 @@
           "url": "https://github.com/sponsors/ai"
         }
       ],
-      "license": "MIT",
       "dependencies": {
-        "nanoid": "^3.3.7",
+        "nanoid": "^3.3.8",
         "picocolors": "^1.1.1",
         "source-map-js": "^1.2.1"
       },
@@ -1002,11 +984,10 @@
       "license": "MIT"
     },
     "node_modules/rollup": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.28.0.tgz",
-      "integrity": "sha512-G9GOrmgWHBma4YfCcX8PjH0qhXSdH8B4HDE2o4/jaxj93S4DPCIDoLcXz99eWMji4hB29UFCEd7B2gwGJDR9cQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.34.6.tgz",
+      "integrity": "sha512-wc2cBWqJgkU3Iz5oztRkQbfVkbxoz5EhnCGOrnJvnLnQ7O0WhQUYyv18qQI79O8L7DdHrrlJNeCHd4VGpnaXKQ==",
       "dev": true,
-      "license": "MIT",
       "dependencies": {
         "@types/estree": "1.0.6"
       },
@@ -1018,24 +999,25 @@
         "npm": ">=8.0.0"
       },
       "optionalDependencies": {
-        "@rollup/rollup-android-arm-eabi": "4.28.0",
-        "@rollup/rollup-android-arm64": "4.28.0",
-        "@rollup/rollup-darwin-arm64": "4.28.0",
-        "@rollup/rollup-darwin-x64": "4.28.0",
-        "@rollup/rollup-freebsd-arm64": "4.28.0",
-        "@rollup/rollup-freebsd-x64": "4.28.0",
-        "@rollup/rollup-linux-arm-gnueabihf": "4.28.0",
-        "@rollup/rollup-linux-arm-musleabihf": "4.28.0",
-        "@rollup/rollup-linux-arm64-gnu": "4.28.0",
-        "@rollup/rollup-linux-arm64-musl": "4.28.0",
-        "@rollup/rollup-linux-powerpc64le-gnu": "4.28.0",
-        "@rollup/rollup-linux-riscv64-gnu": "4.28.0",
-        "@rollup/rollup-linux-s390x-gnu": "4.28.0",
-        "@rollup/rollup-linux-x64-gnu": "4.28.0",
-        "@rollup/rollup-linux-x64-musl": "4.28.0",
-        "@rollup/rollup-win32-arm64-msvc": "4.28.0",
-        "@rollup/rollup-win32-ia32-msvc": "4.28.0",
-        "@rollup/rollup-win32-x64-msvc": "4.28.0",
+        "@rollup/rollup-android-arm-eabi": "4.34.6",
+        "@rollup/rollup-android-arm64": "4.34.6",
+        "@rollup/rollup-darwin-arm64": "4.34.6",
+        "@rollup/rollup-darwin-x64": "4.34.6",
+        "@rollup/rollup-freebsd-arm64": "4.34.6",
+        "@rollup/rollup-freebsd-x64": "4.34.6",
+        "@rollup/rollup-linux-arm-gnueabihf": "4.34.6",
+        "@rollup/rollup-linux-arm-musleabihf": "4.34.6",
+        "@rollup/rollup-linux-arm64-gnu": "4.34.6",
+        "@rollup/rollup-linux-arm64-musl": "4.34.6",
+        "@rollup/rollup-linux-loongarch64-gnu": "4.34.6",
+        "@rollup/rollup-linux-powerpc64le-gnu": "4.34.6",
+        "@rollup/rollup-linux-riscv64-gnu": "4.34.6",
+        "@rollup/rollup-linux-s390x-gnu": "4.34.6",
+        "@rollup/rollup-linux-x64-gnu": "4.34.6",
+        "@rollup/rollup-linux-x64-musl": "4.34.6",
+        "@rollup/rollup-win32-arm64-msvc": "4.34.6",
+        "@rollup/rollup-win32-ia32-msvc": "4.34.6",
+        "@rollup/rollup-win32-x64-msvc": "4.34.6",
         "fsevents": "~2.3.2"
       }
     },
@@ -1066,7 +1048,6 @@
       "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
       "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==",
       "dev": true,
-      "license": "BSD-3-Clause",
       "engines": {
         "node": ">=0.10.0"
       }
@@ -1101,15 +1082,14 @@
       }
     },
     "node_modules/vite": {
-      "version": "6.0.2",
-      "resolved": "https://registry.npmjs.org/vite/-/vite-6.0.2.tgz",
-      "integrity": "sha512-XdQ+VsY2tJpBsKGs0wf3U/+azx8BBpYRHFAyKm5VeEZNOJZRB63q7Sc8Iup3k0TrN3KO6QgyzFf+opSbfY1y0g==",
+      "version": "6.1.0",
+      "resolved": "https://registry.npmjs.org/vite/-/vite-6.1.0.tgz",
+      "integrity": "sha512-RjjMipCKVoR4hVfPY6GQTgveinjNuyLw+qruksLDvA5ktI1150VmcMBKmQaEWJhg/j6Uaf6dNCNA0AfdzUb/hQ==",
       "dev": true,
-      "license": "MIT",
       "dependencies": {
-        "esbuild": "^0.24.0",
-        "postcss": "^8.4.49",
-        "rollup": "^4.23.0"
+        "esbuild": "^0.24.2",
+        "postcss": "^8.5.1",
+        "rollup": "^4.30.1"
       },
       "bin": {
         "vite": "bin/vite.js"

examples/news-chatbot/client/javascript/package.json

@@ -12,7 +12,7 @@
   "license": "ISC",
   "description": "",
   "devDependencies": {
-    "vite": "^6.0.2"
+    "vite": "^6.0.9"
   },
   "dependencies": {
     "@pipecat-ai/client-js": "^0.3.2",

examples/simple-chatbot/client/javascript/package-lock.json

@@ -13,7 +13,7 @@
         "@pipecat-ai/daily-transport": "^0.3.4"
       },
       "devDependencies": {
-        "vite": "^6.0.2"
+        "vite": "^6.0.9"
       }
     },
     "node_modules/@babel/runtime": {
@@ -45,14 +45,13 @@
       }
     },
     "node_modules/@esbuild/aix-ppc64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.24.0.tgz",
-      "integrity": "sha512-WtKdFM7ls47zkKHFVzMz8opM7LkcsIp9amDUBIAWirg70RM71WRSjdILPsY5Uv1D42ZpUfaPILDlfactHgsRkw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.24.2.tgz",
+      "integrity": "sha512-thpVCb/rhxE/BnMLQ7GReQLLN8q9qbHmI55F4489/ByVg2aQaQ6kbcLb6FHkocZzQhxc4gx0sCk0tJkKBFzDhA==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "aix"
@@ -62,14 +61,13 @@
       }
     },
     "node_modules/@esbuild/android-arm": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.24.0.tgz",
-      "integrity": "sha512-arAtTPo76fJ/ICkXWetLCc9EwEHKaeya4vMrReVlEIUCAUncH7M4bhMQ+M9Vf+FFOZJdTNMXNBrWwW+OXWpSew==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.24.2.tgz",
+      "integrity": "sha512-tmwl4hJkCfNHwFB3nBa8z1Uy3ypZpxqxfTQOcHX+xRByyYgunVbZ9MzUUfb0RxaHIMnbHagwAxuTL+tnNM+1/Q==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -79,14 +77,13 @@
       }
     },
     "node_modules/@esbuild/android-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.24.0.tgz",
-      "integrity": "sha512-Vsm497xFM7tTIPYK9bNTYJyF/lsP590Qc1WxJdlB6ljCbdZKU9SY8i7+Iin4kyhV/KV5J2rOKsBQbB77Ab7L/w==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.24.2.tgz",
+      "integrity": "sha512-cNLgeqCqV8WxfcTIOeL4OAtSmL8JjcN6m09XIgro1Wi7cF4t/THaWEa7eL5CMoMBdjoHOTh/vwTO/o2TRXIyzg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -96,14 +93,13 @@
       }
     },
     "node_modules/@esbuild/android-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.24.0.tgz",
-      "integrity": "sha512-t8GrvnFkiIY7pa7mMgJd7p8p8qqYIz1NYiAoKc75Zyv73L3DZW++oYMSHPRarcotTKuSs6m3hTOa5CKHaS02TQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.24.2.tgz",
+      "integrity": "sha512-B6Q0YQDqMx9D7rvIcsXfmJfvUYLoP722bgfBlO5cGvNVb5V/+Y7nhBE3mHV9OpxBf4eAS2S68KZztiPaWq4XYw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -113,14 +109,13 @@
       }
     },
     "node_modules/@esbuild/darwin-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.0.tgz",
-      "integrity": "sha512-CKyDpRbK1hXwv79soeTJNHb5EiG6ct3efd/FTPdzOWdbZZfGhpbcqIpiD0+vwmpu0wTIL97ZRPZu8vUt46nBSw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.2.tgz",
+      "integrity": "sha512-kj3AnYWc+CekmZnS5IPu9D+HWtUI49hbnyqk0FLEJDbzCIQt7hg7ucF1SQAilhtYpIujfaHr6O0UHlzzSPdOeA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -130,14 +125,13 @@
       }
     },
     "node_modules/@esbuild/darwin-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.24.0.tgz",
-      "integrity": "sha512-rgtz6flkVkh58od4PwTRqxbKH9cOjaXCMZgWD905JOzjFKW+7EiUObfd/Kav+A6Gyud6WZk9w+xu6QLytdi2OA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.24.2.tgz",
+      "integrity": "sha512-WeSrmwwHaPkNR5H3yYfowhZcbriGqooyu3zI/3GGpF8AyUdsrrP0X6KumITGA9WOyiJavnGZUwPGvxvwfWPHIA==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -147,14 +141,13 @@
       }
     },
     "node_modules/@esbuild/freebsd-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.24.0.tgz",
-      "integrity": "sha512-6Mtdq5nHggwfDNLAHkPlyLBpE5L6hwsuXZX8XNmHno9JuL2+bg2BX5tRkwjyfn6sKbxZTq68suOjgWqCicvPXA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-UN8HXjtJ0k/Mj6a9+5u6+2eZ2ERD7Edt1Q9IZiB5UZAIdPnVKDoG7mdTVGhHJIeEml60JteamR3qhsr1r8gXvg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
@@ -164,14 +157,13 @@
       }
     },
     "node_modules/@esbuild/freebsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.24.0.tgz",
-      "integrity": "sha512-D3H+xh3/zphoX8ck4S2RxKR6gHlHDXXzOf6f/9dbFt/NRBDIE33+cVa49Kil4WUjxMGW0ZIYBYtaGCa2+OsQwQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.24.2.tgz",
+      "integrity": "sha512-TvW7wE/89PYW+IevEJXZ5sF6gJRDY/14hyIGFXdIucxCsbRmLUcjseQu1SyTko+2idmCw94TgyaEZi9HUSOe3Q==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
@@ -181,14 +173,13 @@
       }
     },
     "node_modules/@esbuild/linux-arm": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.24.0.tgz",
-      "integrity": "sha512-gJKIi2IjRo5G6Glxb8d3DzYXlxdEj2NlkixPsqePSZMhLudqPhtZ4BUrpIuTjJYXxvF9njql+vRjB2oaC9XpBw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.24.2.tgz",
+      "integrity": "sha512-n0WRM/gWIdU29J57hJyUdIsk0WarGd6To0s+Y+LwvlC55wt+GT/OgkwoXCXvIue1i1sSNWblHEig00GBWiJgfA==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -198,14 +189,13 @@
       }
     },
     "node_modules/@esbuild/linux-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.24.0.tgz",
-      "integrity": "sha512-TDijPXTOeE3eaMkRYpcy3LarIg13dS9wWHRdwYRnzlwlA370rNdZqbcp0WTyyV/k2zSxfko52+C7jU5F9Tfj1g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.24.2.tgz",
+      "integrity": "sha512-7HnAD6074BW43YvvUmE/35Id9/NB7BeX5EoNkK9obndmZBUk8xmJJeU7DwmUeN7tkysslb2eSl6CTrYz6oEMQg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -215,14 +205,13 @@
       }
     },
     "node_modules/@esbuild/linux-ia32": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.24.0.tgz",
-      "integrity": "sha512-K40ip1LAcA0byL05TbCQ4yJ4swvnbzHscRmUilrmP9Am7//0UjPreh4lpYzvThT2Quw66MhjG//20mrufm40mA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.24.2.tgz",
+      "integrity": "sha512-sfv0tGPQhcZOgTKO3oBE9xpHuUqguHvSo4jl+wjnKwFpapx+vUDcawbwPNuBIAYdRAvIDBfZVvXprIj3HA+Ugw==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -232,14 +221,13 @@
       }
     },
     "node_modules/@esbuild/linux-loong64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.24.0.tgz",
-      "integrity": "sha512-0mswrYP/9ai+CU0BzBfPMZ8RVm3RGAN/lmOMgW4aFUSOQBjA31UP8Mr6DDhWSuMwj7jaWOT0p0WoZ6jeHhrD7g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.24.2.tgz",
+      "integrity": "sha512-CN9AZr8kEndGooS35ntToZLTQLHEjtVB5n7dl8ZcTZMonJ7CCfStrYhrzF97eAecqVbVJ7APOEe18RPI4KLhwQ==",
       "cpu": [
         "loong64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -249,14 +237,13 @@
       }
     },
     "node_modules/@esbuild/linux-mips64el": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.24.0.tgz",
-      "integrity": "sha512-hIKvXm0/3w/5+RDtCJeXqMZGkI2s4oMUGj3/jM0QzhgIASWrGO5/RlzAzm5nNh/awHE0A19h/CvHQe6FaBNrRA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.24.2.tgz",
+      "integrity": "sha512-iMkk7qr/wl3exJATwkISxI7kTcmHKE+BlymIAbHO8xanq/TjHaaVThFF6ipWzPHryoFsesNQJPE/3wFJw4+huw==",
       "cpu": [
         "mips64el"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -266,14 +253,13 @@
       }
     },
     "node_modules/@esbuild/linux-ppc64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.24.0.tgz",
-      "integrity": "sha512-HcZh5BNq0aC52UoocJxaKORfFODWXZxtBaaZNuN3PUX3MoDsChsZqopzi5UupRhPHSEHotoiptqikjN/B77mYQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.24.2.tgz",
+      "integrity": "sha512-shsVrgCZ57Vr2L8mm39kO5PPIb+843FStGt7sGGoqiiWYconSxwTiuswC1VJZLCjNiMLAMh34jg4VSEQb+iEbw==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -283,14 +269,13 @@
       }
     },
     "node_modules/@esbuild/linux-riscv64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.24.0.tgz",
-      "integrity": "sha512-bEh7dMn/h3QxeR2KTy1DUszQjUrIHPZKyO6aN1X4BCnhfYhuQqedHaa5MxSQA/06j3GpiIlFGSsy1c7Gf9padw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.24.2.tgz",
+      "integrity": "sha512-4eSFWnU9Hhd68fW16GD0TINewo1L6dRrB+oLNNbYyMUAeOD2yCK5KXGK1GH4qD/kT+bTEXjsyTCiJGHPZ3eM9Q==",
       "cpu": [
         "riscv64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -300,14 +285,13 @@
       }
     },
     "node_modules/@esbuild/linux-s390x": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.24.0.tgz",
-      "integrity": "sha512-ZcQ6+qRkw1UcZGPyrCiHHkmBaj9SiCD8Oqd556HldP+QlpUIe2Wgn3ehQGVoPOvZvtHm8HPx+bH20c9pvbkX3g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.24.2.tgz",
+      "integrity": "sha512-S0Bh0A53b0YHL2XEXC20bHLuGMOhFDO6GN4b3YjRLK//Ep3ql3erpNcPlEFed93hsQAjAQDNsvcK+hV90FubSw==",
       "cpu": [
         "s390x"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -317,14 +301,13 @@
       }
     },
     "node_modules/@esbuild/linux-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.24.0.tgz",
-      "integrity": "sha512-vbutsFqQ+foy3wSSbmjBXXIJ6PL3scghJoM8zCL142cGaZKAdCZHyf+Bpu/MmX9zT9Q0zFBVKb36Ma5Fzfa8xA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.24.2.tgz",
+      "integrity": "sha512-8Qi4nQcCTbLnK9WoMjdC9NiTG6/E38RNICU6sUNqK0QFxCYgoARqVqxdFmWkdonVsvGqWhmm7MO0jyTqLqwj0Q==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -333,15 +316,30 @@
         "node": ">=18"
       }
     },
+    "node_modules/@esbuild/netbsd-arm64": {
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-wuLK/VztRRpMt9zyHSazyCVdCXlpHkKm34WUyinD2lzK07FAHTq0KQvZZlXikNWkDGoT6x3TD51jKQ7gMVpopw==",
+      "cpu": [
+        "arm64"
+      ],
+      "dev": true,
+      "optional": true,
+      "os": [
+        "netbsd"
+      ],
+      "engines": {
+        "node": ">=18"
+      }
+    },
     "node_modules/@esbuild/netbsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.24.0.tgz",
-      "integrity": "sha512-hjQ0R/ulkO8fCYFsG0FZoH+pWgTTDreqpqY7UnQntnaKv95uP5iW3+dChxnx7C3trQQU40S+OgWhUVwCjVFLvg==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.24.2.tgz",
+      "integrity": "sha512-VefFaQUc4FMmJuAxmIHgUmfNiLXY438XrL4GDNV1Y1H/RW3qow68xTwjZKfj/+Plp9NANmzbH5R40Meudu8mmw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "netbsd"
@@ -351,14 +349,13 @@
       }
     },
     "node_modules/@esbuild/openbsd-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.24.0.tgz",
-      "integrity": "sha512-MD9uzzkPQbYehwcN583yx3Tu5M8EIoTD+tUgKF982WYL9Pf5rKy9ltgD0eUgs8pvKnmizxjXZyLt0z6DC3rRXg==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-YQbi46SBct6iKnszhSvdluqDmxCJA+Pu280Av9WICNwQmMxV7nLRHZfjQzwbPs3jeWnuAhE9Jy0NrnJ12Oz+0A==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "openbsd"
@@ -368,14 +365,13 @@
       }
     },
     "node_modules/@esbuild/openbsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.24.0.tgz",
-      "integrity": "sha512-4ir0aY1NGUhIC1hdoCzr1+5b43mw99uNwVzhIq1OY3QcEwPDO3B7WNXBzaKY5Nsf1+N11i1eOfFcq+D/gOS15Q==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.24.2.tgz",
+      "integrity": "sha512-+iDS6zpNM6EnJyWv0bMGLWSWeXGN/HTaF/LXHXHwejGsVi+ooqDfMCCTerNFxEkM3wYVcExkeGXNqshc9iMaOA==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "openbsd"
@@ -385,14 +381,13 @@
       }
     },
     "node_modules/@esbuild/sunos-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.24.0.tgz",
-      "integrity": "sha512-jVzdzsbM5xrotH+W5f1s+JtUy1UWgjU0Cf4wMvffTB8m6wP5/kx0KiaLHlbJO+dMgtxKV8RQ/JvtlFcdZ1zCPA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.24.2.tgz",
+      "integrity": "sha512-hTdsW27jcktEvpwNHJU4ZwWFGkz2zRJUz8pvddmXPtXDzVKTTINmlmga3ZzwcuMpUvLw7JkLy9QLKyGpD2Yxig==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "sunos"
@@ -402,14 +397,13 @@
       }
     },
     "node_modules/@esbuild/win32-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.24.0.tgz",
-      "integrity": "sha512-iKc8GAslzRpBytO2/aN3d2yb2z8XTVfNV0PjGlCxKo5SgWmNXx82I/Q3aG1tFfS+A2igVCY97TJ8tnYwpUWLCA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.24.2.tgz",
+      "integrity": "sha512-LihEQ2BBKVFLOC9ZItT9iFprsE9tqjDjnbulhHoFxYQtQfai7qfluVODIYxt1PgdoyQkz23+01rzwNwYfutxUQ==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -419,14 +413,13 @@
       }
     },
     "node_modules/@esbuild/win32-ia32": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.24.0.tgz",
-      "integrity": "sha512-vQW36KZolfIudCcTnaTpmLQ24Ha1RjygBo39/aLkM2kmjkWmZGEJ5Gn9l5/7tzXA42QGIoWbICfg6KLLkIw6yw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.24.2.tgz",
+      "integrity": "sha512-q+iGUwfs8tncmFC9pcnD5IvRHAzmbwQ3GPS5/ceCyHdjXubwQWI12MKWSNSMYLJMq23/IUCvJMS76PDqXe1fxA==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -436,14 +429,13 @@
       }
     },
     "node_modules/@esbuild/win32-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.24.0.tgz",
-      "integrity": "sha512-7IAFPrjSQIJrGsK6flwg7NFmwBoSTyF3rl7If0hNUFQU4ilTsEPL6GuMuU9BfIWVVGuRnuIidkSMC+c0Otu8IA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.24.2.tgz",
+      "integrity": "sha512-7VTgWzgMGvup6aSqDPLiW5zHaxYJGTO4OokMjIlrCtf+VpEL+cXKtCvg723iguPYI5oaUNdS+/V7OU2gvXVWEg==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -478,252 +470,247 @@
       }
     },
     "node_modules/@rollup/rollup-android-arm-eabi": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.28.0.tgz",
-      "integrity": "sha512-wLJuPLT6grGZsy34g4N1yRfYeouklTgPhH1gWXCYspenKYD0s3cR99ZevOGw5BexMNywkbV3UkjADisozBmpPQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.34.6.tgz",
+      "integrity": "sha512-+GcCXtOQoWuC7hhX1P00LqjjIiS/iOouHXhMdiDSnq/1DGTox4SpUvO52Xm+div6+106r+TcvOeo/cxvyEyTgg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
       ]
     },
     "node_modules/@rollup/rollup-android-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.28.0.tgz",
-      "integrity": "sha512-eiNkznlo0dLmVG/6wf+Ifi/v78G4d4QxRhuUl+s8EWZpDewgk7PX3ZyECUXU0Zq/Ca+8nU8cQpNC4Xgn2gFNDA==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.34.6.tgz",
+      "integrity": "sha512-E8+2qCIjciYUnCa1AiVF1BkRgqIGW9KzJeesQqVfyRITGQN+dFuoivO0hnro1DjT74wXLRZ7QF8MIbz+luGaJA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
       ]
     },
     "node_modules/@rollup/rollup-darwin-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.28.0.tgz",
-      "integrity": "sha512-lmKx9yHsppblnLQZOGxdO66gT77bvdBtr/0P+TPOseowE7D9AJoBw8ZDULRasXRWf1Z86/gcOdpBrV6VDUY36Q==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.34.6.tgz",
+      "integrity": "sha512-z9Ib+OzqN3DZEjX7PDQMHEhtF+t6Mi2z/ueChQPLS/qUMKY7Ybn5A2ggFoKRNRh1q1T03YTQfBTQCJZiepESAg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
       ]
     },
     "node_modules/@rollup/rollup-darwin-x64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.28.0.tgz",
-      "integrity": "sha512-8hxgfReVs7k9Js1uAIhS6zq3I+wKQETInnWQtgzt8JfGx51R1N6DRVy3F4o0lQwumbErRz52YqwjfvuwRxGv1w==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.34.6.tgz",
+      "integrity": "sha512-PShKVY4u0FDAR7jskyFIYVyHEPCPnIQY8s5OcXkdU8mz3Y7eXDJPdyM/ZWjkYdR2m0izD9HHWA8sGcXn+Qrsyg==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
       ]
     },
     "node_modules/@rollup/rollup-freebsd-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.28.0.tgz",
-      "integrity": "sha512-lA1zZB3bFx5oxu9fYud4+g1mt+lYXCoch0M0V/xhqLoGatbzVse0wlSQ1UYOWKpuSu3gyN4qEc0Dxf/DII1bhQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.34.6.tgz",
+      "integrity": "sha512-YSwyOqlDAdKqs0iKuqvRHLN4SrD2TiswfoLfvYXseKbL47ht1grQpq46MSiQAx6rQEN8o8URtpXARCpqabqxGQ==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
       ]
     },
     "node_modules/@rollup/rollup-freebsd-x64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.28.0.tgz",
-      "integrity": "sha512-aI2plavbUDjCQB/sRbeUZWX9qp12GfYkYSJOrdYTL/C5D53bsE2/nBPuoiJKoWp5SN78v2Vr8ZPnB+/VbQ2pFA==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.34.6.tgz",
+      "integrity": "sha512-HEP4CgPAY1RxXwwL5sPFv6BBM3tVeLnshF03HMhJYCNc6kvSqBgTMmsEjb72RkZBAWIqiPUyF1JpEBv5XT9wKQ==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm-gnueabihf": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.28.0.tgz",
-      "integrity": "sha512-WXveUPKtfqtaNvpf0iOb0M6xC64GzUX/OowbqfiCSXTdi/jLlOmH0Ba94/OkiY2yTGTwteo4/dsHRfh5bDCZ+w==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.34.6.tgz",
+      "integrity": "sha512-88fSzjC5xeH9S2Vg3rPgXJULkHcLYMkh8faix8DX4h4TIAL65ekwuQMA/g2CXq8W+NJC43V6fUpYZNjaX3+IIg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm-musleabihf": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.28.0.tgz",
-      "integrity": "sha512-yLc3O2NtOQR67lI79zsSc7lk31xjwcaocvdD1twL64PK1yNaIqCeWI9L5B4MFPAVGEVjH5k1oWSGuYX1Wutxpg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.34.6.tgz",
+      "integrity": "sha512-wM4ztnutBqYFyvNeR7Av+reWI/enK9tDOTKNF+6Kk2Q96k9bwhDDOlnCUNRPvromlVXo04riSliMBs/Z7RteEg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.28.0.tgz",
-      "integrity": "sha512-+P9G9hjEpHucHRXqesY+3X9hD2wh0iNnJXX/QhS/J5vTdG6VhNYMxJ2rJkQOxRUd17u5mbMLHM7yWGZdAASfcg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.34.6.tgz",
+      "integrity": "sha512-9RyprECbRa9zEjXLtvvshhw4CMrRa3K+0wcp3KME0zmBe1ILmvcVHnypZ/aIDXpRyfhSYSuN4EPdCCj5Du8FIA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm64-musl": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.28.0.tgz",
-      "integrity": "sha512-1xsm2rCKSTpKzi5/ypT5wfc+4bOGa/9yI/eaOLW0oMs7qpC542APWhl4A37AENGZ6St6GBMWhCCMM6tXgTIplw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.34.6.tgz",
+      "integrity": "sha512-qTmklhCTyaJSB05S+iSovfo++EwnIEZxHkzv5dep4qoszUMX5Ca4WM4zAVUMbfdviLgCSQOu5oU8YoGk1s6M9Q==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
+      "optional": true,
+      "os": [
+        "linux"
+      ]
+    },
+    "node_modules/@rollup/rollup-linux-loongarch64-gnu": {
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loongarch64-gnu/-/rollup-linux-loongarch64-gnu-4.34.6.tgz",
+      "integrity": "sha512-4Qmkaps9yqmpjY5pvpkfOerYgKNUGzQpFxV6rnS7c/JfYbDSU0y6WpbbredB5cCpLFGJEqYX40WUmxMkwhWCjw==",
+      "cpu": [
+        "loong64"
+      ],
+      "dev": true,
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-powerpc64le-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-powerpc64le-gnu/-/rollup-linux-powerpc64le-gnu-4.28.0.tgz",
-      "integrity": "sha512-zgWxMq8neVQeXL+ouSf6S7DoNeo6EPgi1eeqHXVKQxqPy1B2NvTbaOUWPn/7CfMKL7xvhV0/+fq/Z/J69g1WAQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-powerpc64le-gnu/-/rollup-linux-powerpc64le-gnu-4.34.6.tgz",
+      "integrity": "sha512-Zsrtux3PuaxuBTX/zHdLaFmcofWGzaWW1scwLU3ZbW/X+hSsFbz9wDIp6XvnT7pzYRl9MezWqEqKy7ssmDEnuQ==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-riscv64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.28.0.tgz",
-      "integrity": "sha512-VEdVYacLniRxbRJLNtzwGt5vwS0ycYshofI7cWAfj7Vg5asqj+pt+Q6x4n+AONSZW/kVm+5nklde0qs2EUwU2g==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.34.6.tgz",
+      "integrity": "sha512-aK+Zp+CRM55iPrlyKiU3/zyhgzWBxLVrw2mwiQSYJRobCURb781+XstzvA8Gkjg/hbdQFuDw44aUOxVQFycrAg==",
       "cpu": [
         "riscv64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-s390x-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.28.0.tgz",
-      "integrity": "sha512-LQlP5t2hcDJh8HV8RELD9/xlYtEzJkm/aWGsauvdO2ulfl3QYRjqrKW+mGAIWP5kdNCBheqqqYIGElSRCaXfpw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.34.6.tgz",
+      "integrity": "sha512-WoKLVrY9ogmaYPXwTH326+ErlCIgMmsoRSx6bO+l68YgJnlOXhygDYSZe/qbUJCSiCiZAQ+tKm88NcWuUXqOzw==",
       "cpu": [
         "s390x"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-x64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.28.0.tgz",
-      "integrity": "sha512-Nl4KIzteVEKE9BdAvYoTkW19pa7LR/RBrT6F1dJCV/3pbjwDcaOq+edkP0LXuJ9kflW/xOK414X78r+K84+msw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.34.6.tgz",
+      "integrity": "sha512-Sht4aFvmA4ToHd2vFzwMFaQCiYm2lDFho5rPcvPBT5pCdC+GwHG6CMch4GQfmWTQ1SwRKS0dhDYb54khSrjDWw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-x64-musl": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.28.0.tgz",
-      "integrity": "sha512-eKpJr4vBDOi4goT75MvW+0dXcNUqisK4jvibY9vDdlgLx+yekxSm55StsHbxUsRxSTt3JEQvlr3cGDkzcSP8bw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.34.6.tgz",
+      "integrity": "sha512-zmmpOQh8vXc2QITsnCiODCDGXFC8LMi64+/oPpPx5qz3pqv0s6x46ps4xoycfUiVZps5PFn1gksZzo4RGTKT+A==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-win32-arm64-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.28.0.tgz",
-      "integrity": "sha512-Vi+WR62xWGsE/Oj+mD0FNAPY2MEox3cfyG0zLpotZdehPFXwz6lypkGs5y38Jd/NVSbOD02aVad6q6QYF7i8Bg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.34.6.tgz",
+      "integrity": "sha512-3/q1qUsO/tLqGBaD4uXsB6coVGB3usxw3qyeVb59aArCgedSF66MPdgRStUd7vbZOsko/CgVaY5fo2vkvPLWiA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
       ]
     },
     "node_modules/@rollup/rollup-win32-ia32-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.28.0.tgz",
-      "integrity": "sha512-kN/Vpip8emMLn/eOza+4JwqDZBL6MPNpkdaEsgUtW1NYN3DZvZqSQrbKzJcTL6hd8YNmFTn7XGWMwccOcJBL0A==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.34.6.tgz",
+      "integrity": "sha512-oLHxuyywc6efdKVTxvc0135zPrRdtYVjtVD5GUm55I3ODxhU/PwkQFD97z16Xzxa1Fz0AEe4W/2hzRtd+IfpOA==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
       ]
     },
     "node_modules/@rollup/rollup-win32-x64-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.28.0.tgz",
-      "integrity": "sha512-Bvno2/aZT6usSa7lRDL2+hMjVAGjuqaymF1ApZm31JXzniR/hvr14jpU+/z4X6Gt5BPlzosscyJZGUvguXIqeQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.34.6.tgz",
+      "integrity": "sha512-0PVwmgzZ8+TZ9oGBmdZoQVXflbvuwzN/HRclujpl4N/q3i+y0lqLw8n1bXA8ru3sApDjlmONaNAuYr38y1Kr9w==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -808,8 +795,7 @@
       "version": "1.0.6",
       "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.6.tgz",
       "integrity": "sha512-AYnb1nQyY49te+VRAVgmzfcgjYS91mY5P0TKUDCLEM+gNnA+3T6rWITXRLYCpahpqSQbN5cE+gHpnPyXjHWxcw==",
-      "dev": true,
-      "license": "MIT"
+      "dev": true
     },
     "node_modules/@types/events": {
       "version": "3.0.3",
@@ -847,12 +833,11 @@
       }
     },
     "node_modules/esbuild": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.0.tgz",
-      "integrity": "sha512-FuLPevChGDshgSicjisSooU0cemp/sGXR841D5LHMB7mTVOmsEHcAxaH3irL53+8YDIeVNQEySh4DaYU/iuPqQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.2.tgz",
+      "integrity": "sha512-+9egpBW8I3CD5XPe0n6BfT5fxLzxrlDzqydF3aviG+9ni1lDC/OvMHcxqEFV0+LANZG5R1bFMWfUrjVsdwxJvA==",
       "dev": true,
       "hasInstallScript": true,
-      "license": "MIT",
       "bin": {
         "esbuild": "bin/esbuild"
       },
@@ -860,30 +845,31 @@
         "node": ">=18"
       },
       "optionalDependencies": {
-        "@esbuild/aix-ppc64": "0.24.0",
-        "@esbuild/android-arm": "0.24.0",
-        "@esbuild/android-arm64": "0.24.0",
-        "@esbuild/android-x64": "0.24.0",
-        "@esbuild/darwin-arm64": "0.24.0",
-        "@esbuild/darwin-x64": "0.24.0",
-        "@esbuild/freebsd-arm64": "0.24.0",
-        "@esbuild/freebsd-x64": "0.24.0",
-        "@esbuild/linux-arm": "0.24.0",
-        "@esbuild/linux-arm64": "0.24.0",
-        "@esbuild/linux-ia32": "0.24.0",
-        "@esbuild/linux-loong64": "0.24.0",
-        "@esbuild/linux-mips64el": "0.24.0",
-        "@esbuild/linux-ppc64": "0.24.0",
-        "@esbuild/linux-riscv64": "0.24.0",
-        "@esbuild/linux-s390x": "0.24.0",
-        "@esbuild/linux-x64": "0.24.0",
-        "@esbuild/netbsd-x64": "0.24.0",
-        "@esbuild/openbsd-arm64": "0.24.0",
-        "@esbuild/openbsd-x64": "0.24.0",
-        "@esbuild/sunos-x64": "0.24.0",
-        "@esbuild/win32-arm64": "0.24.0",
-        "@esbuild/win32-ia32": "0.24.0",
-        "@esbuild/win32-x64": "0.24.0"
+        "@esbuild/aix-ppc64": "0.24.2",
+        "@esbuild/android-arm": "0.24.2",
+        "@esbuild/android-arm64": "0.24.2",
+        "@esbuild/android-x64": "0.24.2",
+        "@esbuild/darwin-arm64": "0.24.2",
+        "@esbuild/darwin-x64": "0.24.2",
+        "@esbuild/freebsd-arm64": "0.24.2",
+        "@esbuild/freebsd-x64": "0.24.2",
+        "@esbuild/linux-arm": "0.24.2",
+        "@esbuild/linux-arm64": "0.24.2",
+        "@esbuild/linux-ia32": "0.24.2",
+        "@esbuild/linux-loong64": "0.24.2",
+        "@esbuild/linux-mips64el": "0.24.2",
+        "@esbuild/linux-ppc64": "0.24.2",
+        "@esbuild/linux-riscv64": "0.24.2",
+        "@esbuild/linux-s390x": "0.24.2",
+        "@esbuild/linux-x64": "0.24.2",
+        "@esbuild/netbsd-arm64": "0.24.2",
+        "@esbuild/netbsd-x64": "0.24.2",
+        "@esbuild/openbsd-arm64": "0.24.2",
+        "@esbuild/openbsd-x64": "0.24.2",
+        "@esbuild/sunos-x64": "0.24.2",
+        "@esbuild/win32-arm64": "0.24.2",
+        "@esbuild/win32-ia32": "0.24.2",
+        "@esbuild/win32-x64": "0.24.2"
       }
     },
     "node_modules/events": {
@@ -901,7 +887,6 @@
       "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
       "dev": true,
       "hasInstallScript": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -951,7 +936,6 @@
           "url": "https://github.com/sponsors/ai"
         }
       ],
-      "license": "MIT",
       "bin": {
         "nanoid": "bin/nanoid.cjs"
       },
@@ -963,13 +947,12 @@
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
       "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==",
-      "dev": true,
-      "license": "ISC"
+      "dev": true
     },
     "node_modules/postcss": {
-      "version": "8.4.49",
-      "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.49.tgz",
-      "integrity": "sha512-OCVPnIObs4N29kxTjzLfUryOkvZEq+pf8jTF0lg8E7uETuWHA+v7j3c/xJmiqpX450191LlmZfUKkXxkTry7nA==",
+      "version": "8.5.2",
+      "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.2.tgz",
+      "integrity": "sha512-MjOadfU3Ys9KYoX0AdkBlFEF1Vx37uCCeN4ZHnmwm9FfpbsGWMZeBLMmmpY+6Ocqod7mkdZ0DT31OlbsFrLlkA==",
       "dev": true,
       "funding": [
         {
@@ -985,9 +968,8 @@
           "url": "https://github.com/sponsors/ai"
         }
       ],
-      "license": "MIT",
       "dependencies": {
-        "nanoid": "^3.3.7",
+        "nanoid": "^3.3.8",
         "picocolors": "^1.1.1",
         "source-map-js": "^1.2.1"
       },
@@ -1002,11 +984,10 @@
       "license": "MIT"
     },
     "node_modules/rollup": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.28.0.tgz",
-      "integrity": "sha512-G9GOrmgWHBma4YfCcX8PjH0qhXSdH8B4HDE2o4/jaxj93S4DPCIDoLcXz99eWMji4hB29UFCEd7B2gwGJDR9cQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.34.6.tgz",
+      "integrity": "sha512-wc2cBWqJgkU3Iz5oztRkQbfVkbxoz5EhnCGOrnJvnLnQ7O0WhQUYyv18qQI79O8L7DdHrrlJNeCHd4VGpnaXKQ==",
       "dev": true,
-      "license": "MIT",
       "dependencies": {
         "@types/estree": "1.0.6"
       },
@@ -1018,24 +999,25 @@
         "npm": ">=8.0.0"
       },
       "optionalDependencies": {
-        "@rollup/rollup-android-arm-eabi": "4.28.0",
-        "@rollup/rollup-android-arm64": "4.28.0",
-        "@rollup/rollup-darwin-arm64": "4.28.0",
-        "@rollup/rollup-darwin-x64": "4.28.0",
-        "@rollup/rollup-freebsd-arm64": "4.28.0",
-        "@rollup/rollup-freebsd-x64": "4.28.0",
-        "@rollup/rollup-linux-arm-gnueabihf": "4.28.0",
-        "@rollup/rollup-linux-arm-musleabihf": "4.28.0",
-        "@rollup/rollup-linux-arm64-gnu": "4.28.0",
-        "@rollup/rollup-linux-arm64-musl": "4.28.0",
-        "@rollup/rollup-linux-powerpc64le-gnu": "4.28.0",
-        "@rollup/rollup-linux-riscv64-gnu": "4.28.0",
-        "@rollup/rollup-linux-s390x-gnu": "4.28.0",
-        "@rollup/rollup-linux-x64-gnu": "4.28.0",
-        "@rollup/rollup-linux-x64-musl": "4.28.0",
-        "@rollup/rollup-win32-arm64-msvc": "4.28.0",
-        "@rollup/rollup-win32-ia32-msvc": "4.28.0",
-        "@rollup/rollup-win32-x64-msvc": "4.28.0",
+        "@rollup/rollup-android-arm-eabi": "4.34.6",
+        "@rollup/rollup-android-arm64": "4.34.6",
+        "@rollup/rollup-darwin-arm64": "4.34.6",
+        "@rollup/rollup-darwin-x64": "4.34.6",
+        "@rollup/rollup-freebsd-arm64": "4.34.6",
+        "@rollup/rollup-freebsd-x64": "4.34.6",
+        "@rollup/rollup-linux-arm-gnueabihf": "4.34.6",
+        "@rollup/rollup-linux-arm-musleabihf": "4.34.6",
+        "@rollup/rollup-linux-arm64-gnu": "4.34.6",
+        "@rollup/rollup-linux-arm64-musl": "4.34.6",
+        "@rollup/rollup-linux-loongarch64-gnu": "4.34.6",
+        "@rollup/rollup-linux-powerpc64le-gnu": "4.34.6",
+        "@rollup/rollup-linux-riscv64-gnu": "4.34.6",
+        "@rollup/rollup-linux-s390x-gnu": "4.34.6",
+        "@rollup/rollup-linux-x64-gnu": "4.34.6",
+        "@rollup/rollup-linux-x64-musl": "4.34.6",
+        "@rollup/rollup-win32-arm64-msvc": "4.34.6",
+        "@rollup/rollup-win32-ia32-msvc": "4.34.6",
+        "@rollup/rollup-win32-x64-msvc": "4.34.6",
         "fsevents": "~2.3.2"
       }
     },
@@ -1066,7 +1048,6 @@
       "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
       "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==",
       "dev": true,
-      "license": "BSD-3-Clause",
       "engines": {
         "node": ">=0.10.0"
       }
@@ -1101,15 +1082,14 @@
       }
     },
     "node_modules/vite": {
-      "version": "6.0.2",
-      "resolved": "https://registry.npmjs.org/vite/-/vite-6.0.2.tgz",
-      "integrity": "sha512-XdQ+VsY2tJpBsKGs0wf3U/+azx8BBpYRHFAyKm5VeEZNOJZRB63q7Sc8Iup3k0TrN3KO6QgyzFf+opSbfY1y0g==",
+      "version": "6.1.0",
+      "resolved": "https://registry.npmjs.org/vite/-/vite-6.1.0.tgz",
+      "integrity": "sha512-RjjMipCKVoR4hVfPY6GQTgveinjNuyLw+qruksLDvA5ktI1150VmcMBKmQaEWJhg/j6Uaf6dNCNA0AfdzUb/hQ==",
       "dev": true,
-      "license": "MIT",
       "dependencies": {
-        "esbuild": "^0.24.0",
-        "postcss": "^8.4.49",
-        "rollup": "^4.23.0"
+        "esbuild": "^0.24.2",
+        "postcss": "^8.5.1",
+        "rollup": "^4.30.1"
       },
       "bin": {
         "vite": "bin/vite.js"

examples/simple-chatbot/client/javascript/package.json

@@ -12,7 +12,7 @@
   "license": "ISC",
   "description": "",
   "devDependencies": {
-    "vite": "^6.0.2"
+    "vite": "^6.0.9"
   },
   "dependencies": {
     "@pipecat-ai/client-js": "^0.3.2",

examples/simple-chatbot/client/react/package-lock.json

@@ -25,7 +25,7 @@
         "globals": "^15.12.0",
         "typescript": "~5.6.2",
         "typescript-eslint": "^8.15.0",
-        "vite": "^6.0.1"
+        "vite": "^6.0.9"
       }
     },
     "node_modules/@ampproject/remapping": {
@@ -353,14 +353,13 @@
       }
     },
     "node_modules/@esbuild/aix-ppc64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.24.0.tgz",
-      "integrity": "sha512-WtKdFM7ls47zkKHFVzMz8opM7LkcsIp9amDUBIAWirg70RM71WRSjdILPsY5Uv1D42ZpUfaPILDlfactHgsRkw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.24.2.tgz",
+      "integrity": "sha512-thpVCb/rhxE/BnMLQ7GReQLLN8q9qbHmI55F4489/ByVg2aQaQ6kbcLb6FHkocZzQhxc4gx0sCk0tJkKBFzDhA==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "aix"
@@ -370,14 +369,13 @@
       }
     },
     "node_modules/@esbuild/android-arm": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.24.0.tgz",
-      "integrity": "sha512-arAtTPo76fJ/ICkXWetLCc9EwEHKaeya4vMrReVlEIUCAUncH7M4bhMQ+M9Vf+FFOZJdTNMXNBrWwW+OXWpSew==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.24.2.tgz",
+      "integrity": "sha512-tmwl4hJkCfNHwFB3nBa8z1Uy3ypZpxqxfTQOcHX+xRByyYgunVbZ9MzUUfb0RxaHIMnbHagwAxuTL+tnNM+1/Q==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -387,14 +385,13 @@
       }
     },
     "node_modules/@esbuild/android-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.24.0.tgz",
-      "integrity": "sha512-Vsm497xFM7tTIPYK9bNTYJyF/lsP590Qc1WxJdlB6ljCbdZKU9SY8i7+Iin4kyhV/KV5J2rOKsBQbB77Ab7L/w==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.24.2.tgz",
+      "integrity": "sha512-cNLgeqCqV8WxfcTIOeL4OAtSmL8JjcN6m09XIgro1Wi7cF4t/THaWEa7eL5CMoMBdjoHOTh/vwTO/o2TRXIyzg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -404,14 +401,13 @@
       }
     },
     "node_modules/@esbuild/android-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.24.0.tgz",
-      "integrity": "sha512-t8GrvnFkiIY7pa7mMgJd7p8p8qqYIz1NYiAoKc75Zyv73L3DZW++oYMSHPRarcotTKuSs6m3hTOa5CKHaS02TQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.24.2.tgz",
+      "integrity": "sha512-B6Q0YQDqMx9D7rvIcsXfmJfvUYLoP722bgfBlO5cGvNVb5V/+Y7nhBE3mHV9OpxBf4eAS2S68KZztiPaWq4XYw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
@@ -421,14 +417,13 @@
       }
     },
     "node_modules/@esbuild/darwin-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.0.tgz",
-      "integrity": "sha512-CKyDpRbK1hXwv79soeTJNHb5EiG6ct3efd/FTPdzOWdbZZfGhpbcqIpiD0+vwmpu0wTIL97ZRPZu8vUt46nBSw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.24.2.tgz",
+      "integrity": "sha512-kj3AnYWc+CekmZnS5IPu9D+HWtUI49hbnyqk0FLEJDbzCIQt7hg7ucF1SQAilhtYpIujfaHr6O0UHlzzSPdOeA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -438,14 +433,13 @@
       }
     },
     "node_modules/@esbuild/darwin-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.24.0.tgz",
-      "integrity": "sha512-rgtz6flkVkh58od4PwTRqxbKH9cOjaXCMZgWD905JOzjFKW+7EiUObfd/Kav+A6Gyud6WZk9w+xu6QLytdi2OA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.24.2.tgz",
+      "integrity": "sha512-WeSrmwwHaPkNR5H3yYfowhZcbriGqooyu3zI/3GGpF8AyUdsrrP0X6KumITGA9WOyiJavnGZUwPGvxvwfWPHIA==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -455,14 +449,13 @@
       }
     },
     "node_modules/@esbuild/freebsd-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.24.0.tgz",
-      "integrity": "sha512-6Mtdq5nHggwfDNLAHkPlyLBpE5L6hwsuXZX8XNmHno9JuL2+bg2BX5tRkwjyfn6sKbxZTq68suOjgWqCicvPXA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-UN8HXjtJ0k/Mj6a9+5u6+2eZ2ERD7Edt1Q9IZiB5UZAIdPnVKDoG7mdTVGhHJIeEml60JteamR3qhsr1r8gXvg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
@@ -472,14 +465,13 @@
       }
     },
     "node_modules/@esbuild/freebsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.24.0.tgz",
-      "integrity": "sha512-D3H+xh3/zphoX8ck4S2RxKR6gHlHDXXzOf6f/9dbFt/NRBDIE33+cVa49Kil4WUjxMGW0ZIYBYtaGCa2+OsQwQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.24.2.tgz",
+      "integrity": "sha512-TvW7wE/89PYW+IevEJXZ5sF6gJRDY/14hyIGFXdIucxCsbRmLUcjseQu1SyTko+2idmCw94TgyaEZi9HUSOe3Q==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
@@ -489,14 +481,13 @@
       }
     },
     "node_modules/@esbuild/linux-arm": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.24.0.tgz",
-      "integrity": "sha512-gJKIi2IjRo5G6Glxb8d3DzYXlxdEj2NlkixPsqePSZMhLudqPhtZ4BUrpIuTjJYXxvF9njql+vRjB2oaC9XpBw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.24.2.tgz",
+      "integrity": "sha512-n0WRM/gWIdU29J57hJyUdIsk0WarGd6To0s+Y+LwvlC55wt+GT/OgkwoXCXvIue1i1sSNWblHEig00GBWiJgfA==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -506,14 +497,13 @@
       }
     },
     "node_modules/@esbuild/linux-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.24.0.tgz",
-      "integrity": "sha512-TDijPXTOeE3eaMkRYpcy3LarIg13dS9wWHRdwYRnzlwlA370rNdZqbcp0WTyyV/k2zSxfko52+C7jU5F9Tfj1g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.24.2.tgz",
+      "integrity": "sha512-7HnAD6074BW43YvvUmE/35Id9/NB7BeX5EoNkK9obndmZBUk8xmJJeU7DwmUeN7tkysslb2eSl6CTrYz6oEMQg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -523,14 +513,13 @@
       }
     },
     "node_modules/@esbuild/linux-ia32": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.24.0.tgz",
-      "integrity": "sha512-K40ip1LAcA0byL05TbCQ4yJ4swvnbzHscRmUilrmP9Am7//0UjPreh4lpYzvThT2Quw66MhjG//20mrufm40mA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.24.2.tgz",
+      "integrity": "sha512-sfv0tGPQhcZOgTKO3oBE9xpHuUqguHvSo4jl+wjnKwFpapx+vUDcawbwPNuBIAYdRAvIDBfZVvXprIj3HA+Ugw==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -540,14 +529,13 @@
       }
     },
     "node_modules/@esbuild/linux-loong64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.24.0.tgz",
-      "integrity": "sha512-0mswrYP/9ai+CU0BzBfPMZ8RVm3RGAN/lmOMgW4aFUSOQBjA31UP8Mr6DDhWSuMwj7jaWOT0p0WoZ6jeHhrD7g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.24.2.tgz",
+      "integrity": "sha512-CN9AZr8kEndGooS35ntToZLTQLHEjtVB5n7dl8ZcTZMonJ7CCfStrYhrzF97eAecqVbVJ7APOEe18RPI4KLhwQ==",
       "cpu": [
         "loong64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -557,14 +545,13 @@
       }
     },
     "node_modules/@esbuild/linux-mips64el": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.24.0.tgz",
-      "integrity": "sha512-hIKvXm0/3w/5+RDtCJeXqMZGkI2s4oMUGj3/jM0QzhgIASWrGO5/RlzAzm5nNh/awHE0A19h/CvHQe6FaBNrRA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.24.2.tgz",
+      "integrity": "sha512-iMkk7qr/wl3exJATwkISxI7kTcmHKE+BlymIAbHO8xanq/TjHaaVThFF6ipWzPHryoFsesNQJPE/3wFJw4+huw==",
       "cpu": [
         "mips64el"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -574,14 +561,13 @@
       }
     },
     "node_modules/@esbuild/linux-ppc64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.24.0.tgz",
-      "integrity": "sha512-HcZh5BNq0aC52UoocJxaKORfFODWXZxtBaaZNuN3PUX3MoDsChsZqopzi5UupRhPHSEHotoiptqikjN/B77mYQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.24.2.tgz",
+      "integrity": "sha512-shsVrgCZ57Vr2L8mm39kO5PPIb+843FStGt7sGGoqiiWYconSxwTiuswC1VJZLCjNiMLAMh34jg4VSEQb+iEbw==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -591,14 +577,13 @@
       }
     },
     "node_modules/@esbuild/linux-riscv64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.24.0.tgz",
-      "integrity": "sha512-bEh7dMn/h3QxeR2KTy1DUszQjUrIHPZKyO6aN1X4BCnhfYhuQqedHaa5MxSQA/06j3GpiIlFGSsy1c7Gf9padw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.24.2.tgz",
+      "integrity": "sha512-4eSFWnU9Hhd68fW16GD0TINewo1L6dRrB+oLNNbYyMUAeOD2yCK5KXGK1GH4qD/kT+bTEXjsyTCiJGHPZ3eM9Q==",
       "cpu": [
         "riscv64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -608,14 +593,13 @@
       }
     },
     "node_modules/@esbuild/linux-s390x": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.24.0.tgz",
-      "integrity": "sha512-ZcQ6+qRkw1UcZGPyrCiHHkmBaj9SiCD8Oqd556HldP+QlpUIe2Wgn3ehQGVoPOvZvtHm8HPx+bH20c9pvbkX3g==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.24.2.tgz",
+      "integrity": "sha512-S0Bh0A53b0YHL2XEXC20bHLuGMOhFDO6GN4b3YjRLK//Ep3ql3erpNcPlEFed93hsQAjAQDNsvcK+hV90FubSw==",
       "cpu": [
         "s390x"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -625,14 +609,13 @@
       }
     },
     "node_modules/@esbuild/linux-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.24.0.tgz",
-      "integrity": "sha512-vbutsFqQ+foy3wSSbmjBXXIJ6PL3scghJoM8zCL142cGaZKAdCZHyf+Bpu/MmX9zT9Q0zFBVKb36Ma5Fzfa8xA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.24.2.tgz",
+      "integrity": "sha512-8Qi4nQcCTbLnK9WoMjdC9NiTG6/E38RNICU6sUNqK0QFxCYgoARqVqxdFmWkdonVsvGqWhmm7MO0jyTqLqwj0Q==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
@@ -641,15 +624,30 @@
         "node": ">=18"
       }
     },
+    "node_modules/@esbuild/netbsd-arm64": {
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-wuLK/VztRRpMt9zyHSazyCVdCXlpHkKm34WUyinD2lzK07FAHTq0KQvZZlXikNWkDGoT6x3TD51jKQ7gMVpopw==",
+      "cpu": [
+        "arm64"
+      ],
+      "dev": true,
+      "optional": true,
+      "os": [
+        "netbsd"
+      ],
+      "engines": {
+        "node": ">=18"
+      }
+    },
     "node_modules/@esbuild/netbsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.24.0.tgz",
-      "integrity": "sha512-hjQ0R/ulkO8fCYFsG0FZoH+pWgTTDreqpqY7UnQntnaKv95uP5iW3+dChxnx7C3trQQU40S+OgWhUVwCjVFLvg==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.24.2.tgz",
+      "integrity": "sha512-VefFaQUc4FMmJuAxmIHgUmfNiLXY438XrL4GDNV1Y1H/RW3qow68xTwjZKfj/+Plp9NANmzbH5R40Meudu8mmw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "netbsd"
@@ -659,14 +657,13 @@
       }
     },
     "node_modules/@esbuild/openbsd-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.24.0.tgz",
-      "integrity": "sha512-MD9uzzkPQbYehwcN583yx3Tu5M8EIoTD+tUgKF982WYL9Pf5rKy9ltgD0eUgs8pvKnmizxjXZyLt0z6DC3rRXg==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.24.2.tgz",
+      "integrity": "sha512-YQbi46SBct6iKnszhSvdluqDmxCJA+Pu280Av9WICNwQmMxV7nLRHZfjQzwbPs3jeWnuAhE9Jy0NrnJ12Oz+0A==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "openbsd"
@@ -676,14 +673,13 @@
       }
     },
     "node_modules/@esbuild/openbsd-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.24.0.tgz",
-      "integrity": "sha512-4ir0aY1NGUhIC1hdoCzr1+5b43mw99uNwVzhIq1OY3QcEwPDO3B7WNXBzaKY5Nsf1+N11i1eOfFcq+D/gOS15Q==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.24.2.tgz",
+      "integrity": "sha512-+iDS6zpNM6EnJyWv0bMGLWSWeXGN/HTaF/LXHXHwejGsVi+ooqDfMCCTerNFxEkM3wYVcExkeGXNqshc9iMaOA==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "openbsd"
@@ -693,14 +689,13 @@
       }
     },
     "node_modules/@esbuild/sunos-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.24.0.tgz",
-      "integrity": "sha512-jVzdzsbM5xrotH+W5f1s+JtUy1UWgjU0Cf4wMvffTB8m6wP5/kx0KiaLHlbJO+dMgtxKV8RQ/JvtlFcdZ1zCPA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.24.2.tgz",
+      "integrity": "sha512-hTdsW27jcktEvpwNHJU4ZwWFGkz2zRJUz8pvddmXPtXDzVKTTINmlmga3ZzwcuMpUvLw7JkLy9QLKyGpD2Yxig==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "sunos"
@@ -710,14 +705,13 @@
       }
     },
     "node_modules/@esbuild/win32-arm64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.24.0.tgz",
-      "integrity": "sha512-iKc8GAslzRpBytO2/aN3d2yb2z8XTVfNV0PjGlCxKo5SgWmNXx82I/Q3aG1tFfS+A2igVCY97TJ8tnYwpUWLCA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.24.2.tgz",
+      "integrity": "sha512-LihEQ2BBKVFLOC9ZItT9iFprsE9tqjDjnbulhHoFxYQtQfai7qfluVODIYxt1PgdoyQkz23+01rzwNwYfutxUQ==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -727,14 +721,13 @@
       }
     },
     "node_modules/@esbuild/win32-ia32": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.24.0.tgz",
-      "integrity": "sha512-vQW36KZolfIudCcTnaTpmLQ24Ha1RjygBo39/aLkM2kmjkWmZGEJ5Gn9l5/7tzXA42QGIoWbICfg6KLLkIw6yw==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.24.2.tgz",
+      "integrity": "sha512-q+iGUwfs8tncmFC9pcnD5IvRHAzmbwQ3GPS5/ceCyHdjXubwQWI12MKWSNSMYLJMq23/IUCvJMS76PDqXe1fxA==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -744,14 +737,13 @@
       }
     },
     "node_modules/@esbuild/win32-x64": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.24.0.tgz",
-      "integrity": "sha512-7IAFPrjSQIJrGsK6flwg7NFmwBoSTyF3rl7If0hNUFQU4ilTsEPL6GuMuU9BfIWVVGuRnuIidkSMC+c0Otu8IA==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.24.2.tgz",
+      "integrity": "sha512-7VTgWzgMGvup6aSqDPLiW5zHaxYJGTO4OokMjIlrCtf+VpEL+cXKtCvg723iguPYI5oaUNdS+/V7OU2gvXVWEg==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -1097,252 +1089,247 @@
       }
     },
     "node_modules/@rollup/rollup-android-arm-eabi": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.28.0.tgz",
-      "integrity": "sha512-wLJuPLT6grGZsy34g4N1yRfYeouklTgPhH1gWXCYspenKYD0s3cR99ZevOGw5BexMNywkbV3UkjADisozBmpPQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.34.6.tgz",
+      "integrity": "sha512-+GcCXtOQoWuC7hhX1P00LqjjIiS/iOouHXhMdiDSnq/1DGTox4SpUvO52Xm+div6+106r+TcvOeo/cxvyEyTgg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
       ]
     },
     "node_modules/@rollup/rollup-android-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.28.0.tgz",
-      "integrity": "sha512-eiNkznlo0dLmVG/6wf+Ifi/v78G4d4QxRhuUl+s8EWZpDewgk7PX3ZyECUXU0Zq/Ca+8nU8cQpNC4Xgn2gFNDA==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.34.6.tgz",
+      "integrity": "sha512-E8+2qCIjciYUnCa1AiVF1BkRgqIGW9KzJeesQqVfyRITGQN+dFuoivO0hnro1DjT74wXLRZ7QF8MIbz+luGaJA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "android"
       ]
     },
     "node_modules/@rollup/rollup-darwin-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.28.0.tgz",
-      "integrity": "sha512-lmKx9yHsppblnLQZOGxdO66gT77bvdBtr/0P+TPOseowE7D9AJoBw8ZDULRasXRWf1Z86/gcOdpBrV6VDUY36Q==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.34.6.tgz",
+      "integrity": "sha512-z9Ib+OzqN3DZEjX7PDQMHEhtF+t6Mi2z/ueChQPLS/qUMKY7Ybn5A2ggFoKRNRh1q1T03YTQfBTQCJZiepESAg==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
       ]
     },
     "node_modules/@rollup/rollup-darwin-x64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.28.0.tgz",
-      "integrity": "sha512-8hxgfReVs7k9Js1uAIhS6zq3I+wKQETInnWQtgzt8JfGx51R1N6DRVy3F4o0lQwumbErRz52YqwjfvuwRxGv1w==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.34.6.tgz",
+      "integrity": "sha512-PShKVY4u0FDAR7jskyFIYVyHEPCPnIQY8s5OcXkdU8mz3Y7eXDJPdyM/ZWjkYdR2m0izD9HHWA8sGcXn+Qrsyg==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
       ]
     },
     "node_modules/@rollup/rollup-freebsd-arm64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.28.0.tgz",
-      "integrity": "sha512-lA1zZB3bFx5oxu9fYud4+g1mt+lYXCoch0M0V/xhqLoGatbzVse0wlSQ1UYOWKpuSu3gyN4qEc0Dxf/DII1bhQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.34.6.tgz",
+      "integrity": "sha512-YSwyOqlDAdKqs0iKuqvRHLN4SrD2TiswfoLfvYXseKbL47ht1grQpq46MSiQAx6rQEN8o8URtpXARCpqabqxGQ==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
       ]
     },
     "node_modules/@rollup/rollup-freebsd-x64": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.28.0.tgz",
-      "integrity": "sha512-aI2plavbUDjCQB/sRbeUZWX9qp12GfYkYSJOrdYTL/C5D53bsE2/nBPuoiJKoWp5SN78v2Vr8ZPnB+/VbQ2pFA==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.34.6.tgz",
+      "integrity": "sha512-HEP4CgPAY1RxXwwL5sPFv6BBM3tVeLnshF03HMhJYCNc6kvSqBgTMmsEjb72RkZBAWIqiPUyF1JpEBv5XT9wKQ==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "freebsd"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm-gnueabihf": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.28.0.tgz",
-      "integrity": "sha512-WXveUPKtfqtaNvpf0iOb0M6xC64GzUX/OowbqfiCSXTdi/jLlOmH0Ba94/OkiY2yTGTwteo4/dsHRfh5bDCZ+w==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.34.6.tgz",
+      "integrity": "sha512-88fSzjC5xeH9S2Vg3rPgXJULkHcLYMkh8faix8DX4h4TIAL65ekwuQMA/g2CXq8W+NJC43V6fUpYZNjaX3+IIg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm-musleabihf": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.28.0.tgz",
-      "integrity": "sha512-yLc3O2NtOQR67lI79zsSc7lk31xjwcaocvdD1twL64PK1yNaIqCeWI9L5B4MFPAVGEVjH5k1oWSGuYX1Wutxpg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.34.6.tgz",
+      "integrity": "sha512-wM4ztnutBqYFyvNeR7Av+reWI/enK9tDOTKNF+6Kk2Q96k9bwhDDOlnCUNRPvromlVXo04riSliMBs/Z7RteEg==",
       "cpu": [
         "arm"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.28.0.tgz",
-      "integrity": "sha512-+P9G9hjEpHucHRXqesY+3X9hD2wh0iNnJXX/QhS/J5vTdG6VhNYMxJ2rJkQOxRUd17u5mbMLHM7yWGZdAASfcg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.34.6.tgz",
+      "integrity": "sha512-9RyprECbRa9zEjXLtvvshhw4CMrRa3K+0wcp3KME0zmBe1ILmvcVHnypZ/aIDXpRyfhSYSuN4EPdCCj5Du8FIA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-arm64-musl": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.28.0.tgz",
-      "integrity": "sha512-1xsm2rCKSTpKzi5/ypT5wfc+4bOGa/9yI/eaOLW0oMs7qpC542APWhl4A37AENGZ6St6GBMWhCCMM6tXgTIplw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.34.6.tgz",
+      "integrity": "sha512-qTmklhCTyaJSB05S+iSovfo++EwnIEZxHkzv5dep4qoszUMX5Ca4WM4zAVUMbfdviLgCSQOu5oU8YoGk1s6M9Q==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
+      "optional": true,
+      "os": [
+        "linux"
+      ]
+    },
+    "node_modules/@rollup/rollup-linux-loongarch64-gnu": {
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loongarch64-gnu/-/rollup-linux-loongarch64-gnu-4.34.6.tgz",
+      "integrity": "sha512-4Qmkaps9yqmpjY5pvpkfOerYgKNUGzQpFxV6rnS7c/JfYbDSU0y6WpbbredB5cCpLFGJEqYX40WUmxMkwhWCjw==",
+      "cpu": [
+        "loong64"
+      ],
+      "dev": true,
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-powerpc64le-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-powerpc64le-gnu/-/rollup-linux-powerpc64le-gnu-4.28.0.tgz",
-      "integrity": "sha512-zgWxMq8neVQeXL+ouSf6S7DoNeo6EPgi1eeqHXVKQxqPy1B2NvTbaOUWPn/7CfMKL7xvhV0/+fq/Z/J69g1WAQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-powerpc64le-gnu/-/rollup-linux-powerpc64le-gnu-4.34.6.tgz",
+      "integrity": "sha512-Zsrtux3PuaxuBTX/zHdLaFmcofWGzaWW1scwLU3ZbW/X+hSsFbz9wDIp6XvnT7pzYRl9MezWqEqKy7ssmDEnuQ==",
       "cpu": [
         "ppc64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-riscv64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.28.0.tgz",
-      "integrity": "sha512-VEdVYacLniRxbRJLNtzwGt5vwS0ycYshofI7cWAfj7Vg5asqj+pt+Q6x4n+AONSZW/kVm+5nklde0qs2EUwU2g==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.34.6.tgz",
+      "integrity": "sha512-aK+Zp+CRM55iPrlyKiU3/zyhgzWBxLVrw2mwiQSYJRobCURb781+XstzvA8Gkjg/hbdQFuDw44aUOxVQFycrAg==",
       "cpu": [
         "riscv64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-s390x-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.28.0.tgz",
-      "integrity": "sha512-LQlP5t2hcDJh8HV8RELD9/xlYtEzJkm/aWGsauvdO2ulfl3QYRjqrKW+mGAIWP5kdNCBheqqqYIGElSRCaXfpw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.34.6.tgz",
+      "integrity": "sha512-WoKLVrY9ogmaYPXwTH326+ErlCIgMmsoRSx6bO+l68YgJnlOXhygDYSZe/qbUJCSiCiZAQ+tKm88NcWuUXqOzw==",
       "cpu": [
         "s390x"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-x64-gnu": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.28.0.tgz",
-      "integrity": "sha512-Nl4KIzteVEKE9BdAvYoTkW19pa7LR/RBrT6F1dJCV/3pbjwDcaOq+edkP0LXuJ9kflW/xOK414X78r+K84+msw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.34.6.tgz",
+      "integrity": "sha512-Sht4aFvmA4ToHd2vFzwMFaQCiYm2lDFho5rPcvPBT5pCdC+GwHG6CMch4GQfmWTQ1SwRKS0dhDYb54khSrjDWw==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-linux-x64-musl": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.28.0.tgz",
-      "integrity": "sha512-eKpJr4vBDOi4goT75MvW+0dXcNUqisK4jvibY9vDdlgLx+yekxSm55StsHbxUsRxSTt3JEQvlr3cGDkzcSP8bw==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.34.6.tgz",
+      "integrity": "sha512-zmmpOQh8vXc2QITsnCiODCDGXFC8LMi64+/oPpPx5qz3pqv0s6x46ps4xoycfUiVZps5PFn1gksZzo4RGTKT+A==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "linux"
       ]
     },
     "node_modules/@rollup/rollup-win32-arm64-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.28.0.tgz",
-      "integrity": "sha512-Vi+WR62xWGsE/Oj+mD0FNAPY2MEox3cfyG0zLpotZdehPFXwz6lypkGs5y38Jd/NVSbOD02aVad6q6QYF7i8Bg==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.34.6.tgz",
+      "integrity": "sha512-3/q1qUsO/tLqGBaD4uXsB6coVGB3usxw3qyeVb59aArCgedSF66MPdgRStUd7vbZOsko/CgVaY5fo2vkvPLWiA==",
       "cpu": [
         "arm64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
       ]
     },
     "node_modules/@rollup/rollup-win32-ia32-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.28.0.tgz",
-      "integrity": "sha512-kN/Vpip8emMLn/eOza+4JwqDZBL6MPNpkdaEsgUtW1NYN3DZvZqSQrbKzJcTL6hd8YNmFTn7XGWMwccOcJBL0A==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.34.6.tgz",
+      "integrity": "sha512-oLHxuyywc6efdKVTxvc0135zPrRdtYVjtVD5GUm55I3ODxhU/PwkQFD97z16Xzxa1Fz0AEe4W/2hzRtd+IfpOA==",
       "cpu": [
         "ia32"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
       ]
     },
     "node_modules/@rollup/rollup-win32-x64-msvc": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.28.0.tgz",
-      "integrity": "sha512-Bvno2/aZT6usSa7lRDL2+hMjVAGjuqaymF1ApZm31JXzniR/hvr14jpU+/z4X6Gt5BPlzosscyJZGUvguXIqeQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.34.6.tgz",
+      "integrity": "sha512-0PVwmgzZ8+TZ9oGBmdZoQVXflbvuwzN/HRclujpl4N/q3i+y0lqLw8n1bXA8ru3sApDjlmONaNAuYr38y1Kr9w==",
       "cpu": [
         "x64"
       ],
       "dev": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "win32"
@@ -2066,12 +2053,11 @@
       "license": "ISC"
     },
     "node_modules/esbuild": {
-      "version": "0.24.0",
-      "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.0.tgz",
-      "integrity": "sha512-FuLPevChGDshgSicjisSooU0cemp/sGXR841D5LHMB7mTVOmsEHcAxaH3irL53+8YDIeVNQEySh4DaYU/iuPqQ==",
+      "version": "0.24.2",
+      "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.24.2.tgz",
+      "integrity": "sha512-+9egpBW8I3CD5XPe0n6BfT5fxLzxrlDzqydF3aviG+9ni1lDC/OvMHcxqEFV0+LANZG5R1bFMWfUrjVsdwxJvA==",
       "dev": true,
       "hasInstallScript": true,
-      "license": "MIT",
       "bin": {
         "esbuild": "bin/esbuild"
       },
@@ -2079,30 +2065,31 @@
         "node": ">=18"
       },
       "optionalDependencies": {
-        "@esbuild/aix-ppc64": "0.24.0",
-        "@esbuild/android-arm": "0.24.0",
-        "@esbuild/android-arm64": "0.24.0",
-        "@esbuild/android-x64": "0.24.0",
-        "@esbuild/darwin-arm64": "0.24.0",
-        "@esbuild/darwin-x64": "0.24.0",
-        "@esbuild/freebsd-arm64": "0.24.0",
-        "@esbuild/freebsd-x64": "0.24.0",
-        "@esbuild/linux-arm": "0.24.0",
-        "@esbuild/linux-arm64": "0.24.0",
-        "@esbuild/linux-ia32": "0.24.0",
-        "@esbuild/linux-loong64": "0.24.0",
-        "@esbuild/linux-mips64el": "0.24.0",
-        "@esbuild/linux-ppc64": "0.24.0",
-        "@esbuild/linux-riscv64": "0.24.0",
-        "@esbuild/linux-s390x": "0.24.0",
-        "@esbuild/linux-x64": "0.24.0",
-        "@esbuild/netbsd-x64": "0.24.0",
-        "@esbuild/openbsd-arm64": "0.24.0",
-        "@esbuild/openbsd-x64": "0.24.0",
-        "@esbuild/sunos-x64": "0.24.0",
-        "@esbuild/win32-arm64": "0.24.0",
-        "@esbuild/win32-ia32": "0.24.0",
-        "@esbuild/win32-x64": "0.24.0"
+        "@esbuild/aix-ppc64": "0.24.2",
+        "@esbuild/android-arm": "0.24.2",
+        "@esbuild/android-arm64": "0.24.2",
+        "@esbuild/android-x64": "0.24.2",
+        "@esbuild/darwin-arm64": "0.24.2",
+        "@esbuild/darwin-x64": "0.24.2",
+        "@esbuild/freebsd-arm64": "0.24.2",
+        "@esbuild/freebsd-x64": "0.24.2",
+        "@esbuild/linux-arm": "0.24.2",
+        "@esbuild/linux-arm64": "0.24.2",
+        "@esbuild/linux-ia32": "0.24.2",
+        "@esbuild/linux-loong64": "0.24.2",
+        "@esbuild/linux-mips64el": "0.24.2",
+        "@esbuild/linux-ppc64": "0.24.2",
+        "@esbuild/linux-riscv64": "0.24.2",
+        "@esbuild/linux-s390x": "0.24.2",
+        "@esbuild/linux-x64": "0.24.2",
+        "@esbuild/netbsd-arm64": "0.24.2",
+        "@esbuild/netbsd-x64": "0.24.2",
+        "@esbuild/openbsd-arm64": "0.24.2",
+        "@esbuild/openbsd-x64": "0.24.2",
+        "@esbuild/sunos-x64": "0.24.2",
+        "@esbuild/win32-arm64": "0.24.2",
+        "@esbuild/win32-ia32": "0.24.2",
+        "@esbuild/win32-x64": "0.24.2"
       }
     },
     "node_modules/escalade": {
@@ -2445,7 +2432,6 @@
       "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
       "dev": true,
       "hasInstallScript": true,
-      "license": "MIT",
       "optional": true,
       "os": [
         "darwin"
@@ -2825,7 +2811,6 @@
           "url": "https://github.com/sponsors/ai"
         }
       ],
-      "license": "MIT",
       "bin": {
         "nanoid": "bin/nanoid.cjs"
       },
@@ -2951,9 +2936,9 @@
       }
     },
     "node_modules/postcss": {
-      "version": "8.4.49",
-      "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.49.tgz",
-      "integrity": "sha512-OCVPnIObs4N29kxTjzLfUryOkvZEq+pf8jTF0lg8E7uETuWHA+v7j3c/xJmiqpX450191LlmZfUKkXxkTry7nA==",
+      "version": "8.5.2",
+      "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.2.tgz",
+      "integrity": "sha512-MjOadfU3Ys9KYoX0AdkBlFEF1Vx37uCCeN4ZHnmwm9FfpbsGWMZeBLMmmpY+6Ocqod7mkdZ0DT31OlbsFrLlkA==",
       "dev": true,
       "funding": [
         {
@@ -2969,9 +2954,8 @@
           "url": "https://github.com/sponsors/ai"
         }
       ],
-      "license": "MIT",
       "dependencies": {
-        "nanoid": "^3.3.7",
+        "nanoid": "^3.3.8",
         "picocolors": "^1.1.1",
         "source-map-js": "^1.2.1"
       },
@@ -3083,11 +3067,10 @@
       }
     },
     "node_modules/rollup": {
-      "version": "4.28.0",
-      "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.28.0.tgz",
-      "integrity": "sha512-G9GOrmgWHBma4YfCcX8PjH0qhXSdH8B4HDE2o4/jaxj93S4DPCIDoLcXz99eWMji4hB29UFCEd7B2gwGJDR9cQ==",
+      "version": "4.34.6",
+      "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.34.6.tgz",
+      "integrity": "sha512-wc2cBWqJgkU3Iz5oztRkQbfVkbxoz5EhnCGOrnJvnLnQ7O0WhQUYyv18qQI79O8L7DdHrrlJNeCHd4VGpnaXKQ==",
       "dev": true,
-      "license": "MIT",
       "dependencies": {
         "@types/estree": "1.0.6"
       },
@@ -3099,24 +3082,25 @@
         "npm": ">=8.0.0"
       },
       "optionalDependencies": {
-        "@rollup/rollup-android-arm-eabi": "4.28.0",
-        "@rollup/rollup-android-arm64": "4.28.0",
-        "@rollup/rollup-darwin-arm64": "4.28.0",
-        "@rollup/rollup-darwin-x64": "4.28.0",
-        "@rollup/rollup-freebsd-arm64": "4.28.0",
-        "@rollup/rollup-freebsd-x64": "4.28.0",
-        "@rollup/rollup-linux-arm-gnueabihf": "4.28.0",
-        "@rollup/rollup-linux-arm-musleabihf": "4.28.0",
-        "@rollup/rollup-linux-arm64-gnu": "4.28.0",
-        "@rollup/rollup-linux-arm64-musl": "4.28.0",
-        "@rollup/rollup-linux-powerpc64le-gnu": "4.28.0",
-        "@rollup/rollup-linux-riscv64-gnu": "4.28.0",
-        "@rollup/rollup-linux-s390x-gnu": "4.28.0",
-        "@rollup/rollup-linux-x64-gnu": "4.28.0",
-        "@rollup/rollup-linux-x64-musl": "4.28.0",
-        "@rollup/rollup-win32-arm64-msvc": "4.28.0",
-        "@rollup/rollup-win32-ia32-msvc": "4.28.0",
-        "@rollup/rollup-win32-x64-msvc": "4.28.0",
+        "@rollup/rollup-android-arm-eabi": "4.34.6",
+        "@rollup/rollup-android-arm64": "4.34.6",
+        "@rollup/rollup-darwin-arm64": "4.34.6",
+        "@rollup/rollup-darwin-x64": "4.34.6",
+        "@rollup/rollup-freebsd-arm64": "4.34.6",
+        "@rollup/rollup-freebsd-x64": "4.34.6",
+        "@rollup/rollup-linux-arm-gnueabihf": "4.34.6",
+        "@rollup/rollup-linux-arm-musleabihf": "4.34.6",
+        "@rollup/rollup-linux-arm64-gnu": "4.34.6",
+        "@rollup/rollup-linux-arm64-musl": "4.34.6",
+        "@rollup/rollup-linux-loongarch64-gnu": "4.34.6",
+        "@rollup/rollup-linux-powerpc64le-gnu": "4.34.6",
+        "@rollup/rollup-linux-riscv64-gnu": "4.34.6",
+        "@rollup/rollup-linux-s390x-gnu": "4.34.6",
+        "@rollup/rollup-linux-x64-gnu": "4.34.6",
+        "@rollup/rollup-linux-x64-musl": "4.34.6",
+        "@rollup/rollup-win32-arm64-msvc": "4.34.6",
+        "@rollup/rollup-win32-ia32-msvc": "4.34.6",
+        "@rollup/rollup-win32-x64-msvc": "4.34.6",
         "fsevents": "~2.3.2"
       }
     },
@@ -3213,7 +3197,6 @@
       "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
       "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==",
       "dev": true,
-      "license": "BSD-3-Clause",
       "engines": {
         "node": ">=0.10.0"
       }
@@ -3395,15 +3378,14 @@
       }
     },
     "node_modules/vite": {
-      "version": "6.0.2",
-      "resolved": "https://registry.npmjs.org/vite/-/vite-6.0.2.tgz",
-      "integrity": "sha512-XdQ+VsY2tJpBsKGs0wf3U/+azx8BBpYRHFAyKm5VeEZNOJZRB63q7Sc8Iup3k0TrN3KO6QgyzFf+opSbfY1y0g==",
+      "version": "6.1.0",
+      "resolved": "https://registry.npmjs.org/vite/-/vite-6.1.0.tgz",
+      "integrity": "sha512-RjjMipCKVoR4hVfPY6GQTgveinjNuyLw+qruksLDvA5ktI1150VmcMBKmQaEWJhg/j6Uaf6dNCNA0AfdzUb/hQ==",
       "dev": true,
-      "license": "MIT",
       "dependencies": {
-        "esbuild": "^0.24.0",
-        "postcss": "^8.4.49",
-        "rollup": "^4.23.0"
+        "esbuild": "^0.24.2",
+        "postcss": "^8.5.1",
+        "rollup": "^4.30.1"
       },
       "bin": {
         "vite": "bin/vite.js"

examples/simple-chatbot/client/react/package.json

@@ -27,6 +27,6 @@
     "globals": "^15.12.0",
     "typescript": "~5.6.2",
     "typescript-eslint": "^8.15.0",
-    "vite": "^6.0.1"
+    "vite": "^6.0.9"
   }
 }

pyproject.toml

@@ -55,7 +55,7 @@ elevenlabs = [ "websockets~=13.1" ]
 fal = [ "fal-client~=0.5.6" ]
 fish = [ "ormsgpack~=1.7.0", "websockets~=13.1" ]
 gladia = [ "websockets~=13.1" ]
-google = [ "google-generativeai~=0.8.3", "google-cloud-texttospeech~=2.24.0", "google-genai~=1.0.0" ]
+google = [ "google-cloud-speech~=2.31.0", "google-cloud-texttospeech~=2.25.0", "google-genai~=1.2.0", "google-generativeai~=0.8.4" ]
 grok = [ "openai~=1.59.6" ]
 groq = [ "openai~=1.59.6" ]
 gstreamer = [ "pygobject~=3.50.0" ]
@@ -73,6 +73,7 @@ openai = [ "openai~=1.59.6", "websockets~=13.1", "python-deepcompare~=2.1.0" ]
 openpipe = [ "openpipe~=4.45.0" ]
 perplexity = [ "openai~=1.59.6" ]
 playht = [ "pyht~=0.1.6", "websockets~=13.1" ]
+rime = [ "websockets~=13.1" ]
 riva = [ "nvidia-riva-client~=2.18.0" ]
 sentry = [ "sentry-sdk~=2.20.0" ]
 silero = [ "onnxruntime~=1.20.1" ]
@@ -112,3 +113,8 @@ select = [
 
 [tool.ruff.lint.pydocstyle]
 convention = "google"
+
+[tool.coverage.run]
+command_line = "--module pytest"
+source = ["src"]
+omit = ["*/tests/*"]

src/pipecat/frames/frames.py

@@ -565,6 +565,22 @@ class UserStoppedSpeakingFrame(SystemFrame):
     pass
 
 
+@dataclass
+class EmulateUserStartedSpeakingFrame(SystemFrame):
+    """Emitted by internal processors upstream to emulate VAD behavior when a
+    user starts speaking."""
+
+    pass
+
+
+@dataclass
+class EmulateUserStoppedSpeakingFrame(SystemFrame):
+    """Emitted by internal processors upstream to emulate VAD behavior when a
+    user stops speaking."""
+
+    pass
+
+
 @dataclass
 class BotInterruptionFrame(SystemFrame):
     """Emitted by when the bot should be interrupted. This will mainly cause the
@@ -618,6 +634,13 @@ class FunctionCallInProgressFrame(SystemFrame):
     arguments: str
 
 
+@dataclass
+class STTMuteFrame(SystemFrame):
+    """System frame to mute/unmute the STT service."""
+
+    mute: bool
+
+
 @dataclass
 class TransportMessageUrgentFrame(SystemFrame):
     message: Any
@@ -752,13 +775,6 @@ class TTSUpdateSettingsFrame(ServiceUpdateSettingsFrame):
     pass
 
 
-@dataclass
-class STTMuteFrame(ControlFrame):
-    """Control frame to mute/unmute the STT service."""
-
-    mute: bool
-
-
 @dataclass
 class STTUpdateSettingsFrame(ServiceUpdateSettingsFrame):
     pass

src/pipecat/processors/aggregators/llm_response.py

@@ -4,9 +4,16 @@
 # SPDX-License-Identifier: BSD 2-Clause License
 #
 
-from typing import List, Optional, Type
+import asyncio
+import time
+from abc import abstractmethod
+from typing import List
 
 from pipecat.frames.frames import (
+    CancelFrame,
+    EmulateUserStartedSpeakingFrame,
+    EmulateUserStoppedSpeakingFrame,
+    EndFrame,
     Frame,
     InterimTranscriptionFrame,
     LLMFullResponseEndFrame,
@@ -15,6 +22,7 @@ from pipecat.frames.frames import (
     LLMMessagesFrame,
     LLMMessagesUpdateFrame,
     LLMSetToolsFrame,
+    StartFrame,
     StartInterruptionFrame,
     TextFrame,
     TranscriptionFrame,
@@ -28,121 +36,105 @@ from pipecat.processors.aggregators.openai_llm_context import (
 from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
 
 
-class LLMResponseAggregator(FrameProcessor):
+class BaseLLMResponseAggregator(FrameProcessor):
+    """This is the base class for all LLM response aggregators. These
+    aggregators process incoming frames and aggregate content until they are
+    ready to push the aggregation. In the case of a user, an aggregation might
+    be a full transcription received from the STT service.
+
+    The LLM response aggregators also keep a store (e.g. a message list or an
+    LLM context) of the current conversation, that is, it stores the messages
+    said by the user or by the bot.
+
+    """
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+    @property
+    @abstractmethod
+    def messages(self) -> List[dict]:
+        """Returns the messages from the current conversation."""
+        pass
+
+    @property
+    @abstractmethod
+    def role(self) -> str:
+        """Returns the role (e.g. user, assistant...) for this aggregator."""
+        pass
+
+    @abstractmethod
+    def add_messages(self, messages):
+        """Add the given messages to the conversation."""
+        pass
+
+    @abstractmethod
+    def set_messages(self, messages):
+        """Reset the conversation with the given messages."""
+        pass
+
+    @abstractmethod
+    def set_tools(self, tools):
+        """Set LLM tools to be used in the current conversation."""
+        pass
+
+    @abstractmethod
+    def reset(self):
+        """Reset the internals of this aggregator. This should not modify the
+        internal messages."""
+        pass
+
+    @abstractmethod
+    async def push_aggregation(self):
+        pass
+
+
+class LLMResponseAggregator(BaseLLMResponseAggregator):
+    """This is a base LLM aggregator that uses a simple list of messages to
+    store the conversation. It pushes `LLMMessagesFrame` as an aggregation
+    frame.
+
+    """
+
     def __init__(
         self,
         *,
         messages: List[dict],
-        role: str,
-        start_frame,
-        end_frame,
-        accumulator_frame: Type[TextFrame],
-        interim_accumulator_frame: Optional[Type[TextFrame]] = None,
-        handle_interruptions: bool = False,
-        expect_stripped_words: bool = True,  # if True, need to add spaces between words
+        role: str = "user",
+        **kwargs,
     ):
-        super().__init__()
+        super().__init__(**kwargs)
 
         self._messages = messages
         self._role = role
-        self._start_frame = start_frame
-        self._end_frame = end_frame
-        self._accumulator_frame = accumulator_frame
-        self._interim_accumulator_frame = interim_accumulator_frame
-        self._handle_interruptions = handle_interruptions
-        self._expect_stripped_words = expect_stripped_words
 
-        # Reset our accumulator state.
-        self._reset()
+        self._aggregation = ""
+
+        self.reset()
 
     @property
-    def messages(self):
+    def messages(self) -> List[dict]:
         return self._messages
 
     @property
-    def role(self):
+    def role(self) -> str:
         return self._role
 
-    #
-    # Frame processor
-    #
+    def add_messages(self, messages):
+        self._messages.extend(messages)
 
-    # Use cases implemented:
-    #
-    # S: Start, E: End, T: Transcription, I: Interim, X: Text
-    #
-    #        S E -> None
-    #      S T E -> X
-    #    S I T E -> X
-    #    S I E T -> X
-    #  S I E I T -> X
-    #      S E T -> X
-    #    S E I T -> X
-    #
-    # The following case would not be supported:
-    #
-    #    S I E T1 I T2 -> X
-    #
-    # and T2 would be dropped.
+    def set_messages(self, messages):
+        self.reset()
+        self._messages.clear()
+        self._messages.extend(messages)
 
-    async def process_frame(self, frame: Frame, direction: FrameDirection):
-        await super().process_frame(frame, direction)
+    def set_tools(self, tools):
+        pass
 
-        send_aggregation = False
+    def reset(self):
+        self._aggregation = ""
 
-        if isinstance(frame, self._start_frame):
-            self._aggregation = ""
-            self._aggregating = True
-            self._seen_start_frame = True
-            self._seen_end_frame = False
-            self._seen_interim_results = False
-            await self.push_frame(frame, direction)
-        elif isinstance(frame, self._end_frame):
-            self._seen_end_frame = True
-            self._seen_start_frame = False
-
-            # We might have received the end frame but we might still be
-            # aggregating (i.e. we have seen interim results but not the final
-            # text).
-            self._aggregating = self._seen_interim_results or len(self._aggregation) == 0
-
-            # Send the aggregation if we are not aggregating anymore (i.e. no
-            # more interim results received).
-            send_aggregation = not self._aggregating
-            await self.push_frame(frame, direction)
-        elif isinstance(frame, self._accumulator_frame):
-            if self._aggregating:
-                if self._expect_stripped_words:
-                    self._aggregation += f" {frame.text}" if self._aggregation else frame.text
-                else:
-                    self._aggregation += frame.text
-                # We have recevied a complete sentence, so if we have seen the
-                # end frame and we were still aggregating, it means we should
-                # send the aggregation.
-                send_aggregation = self._seen_end_frame
-
-            # We just got our final result, so let's reset interim results.
-            self._seen_interim_results = False
-        elif self._interim_accumulator_frame and isinstance(frame, self._interim_accumulator_frame):
-            self._seen_interim_results = True
-        elif self._handle_interruptions and isinstance(frame, StartInterruptionFrame):
-            await self._push_aggregation()
-            # Reset anyways
-            self._reset()
-            await self.push_frame(frame, direction)
-        elif isinstance(frame, LLMMessagesAppendFrame):
-            self._add_messages(frame.messages)
-        elif isinstance(frame, LLMMessagesUpdateFrame):
-            self._set_messages(frame.messages)
-        elif isinstance(frame, LLMSetToolsFrame):
-            self._set_tools(frame.tools)
-        else:
-            await self.push_frame(frame, direction)
-
-        if send_aggregation:
-            await self._push_aggregation()
-
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if len(self._aggregation) > 0:
             self._messages.append({"role": self._role, "content": self._aggregation})
 
@@ -153,109 +145,27 @@ class LLMResponseAggregator(FrameProcessor):
             frame = LLMMessagesFrame(self._messages)
             await self.push_frame(frame)
 
-    # TODO-CB: Types
-    def _add_messages(self, messages):
-        self._messages.extend(messages)
 
-    def _set_messages(self, messages):
-        self._reset()
-        self._messages.clear()
-        self._messages.extend(messages)
+class LLMContextResponseAggregator(BaseLLMResponseAggregator):
+    """This is a base LLM aggregator that uses an LLM context to store the
+    conversation. It pushes `OpenAILLMContextFrame` as an aggregation frame.
 
-    def _set_tools(self, tools):
-        # noop in the base class
-        pass
-
-    def _reset(self):
-        self._aggregation = ""
-        self._aggregating = False
-        self._seen_start_frame = False
-        self._seen_end_frame = False
-        self._seen_interim_results = False
-
-
-class LLMAssistantResponseAggregator(LLMResponseAggregator):
-    def __init__(self, messages: List[dict] = []):
-        super().__init__(
-            messages=messages,
-            role="assistant",
-            start_frame=LLMFullResponseStartFrame,
-            end_frame=LLMFullResponseEndFrame,
-            accumulator_frame=TextFrame,
-            handle_interruptions=True,
-        )
-
-
-class LLMUserResponseAggregator(LLMResponseAggregator):
-    def __init__(self, messages: List[dict] = []):
-        super().__init__(
-            messages=messages,
-            role="user",
-            start_frame=UserStartedSpeakingFrame,
-            end_frame=UserStoppedSpeakingFrame,
-            accumulator_frame=TranscriptionFrame,
-            interim_accumulator_frame=InterimTranscriptionFrame,
-        )
-
-
-class LLMFullResponseAggregator(FrameProcessor):
-    """This class aggregates Text frames until it receives a
-    LLMFullResponseEndFrame, then emits the concatenated text as
-    a single text frame.
-
-    given the following frames:
-
-        TextFrame("Hello,")
-        TextFrame(" world.")
-        TextFrame(" I am")
-        TextFrame(" an LLM.")
-        LLMFullResponseEndFrame()]
-
-    this processor will yield nothing for the first 4 frames, then
-
-        TextFrame("Hello, world. I am an LLM.")
-        LLMFullResponseEndFrame()
-
-    when passed the last frame.
-
-    >>> async def print_frames(aggregator, frame):
-    ...     async for frame in aggregator.process_frame(frame):
-    ...         if isinstance(frame, TextFrame):
-    ...             print(frame.text)
-    ...         else:
-    ...             print(frame.__class__.__name__)
-
-    >>> aggregator = LLMFullResponseAggregator()
-    >>> asyncio.run(print_frames(aggregator, TextFrame("Hello,")))
-    >>> asyncio.run(print_frames(aggregator, TextFrame(" world.")))
-    >>> asyncio.run(print_frames(aggregator, TextFrame(" I am")))
-    >>> asyncio.run(print_frames(aggregator, TextFrame(" an LLM.")))
-    >>> asyncio.run(print_frames(aggregator, LLMFullResponseEndFrame()))
-    Hello, world. I am an LLM.
-    LLMFullResponseEndFrame
     """
 
-    def __init__(self):
-        super().__init__()
-        self._aggregation = ""
-
-    async def process_frame(self, frame: Frame, direction: FrameDirection):
-        await super().process_frame(frame, direction)
-
-        if isinstance(frame, TextFrame):
-            self._aggregation += frame.text
-        elif isinstance(frame, LLMFullResponseEndFrame):
-            await self.push_frame(TextFrame(self._aggregation))
-            await self.push_frame(frame)
-            self._aggregation = ""
-        else:
-            await self.push_frame(frame, direction)
-
-
-class LLMContextAggregator(LLMResponseAggregator):
-    def __init__(self, *, context: OpenAILLMContext, **kwargs):
+    def __init__(self, *, context: OpenAILLMContext, role: str, **kwargs):
         super().__init__(**kwargs)
         self._context = context
+        self._role = role
+
+        self._aggregation = ""
+
+    @property
+    def messages(self) -> List[dict]:
+        return self._context.get_messages()
+
+    @property
+    def role(self) -> str:
+        return self._role
 
     @property
     def context(self):
@@ -264,23 +174,25 @@ class LLMContextAggregator(LLMResponseAggregator):
     def get_context_frame(self) -> OpenAILLMContextFrame:
         return OpenAILLMContextFrame(context=self._context)
 
-    async def push_context_frame(self):
+    async def push_context_frame(self, direction: FrameDirection = FrameDirection.DOWNSTREAM):
         frame = self.get_context_frame()
-        await self.push_frame(frame)
+        await self.push_frame(frame, direction)
 
-    # TODO-CB: Types
-    def _add_messages(self, messages):
+    def add_messages(self, messages):
         self._context.add_messages(messages)
 
-    def _set_messages(self, messages):
+    def set_messages(self, messages):
         self._context.set_messages(messages)
 
-    def _set_tools(self, tools: List):
+    def set_tools(self, tools: List):
         self._context.set_tools(tools)
 
-    async def _push_aggregation(self):
+    def reset(self):
+        self._aggregation = ""
+
+    async def push_aggregation(self):
         if len(self._aggregation) > 0:
-            self._context.add_message({"role": self._role, "content": self._aggregation})
+            self._context.add_message({"role": self.role, "content": self._aggregation})
 
             # Reset the aggregation. Reset it before pushing it down, otherwise
             # if the tasks gets cancelled we won't be able to clear things up.
@@ -290,31 +202,239 @@ class LLMContextAggregator(LLMResponseAggregator):
             await self.push_frame(frame)
 
             # Reset our accumulator state.
-            self._reset()
+            self.reset()
 
 
-class LLMAssistantContextAggregator(LLMContextAggregator):
-    def __init__(self, context: OpenAILLMContext, *, expect_stripped_words: bool = True):
-        super().__init__(
-            messages=[],
-            context=context,
-            role="assistant",
-            start_frame=LLMFullResponseStartFrame,
-            end_frame=LLMFullResponseEndFrame,
-            accumulator_frame=TextFrame,
-            handle_interruptions=True,
-            expect_stripped_words=expect_stripped_words,
-        )
+class LLMUserContextAggregator(LLMContextResponseAggregator):
+    """This is a user LLM aggregator that uses an LLM context to store the
+    conversation. It aggregates transcriptions from the STT service and it has
+    logic to handle multiple scenarios where transcriptions are received between
+    VAD events (`UserStartedSpeakingFrame` and `UserStoppedSpeakingFrame`) or
+    even outside or no VAD events at all.
+
+    """
+
+    def __init__(
+        self,
+        context: OpenAILLMContext,
+        aggregation_timeout: float = 1.0,
+        bot_interruption_timeout: float = 2.0,
+        **kwargs,
+    ):
+        super().__init__(context=context, role="user", **kwargs)
+        self._aggregation_timeout = aggregation_timeout
+        self._bot_interruption_timeout = bot_interruption_timeout
+
+        self._seen_interim_results = False
+        self._user_speaking = False
+        self._last_user_speaking_time = 0
+        self._emulating_vad = False
+
+        self._aggregation_event = asyncio.Event()
+        self._aggregation_task = None
+
+        self.reset()
+
+    def reset(self):
+        super().reset()
+        self._seen_interim_results = False
+
+    async def process_frame(self, frame: Frame, direction: FrameDirection):
+        await super().process_frame(frame, direction)
+
+        if isinstance(frame, StartFrame):
+            await self._start(frame)
+            await self.push_frame(frame, direction)
+        elif isinstance(frame, EndFrame):
+            await self._stop(frame)
+            await self.push_frame(frame, direction)
+        elif isinstance(frame, CancelFrame):
+            await self._cancel(frame)
+            await self.push_frame(frame, direction)
+        elif isinstance(frame, UserStartedSpeakingFrame):
+            await self._handle_user_started_speaking(frame)
+            await self.push_frame(frame, direction)
+        elif isinstance(frame, UserStoppedSpeakingFrame):
+            await self._handle_user_stopped_speaking(frame)
+            await self.push_frame(frame, direction)
+        elif isinstance(frame, TranscriptionFrame):
+            await self._handle_transcription(frame)
+        elif isinstance(frame, InterimTranscriptionFrame):
+            await self._handle_interim_transcription(frame)
+        elif isinstance(frame, LLMMessagesAppendFrame):
+            self.add_messages(frame.messages)
+        elif isinstance(frame, LLMMessagesUpdateFrame):
+            self.set_messages(frame.messages)
+        elif isinstance(frame, LLMSetToolsFrame):
+            self.set_tools(frame.tools)
+        else:
+            await self.push_frame(frame, direction)
+
+    async def _start(self, frame: StartFrame):
+        self._create_aggregation_task()
+
+    async def _stop(self, frame: EndFrame):
+        await self._cancel_aggregation_task()
+
+    async def _cancel(self, frame: CancelFrame):
+        await self._cancel_aggregation_task()
+
+    async def _handle_user_started_speaking(self, _: UserStartedSpeakingFrame):
+        self._last_user_speaking_time = time.time()
+        self._user_speaking = True
+
+    async def _handle_user_stopped_speaking(self, _: UserStoppedSpeakingFrame):
+        self._last_user_speaking_time = time.time()
+        self._user_speaking = False
+        if not self._seen_interim_results:
+            await self.push_aggregation()
+
+    async def _handle_transcription(self, frame: TranscriptionFrame):
+        self._aggregation += f" {frame.text}" if self._aggregation else frame.text
+        # We just got a final result, so let's reset interim results.
+        self._seen_interim_results = False
+        # Reset aggregation timer.
+        self._aggregation_event.set()
+
+    async def _handle_interim_transcription(self, _: InterimTranscriptionFrame):
+        self._seen_interim_results = True
+        # Reset aggregation timer.
+        self._aggregation_event.set()
+
+    def _create_aggregation_task(self):
+        self._aggregation_task = self.create_task(self._aggregation_task_handler())
+
+    async def _cancel_aggregation_task(self):
+        if self._aggregation_task:
+            await self.cancel_task(self._aggregation_task)
+            self._aggregation_task = None
+
+    async def _aggregation_task_handler(self):
+        while True:
+            try:
+                await asyncio.wait_for(self._aggregation_event.wait(), self._aggregation_timeout)
+                await self._maybe_push_bot_interruption()
+            except asyncio.TimeoutError:
+                if not self._user_speaking:
+                    await self.push_aggregation()
+
+                # If we are emulating VAD we still need to send the user stopped
+                # speaking frame.
+                if self._emulating_vad:
+                    await self.push_frame(
+                        EmulateUserStoppedSpeakingFrame(), FrameDirection.UPSTREAM
+                    )
+                    self._emulating_vad = False
+            finally:
+                self._aggregation_event.clear()
+
+    async def _maybe_push_bot_interruption(self):
+        """If the user stopped speaking a while back and we got a transcription
+        frame we might want to interrupt the bot.
+
+        """
+        if not self._user_speaking:
+            diff_time = time.time() - self._last_user_speaking_time
+            if diff_time > self._bot_interruption_timeout:
+                # If we reach this case we received a transcription but VAD was
+                # not able to detect voice (e.g. when you whisper a short
+                # utterance). So, we need to emulate VAD (i.e. user
+                # start/stopped speaking).
+                await self.push_frame(EmulateUserStartedSpeakingFrame(), FrameDirection.UPSTREAM)
+                self._emulating_vad = True
+
+                # Reset time so we don't interrupt again right away.
+                self._last_user_speaking_time = time.time()
 
 
-class LLMUserContextAggregator(LLMContextAggregator):
-    def __init__(self, context: OpenAILLMContext):
-        super().__init__(
-            messages=[],
-            context=context,
-            role="user",
-            start_frame=UserStartedSpeakingFrame,
-            end_frame=UserStoppedSpeakingFrame,
-            accumulator_frame=TranscriptionFrame,
-            interim_accumulator_frame=InterimTranscriptionFrame,
-        )
+class LLMAssistantContextAggregator(LLMContextResponseAggregator):
+    """This is an assistant LLM aggregator that uses an LLM context to store the
+    conversation. It aggregates text frames received between
+    `LLMFullResponseStartFrame` and `LLMFullResponseEndFrame`.
+
+    """
+
+    def __init__(self, context: OpenAILLMContext, *, expect_stripped_words: bool = True, **kwargs):
+        super().__init__(context=context, role="assistant", **kwargs)
+        self._expect_stripped_words = expect_stripped_words
+
+        self._started = False
+
+        self.reset()
+
+    async def process_frame(self, frame: Frame, direction: FrameDirection):
+        await super().process_frame(frame, direction)
+
+        if isinstance(frame, StartInterruptionFrame):
+            await self.push_aggregation()
+            # Reset anyways
+            self.reset()
+            await self.push_frame(frame, direction)
+        elif isinstance(frame, LLMFullResponseStartFrame):
+            await self._handle_llm_start(frame)
+        elif isinstance(frame, LLMFullResponseEndFrame):
+            await self._handle_llm_end(frame)
+        elif isinstance(frame, TextFrame):
+            await self._handle_text(frame)
+        elif isinstance(frame, LLMMessagesAppendFrame):
+            self.add_messages(frame.messages)
+        elif isinstance(frame, LLMMessagesUpdateFrame):
+            self.set_messages(frame.messages)
+        elif isinstance(frame, LLMSetToolsFrame):
+            self.set_tools(frame.tools)
+        else:
+            await self.push_frame(frame, direction)
+
+    async def _handle_llm_start(self, _: LLMFullResponseStartFrame):
+        self._started = True
+
+    async def _handle_llm_end(self, _: LLMFullResponseEndFrame):
+        self._started = False
+        await self.push_aggregation()
+
+    async def _handle_text(self, frame: TextFrame):
+        if not self._started:
+            return
+
+        if self._expect_stripped_words:
+            self._aggregation += f" {frame.text}" if self._aggregation else frame.text
+        else:
+            self._aggregation += frame.text
+
+
+class LLMUserResponseAggregator(LLMUserContextAggregator):
+    def __init__(self, messages: List[dict] = [], **kwargs):
+        super().__init__(context=OpenAILLMContext(messages), **kwargs)
+
+    async def push_aggregation(self):
+        if len(self._aggregation) > 0:
+            self._context.add_message({"role": self.role, "content": self._aggregation})
+
+            # Reset the aggregation. Reset it before pushing it down, otherwise
+            # if the tasks gets cancelled we won't be able to clear things up.
+            self._aggregation = ""
+
+            frame = LLMMessagesFrame(self._context.messages)
+            await self.push_frame(frame)
+
+            # Reset our accumulator state.
+            self.reset()
+
+
+class LLMAssistantResponseAggregator(LLMAssistantContextAggregator):
+    def __init__(self, messages: List[dict] = [], **kwargs):
+        super().__init__(context=OpenAILLMContext(messages), **kwargs)
+
+    async def push_aggregation(self):
+        if len(self._aggregation) > 0:
+            self._context.add_message({"role": self.role, "content": self._aggregation})
+
+            # Reset the aggregation. Reset it before pushing it down, otherwise
+            # if the tasks gets cancelled we won't be able to clear things up.
+            self._aggregation = ""
+
+            frame = LLMMessagesFrame(self._context.messages)
+            await self.push_frame(frame)
+
+            # Reset our accumulator state.
+            self.reset()

src/pipecat/processors/aggregators/user_response.py

@@ -4,131 +4,15 @@
 # SPDX-License-Identifier: BSD 2-Clause License
 #
 
-from typing import Optional
-
-from pipecat.frames.frames import (
-    Frame,
-    InterimTranscriptionFrame,
-    StartInterruptionFrame,
-    TextFrame,
-    TranscriptionFrame,
-    UserStartedSpeakingFrame,
-    UserStoppedSpeakingFrame,
-)
-from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
+from pipecat.frames.frames import TextFrame
+from pipecat.processors.aggregators.llm_response import LLMUserResponseAggregator
 
 
-class ResponseAggregator(FrameProcessor):
-    """This frame processor aggregates frames between a start and an end frame
-    into complete text frame sentences.
+class UserResponseAggregator(LLMUserResponseAggregator):
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
 
-    For example, frame input/output:
-        UserStartedSpeakingFrame() -> None
-        TranscriptionFrame("Hello,") -> None
-        TranscriptionFrame(" world.") -> None
-        UserStoppedSpeakingFrame() -> TextFrame("Hello world.")
-
-    Doctest: FIXME to work with asyncio
-    >>> async def print_frames(aggregator, frame):
-    ...     async for frame in aggregator.process_frame(frame):
-    ...         if isinstance(frame, TextFrame):
-    ...             print(frame.text)
-
-    >>> aggregator = ResponseAggregator(start_frame = UserStartedSpeakingFrame,
-    ...                                 end_frame=UserStoppedSpeakingFrame,
-    ...                                 accumulator_frame=TranscriptionFrame,
-    ...                                 pass_through=False)
-    >>> asyncio.run(print_frames(aggregator, UserStartedSpeakingFrame()))
-    >>> asyncio.run(print_frames(aggregator, TranscriptionFrame("Hello,", 1, 1)))
-    >>> asyncio.run(print_frames(aggregator, TranscriptionFrame("world.",  1, 2)))
-    >>> asyncio.run(print_frames(aggregator, UserStoppedSpeakingFrame()))
-    Hello, world.
-
-    """
-
-    def __init__(
-        self,
-        *,
-        start_frame,
-        end_frame,
-        accumulator_frame: TextFrame,
-        interim_accumulator_frame: Optional[TextFrame] = None,
-    ):
-        super().__init__()
-
-        self._start_frame = start_frame
-        self._end_frame = end_frame
-        self._accumulator_frame = accumulator_frame
-        self._interim_accumulator_frame = interim_accumulator_frame
-
-        # Reset our accumulator state.
-        self._reset()
-
-    #
-    # Frame processor
-    #
-
-    # Use cases implemented:
-    #
-    # S: Start, E: End, T: Transcription, I: Interim, X: Text
-    #
-    #        S E -> None
-    #      S T E -> X
-    #    S I T E -> X
-    #    S I E T -> X
-    #  S I E I T -> X
-    #      S E T -> X
-    #    S E I T -> X
-    #
-    # The following case would not be supported:
-    #
-    #    S I E T1 I T2 -> X
-    #
-    # and T2 would be dropped.
-
-    async def process_frame(self, frame: Frame, direction: FrameDirection):
-        await super().process_frame(frame, direction)
-
-        send_aggregation = False
-
-        if isinstance(frame, self._start_frame):
-            self._aggregating = True
-            self._seen_start_frame = True
-            self._seen_end_frame = False
-            self._seen_interim_results = False
-            await self.push_frame(frame, direction)
-        elif isinstance(frame, self._end_frame):
-            self._seen_end_frame = True
-            self._seen_start_frame = False
-
-            # We might have received the end frame but we might still be
-            # aggregating (i.e. we have seen interim results but not the final
-            # text).
-            self._aggregating = self._seen_interim_results or len(self._aggregation) == 0
-
-            # Send the aggregation if we are not aggregating anymore (i.e. no
-            # more interim results received).
-            send_aggregation = not self._aggregating
-            await self.push_frame(frame, direction)
-        elif isinstance(frame, self._accumulator_frame):
-            if self._aggregating:
-                self._aggregation += f" {frame.text}"
-                # We have recevied a complete sentence, so if we have seen the
-                # end frame and we were still aggregating, it means we should
-                # send the aggregation.
-                send_aggregation = self._seen_end_frame
-
-            # We just got our final result, so let's reset interim results.
-            self._seen_interim_results = False
-        elif self._interim_accumulator_frame and isinstance(frame, self._interim_accumulator_frame):
-            self._seen_interim_results = True
-        else:
-            await self.push_frame(frame, direction)
-
-        if send_aggregation:
-            await self._push_aggregation()
-
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if len(self._aggregation) > 0:
             frame = TextFrame(self._aggregation.strip())
 
@@ -139,21 +23,4 @@ class ResponseAggregator(FrameProcessor):
             await self.push_frame(frame)
 
             # Reset our accumulator state.
-            self._reset()
-
-    def _reset(self):
-        self._aggregation = ""
-        self._aggregating = False
-        self._seen_start_frame = False
-        self._seen_end_frame = False
-        self._seen_interim_results = False
-
-
-class UserResponseAggregator(ResponseAggregator):
-    def __init__(self):
-        super().__init__(
-            start_frame=UserStartedSpeakingFrame,
-            end_frame=UserStoppedSpeakingFrame,
-            accumulator_frame=TranscriptionFrame,
-            interim_accumulator_frame=InterimTranscriptionFrame,
-        )
+            self.reset()

src/pipecat/processors/filters/stt_mute_filter.py

@@ -23,6 +23,7 @@ from pipecat.frames.frames import (
     Frame,
     FunctionCallInProgressFrame,
     FunctionCallResultFrame,
+    StartFrame,
     StartInterruptionFrame,
     StopInterruptionFrame,
     STTMuteFrame,
@@ -37,16 +38,18 @@ class STTMuteStrategy(Enum):
     """Strategies determining when STT should be muted.
 
     Attributes:
-        FIRST_SPEECH: Mute only during first bot speech
+        FIRST_SPEECH: Mute only during first detected bot speech
+        MUTE_UNTIL_FIRST_BOT_COMPLETE: Start muted and remain muted until first bot speech completes
         FUNCTION_CALL: Mute during function calls
         ALWAYS: Mute during all bot speech
         CUSTOM: Allow custom logic via callback
     """
 
-    FIRST_SPEECH = "first_speech"  # Mute only during first bot speech
-    FUNCTION_CALL = "function_call"  # Mute during function calls
-    ALWAYS = "always"  # Mute during all bot speech
-    CUSTOM = "custom"  # Allow custom logic via callback
+    FIRST_SPEECH = "first_speech"
+    MUTE_UNTIL_FIRST_BOT_COMPLETE = "mute_until_first_bot_complete"
+    FUNCTION_CALL = "function_call"
+    ALWAYS = "always"
+    CUSTOM = "custom"
 
 
 @dataclass
@@ -57,12 +60,25 @@ class STTMuteConfig:
         strategies: Set of muting strategies to apply
         should_mute_callback: Optional callback for custom muting logic.
             Only required when using STTMuteStrategy.CUSTOM
+
+    Note:
+        MUTE_UNTIL_FIRST_BOT_COMPLETE and FIRST_SPEECH strategies should not be used together
+        as they handle the first bot speech differently.
     """
 
     strategies: set[STTMuteStrategy]
     # Optional callback for custom muting logic
     should_mute_callback: Optional[Callable[["STTMuteFilter"], Awaitable[bool]]] = None
 
+    def __post_init__(self):
+        if (
+            STTMuteStrategy.MUTE_UNTIL_FIRST_BOT_COMPLETE in self.strategies
+            and STTMuteStrategy.FIRST_SPEECH in self.strategies
+        ):
+            raise ValueError(
+                "MUTE_UNTIL_FIRST_BOT_COMPLETE and FIRST_SPEECH strategies should not be used together"
+            )
+
 
 class STTMuteFilter(FrameProcessor):
     """A processor that handles STT muting and interruption control.
@@ -93,7 +109,7 @@ class STTMuteFilter(FrameProcessor):
         self._first_speech_handled = False
         self._bot_is_speaking = False
         self._function_call_in_progress = False
-        self._is_muted = False
+        self._is_muted = False  # Initialize as unmuted, will set state on StartFrame if needed
 
     @property
     def is_muted(self) -> bool:
@@ -124,6 +140,10 @@ class STTMuteFilter(FrameProcessor):
                         self._first_speech_handled = True
                         return True
 
+                case STTMuteStrategy.MUTE_UNTIL_FIRST_BOT_COMPLETE:
+                    if not self._first_speech_handled:
+                        return True
+
                 case STTMuteStrategy.CUSTOM:
                     if self._bot_is_speaking and self._config.should_mute_callback:
                         should_mute = await self._config.should_mute_callback(self)
@@ -133,25 +153,31 @@ class STTMuteFilter(FrameProcessor):
         return False
 
     async def process_frame(self, frame: Frame, direction: FrameDirection):
+        """Processes incoming frames and manages muting state."""
         await super().process_frame(frame, direction)
 
-        """Processes incoming frames and manages muting state."""
-        # Handle function call state changes
-        if isinstance(frame, FunctionCallInProgressFrame):
+        # Determine if we need to change mute state based on frame type
+        should_mute = None
+
+        # Process frames to determine mute state
+        if isinstance(frame, StartFrame):
+            should_mute = await self._should_mute()
+        elif isinstance(frame, FunctionCallInProgressFrame):
             self._function_call_in_progress = True
-            await self._handle_mute_state(await self._should_mute())
+            should_mute = await self._should_mute()
         elif isinstance(frame, FunctionCallResultFrame):
             self._function_call_in_progress = False
-            await self._handle_mute_state(await self._should_mute())
-        # Handle bot speaking state changes
+            should_mute = await self._should_mute()
         elif isinstance(frame, BotStartedSpeakingFrame):
             self._bot_is_speaking = True
-            await self._handle_mute_state(await self._should_mute())
+            should_mute = await self._should_mute()
         elif isinstance(frame, BotStoppedSpeakingFrame):
             self._bot_is_speaking = False
-            await self._handle_mute_state(await self._should_mute())
+            if not self._first_speech_handled:
+                self._first_speech_handled = True
+            should_mute = await self._should_mute()
 
-        # Handle frame propagation
+        # Then push the original frame
         if isinstance(
             frame,
             (
@@ -169,3 +195,7 @@ class STTMuteFilter(FrameProcessor):
         else:
             # Pass all other frames through
             await self.push_frame(frame, direction)
+
+        # Finally handle mute state change if needed
+        if should_mute is not None and should_mute != self.is_muted:
+            await self._handle_mute_state(should_mute)

src/pipecat/processors/frame_processor.py

@@ -73,10 +73,11 @@ class FrameProcessor:
         self._metrics.set_processor_name(self.name)
 
         # Processors have an input queue. The input queue will be processed
-        # immediately (default) or it will block if `pause_processing_frames()` is
-        # called. To resume processing frames we need to call
-        # `resume_processing_frames()`.
+        # immediately (default) or it will block if `pause_processing_frames()`
+        # is called. To resume processing frames we need to call
+        # `resume_processing_frames()` which will wake up the event.
         self.__should_block_frames = False
+        self.__input_event = asyncio.Event()
         self.__input_frame_task: Optional[asyncio.Task] = None
 
         # Every processor in Pipecat should only output frames from a single
@@ -335,8 +336,8 @@ class FrameProcessor:
     def __create_input_task(self):
         if not self.__input_frame_task:
             self.__should_block_frames = False
+            self.__input_event.clear()
             self.__input_queue = asyncio.Queue()
-            self.__input_event = asyncio.Event()
             self.__input_frame_task = self.create_task(self.__input_frame_task_handler())
 
     async def __cancel_input_task(self):

src/pipecat/processors/frameworks/rtvi.py

@@ -5,6 +5,7 @@
 #
 
 import asyncio
+import base64
 from dataclasses import dataclass
 from typing import (
     Any,
@@ -31,6 +32,7 @@ from pipecat.frames.frames import (
     ErrorFrame,
     Frame,
     FunctionCallResultFrame,
+    InputAudioRawFrame,
     InterimTranscriptionFrame,
     LLMFullResponseEndFrame,
     LLMFullResponseStartFrame,
@@ -58,7 +60,9 @@ from pipecat.processors.aggregators.openai_llm_context import (
     OpenAILLMContextFrame,
 )
 from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
+from pipecat.transports.base_input import BaseInputTransport
 from pipecat.transports.base_output import BaseOutputTransport
+from pipecat.transports.base_transport import BaseTransport
 from pipecat.utils.string import match_endofsentence
 
 RTVI_PROTOCOL_VERSION = "0.3.0"
@@ -819,6 +823,7 @@ class RTVIProcessor(FrameProcessor):
         self,
         *,
         config: RTVIConfig = RTVIConfig(config=[]),
+        transport: Optional[BaseTransport] = None,
         **kwargs,
     ):
         super().__init__(**kwargs)
@@ -844,6 +849,14 @@ class RTVIProcessor(FrameProcessor):
         self._register_event_handler("on_bot_started")
         self._register_event_handler("on_client_ready")
 
+        self._input_transport = None
+        self._transport = transport
+        if self._transport:
+            input_transport = self._transport.input()
+            if isinstance(input_transport, BaseInputTransport):
+                self._input_transport = input_transport
+                self._input_transport.enable_audio_in_stream_on_start(False)
+
     def observer(self) -> RTVIObserver:
         import warnings
 
@@ -1013,6 +1026,8 @@ class RTVIProcessor(FrameProcessor):
                 case "llm-function-call-result":
                     data = RTVILLMFunctionCallResultData.model_validate(message.data)
                     await self._handle_function_call_result(data)
+                case "raw-audio" | "raw-audio-batch":
+                    await self._handle_audio_buffer(message.data)
 
                 case _:
                     await self._send_error_response(message.id, f"Unsupported type {message.type}")
@@ -1025,9 +1040,34 @@ class RTVIProcessor(FrameProcessor):
             logger.warning(f"Exception processing message: {e}")
 
     async def _handle_client_ready(self, request_id: str):
+        logger.debug("Received client-ready")
+        if self._input_transport:
+            self._input_transport.start_audio_in_streaming()
+
         self._client_ready_id = request_id
         await self.set_client_ready()
 
+    async def _handle_audio_buffer(self, data):
+        if not self._input_transport:
+            return
+
+        # Extract audio batch ensuring it's a list
+        audio_list = data.get("base64AudioBatch") or [data.get("base64Audio")]
+
+        try:
+            for base64_audio in filter(None, audio_list):  # Filter out None values
+                pcm_bytes = base64.b64decode(base64_audio)
+                frame = InputAudioRawFrame(
+                    audio=pcm_bytes,
+                    sample_rate=data["sampleRate"],
+                    num_channels=data["numChannels"],
+                )
+                await self._input_transport.push_audio_frame(frame)
+
+        except (KeyError, TypeError, ValueError) as e:
+            # Handle missing keys, decoding errors, and invalid types
+            logger.error(f"Error processing audio buffer: {e}")
+
     async def _handle_describe_config(self, request_id: str):
         services = list(self._registered_services.values())
         message = RTVIDescribeConfig(id=request_id, data=RTVIDescribeConfigData(config=services))

src/pipecat/services/ai_services.py

@@ -15,6 +15,7 @@ from loguru import logger
 from pipecat.audio.utils import calculate_audio_volume, exp_smoothing
 from pipecat.frames.frames import (
     AudioRawFrame,
+    BotStoppedSpeakingFrame,
     CancelFrame,
     EndFrame,
     ErrorFrame,
@@ -75,13 +76,13 @@ class AIService(FrameProcessor):
         )
 
         for key, value in settings.items():
-            print("Update request for:", key, value)
+            logger.debug("Update request for:", key, value)
 
             if key in self._settings:
                 logger.info(f"Updating LLM setting {key} to: [{value}]")
                 self._settings[key] = value
             elif key in SessionProperties.model_fields:
-                print("Attempting to update", key, value)
+                logger.debug("Attempting to update", key, value)
 
                 try:
                     from pipecat.services.openai_realtime_beta.events import (
@@ -212,6 +213,8 @@ class TTSService(AIService):
         push_silence_after_stop: bool = False,
         # if push_silence_after_stop is True, send this amount of audio silence
         silence_time_s: float = 2.0,
+        # if True, we will pause processing frames while we are receiving audio
+        pause_frame_processing: bool = False,
         # TTS output sample rate
         sample_rate: Optional[int] = None,
         text_filter: Optional[BaseTextFilter] = None,
@@ -224,6 +227,7 @@ class TTSService(AIService):
         self._stop_frame_timeout_s: float = stop_frame_timeout_s
         self._push_silence_after_stop: bool = push_silence_after_stop
         self._silence_time_s: float = silence_time_s
+        self._pause_frame_processing: bool = pause_frame_processing
         self._init_sample_rate = sample_rate
         self._sample_rate = 0
         self._voice_id: str = ""
@@ -234,6 +238,7 @@ class TTSService(AIService):
         self._stop_frame_queue: asyncio.Queue = asyncio.Queue()
 
         self._current_sentence: str = ""
+        self._processing_text: bool = False
 
     @property
     def sample_rate(self) -> int:
@@ -299,6 +304,7 @@ class TTSService(AIService):
 
     async def process_frame(self, frame: Frame, direction: FrameDirection):
         await super().process_frame(frame, direction)
+
         if (
             isinstance(frame, TextFrame)
             and not isinstance(frame, InterimTranscriptionFrame)
@@ -307,9 +313,16 @@ class TTSService(AIService):
             await self._process_text_frame(frame)
         elif isinstance(frame, StartInterruptionFrame):
             await self._handle_interruption(frame, direction)
+            await self.push_frame(frame, direction)
         elif isinstance(frame, (LLMFullResponseEndFrame, EndFrame)):
+            # We pause processing incoming frames if the LLM response included
+            # text (it might be that it's only a function calling response). We
+            # pause to avoid audio overlapping.
+            await self._maybe_pause_frame_processing()
+
             sentence = self._current_sentence
             self._current_sentence = ""
+            self._processing_text = False
             await self._push_tts_frames(sentence)
             if isinstance(frame, LLMFullResponseEndFrame):
                 if self._push_text_frames:
@@ -318,9 +331,16 @@ class TTSService(AIService):
                 await self.push_frame(frame, direction)
         elif isinstance(frame, TTSSpeakFrame):
             await self._push_tts_frames(frame.text)
+            # We pause processing incoming frames because we are sending data to
+            # the TTS. We pause to avoid audio overlapping.
+            await self._maybe_pause_frame_processing()
             await self.flush_audio()
+            self._processing_text = False
         elif isinstance(frame, TTSUpdateSettingsFrame):
             await self._update_settings(frame.settings)
+        elif isinstance(frame, BotStoppedSpeakingFrame):
+            await self._maybe_resume_frame_processing()
+            await self.push_frame(frame, direction)
         else:
             await self.push_frame(frame, direction)
 
@@ -347,9 +367,17 @@ class TTSService(AIService):
 
     async def _handle_interruption(self, frame: StartInterruptionFrame, direction: FrameDirection):
         self._current_sentence = ""
+        self._processing_text = False
         if self._text_filter:
             self._text_filter.handle_interruption()
-        await self.push_frame(frame, direction)
+
+    async def _maybe_pause_frame_processing(self):
+        if self._processing_text and self._pause_frame_processing:
+            await self.pause_processing_frames()
+
+    async def _maybe_resume_frame_processing(self):
+        if self._pause_frame_processing:
+            await self.resume_processing_frames()
 
     async def _process_text_frame(self, frame: TextFrame):
         text: Optional[str] = None
@@ -371,6 +399,11 @@ class TTSService(AIService):
         if not text.strip():
             return
 
+        # This is just a flag that indicates if we sent something to the TTS
+        # service. It will be cleared if we sent text because of a TTSSpeakFrame
+        # or when we received an LLMFullResponseEndFrame
+        self._processing_text = True
+
         await self.start_processing_metrics()
         if self._text_filter:
             self._text_filter.reset_interruption()
@@ -419,7 +452,7 @@ class WordTTSService(TTSService):
 
     async def start(self, frame: StartFrame):
         await super().start(frame)
-        await self._create_words_task()
+        self._create_words_task()
 
     async def stop(self, frame: EndFrame):
         await super().stop(frame)
@@ -439,7 +472,7 @@ class WordTTSService(TTSService):
         await super()._handle_interruption(frame, direction)
         self.reset_word_timestamps()
 
-    async def _create_words_task(self):
+    def _create_words_task(self):
         self._words_task = self.create_task(self._words_task_handler())
 
     async def _stop_words_task(self):
@@ -469,6 +502,115 @@ class WordTTSService(TTSService):
             self._words_queue.task_done()
 
 
+class AudioContextWordTTSService(WordTTSService):
+    """This services allow us to send multiple TTS request to the services. Each
+    request could be multiple sentences long which are grouped by context. For
+    this to work, the TTS service needs to support handling multiple requests at
+    once (i.e. multiple simultaneous contexts).
+
+    The audio received from the TTS will be played in context order. That is, if
+    we requested audio for a context "A" and then audio for context "B", the
+    audio from context ID "A" will be played first.
+
+    """
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        self._contexts_queue = asyncio.Queue()
+        self._contexts: Dict[str, asyncio.Queue] = {}
+        self._audio_context_task = None
+
+    async def create_audio_context(self, context_id: str):
+        """Create a new audio context."""
+        await self._contexts_queue.put(context_id)
+        self._contexts[context_id] = asyncio.Queue()
+        logger.trace(f"{self} created audio context {context_id}")
+
+    async def append_to_audio_context(self, context_id: str, frame: TTSAudioRawFrame):
+        """Append audio to an existing context."""
+        if self.audio_context_available(context_id):
+            logger.trace(f"{self} appending audio {frame} to audio context {context_id}")
+            await self._contexts[context_id].put(frame)
+        else:
+            logger.warning(f"{self} unable to append audio to context {context_id}")
+
+    async def remove_audio_context(self, context_id: str):
+        """Remove an existing audio context."""
+        if self.audio_context_available(context_id):
+            # We just mark the audio context for deletion by appending
+            # None. Once we reach None while handling audio we know we can
+            # safely remove the context.
+            logger.trace(f"{self} marking audio context {context_id} for deletion")
+            await self._contexts[context_id].put(None)
+        else:
+            logger.warning(f"{self} unable to remove context {context_id}")
+
+    def audio_context_available(self, context_id: str) -> bool:
+        """Checks whether the given audio context is registered."""
+        return context_id in self._contexts
+
+    async def start(self, frame: StartFrame):
+        await super().start(frame)
+        self._create_audio_context_task()
+
+    async def stop(self, frame: EndFrame):
+        await super().stop(frame)
+        await self._stop_audio_context_task()
+
+    async def cancel(self, frame: CancelFrame):
+        await super().cancel(frame)
+        await self._stop_audio_context_task()
+
+    async def _handle_interruption(self, frame: StartInterruptionFrame, direction: FrameDirection):
+        await super()._handle_interruption(frame, direction)
+        await self._stop_audio_context_task()
+        self._create_audio_context_task()
+
+    def _create_audio_context_task(self):
+        self._contexts_queue = asyncio.Queue()
+        self._contexts: Dict[str, asyncio.Queue] = {}
+        self._audio_context_task = self.create_task(self._audio_context_task_handler())
+
+    async def _stop_audio_context_task(self):
+        if self._audio_context_task:
+            await self.cancel_task(self._audio_context_task)
+            self._audio_context_task = None
+
+    async def _audio_context_task_handler(self):
+        """In this task we process audio contexts in order."""
+        while True:
+            context_id = await self._contexts_queue.get()
+
+            # Process the audio context until the context doesn't have more
+            # audio available (i.e. we find None).
+            await self._handle_audio_context(context_id)
+
+            # We just finished processing the context, so we can safely remove it.
+            del self._contexts[context_id]
+            self._contexts_queue.task_done()
+
+            # Append some silence between sentences.
+            silence = b"\x00" * self.sample_rate
+            frame = TTSAudioRawFrame(audio=silence, sample_rate=self.sample_rate, num_channels=1)
+            await self.push_frame(frame)
+
+    async def _handle_audio_context(self, context_id: str):
+        # If we don't receive any audio during this time, we consider the context finished.
+        AUDIO_CONTEXT_TIMEOUT = 3.0
+        queue = self._contexts[context_id]
+        running = True
+        while running:
+            try:
+                frame = await asyncio.wait_for(queue.get(), timeout=AUDIO_CONTEXT_TIMEOUT)
+                if frame:
+                    await self.push_frame(frame)
+                running = frame is not None
+            except asyncio.TimeoutError:
+                # We didn't get audio, so let's consider this context finished.
+                logger.trace(f"{self} time out on audio context {context_id}")
+                break
+
+
 class STTService(AIService):
     """STTService is a base class for speech-to-text services."""
 
@@ -525,9 +667,13 @@ class STTService(AIService):
             else:
                 logger.warning(f"Unknown setting for STT service: {key}")
 
-    async def process_audio_frame(self, frame: AudioRawFrame):
-        if not self._muted:
-            await self.process_generator(self.run_stt(frame.audio))
+    async def process_audio_frame(self, frame: AudioRawFrame, direction: FrameDirection):
+        if self._muted:
+            return
+
+        await self.process_generator(self.run_stt(frame.audio))
+        if self._audio_passthrough:
+            await self.push_frame(frame, direction)
 
     async def process_frame(self, frame: Frame, direction: FrameDirection):
         """Processes a frame of audio data, either buffering or transcribing it."""
@@ -537,9 +683,7 @@ class STTService(AIService):
             # In this service we accumulate audio internally and at the end we
             # push a TextFrame. We also push audio downstream in case someone
             # else needs it.
-            await self.process_audio_frame(frame)
-            if self._audio_passthrough:
-                await self.push_frame(frame, direction)
+            await self.process_audio_frame(frame, direction)
         elif isinstance(frame, STTUpdateSettingsFrame):
             await self._update_settings(frame.settings)
         elif isinstance(frame, STTMuteFrame):
@@ -575,7 +719,7 @@ class SegmentedSTTService(STTService):
         self._smoothing_factor = 0.2
         self._prev_volume = 0
 
-    async def process_audio_frame(self, frame: AudioRawFrame):
+    async def process_audio_frame(self, frame: AudioRawFrame, direction: FrameDirection):
         # Try to filter out empty background noise
         volume = self._get_smoothed_volume(frame)
         if volume >= self._min_volume:

src/pipecat/services/anthropic.py

@@ -126,9 +126,11 @@ class AnthropicLLMService(LLMService):
     def create_context_aggregator(
         context: OpenAILLMContext, *, assistant_expect_stripped_words: bool = True
     ) -> AnthropicContextAggregatorPair:
+        if isinstance(context, OpenAILLMContext):
+            context = AnthropicLLMContext.from_openai_context(context)
         user = AnthropicUserContextAggregator(context)
         assistant = AnthropicAssistantContextAggregator(
-            user, expect_stripped_words=assistant_expect_stripped_words
+            context, expect_stripped_words=assistant_expect_stripped_words
         )
         return AnthropicContextAggregatorPair(_user=user, _assistant=assistant)
 
@@ -651,11 +653,8 @@ class AnthropicLLMContext(OpenAILLMContext):
 
 
 class AnthropicUserContextAggregator(LLMUserContextAggregator):
-    def __init__(self, context: OpenAILLMContext | AnthropicLLMContext):
-        super().__init__(context=context)
-
-        if isinstance(context, OpenAILLMContext):
-            self._context = AnthropicLLMContext.from_openai_context(context)
+    def __init__(self, context: OpenAILLMContext | AnthropicLLMContext, **kwargs):
+        super().__init__(context=context, **kwargs)
 
     async def process_frame(self, frame, direction):
         await super().process_frame(frame, direction)
@@ -703,9 +702,8 @@ class AnthropicUserContextAggregator(LLMUserContextAggregator):
 
 
 class AnthropicAssistantContextAggregator(LLMAssistantContextAggregator):
-    def __init__(self, user_context_aggregator: AnthropicUserContextAggregator, **kwargs):
-        super().__init__(context=user_context_aggregator._context, **kwargs)
-        self._user_context_aggregator = user_context_aggregator
+    def __init__(self, context: OpenAILLMContext | AnthropicLLMContext, **kwargs):
+        super().__init__(context=context, **kwargs)
         self._function_call_in_progress = None
         self._function_call_result = None
         self._pending_image_frame_message = None
@@ -725,7 +723,7 @@ class AnthropicAssistantContextAggregator(LLMAssistantContextAggregator):
             ):
                 self._function_call_in_progress = None
                 self._function_call_result = frame
-                await self._push_aggregation()
+                await self.push_aggregation()
             else:
                 logger.warning(
                     "FunctionCallResultFrame tool_call_id != InProgressFrame tool_call_id"
@@ -734,9 +732,9 @@ class AnthropicAssistantContextAggregator(LLMAssistantContextAggregator):
                 self._function_call_result = None
         elif isinstance(frame, AnthropicImageMessageFrame):
             self._pending_image_frame_message = frame
-            await self._push_aggregation()
+            await self.push_aggregation()
 
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if not (
             self._aggregation or self._function_call_result or self._pending_image_frame_message
         ):
@@ -746,7 +744,7 @@ class AnthropicAssistantContextAggregator(LLMAssistantContextAggregator):
         properties: Optional[FunctionCallResultProperties] = None
 
         aggregation = self._aggregation
-        self._reset()
+        self.reset()
 
         try:
             if self._function_call_result:
@@ -799,15 +797,14 @@ class AnthropicAssistantContextAggregator(LLMAssistantContextAggregator):
                 run_llm = True
 
             if run_llm:
-                await self._user_context_aggregator.push_context_frame()
+                await self.push_context_frame(FrameDirection.UPSTREAM)
 
             # Emit the on_context_updated callback once the function call result is added to the context
             if properties and properties.on_context_updated is not None:
                 await properties.on_context_updated()
 
             # Push context frame
-            frame = OpenAILLMContextFrame(self._context)
-            await self.push_frame(frame)
+            await self.push_context_frame()
 
             # Push timestamp frame with current time
             timestamp_frame = OpenAILLMContextAssistantTimestampFrame(timestamp=time_now_iso8601())

src/pipecat/services/base_whisper.py

@@ -111,6 +111,8 @@ class BaseWhisperSTTService(SegmentedSTTService):
         api_key: Service API key. Defaults to None.
         base_url: Service API base URL. Defaults to None.
         language: Language of the audio input. Defaults to English.
+        prompt: Optional text to guide the model's style or continue a previous segment.
+        temperature: Sampling temperature between 0 and 1. Defaults to 0.0.
         **kwargs: Additional arguments passed to SegmentedSTTService.
     """
 
@@ -121,12 +123,16 @@ class BaseWhisperSTTService(SegmentedSTTService):
         api_key: Optional[str] = None,
         base_url: Optional[str] = None,
         language: Optional[Language] = Language.EN,
+        prompt: Optional[str] = None,
+        temperature: Optional[float] = None,
         **kwargs,
     ):
         super().__init__(**kwargs)
         self.set_model_name(model)
         self._client = self._create_client(api_key, base_url)
         self._language = self.language_to_service_language(language or Language.EN)
+        self._prompt = prompt
+        self._temperature = temperature
 
     def _create_client(self, api_key: Optional[str], base_url: Optional[str]):
         return AsyncOpenAI(api_key=api_key, base_url=base_url)

src/pipecat/services/cartesia.py

@@ -27,7 +27,7 @@ from pipecat.frames.frames import (
     TTSStoppedFrame,
 )
 from pipecat.processors.frame_processor import FrameDirection
-from pipecat.services.ai_services import TTSService, WordTTSService
+from pipecat.services.ai_services import AudioContextWordTTSService, TTSService
 from pipecat.services.websocket_service import WebsocketService
 from pipecat.transcriptions.language import Language
 
@@ -75,7 +75,7 @@ def language_to_cartesia_language(language: Language) -> Optional[str]:
     return result
 
 
-class CartesiaTTSService(WordTTSService, WebsocketService):
+class CartesiaTTSService(AudioContextWordTTSService, WebsocketService):
     class InputParams(BaseModel):
         language: Optional[Language] = Language.EN
         speed: Optional[Union[str, float]] = ""
@@ -105,10 +105,11 @@ class CartesiaTTSService(WordTTSService, WebsocketService):
         # if we're interrupted. Cartesia gives us word-by-word timestamps. We
         # can use those to generate text frames ourselves aligned with the
         # playout timing of the audio!
-        WordTTSService.__init__(
+        AudioContextWordTTSService.__init__(
             self,
             aggregate_sentences=True,
             push_text_frames=False,
+            pause_frame_processing=True,
             sample_rate=sample_rate,
             **kwargs,
         )
@@ -191,12 +192,12 @@ class CartesiaTTSService(WordTTSService, WebsocketService):
         self._receive_task = self.create_task(self._receive_task_handler(self.push_error))
 
     async def _disconnect(self):
-        await self._disconnect_websocket()
-
         if self._receive_task:
             await self.cancel_task(self._receive_task)
             self._receive_task = None
 
+        await self._disconnect_websocket()
+
     async def _connect_websocket(self):
         try:
             logger.debug("Connecting to Cartesia")
@@ -239,21 +240,19 @@ class CartesiaTTSService(WordTTSService, WebsocketService):
         logger.trace(f"{self}: flushing audio")
         msg = self._build_msg(text="", continue_transcript=False)
         await self._websocket.send(msg)
+        self._context_id = None
 
     async def _receive_messages(self):
         async for message in self._get_websocket():
             msg = json.loads(message)
-            if not msg or msg["context_id"] != self._context_id:
+            if not msg or not self.audio_context_available(msg["context_id"]):
                 continue
             if msg["type"] == "done":
                 await self.stop_ttfb_metrics()
-                # Unset _context_id but not the _context_id_start_timestamp
-                # because we are likely still playing out audio and need the
-                # timestamp to set send context frames.
-                self._context_id = None
                 await self.add_word_timestamps(
                     [("TTSStoppedFrame", 0), ("LLMFullResponseEndFrame", 0), ("Reset", 0)]
                 )
+                await self.remove_audio_context(msg["context_id"])
             elif msg["type"] == "timestamps":
                 await self.add_word_timestamps(
                     list(zip(msg["word_timestamps"]["words"], msg["word_timestamps"]["start"]))
@@ -266,28 +265,16 @@ class CartesiaTTSService(WordTTSService, WebsocketService):
                     sample_rate=self.sample_rate,
                     num_channels=1,
                 )
-                await self.push_frame(frame)
+                await self.append_to_audio_context(msg["context_id"], frame)
             elif msg["type"] == "error":
                 logger.error(f"{self} error: {msg}")
                 await self.push_frame(TTSStoppedFrame())
                 await self.stop_all_metrics()
                 await self.push_error(ErrorFrame(f"{self} error: {msg['error']}"))
+                self._context_id = None
             else:
                 logger.error(f"{self} error, unknown message type: {msg}")
 
-    async def process_frame(self, frame: Frame, direction: FrameDirection):
-        await super().process_frame(frame, direction)
-
-        # If we received a TTSSpeakFrame and the LLM response included text (it
-        # might be that it's only a function calling response) we pause
-        # processing more frames until we receive a BotStoppedSpeakingFrame.
-        if isinstance(frame, TTSSpeakFrame):
-            await self.pause_processing_frames()
-        elif isinstance(frame, LLMFullResponseEndFrame) and self._context_id:
-            await self.pause_processing_frames()
-        elif isinstance(frame, BotStoppedSpeakingFrame):
-            await self.resume_processing_frames()
-
     async def run_tts(self, text: str) -> AsyncGenerator[Frame, None]:
         logger.debug(f"Generating TTS: [{text}]")
 
@@ -299,6 +286,7 @@ class CartesiaTTSService(WordTTSService, WebsocketService):
                 await self.start_ttfb_metrics()
                 yield TTSStartedFrame()
                 self._context_id = str(uuid.uuid4())
+                await self.create_audio_context(self._context_id)
 
             msg = self._build_msg(text=text or " ")  # Text must contain at least one character
 

src/pipecat/services/elevenlabs.py

@@ -192,6 +192,7 @@ class ElevenLabsTTSService(WordTTSService, WebsocketService):
             push_text_frames=False,
             push_stop_frames=True,
             stop_frame_timeout_s=2.0,
+            pause_frame_processing=True,
             sample_rate=sample_rate,
             **kwargs,
         )
@@ -289,19 +290,6 @@ class ElevenLabsTTSService(WordTTSService, WebsocketService):
             if isinstance(frame, TTSStoppedFrame):
                 await self.add_word_timestamps([("LLMFullResponseEndFrame", 0), ("Reset", 0)])
 
-    async def process_frame(self, frame: Frame, direction: FrameDirection):
-        await super().process_frame(frame, direction)
-
-        # If we received a TTSSpeakFrame and the LLM response included text (it
-        # might be that it's only a function calling response) we pause
-        # processing more frames until we receive a BotStoppedSpeakingFrame.
-        if isinstance(frame, TTSSpeakFrame):
-            await self.pause_processing_frames()
-        elif isinstance(frame, LLMFullResponseEndFrame) and self._started:
-            await self.pause_processing_frames()
-        elif isinstance(frame, BotStoppedSpeakingFrame):
-            await self.resume_processing_frames()
-
     async def _connect(self):
         await self._connect_websocket()
 
@@ -370,8 +358,13 @@ class ElevenLabsTTSService(WordTTSService, WebsocketService):
         except Exception as e:
             logger.error(f"{self} error closing websocket: {e}")
 
+    def _get_websocket(self):
+        if self._websocket:
+            return self._websocket
+        raise Exception("Websocket not connected")
+
     async def _receive_messages(self):
-        async for message in self._websocket:
+        async for message in self._get_websocket():
             msg = json.loads(message)
             if msg.get("audio"):
                 await self.stop_ttfb_metrics()

src/pipecat/services/fal.py

@@ -4,6 +4,7 @@
 # SPDX-License-Identifier: BSD 2-Clause License
 #
 
+import asyncio
 import io
 import os
 from typing import AsyncGenerator, Dict, Optional, Union
@@ -53,6 +54,11 @@ class FalImageGenService(ImageGenService):
             os.environ["FAL_KEY"] = key
 
     async def run_image_gen(self, prompt: str) -> AsyncGenerator[Frame, None]:
+        def load_image_bytes(encoded_image: bytes):
+            buffer = io.BytesIO(encoded_image)
+            image = Image.open(buffer)
+            return (image.tobytes(), image.size, image.format)
+
         logger.debug(f"Generating image from prompt: {prompt}")
 
         response = await fal_client.run_async(
@@ -73,10 +79,8 @@ class FalImageGenService(ImageGenService):
         logger.debug(f"Downloading image {image_url} ...")
         async with self._aiohttp_session.get(image_url) as response:
             logger.debug(f"Downloaded image {image_url}")
-            image_stream = io.BytesIO(await response.content.read())
-            image = Image.open(image_stream)
+            encoded_image = await response.content.read()
+            (image_bytes, size, format) = await asyncio.to_thread(load_image_bytes, encoded_image)
 
-            frame = URLImageRawFrame(
-                url=image_url, image=image.tobytes(), size=image.size, format=image.format
-            )
+            frame = URLImageRawFrame(url=image_url, image=image_bytes, size=size, format=format)
             yield frame

src/pipecat/services/fish.py

@@ -60,7 +60,7 @@ class FishAudioTTSService(TTSService, WebsocketService):
         params: InputParams = InputParams(),
         **kwargs,
     ):
-        super().__init__(sample_rate=sample_rate, **kwargs)
+        super().__init__(pause_frame_processing=True, sample_rate=sample_rate, **kwargs)
 
         self._api_key = api_key
         self._base_url = "wss://api.fish.audio/v1/tts/live"
@@ -166,16 +166,6 @@ class FishAudioTTSService(TTSService, WebsocketService):
             except Exception as e:
                 logger.error(f"Error processing message: {e}")
 
-    async def process_frame(self, frame: Frame, direction: FrameDirection):
-        await super().process_frame(frame, direction)
-
-        if isinstance(frame, TTSSpeakFrame):
-            await self.pause_processing_frames()
-        elif isinstance(frame, LLMFullResponseEndFrame) and self._request_id:
-            await self.pause_processing_frames()
-        elif isinstance(frame, BotStoppedSpeakingFrame):
-            await self.resume_processing_frames()
-
     async def _handle_interruption(self, frame: StartInterruptionFrame, direction: FrameDirection):
         await super()._handle_interruption(frame, direction)
         await self.stop_all_metrics()

src/pipecat/services/gemini_multimodal_live/gemini.py

@@ -115,10 +115,10 @@ class GeminiMultimodalLiveUserContextAggregator(OpenAIUserContextAggregator):
 
 
 class GeminiMultimodalLiveAssistantContextAggregator(OpenAIAssistantContextAggregator):
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         # We don't want to store any images in the context. Revisit this later when the API evolves.
         self._pending_image_frame_message = None
-        await super()._push_aggregation()
+        await super().push_aggregation()
 
 
 @dataclass
@@ -706,6 +706,6 @@ class GeminiMultimodalLiveLLMService(LLMService):
         GeminiMultimodalLiveContext.upgrade(context)
         user = GeminiMultimodalLiveUserContextAggregator(context)
         assistant = GeminiMultimodalLiveAssistantContextAggregator(
-            user, expect_stripped_words=assistant_expect_stripped_words
+            context, expect_stripped_words=assistant_expect_stripped_words
         )
         return GeminiMultimodalLiveContextAggregatorPair(_user=user, _assistant=assistant)

src/pipecat/services/google/google.py

@@ -8,24 +8,35 @@ import asyncio
 import base64
 import io
 import json
+import os
+import time
+
+# Suppress gRPC fork warnings
+os.environ["GRPC_ENABLE_FORK_SUPPORT"] = "false"
+
 from dataclasses import dataclass
-from typing import Any, AsyncGenerator, Dict, List, Literal, Optional
+from typing import Any, AsyncGenerator, Dict, List, Literal, Optional, Union
 
 from loguru import logger
 from PIL import Image
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, field_validator
 
 from pipecat.frames.frames import (
     AudioRawFrame,
+    CancelFrame,
+    EndFrame,
     ErrorFrame,
     Frame,
     FunctionCallResultProperties,
+    InterimTranscriptionFrame,
     LLMFullResponseEndFrame,
     LLMFullResponseStartFrame,
     LLMMessagesFrame,
     LLMTextFrame,
     LLMUpdateSettingsFrame,
     OpenAILLMContextAssistantTimestampFrame,
+    StartFrame,
+    TranscriptionFrame,
     TTSAudioRawFrame,
     TTSStartedFrame,
     TTSStoppedFrame,
@@ -38,7 +49,7 @@ from pipecat.processors.aggregators.openai_llm_context import (
     OpenAILLMContextFrame,
 )
 from pipecat.processors.frame_processor import FrameDirection
-from pipecat.services.ai_services import ImageGenService, LLMService, TTSService
+from pipecat.services.ai_services import ImageGenService, LLMService, STTService, TTSService
 from pipecat.services.google.frames import LLMSearchResponseFrame
 from pipecat.services.openai import (
     OpenAIAssistantContextAggregator,
@@ -51,10 +62,13 @@ try:
     import google.ai.generativelanguage as glm
     import google.generativeai as gai
     from google import genai
-    from google.cloud import texttospeech_v1
+    from google.api_core.client_options import ClientOptions
+    from google.cloud import speech_v2, texttospeech_v1
+    from google.cloud.speech_v2.types import cloud_speech
     from google.genai import types
     from google.generativeai.types import GenerationConfig
     from google.oauth2 import service_account
+
 except ModuleNotFoundError as e:
     logger.error(f"Exception: {e}")
     logger.error(
@@ -63,7 +77,7 @@ except ModuleNotFoundError as e:
     raise Exception(f"Missing module: {e}")
 
 
-def language_to_google_language(language: Language) -> Optional[str]:
+def language_to_google_tts_language(language: Language) -> Optional[str]:
     language_map = {
         # Afrikaans
         Language.AF: "af-ZA",
@@ -223,8 +237,307 @@ def language_to_google_language(language: Language) -> Optional[str]:
     return language_map.get(language)
 
 
+def language_to_google_stt_language(language: Language) -> Optional[str]:
+    """Maps Language enum to Google Speech-to-Text V2 language codes.
+
+    Args:
+        language: Language enum value.
+
+    Returns:
+        Optional[str]: Google STT language code or None if not supported.
+    """
+    language_map = {
+        # Afrikaans
+        Language.AF: "af-ZA",
+        Language.AF_ZA: "af-ZA",
+        # Albanian
+        Language.SQ: "sq-AL",
+        Language.SQ_AL: "sq-AL",
+        # Amharic
+        Language.AM: "am-ET",
+        Language.AM_ET: "am-ET",
+        # Arabic
+        Language.AR: "ar-EG",  # Default to Egypt
+        Language.AR_AE: "ar-AE",
+        Language.AR_BH: "ar-BH",
+        Language.AR_DZ: "ar-DZ",
+        Language.AR_EG: "ar-EG",
+        Language.AR_IQ: "ar-IQ",
+        Language.AR_JO: "ar-JO",
+        Language.AR_KW: "ar-KW",
+        Language.AR_LB: "ar-LB",
+        Language.AR_MA: "ar-MA",
+        Language.AR_OM: "ar-OM",
+        Language.AR_QA: "ar-QA",
+        Language.AR_SA: "ar-SA",
+        Language.AR_SY: "ar-SY",
+        Language.AR_TN: "ar-TN",
+        Language.AR_YE: "ar-YE",
+        # Armenian
+        Language.HY: "hy-AM",
+        Language.HY_AM: "hy-AM",
+        # Azerbaijani
+        Language.AZ: "az-AZ",
+        Language.AZ_AZ: "az-AZ",
+        # Basque
+        Language.EU: "eu-ES",
+        Language.EU_ES: "eu-ES",
+        # Bengali
+        Language.BN: "bn-IN",  # Default to India
+        Language.BN_BD: "bn-BD",
+        Language.BN_IN: "bn-IN",
+        # Bosnian
+        Language.BS: "bs-BA",
+        Language.BS_BA: "bs-BA",
+        # Bulgarian
+        Language.BG: "bg-BG",
+        Language.BG_BG: "bg-BG",
+        # Burmese
+        Language.MY: "my-MM",
+        Language.MY_MM: "my-MM",
+        # Catalan
+        Language.CA: "ca-ES",
+        Language.CA_ES: "ca-ES",
+        # Chinese
+        Language.ZH: "cmn-Hans-CN",  # Default to Simplified Chinese
+        Language.ZH_CN: "cmn-Hans-CN",
+        Language.ZH_HK: "cmn-Hans-HK",
+        Language.ZH_TW: "cmn-Hant-TW",
+        Language.YUE: "yue-Hant-HK",  # Cantonese
+        Language.YUE_CN: "yue-Hant-HK",
+        # Croatian
+        Language.HR: "hr-HR",
+        Language.HR_HR: "hr-HR",
+        # Czech
+        Language.CS: "cs-CZ",
+        Language.CS_CZ: "cs-CZ",
+        # Danish
+        Language.DA: "da-DK",
+        Language.DA_DK: "da-DK",
+        # Dutch
+        Language.NL: "nl-NL",  # Default to Netherlands
+        Language.NL_BE: "nl-BE",
+        Language.NL_NL: "nl-NL",
+        # English
+        Language.EN: "en-US",  # Default to US
+        Language.EN_AU: "en-AU",
+        Language.EN_CA: "en-CA",
+        Language.EN_GB: "en-GB",
+        Language.EN_GH: "en-GH",
+        Language.EN_HK: "en-HK",
+        Language.EN_IN: "en-IN",
+        Language.EN_IE: "en-IE",
+        Language.EN_KE: "en-KE",
+        Language.EN_NG: "en-NG",
+        Language.EN_NZ: "en-NZ",
+        Language.EN_PH: "en-PH",
+        Language.EN_SG: "en-SG",
+        Language.EN_TZ: "en-TZ",
+        Language.EN_US: "en-US",
+        Language.EN_ZA: "en-ZA",
+        # Estonian
+        Language.ET: "et-EE",
+        Language.ET_EE: "et-EE",
+        # Filipino
+        Language.FIL: "fil-PH",
+        Language.FIL_PH: "fil-PH",
+        # Finnish
+        Language.FI: "fi-FI",
+        Language.FI_FI: "fi-FI",
+        # French
+        Language.FR: "fr-FR",  # Default to France
+        Language.FR_BE: "fr-BE",
+        Language.FR_CA: "fr-CA",
+        Language.FR_CH: "fr-CH",
+        Language.FR_FR: "fr-FR",
+        # Galician
+        Language.GL: "gl-ES",
+        Language.GL_ES: "gl-ES",
+        # Georgian
+        Language.KA: "ka-GE",
+        Language.KA_GE: "ka-GE",
+        # German
+        Language.DE: "de-DE",  # Default to Germany
+        Language.DE_AT: "de-AT",
+        Language.DE_CH: "de-CH",
+        Language.DE_DE: "de-DE",
+        # Greek
+        Language.EL: "el-GR",
+        Language.EL_GR: "el-GR",
+        # Gujarati
+        Language.GU: "gu-IN",
+        Language.GU_IN: "gu-IN",
+        # Hebrew
+        Language.HE: "iw-IL",
+        Language.HE_IL: "iw-IL",
+        # Hindi
+        Language.HI: "hi-IN",
+        Language.HI_IN: "hi-IN",
+        # Hungarian
+        Language.HU: "hu-HU",
+        Language.HU_HU: "hu-HU",
+        # Icelandic
+        Language.IS: "is-IS",
+        Language.IS_IS: "is-IS",
+        # Indonesian
+        Language.ID: "id-ID",
+        Language.ID_ID: "id-ID",
+        # Italian
+        Language.IT: "it-IT",
+        Language.IT_IT: "it-IT",
+        Language.IT_CH: "it-CH",
+        # Japanese
+        Language.JA: "ja-JP",
+        Language.JA_JP: "ja-JP",
+        # Javanese
+        Language.JV: "jv-ID",
+        Language.JV_ID: "jv-ID",
+        # Kannada
+        Language.KN: "kn-IN",
+        Language.KN_IN: "kn-IN",
+        # Kazakh
+        Language.KK: "kk-KZ",
+        Language.KK_KZ: "kk-KZ",
+        # Khmer
+        Language.KM: "km-KH",
+        Language.KM_KH: "km-KH",
+        # Korean
+        Language.KO: "ko-KR",
+        Language.KO_KR: "ko-KR",
+        # Lao
+        Language.LO: "lo-LA",
+        Language.LO_LA: "lo-LA",
+        # Latvian
+        Language.LV: "lv-LV",
+        Language.LV_LV: "lv-LV",
+        # Lithuanian
+        Language.LT: "lt-LT",
+        Language.LT_LT: "lt-LT",
+        # Macedonian
+        Language.MK: "mk-MK",
+        Language.MK_MK: "mk-MK",
+        # Malay
+        Language.MS: "ms-MY",
+        Language.MS_MY: "ms-MY",
+        # Malayalam
+        Language.ML: "ml-IN",
+        Language.ML_IN: "ml-IN",
+        # Marathi
+        Language.MR: "mr-IN",
+        Language.MR_IN: "mr-IN",
+        # Mongolian
+        Language.MN: "mn-MN",
+        Language.MN_MN: "mn-MN",
+        # Nepali
+        Language.NE: "ne-NP",
+        Language.NE_NP: "ne-NP",
+        # Norwegian
+        Language.NO: "no-NO",
+        Language.NB: "no-NO",
+        Language.NB_NO: "no-NO",
+        # Persian
+        Language.FA: "fa-IR",
+        Language.FA_IR: "fa-IR",
+        # Polish
+        Language.PL: "pl-PL",
+        Language.PL_PL: "pl-PL",
+        # Portuguese
+        Language.PT: "pt-PT",  # Default to Portugal
+        Language.PT_BR: "pt-BR",
+        Language.PT_PT: "pt-PT",
+        # Punjabi
+        Language.PA: "pa-Guru-IN",
+        Language.PA_IN: "pa-Guru-IN",
+        # Romanian
+        Language.RO: "ro-RO",
+        Language.RO_RO: "ro-RO",
+        # Russian
+        Language.RU: "ru-RU",
+        Language.RU_RU: "ru-RU",
+        # Serbian
+        Language.SR: "sr-RS",
+        Language.SR_RS: "sr-RS",
+        # Sinhala
+        Language.SI: "si-LK",
+        Language.SI_LK: "si-LK",
+        # Slovak
+        Language.SK: "sk-SK",
+        Language.SK_SK: "sk-SK",
+        # Slovenian
+        Language.SL: "sl-SI",
+        Language.SL_SI: "sl-SI",
+        # Spanish
+        Language.ES: "es-ES",  # Default to Spain
+        Language.ES_AR: "es-AR",
+        Language.ES_BO: "es-BO",
+        Language.ES_CL: "es-CL",
+        Language.ES_CO: "es-CO",
+        Language.ES_CR: "es-CR",
+        Language.ES_DO: "es-DO",
+        Language.ES_EC: "es-EC",
+        Language.ES_ES: "es-ES",
+        Language.ES_GT: "es-GT",
+        Language.ES_HN: "es-HN",
+        Language.ES_MX: "es-MX",
+        Language.ES_NI: "es-NI",
+        Language.ES_PA: "es-PA",
+        Language.ES_PE: "es-PE",
+        Language.ES_PR: "es-PR",
+        Language.ES_PY: "es-PY",
+        Language.ES_SV: "es-SV",
+        Language.ES_US: "es-US",
+        Language.ES_UY: "es-UY",
+        Language.ES_VE: "es-VE",
+        # Sundanese
+        Language.SU: "su-ID",
+        Language.SU_ID: "su-ID",
+        # Swahili
+        Language.SW: "sw-TZ",  # Default to Tanzania
+        Language.SW_KE: "sw-KE",
+        Language.SW_TZ: "sw-TZ",
+        # Swedish
+        Language.SV: "sv-SE",
+        Language.SV_SE: "sv-SE",
+        # Tamil
+        Language.TA: "ta-IN",  # Default to India
+        Language.TA_IN: "ta-IN",
+        Language.TA_MY: "ta-MY",
+        Language.TA_SG: "ta-SG",
+        Language.TA_LK: "ta-LK",
+        # Telugu
+        Language.TE: "te-IN",
+        Language.TE_IN: "te-IN",
+        # Thai
+        Language.TH: "th-TH",
+        Language.TH_TH: "th-TH",
+        # Turkish
+        Language.TR: "tr-TR",
+        Language.TR_TR: "tr-TR",
+        # Ukrainian
+        Language.UK: "uk-UA",
+        Language.UK_UA: "uk-UA",
+        # Urdu
+        Language.UR: "ur-IN",  # Default to India
+        Language.UR_IN: "ur-IN",
+        Language.UR_PK: "ur-PK",
+        # Uzbek
+        Language.UZ: "uz-UZ",
+        Language.UZ_UZ: "uz-UZ",
+        # Vietnamese
+        Language.VI: "vi-VN",
+        Language.VI_VN: "vi-VN",
+        # Xhosa
+        Language.XH: "xh-ZA",
+        # Zulu
+        Language.ZU: "zu-ZA",
+        Language.ZU_ZA: "zu-ZA",
+    }
+
+    return language_map.get(language)
+
+
 class GoogleUserContextAggregator(OpenAIUserContextAggregator):
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if len(self._aggregation) > 0:
             self._context.add_message(
                 glm.Content(role="user", parts=[glm.Part(text=self._aggregation)])
@@ -239,11 +552,11 @@ class GoogleUserContextAggregator(OpenAIUserContextAggregator):
             await self.push_frame(frame)
 
             # Reset our accumulator state.
-            self._reset()
+            self.reset()
 
 
 class GoogleAssistantContextAggregator(OpenAIAssistantContextAggregator):
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if not (
             self._aggregation or self._function_call_result or self._pending_image_frame_message
         ):
@@ -253,7 +566,7 @@ class GoogleAssistantContextAggregator(OpenAIAssistantContextAggregator):
         properties: Optional[FunctionCallResultProperties] = None
 
         aggregation = self._aggregation
-        self._reset()
+        self.reset()
 
         try:
             if self._function_call_result:
@@ -313,15 +626,14 @@ class GoogleAssistantContextAggregator(OpenAIAssistantContextAggregator):
                 run_llm = True
 
             if run_llm:
-                await self._user_context_aggregator.push_context_frame()
+                await self.push_context_frame(FrameDirection.UPSTREAM)
 
             # Emit the on_context_updated callback once the function call result is added to the context
             if properties and properties.on_context_updated is not None:
                 await properties.on_context_updated()
 
             # Push context frame
-            frame = OpenAILLMContextFrame(self._context)
-            await self.push_frame(frame)
+            await self.push_context_frame()
 
             # Push timestamp frame with current time
             timestamp_frame = OpenAILLMContextAssistantTimestampFrame(timestamp=time_now_iso8601())
@@ -604,9 +916,9 @@ class GoogleLLMContext(OpenAILLMContext):
         # Check if we only have function-related messages (no regular text)
         has_regular_messages = any(
             len(msg.parts) == 1
-            and hasattr(msg.parts[0], "text")
-            and not hasattr(msg.parts[0], "function_call")
-            and not hasattr(msg.parts[0], "function_response")
+            and not getattr(msg.parts[0], "text", None)
+            and getattr(msg.parts[0], "function_call", None)
+            and getattr(msg.parts[0], "function_response", None)
             for msg in self._messages
         )
 
@@ -621,7 +933,7 @@ class GoogleLLMContext(OpenAILLMContext):
 
 
 class GoogleLLMService(LLMService):
-    """This class implements inference with Google's AI models
+    """This class implements inference with Google's AI models.
 
     This service translates internally from OpenAILLMContext to the messages format
     expected by the Google AI model. We are using the OpenAILLMContext as a lingua
@@ -862,7 +1174,7 @@ class GoogleLLMService(LLMService):
     ) -> GoogleContextAggregatorPair:
         user = GoogleUserContextAggregator(context)
         assistant = GoogleAssistantContextAggregator(
-            user, expect_stripped_words=assistant_expect_stripped_words
+            context, expect_stripped_words=assistant_expect_stripped_words
         )
         return GoogleContextAggregatorPair(_user=user, _assistant=assistant)
 
@@ -927,7 +1239,7 @@ class GoogleTTSService(TTSService):
         return True
 
     def language_to_service_language(self, language: Language) -> Optional[str]:
-        return language_to_google_language(language)
+        return language_to_google_tts_language(language)
 
     def _construct_ssml(self, text: str) -> str:
         ssml = "<speak>"
@@ -1097,3 +1409,451 @@ class GoogleImageGenService(ImageGenService):
         except Exception as e:
             logger.error(f"{self} error generating image: {e}")
             yield ErrorFrame(f"Image generation error: {str(e)}")
+
+
+class GoogleSTTService(STTService):
+    """Google Cloud Speech-to-Text V2 service implementation.
+
+    Provides real-time speech recognition using Google Cloud's Speech-to-Text V2 API
+    with streaming support. Handles audio transcription and optional voice activity detection.
+
+    Attributes:
+        InputParams: Configuration parameters for the STT service.
+    """
+
+    # Google Cloud's STT service has a connection time limit of 5 minutes per stream.
+    # They've shared an "endless streaming" example that guided this implementation:
+    # https://cloud.google.com/speech-to-text/docs/transcribe-streaming-audio#endless-streaming
+
+    STREAMING_LIMIT = 240000  # 4 minutes in milliseconds
+
+    class InputParams(BaseModel):
+        """Configuration parameters for Google Speech-to-Text.
+
+        Attributes:
+            languages: Single language or list of recognition languages. First language is primary.
+            model: Speech recognition model to use.
+            use_separate_recognition_per_channel: Process each audio channel separately.
+            enable_automatic_punctuation: Add punctuation to transcripts.
+            enable_spoken_punctuation: Include spoken punctuation in transcript.
+            enable_spoken_emojis: Include spoken emojis in transcript.
+            profanity_filter: Filter profanity from transcript.
+            enable_word_time_offsets: Include timing information for each word.
+            enable_word_confidence: Include confidence scores for each word.
+            enable_interim_results: Stream partial recognition results.
+            enable_voice_activity_events: Detect voice activity in audio.
+        """
+
+        languages: Union[Language, List[Language]] = Field(default_factory=lambda: [Language.EN_US])
+        model: Optional[str] = "latest_long"
+        use_separate_recognition_per_channel: Optional[bool] = False
+        enable_automatic_punctuation: Optional[bool] = True
+        enable_spoken_punctuation: Optional[bool] = False
+        enable_spoken_emojis: Optional[bool] = False
+        profanity_filter: Optional[bool] = False
+        enable_word_time_offsets: Optional[bool] = False
+        enable_word_confidence: Optional[bool] = False
+        enable_interim_results: Optional[bool] = True
+        enable_voice_activity_events: Optional[bool] = False
+
+        @field_validator("languages", mode="before")
+        @classmethod
+        def validate_languages(cls, v) -> List[Language]:
+            if isinstance(v, Language):
+                return [v]
+            return v
+
+        @property
+        def language_list(self) -> List[Language]:
+            """Get languages as a guaranteed list."""
+            assert isinstance(self.languages, list)
+            return self.languages
+
+    def __init__(
+        self,
+        *,
+        credentials: Optional[str] = None,
+        credentials_path: Optional[str] = None,
+        location: str = "global",
+        sample_rate: Optional[int] = None,
+        params: InputParams = InputParams(),
+        **kwargs,
+    ):
+        """Initialize the Google STT service.
+
+        Args:
+            credentials: JSON string containing Google Cloud service account credentials.
+            credentials_path: Path to service account credentials JSON file.
+            location: Google Cloud location (e.g., "global", "us-central1").
+            sample_rate: Audio sample rate in Hertz.
+            params: Configuration parameters for the service.
+            **kwargs: Additional arguments passed to STTService.
+
+        Raises:
+            ValueError: If neither credentials nor credentials_path is provided.
+            ValueError: If project ID is not found in credentials.
+        """
+        super().__init__(sample_rate=sample_rate, **kwargs)
+
+        self._location = location
+        self._stream = None
+        self._config = None
+        self._request_queue = asyncio.Queue()
+        self._streaming_task = None
+
+        # Used for keep-alive logic
+        self._stream_start_time = 0
+        self._last_audio_input = []
+        self._audio_input = []
+        self._result_end_time = 0
+        self._is_final_end_time = 0
+        self._final_request_end_time = 0
+        self._bridging_offset = 0
+        self._last_transcript_was_final = False
+        self._new_stream = True
+        self._restart_counter = 0
+
+        # Configure client options based on location
+        client_options = None
+        if self._location != "global":
+            client_options = ClientOptions(api_endpoint=f"{self._location}-speech.googleapis.com")
+
+        # Extract project ID and create client
+        if credentials:
+            json_account_info = json.loads(credentials)
+            self._project_id = json_account_info.get("project_id")
+            creds = service_account.Credentials.from_service_account_info(json_account_info)
+        elif credentials_path:
+            with open(credentials_path) as f:
+                json_account_info = json.load(f)
+                self._project_id = json_account_info.get("project_id")
+            creds = service_account.Credentials.from_service_account_file(credentials_path)
+        else:
+            raise ValueError("Either credentials or credentials_path must be provided")
+
+        if not self._project_id:
+            raise ValueError("Project ID not found in credentials")
+
+        self._client = speech_v2.SpeechAsyncClient(credentials=creds, client_options=client_options)
+
+        self._settings = {
+            "language_codes": [
+                self.language_to_service_language(lang) for lang in params.language_list
+            ],
+            "model": params.model,
+            "use_separate_recognition_per_channel": params.use_separate_recognition_per_channel,
+            "enable_automatic_punctuation": params.enable_automatic_punctuation,
+            "enable_spoken_punctuation": params.enable_spoken_punctuation,
+            "enable_spoken_emojis": params.enable_spoken_emojis,
+            "profanity_filter": params.profanity_filter,
+            "enable_word_time_offsets": params.enable_word_time_offsets,
+            "enable_word_confidence": params.enable_word_confidence,
+            "enable_interim_results": params.enable_interim_results,
+            "enable_voice_activity_events": params.enable_voice_activity_events,
+        }
+
+    def language_to_service_language(self, language: Language | List[Language]) -> str | List[str]:
+        """Convert Language enum(s) to Google STT language code(s).
+
+        Args:
+            language: Single Language enum or list of Language enums.
+
+        Returns:
+            str | List[str]: Google STT language code(s).
+        """
+        if isinstance(language, list):
+            return [language_to_google_stt_language(lang) or "en-US" for lang in language]
+        return language_to_google_stt_language(language) or "en-US"
+
+    async def _reconnect_if_needed(self):
+        """Reconnect the stream if it's currently active."""
+        if self._streaming_task:
+            logger.debug("Reconnecting stream due to configuration changes")
+            await self._disconnect()
+            await self._connect()
+
+    async def set_languages(self, languages: List[Language]):
+        """Update the service's recognition languages.
+
+        Args:
+            languages: List of languages for recognition. First language is primary.
+        """
+        logger.debug(f"Switching STT languages to: {languages}")
+        self._settings["language_codes"] = [
+            self.language_to_service_language(lang) for lang in languages
+        ]
+        # Recreate stream with new languages
+        await self._reconnect_if_needed()
+
+    async def set_model(self, model: str):
+        """Update the service's recognition model."""
+        logger.debug(f"Switching STT model to: {model}")
+        await super().set_model(model)
+        self._settings["model"] = model
+        # Recreate stream with new model
+        await self._reconnect_if_needed()
+
+    async def start(self, frame: StartFrame):
+        await super().start(frame)
+        await self._connect()
+
+    async def stop(self, frame: EndFrame):
+        await super().stop(frame)
+        await self._disconnect()
+
+    async def cancel(self, frame: CancelFrame):
+        await super().cancel(frame)
+        await self._disconnect()
+
+    async def update_options(
+        self,
+        *,
+        languages: Optional[List[Language]] = None,
+        model: Optional[str] = None,
+        enable_automatic_punctuation: Optional[bool] = None,
+        enable_spoken_punctuation: Optional[bool] = None,
+        enable_spoken_emojis: Optional[bool] = None,
+        profanity_filter: Optional[bool] = None,
+        enable_word_time_offsets: Optional[bool] = None,
+        enable_word_confidence: Optional[bool] = None,
+        enable_interim_results: Optional[bool] = None,
+        enable_voice_activity_events: Optional[bool] = None,
+        location: Optional[str] = None,
+    ) -> None:
+        """Update service options dynamically.
+
+        Args:
+            languages: New list of recongition languages.
+            model: New recognition model.
+            enable_automatic_punctuation: Enable/disable automatic punctuation.
+            enable_spoken_punctuation: Enable/disable spoken punctuation.
+            enable_spoken_emojis: Enable/disable spoken emojis.
+            profanity_filter: Enable/disable profanity filter.
+            enable_word_time_offsets: Enable/disable word timing info.
+            enable_word_confidence: Enable/disable word confidence scores.
+            enable_interim_results: Enable/disable interim results.
+            enable_voice_activity_events: Enable/disable voice activity detection.
+            location: New Google Cloud location.
+
+        Note:
+            Changes that affect the streaming configuration will cause
+            the stream to be reconnected.
+        """
+        # Update settings with new values
+        if languages is not None:
+            logger.debug(f"Updating language to: {languages}")
+            self._settings["language_codes"] = [
+                self.language_to_service_language(lang) for lang in languages
+            ]
+
+        if model is not None:
+            logger.debug(f"Updating model to: {model}")
+            self._settings["model"] = model
+
+        if enable_automatic_punctuation is not None:
+            logger.debug(f"Updating automatic punctuation to: {enable_automatic_punctuation}")
+            self._settings["enable_automatic_punctuation"] = enable_automatic_punctuation
+
+        if enable_spoken_punctuation is not None:
+            logger.debug(f"Updating spoken punctuation to: {enable_spoken_punctuation}")
+            self._settings["enable_spoken_punctuation"] = enable_spoken_punctuation
+
+        if enable_spoken_emojis is not None:
+            logger.debug(f"Updating spoken emojis to: {enable_spoken_emojis}")
+            self._settings["enable_spoken_emojis"] = enable_spoken_emojis
+
+        if profanity_filter is not None:
+            logger.debug(f"Updating profanity filter to: {profanity_filter}")
+            self._settings["profanity_filter"] = profanity_filter
+
+        if enable_word_time_offsets is not None:
+            logger.debug(f"Updating word time offsets to: {enable_word_time_offsets}")
+            self._settings["enable_word_time_offsets"] = enable_word_time_offsets
+
+        if enable_word_confidence is not None:
+            logger.debug(f"Updating word confidence to: {enable_word_confidence}")
+            self._settings["enable_word_confidence"] = enable_word_confidence
+
+        if enable_interim_results is not None:
+            logger.debug(f"Updating interim results to: {enable_interim_results}")
+            self._settings["enable_interim_results"] = enable_interim_results
+
+        if enable_voice_activity_events is not None:
+            logger.debug(f"Updating voice activity events to: {enable_voice_activity_events}")
+            self._settings["enable_voice_activity_events"] = enable_voice_activity_events
+
+        if location is not None:
+            logger.debug(f"Updating location to: {location}")
+            self._location = location
+
+        # Reconnect the stream for updates
+        await self._reconnect_if_needed()
+
+    async def _connect(self):
+        """Initialize streaming recognition config and stream."""
+        logger.debug("Connecting to Google Speech-to-Text")
+
+        # Set stream start time
+        self._stream_start_time = int(time.time() * 1000)
+        self._new_stream = True
+
+        self._config = cloud_speech.StreamingRecognitionConfig(
+            config=cloud_speech.RecognitionConfig(
+                explicit_decoding_config=cloud_speech.ExplicitDecodingConfig(
+                    encoding=cloud_speech.ExplicitDecodingConfig.AudioEncoding.LINEAR16,
+                    sample_rate_hertz=self.sample_rate,
+                    audio_channel_count=1,
+                ),
+                language_codes=self._settings["language_codes"],
+                model=self._settings["model"],
+                features=cloud_speech.RecognitionFeatures(
+                    enable_automatic_punctuation=self._settings["enable_automatic_punctuation"],
+                    enable_spoken_punctuation=self._settings["enable_spoken_punctuation"],
+                    enable_spoken_emojis=self._settings["enable_spoken_emojis"],
+                    profanity_filter=self._settings["profanity_filter"],
+                    enable_word_time_offsets=self._settings["enable_word_time_offsets"],
+                    enable_word_confidence=self._settings["enable_word_confidence"],
+                ),
+            ),
+            streaming_features=cloud_speech.StreamingRecognitionFeatures(
+                enable_voice_activity_events=self._settings["enable_voice_activity_events"],
+                interim_results=self._settings["enable_interim_results"],
+            ),
+        )
+
+        self._streaming_task = self.create_task(self._stream_audio())
+
+    async def _disconnect(self):
+        """Clean up streaming recognition resources."""
+        if self._streaming_task:
+            logger.debug("Disconnecting from Google Speech-to-Text")
+            # Send sentinel value to stop request generator
+            await self._request_queue.put(None)
+            await self.cancel_task(self._streaming_task)
+            self._streaming_task = None
+            # Clear any remaining items in the queue
+            while not self._request_queue.empty():
+                try:
+                    self._request_queue.get_nowait()
+                    self._request_queue.task_done()
+                except asyncio.QueueEmpty:
+                    break
+
+    async def _request_generator(self):
+        """Generates requests for the streaming recognize method."""
+        recognizer_path = f"projects/{self._project_id}/locations/{self._location}/recognizers/_"
+        logger.trace(f"Using recognizer path: {recognizer_path}")
+
+        try:
+            # Send initial config
+            yield cloud_speech.StreamingRecognizeRequest(
+                recognizer=recognizer_path,
+                streaming_config=self._config,
+            )
+
+            while True:
+                try:
+                    audio_data = await self._request_queue.get()
+                    if audio_data is None:  # Sentinel value to stop
+                        break
+
+                    # Check streaming limit
+                    if (int(time.time() * 1000) - self._stream_start_time) > self.STREAMING_LIMIT:
+                        logger.debug("Streaming limit reached, initiating graceful reconnection")
+                        # Instead of immediate reconnection, we'll break and let the stream close naturally
+                        self._last_audio_input = self._audio_input
+                        self._audio_input = []
+                        self._restart_counter += 1
+                        # Put the current audio chunk back in the queue
+                        await self._request_queue.put(audio_data)
+                        break
+
+                    self._audio_input.append(audio_data)
+                    yield cloud_speech.StreamingRecognizeRequest(audio=audio_data)
+
+                except asyncio.CancelledError:
+                    break
+                finally:
+                    self._request_queue.task_done()
+
+        except Exception as e:
+            logger.error(f"Error in request generator: {e}")
+            raise
+
+    async def _stream_audio(self):
+        """Handle bi-directional streaming with Google STT."""
+        try:
+            while True:
+                try:
+                    # Start bi-directional streaming
+                    streaming_recognize = await self._client.streaming_recognize(
+                        requests=self._request_generator()
+                    )
+
+                    # Process responses
+                    await self._process_responses(streaming_recognize)
+
+                    # If we're here, check if we need to reconnect
+                    if (int(time.time() * 1000) - self._stream_start_time) > self.STREAMING_LIMIT:
+                        logger.debug("Reconnecting stream after timeout")
+                        # Reset stream start time
+                        self._stream_start_time = int(time.time() * 1000)
+                        continue
+                    else:
+                        # Normal stream end
+                        break
+
+                except Exception as e:
+                    logger.error(f"Stream error, attempting to reconnect: {e}")
+                    await asyncio.sleep(1)  # Brief delay before reconnecting
+                    self._stream_start_time = int(time.time() * 1000)
+                    continue
+
+        except Exception as e:
+            logger.error(f"Error in streaming task: {e}")
+            await self.push_frame(ErrorFrame(str(e)))
+
+    async def run_stt(self, audio: bytes) -> AsyncGenerator[Frame, None]:
+        """Process an audio chunk for STT transcription."""
+        if self._streaming_task:
+            # Queue the audio data
+            await self._request_queue.put(audio)
+        yield None
+
+    async def _process_responses(self, streaming_recognize):
+        """Process streaming recognition responses."""
+        try:
+            async for response in streaming_recognize:
+                # Check streaming limit
+                if (int(time.time() * 1000) - self._stream_start_time) > self.STREAMING_LIMIT:
+                    logger.debug("Stream timeout reached in response processing")
+                    break
+
+                if not response.results:
+                    continue
+
+                for result in response.results:
+                    if not result.alternatives:
+                        continue
+
+                    transcript = result.alternatives[0].transcript
+                    if not transcript:
+                        continue
+
+                    primary_language = self._settings["language_codes"][0]
+
+                    if result.is_final:
+                        self._last_transcript_was_final = True
+                        await self.push_frame(
+                            TranscriptionFrame(transcript, "", time_now_iso8601(), primary_language)
+                        )
+                    else:
+                        self._last_transcript_was_final = False
+                        await self.push_frame(
+                            InterimTranscriptionFrame(
+                                transcript, "", time_now_iso8601(), primary_language
+                            )
+                        )
+
+        except Exception as e:
+            logger.error(f"Error processing Google STT responses: {e}")

src/pipecat/services/grok.py

@@ -17,6 +17,7 @@ from pipecat.processors.aggregators.openai_llm_context import (
     OpenAILLMContext,
     OpenAILLMContextFrame,
 )
+from pipecat.processors.frame_processor import FrameDirection
 from pipecat.services.openai import (
     OpenAIAssistantContextAggregator,
     OpenAILLMService,
@@ -27,7 +28,7 @@ from pipecat.services.openai import (
 class GrokAssistantContextAggregator(OpenAIAssistantContextAggregator):
     """Custom assistant context aggregator for Grok that handles empty content requirement."""
 
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if not (
             self._aggregation or self._function_call_result or self._pending_image_frame_message
         ):
@@ -37,7 +38,7 @@ class GrokAssistantContextAggregator(OpenAIAssistantContextAggregator):
         properties: Optional[FunctionCallResultProperties] = None
 
         aggregation = self._aggregation
-        self._reset()
+        self.reset()
 
         try:
             if self._function_call_result:
@@ -91,14 +92,13 @@ class GrokAssistantContextAggregator(OpenAIAssistantContextAggregator):
                 run_llm = True
 
             if run_llm:
-                await self._user_context_aggregator.push_context_frame()
+                await self.push_context_frame(FrameDirection.UPSTREAM)
 
             # Emit the on_context_updated callback once the function call result is added to the context
             if properties and properties.on_context_updated is not None:
                 await properties.on_context_updated()
 
-            frame = OpenAILLMContextFrame(self._context)
-            await self.push_frame(frame)
+            await self.push_context_frame()
 
         except Exception as e:
             logger.error(f"Error processing frame: {e}")
@@ -212,6 +212,6 @@ class GrokLLMService(OpenAILLMService):
     ) -> GrokContextAggregatorPair:
         user = OpenAIUserContextAggregator(context)
         assistant = GrokAssistantContextAggregator(
-            user, expect_stripped_words=assistant_expect_stripped_words
+            context, expect_stripped_words=assistant_expect_stripped_words
         )
         return GrokContextAggregatorPair(_user=user, _assistant=assistant)

src/pipecat/services/groq.py

@@ -54,6 +54,8 @@ class GroqSTTService(BaseWhisperSTTService):
         api_key: Groq API key. Defaults to None.
         base_url: API base URL. Defaults to "https://api.groq.com/openai/v1".
         language: Language of the audio input. Defaults to English.
+        prompt: Optional text to guide the model's style or continue a previous segment.
+        temperature: Optional sampling temperature between 0 and 1. Defaults to 0.0.
         **kwargs: Additional arguments passed to BaseWhisperSTTService.
     """
 
@@ -64,17 +66,35 @@ class GroqSTTService(BaseWhisperSTTService):
         api_key: Optional[str] = None,
         base_url: str = "https://api.groq.com/openai/v1",
         language: Optional[Language] = Language.EN,
+        prompt: Optional[str] = None,
+        temperature: Optional[float] = None,
         **kwargs,
     ):
         super().__init__(
-            model=model, api_key=api_key, base_url=base_url, language=language, **kwargs
+            model=model,
+            api_key=api_key,
+            base_url=base_url,
+            language=language,
+            prompt=prompt,
+            temperature=temperature,
+            **kwargs,
         )
 
     async def _transcribe(self, audio: bytes) -> Transcription:
         assert self._language is not None  # Assigned in the BaseWhisperSTTService class
-        return await self._client.audio.transcriptions.create(
-            file=("audio.wav", audio, "audio/wav"),
-            model=self.model_name,
-            response_format="json",
-            language=self._language,
-        )
+
+        # Build kwargs dict with only set parameters
+        kwargs = {
+            "file": ("audio.wav", audio, "audio/wav"),
+            "model": self.model_name,
+            "response_format": "json",
+            "language": self._language,
+        }
+
+        if self._prompt is not None:
+            kwargs["prompt"] = self._prompt
+
+        if self._temperature is not None:
+            kwargs["temperature"] = self._temperature
+
+        return await self._client.audio.transcriptions.create(**kwargs)

src/pipecat/services/lmnt.py

@@ -73,6 +73,7 @@ class LmntTTSService(TTSService, WebsocketService):
         TTSService.__init__(
             self,
             push_stop_frames=True,
+            pause_frame_processing=True,
             sample_rate=sample_rate,
             **kwargs,
         )

src/pipecat/services/openai.py

@@ -110,7 +110,7 @@ class BaseOpenAILLMService(LLMService):
         seed: Optional[int] = Field(default_factory=lambda: NOT_GIVEN, ge=0)
         temperature: Optional[float] = Field(default_factory=lambda: NOT_GIVEN, ge=0.0, le=2.0)
         # Note: top_k is currently not supported by the OpenAI client library,
-        # so top_k is ignore right now.
+        # so top_k is ignored right now.
         top_k: Optional[int] = Field(default=None, ge=0)
         top_p: Optional[float] = Field(default_factory=lambda: NOT_GIVEN, ge=0.0, le=1.0)
         max_tokens: Optional[int] = Field(default_factory=lambda: NOT_GIVEN, ge=1)
@@ -355,7 +355,7 @@ class OpenAILLMService(BaseOpenAILLMService):
     ) -> OpenAIContextAggregatorPair:
         user = OpenAIUserContextAggregator(context)
         assistant = OpenAIAssistantContextAggregator(
-            user, expect_stripped_words=assistant_expect_stripped_words
+            context, expect_stripped_words=assistant_expect_stripped_words
         )
         return OpenAIContextAggregatorPair(_user=user, _assistant=assistant)
 
@@ -408,6 +408,8 @@ class OpenAISTTService(BaseWhisperSTTService):
         api_key: OpenAI API key. Defaults to None.
         base_url: API base URL. Defaults to None.
         language: Language of the audio input. Defaults to English.
+        prompt: Optional text to guide the model's style or continue a previous segment.
+        temperature: Optional sampling temperature between 0 and 1. Defaults to 0.0.
         **kwargs: Additional arguments passed to BaseWhisperSTTService.
     """
 
@@ -418,17 +420,37 @@ class OpenAISTTService(BaseWhisperSTTService):
         api_key: Optional[str] = None,
         base_url: Optional[str] = None,
         language: Optional[Language] = Language.EN,
+        prompt: Optional[str] = None,
+        temperature: Optional[float] = None,
         **kwargs,
     ):
         super().__init__(
-            model=model, api_key=api_key, base_url=base_url, language=language, **kwargs
+            model=model,
+            api_key=api_key,
+            base_url=base_url,
+            language=language,
+            prompt=prompt,
+            temperature=temperature,
+            **kwargs,
         )
 
     async def _transcribe(self, audio: bytes) -> Transcription:
         assert self._language is not None  # Assigned in the BaseWhisperSTTService class
-        return await self._client.audio.transcriptions.create(
-            file=("audio.wav", audio, "audio/wav"), model=self.model_name, language=self._language
-        )
+
+        # Build kwargs dict with only set parameters
+        kwargs = {
+            "file": ("audio.wav", audio, "audio/wav"),
+            "model": self.model_name,
+            "language": self._language,
+        }
+
+        if self._prompt is not None:
+            kwargs["prompt"] = self._prompt
+
+        if self._temperature is not None:
+            kwargs["temperature"] = self._temperature
+
+        return await self._client.audio.transcriptions.create(**kwargs)
 
 
 class OpenAITTSService(TTSService):
@@ -533,8 +555,8 @@ class OpenAIImageMessageFrame(Frame):
 
 
 class OpenAIUserContextAggregator(LLMUserContextAggregator):
-    def __init__(self, context: OpenAILLMContext):
-        super().__init__(context=context)
+    def __init__(self, context: OpenAILLMContext, **kwargs):
+        super().__init__(context=context, **kwargs)
 
     async def process_frame(self, frame, direction):
         await super().process_frame(frame, direction)
@@ -570,9 +592,8 @@ class OpenAIUserContextAggregator(LLMUserContextAggregator):
 
 
 class OpenAIAssistantContextAggregator(LLMAssistantContextAggregator):
-    def __init__(self, user_context_aggregator: OpenAIUserContextAggregator, **kwargs):
-        super().__init__(context=user_context_aggregator._context, **kwargs)
-        self._user_context_aggregator = user_context_aggregator
+    def __init__(self, context: OpenAILLMContext, **kwargs):
+        super().__init__(context=context, **kwargs)
         self._function_calls_in_progress = {}
         self._function_call_result = None
         self._pending_image_frame_message = None
@@ -592,7 +613,7 @@ class OpenAIAssistantContextAggregator(LLMAssistantContextAggregator):
                 del self._function_calls_in_progress[frame.tool_call_id]
                 self._function_call_result = frame
                 # TODO-CB: Kwin wants us to refactor this out of here but I REFUSE
-                await self._push_aggregation()
+                await self.push_aggregation()
             else:
                 logger.warning(
                     "FunctionCallResultFrame tool_call_id does not match any function call in progress"
@@ -600,9 +621,9 @@ class OpenAIAssistantContextAggregator(LLMAssistantContextAggregator):
                 self._function_call_result = None
         elif isinstance(frame, OpenAIImageMessageFrame):
             self._pending_image_frame_message = frame
-            await self._push_aggregation()
+            await self.push_aggregation()
 
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         if not (
             self._aggregation or self._function_call_result or self._pending_image_frame_message
         ):
@@ -612,7 +633,7 @@ class OpenAIAssistantContextAggregator(LLMAssistantContextAggregator):
         properties: Optional[FunctionCallResultProperties] = None
 
         aggregation = self._aggregation
-        self._reset()
+        self.reset()
 
         try:
             if self._function_call_result:
@@ -664,15 +685,14 @@ class OpenAIAssistantContextAggregator(LLMAssistantContextAggregator):
                 run_llm = True
 
             if run_llm:
-                await self._user_context_aggregator.push_context_frame()
+                await self.push_context_frame(FrameDirection.UPSTREAM)
 
             # Emit the on_context_updated callback once the function call result is added to the context
             if properties and properties.on_context_updated is not None:
                 await properties.on_context_updated()
 
             # Push context frame
-            frame = OpenAILLMContextFrame(self._context)
-            await self.push_frame(frame)
+            await self.push_context_frame()
 
             # Push timestamp frame with current time
             timestamp_frame = OpenAILLMContextAssistantTimestampFrame(timestamp=time_now_iso8601())

src/pipecat/services/openai_realtime_beta/context.py

@@ -166,7 +166,7 @@ class OpenAIRealtimeUserContextAggregator(OpenAIUserContextAggregator):
         if isinstance(frame, LLMSetToolsFrame):
             await self.push_frame(frame, direction)
 
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         # for the moment, ignore all user input coming into the pipeline.
         # todo: think about whether/how to fix this to allow for text input from
         #       upstream (transport/transcription, or other sources)
@@ -174,7 +174,7 @@ class OpenAIRealtimeUserContextAggregator(OpenAIUserContextAggregator):
 
 
 class OpenAIRealtimeAssistantContextAggregator(OpenAIAssistantContextAggregator):
-    async def _push_aggregation(self):
+    async def push_aggregation(self):
         # the only thing we implement here is function calling. in all other cases, messages
         # are added to the context when we receive openai realtime api events
         if not self._function_call_result:
@@ -182,7 +182,7 @@ class OpenAIRealtimeAssistantContextAggregator(OpenAIAssistantContextAggregator)
 
         properties: Optional[FunctionCallResultProperties] = None
 
-        self._reset()
+        self.reset()
         try:
             run_llm = True
             frame = self._function_call_result
@@ -217,8 +217,8 @@ class OpenAIRealtimeAssistantContextAggregator(OpenAIAssistantContextAggregator)
                 # The standard function callback code path pushes the FunctionCallResultFrame from the llm itself,
                 # so we didn't have a chance to add the result to the openai realtime api context. Let's push a
                 # special frame to do that.
-                await self._user_context_aggregator.push_frame(
-                    RealtimeFunctionCallResultFrame(result_frame=frame)
+                await self.push_frame(
+                    RealtimeFunctionCallResultFrame(result_frame=frame), FrameDirection.UPSTREAM
                 )
                 if properties and properties.run_llm is not None:
                     # If the tool call result has a run_llm property, use it
@@ -228,14 +228,13 @@ class OpenAIRealtimeAssistantContextAggregator(OpenAIAssistantContextAggregator)
                     run_llm = not bool(self._function_calls_in_progress)
 
             if run_llm:
-                await self._user_context_aggregator.push_context_frame()
+                await self.push_context_frame(FrameDirection.UPSTREAM)
 
             # Emit the on_context_updated callback once the function call result is added to the context
             if properties and properties.on_context_updated is not None:
                 await properties.on_context_updated()
 
-            frame = OpenAILLMContextFrame(self._context)
-            await self.push_frame(frame)
+            await self.push_context_frame()
 
         except Exception as e:
             logger.error(f"Error processing frame: {e}")

src/pipecat/services/openai_realtime_beta/openai.py

@@ -568,6 +568,6 @@ class OpenAIRealtimeBetaLLMService(LLMService):
         OpenAIRealtimeLLMContext.upgrade_to_realtime(context)
         user = OpenAIRealtimeUserContextAggregator(context)
         assistant = OpenAIRealtimeAssistantContextAggregator(
-            user, expect_stripped_words=assistant_expect_stripped_words
+            context, expect_stripped_words=assistant_expect_stripped_words
         )
         return OpenAIContextAggregatorPair(_user=user, _assistant=assistant)

src/pipecat/services/playht.py

@@ -120,6 +120,7 @@ class PlayHTTTSService(TTSService, WebsocketService):
     ):
         TTSService.__init__(
             self,
+            pause_frame_processing=True,
             sample_rate=sample_rate,
             **kwargs,
         )
@@ -269,19 +270,6 @@ class PlayHTTTSService(TTSService, WebsocketService):
                 except json.JSONDecodeError:
                     logger.error(f"Invalid JSON message: {message}")
 
-    async def process_frame(self, frame: Frame, direction: FrameDirection):
-        await super().process_frame(frame, direction)
-
-        # If we received a TTSSpeakFrame and the LLM response included text (it
-        # might be that it's only a function calling response) we pause
-        # processing more frames until we receive a BotStoppedSpeakingFrame.
-        if isinstance(frame, TTSSpeakFrame):
-            await self.pause_processing_frames()
-        elif isinstance(frame, LLMFullResponseEndFrame) and self._request_id:
-            await self.pause_processing_frames()
-        elif isinstance(frame, BotStoppedSpeakingFrame):
-            await self.resume_processing_frames()
-
     async def run_tts(self, text: str) -> AsyncGenerator[Frame, None]:
         logger.debug(f"Generating TTS: [{text}]")
 

src/pipecat/services/rime.py

@@ -4,6 +4,9 @@
 # SPDX-License-Identifier: BSD 2-Clause License
 #
 
+import base64
+import json
+import uuid
 from typing import AsyncGenerator, Optional
 
 import aiohttp
@@ -11,13 +14,327 @@ from loguru import logger
 from pydantic import BaseModel
 
 from pipecat.frames.frames import (
+    BotStoppedSpeakingFrame,
+    CancelFrame,
+    EndFrame,
     ErrorFrame,
     Frame,
+    LLMFullResponseEndFrame,
+    StartFrame,
+    StartInterruptionFrame,
     TTSAudioRawFrame,
+    TTSSpeakFrame,
     TTSStartedFrame,
     TTSStoppedFrame,
 )
-from pipecat.services.ai_services import TTSService
+from pipecat.processors.frame_processor import FrameDirection
+from pipecat.services.ai_services import AudioContextWordTTSService, TTSService
+from pipecat.services.websocket_service import WebsocketService
+from pipecat.transcriptions.language import Language
+
+try:
+    import websockets
+except ModuleNotFoundError as e:
+    logger.error(f"Exception: {e}")
+    logger.error(
+        "In order to use Rime, you need to `pip install pipecat-ai[rime]`. Also, set `RIME_API_KEY` environment variable."
+    )
+    raise Exception(f"Missing module: {e}")
+
+
+def language_to_rime_language(language: Language) -> str:
+    """Convert pipecat Language to Rime language code.
+
+    Args:
+        language: The pipecat Language enum value.
+
+    Returns:
+        str: Three-letter language code used by Rime (e.g., 'eng' for English).
+    """
+    LANGUAGE_MAP = {
+        Language.EN: "eng",
+        Language.ES: "spa",
+    }
+    return LANGUAGE_MAP.get(language, "eng")
+
+
+class RimeTTSService(AudioContextWordTTSService, WebsocketService):
+    """Text-to-Speech service using Rime's websocket API.
+
+    Uses Rime's websocket JSON API to convert text to speech with word-level timing
+    information. Supports interruptions and maintains context across multiple messages
+    within a turn.
+    """
+
+    class InputParams(BaseModel):
+        """Configuration parameters for Rime TTS service."""
+
+        language: Optional[Language] = Language.EN
+        speed_alpha: Optional[float] = 1.0
+        reduce_latency: Optional[bool] = False
+
+    def __init__(
+        self,
+        *,
+        api_key: str,
+        voice_id: str,
+        url: str = "wss://users-ws.rime.ai/ws2",
+        model: str = "mistv2",
+        sample_rate: Optional[int] = None,
+        params: InputParams = InputParams(),
+        **kwargs,
+    ):
+        """Initialize Rime TTS service.
+
+        Args:
+            api_key: Rime API key for authentication.
+            voice_id: ID of the voice to use.
+            url: Rime websocket API endpoint.
+            model: Model ID to use for synthesis.
+            sample_rate: Audio sample rate in Hz.
+            params: Additional configuration parameters.
+        """
+        # Initialize with parent class settings for proper frame handling
+        AudioContextWordTTSService.__init__(
+            self,
+            aggregate_sentences=True,
+            push_text_frames=False,
+            push_stop_frames=True,
+            stop_frame_timeout_s=2.0,
+            pause_frame_processing=True,
+            sample_rate=sample_rate,
+            **kwargs,
+        )
+        WebsocketService.__init__(self)
+
+        # Store service configuration
+        self._api_key = api_key
+        self._url = url
+        self._voice_id = voice_id
+        self._model = model
+        self._settings = {
+            "speaker": voice_id,
+            "modelId": model,
+            "audioFormat": "pcm",
+            "samplingRate": 0,
+            "lang": self.language_to_service_language(params.language)
+            if params.language
+            else "eng",
+            "speedAlpha": params.speed_alpha,
+            "reduceLatency": params.reduce_latency,
+        }
+
+        # State tracking
+        self._context_id = None  # Tracks current turn
+        self._receive_task = None
+        self._cumulative_time = 0  # Accumulates time across messages
+
+    def can_generate_metrics(self) -> bool:
+        return True
+
+    def language_to_service_language(self, language: Language) -> str | None:
+        """Convert pipecat language to Rime language code."""
+        return language_to_rime_language(language)
+
+    async def set_model(self, model: str):
+        """Update the TTS model."""
+        self._model = model
+        await super().set_model(model)
+
+    def _build_msg(self, text: str = "") -> dict:
+        """Build JSON message for Rime API."""
+        return {"text": text, "contextId": self._context_id}
+
+    def _build_clear_msg(self) -> dict:
+        """Build clear operation message."""
+        return {"operation": "clear"}
+
+    def _build_eos_msg(self) -> dict:
+        """Build end-of-stream operation message."""
+        return {"operation": "eos"}
+
+    async def start(self, frame: StartFrame):
+        """Start the service and establish websocket connection."""
+        await super().start(frame)
+        self._settings["samplingRate"] = self.sample_rate
+        await self._connect()
+
+    async def stop(self, frame: EndFrame):
+        """Stop the service and close connection."""
+        await super().stop(frame)
+        await self._disconnect()
+
+    async def cancel(self, frame: CancelFrame):
+        """Cancel current operation and clean up."""
+        await super().cancel(frame)
+        await self._disconnect()
+
+    async def _connect(self):
+        """Establish websocket connection and start receive task."""
+        await self._connect_websocket()
+        self._receive_task = self.create_task(self._receive_task_handler(self.push_error))
+
+    async def _disconnect(self):
+        """Close websocket connection and clean up tasks."""
+        await self._disconnect_websocket()
+        if self._receive_task:
+            await self.cancel_task(self._receive_task)
+            self._receive_task = None
+
+    async def _connect_websocket(self):
+        """Connect to Rime websocket API with configured settings."""
+        try:
+            params = "&".join(f"{k}={v}" for k, v in self._settings.items())
+            url = f"{self._url}?{params}"
+            headers = {"Authorization": f"Bearer {self._api_key}"}
+            self._websocket = await websockets.connect(url, extra_headers=headers)
+        except Exception as e:
+            logger.error(f"{self} initialization error: {e}")
+            self._websocket = None
+
+    async def _disconnect_websocket(self):
+        """Close websocket connection and reset state."""
+        try:
+            await self.stop_all_metrics()
+            if self._websocket:
+                await self._websocket.send(json.dumps(self._build_eos_msg()))
+                await self._websocket.close()
+                self._websocket = None
+            self._context_id = None
+        except Exception as e:
+            logger.error(f"{self} error closing websocket: {e}")
+
+    def _get_websocket(self):
+        """Get active websocket connection or raise exception."""
+        if self._websocket:
+            return self._websocket
+        raise Exception("Websocket not connected")
+
+    async def _handle_interruption(self, frame: StartInterruptionFrame, direction: FrameDirection):
+        """Handle interruption by clearing current context."""
+        await super()._handle_interruption(frame, direction)
+        await self.stop_all_metrics()
+        if self._context_id:
+            await self._get_websocket().send(json.dumps(self._build_clear_msg()))
+            self._context_id = None
+
+    def _calculate_word_times(self, words: list, starts: list, ends: list) -> list:
+        """Calculate word timing pairs with proper spacing and punctuation.
+
+        Args:
+            words: List of words from Rime.
+            starts: List of start times for each word.
+            ends: List of end times for each word.
+
+        Returns:
+            List of (word, timestamp) pairs with proper timing.
+        """
+        word_pairs = []
+        for i, (word, start_time, _) in enumerate(zip(words, starts, ends)):
+            if not word.strip():
+                continue
+
+            # Adjust timing by adding cumulative time
+            adjusted_start = start_time + self._cumulative_time
+
+            # Handle punctuation by appending to previous word
+            is_punctuation = bool(word.strip(",.!?") == "")
+            if is_punctuation and word_pairs:
+                prev_word, prev_time = word_pairs[-1]
+                word_pairs[-1] = (prev_word + word, prev_time)
+            else:
+                word_pairs.append((word, adjusted_start))
+
+        return word_pairs
+
+    async def flush_audio(self):
+        if not self._context_id or not self._websocket:
+            return
+        logger.trace(f"{self}: flushing audio")
+        self._context_id = None
+
+    async def _receive_messages(self):
+        """Process incoming websocket messages."""
+        async for message in self._get_websocket():
+            msg = json.loads(message)
+
+            if not msg or not self.audio_context_available(msg["contextId"]):
+                continue
+
+            if msg["type"] == "chunk":
+                # Process audio chunk
+                await self.stop_ttfb_metrics()
+                self.start_word_timestamps()
+                frame = TTSAudioRawFrame(
+                    audio=base64.b64decode(msg["data"]),
+                    sample_rate=self.sample_rate,
+                    num_channels=1,
+                )
+                await self.append_to_audio_context(msg["contextId"], frame)
+
+            elif msg["type"] == "timestamps":
+                # Process word timing information
+                timestamps = msg.get("word_timestamps", {})
+                words = timestamps.get("words", [])
+                starts = timestamps.get("start", [])
+                ends = timestamps.get("end", [])
+
+                if words and starts:
+                    # Calculate word timing pairs
+                    word_pairs = self._calculate_word_times(words, starts, ends)
+                    if word_pairs:
+                        await self.add_word_timestamps(word_pairs)
+                        self._cumulative_time = ends[-1] + self._cumulative_time
+                        logger.debug(f"Updated cumulative time to: {self._cumulative_time}")
+
+            elif msg["type"] == "error":
+                logger.error(f"{self} error: {msg}")
+                await self.push_frame(TTSStoppedFrame())
+                await self.stop_all_metrics()
+                await self.push_error(ErrorFrame(f"{self} error: {msg['message']}"))
+                self._context_id = None
+
+    async def push_frame(self, frame: Frame, direction: FrameDirection = FrameDirection.DOWNSTREAM):
+        """Push frame and handle end-of-turn conditions."""
+        await super().push_frame(frame, direction)
+        if isinstance(frame, (TTSStoppedFrame, StartInterruptionFrame)):
+            if isinstance(frame, TTSStoppedFrame):
+                await self.add_word_timestamps([("LLMFullResponseEndFrame", 0), ("Reset", 0)])
+
+    async def run_tts(self, text: str) -> AsyncGenerator[Frame, None]:
+        """Generate speech from text.
+
+        Args:
+            text: The text to convert to speech.
+
+        Yields:
+            Frames containing audio data and timing information.
+        """
+        logger.debug(f"Generating TTS: [{text}]")
+        try:
+            if not self._websocket:
+                await self._connect()
+
+            try:
+                if not self._context_id:
+                    await self.start_ttfb_metrics()
+                    yield TTSStartedFrame()
+                    self._cumulative_time = 0
+                    self._context_id = str(uuid.uuid4())
+                    await self.create_audio_context(self._context_id)
+
+                msg = self._build_msg(text=text)
+                await self._get_websocket().send(json.dumps(msg))
+                await self.start_tts_usage_metrics(text)
+            except Exception as e:
+                logger.error(f"{self} error sending message: {e}")
+                yield TTSStoppedFrame()
+                await self._disconnect()
+                await self._connect()
+                return
+            yield None
+        except Exception as e:
+            logger.error(f"{self} exception: {e}")
 
 
 class RimeHttpTTSService(TTSService):
@@ -33,7 +350,7 @@ class RimeHttpTTSService(TTSService):
         *,
         api_key: str,
         voice_id: str = "eva",
-        model: str = "mist",
+        model: str = "mistv2",
         sample_rate: Optional[int] = None,
         params: InputParams = InputParams(),
         **kwargs,

src/pipecat/services/simli.py

@@ -41,7 +41,7 @@ class SimliVideoService(FrameProcessor):
 
         self._pipecat_resampler_event = asyncio.Event()
         self._pipecat_resampler: AudioResampler = None
-        self._simli_resampler = AudioResampler("s16", 1, 16000)
+        self._simli_resampler = AudioResampler("s16", "mono", 16000)
 
         self._audio_task: asyncio.Task = None
         self._video_task: asyncio.Task = None

src/pipecat/services/websocket_service.py

@@ -10,6 +10,7 @@ from typing import Awaitable, Callable, Optional
 
 import websockets
 from loguru import logger
+from websockets.protocol import State
 
 from pipecat.frames.frames import ErrorFrame
 
@@ -83,11 +84,13 @@ class WebsocketService(ABC):
         while True:
             try:
                 await self._receive_messages()
-                logger.debug(f"{self} connection established successfully")
                 retry_count = 0  # Reset counter on successful message receive
-            except websockets.ConnectionClosed as e:
-                logger.warning(f"{self} connection closed: {e}")
-                break
+                if self._websocket and self._websocket.state == State.CLOSED:
+                    raise websockets.ConnectionClosedOK(
+                        self._websocket.close_rcvd,
+                        self._websocket.close_sent,
+                        self._websocket.close_rcvd_then_sent,
+                    )
             except Exception as e:
                 retry_count += 1
                 if retry_count >= MAX_RETRIES:

src/pipecat/tests/utils.py

@@ -5,6 +5,7 @@
 #
 
 import asyncio
+from dataclasses import dataclass
 from typing import Any, Awaitable, Callable, Dict, Sequence, Tuple
 
 from pipecat.frames.frames import (
@@ -12,6 +13,7 @@ from pipecat.frames.frames import (
     Frame,
     HeartbeatFrame,
     StartFrame,
+    SystemFrame,
 )
 from pipecat.observers.base_observer import BaseObserver
 from pipecat.pipeline.pipeline import Pipeline
@@ -20,6 +22,16 @@ from pipecat.pipeline.task import PipelineParams, PipelineTask
 from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
 
 
+@dataclass
+class SleepFrame(SystemFrame):
+    """This frame is used by test framework to introduce some sleep time before
+    the next frame is pushed. This is useful to control system frames vs data or
+    control frames.
+    """
+
+    sleep: float = 0.1
+
+
 class HeartbeatsObserver(BaseObserver):
     def __init__(
         self,
@@ -44,7 +56,11 @@ class HeartbeatsObserver(BaseObserver):
 
 class QueuedFrameProcessor(FrameProcessor):
     def __init__(
-        self, queue: asyncio.Queue, queue_direction: FrameDirection, ignore_start: bool = True
+        self,
+        *,
+        queue: asyncio.Queue,
+        queue_direction: FrameDirection,
+        ignore_start: bool = True,
     ):
         super().__init__()
         self._queue = queue
@@ -72,21 +88,35 @@ async def run_test(
 ) -> Tuple[Sequence[Frame], Sequence[Frame]]:
     received_up = asyncio.Queue()
     received_down = asyncio.Queue()
-    source = QueuedFrameProcessor(received_up, FrameDirection.UPSTREAM, ignore_start)
-    sink = QueuedFrameProcessor(received_down, FrameDirection.DOWNSTREAM, ignore_start)
+    source = QueuedFrameProcessor(
+        queue=received_up,
+        queue_direction=FrameDirection.UPSTREAM,
+        ignore_start=ignore_start,
+    )
+    sink = QueuedFrameProcessor(
+        queue=received_down,
+        queue_direction=FrameDirection.DOWNSTREAM,
+        ignore_start=ignore_start,
+    )
 
     pipeline = Pipeline([source, processor, sink])
 
     task = PipelineTask(pipeline, params=PipelineParams(start_metadata=start_metadata))
 
-    for frame in frames_to_send:
-        await task.queue_frame(frame)
+    async def push_frames():
+        # Just give a little head start to the runner.
+        await asyncio.sleep(0.01)
+        for frame in frames_to_send:
+            if isinstance(frame, SleepFrame):
+                await asyncio.sleep(frame.sleep)
+            else:
+                await task.queue_frame(frame)
 
-    if send_end_frame:
-        await task.queue_frame(EndFrame())
+        if send_end_frame:
+            await task.queue_frame(EndFrame())
 
     runner = PipelineRunner()
-    await runner.run(task)
+    await asyncio.gather(runner.run(task), push_frames())
 
     #
     # Down frames
@@ -98,6 +128,7 @@ async def run_test(
             received_down_frames.append(frame)
 
     print("received DOWN frames =", received_down_frames)
+    print("expected DOWN frames =", expected_down_frames)
 
     assert len(received_down_frames) == len(expected_down_frames)
 
@@ -113,6 +144,7 @@ async def run_test(
         received_up_frames.append(frame)
 
     print("received UP frames =", received_up_frames)
+    print("expected UP frames =", expected_up_frames)
 
     assert len(received_up_frames) == len(expected_up_frames)
 

src/pipecat/transcriptions/language.py

@@ -101,6 +101,7 @@ class Language(StrEnum):
     EN_AU = "en-AU"
     EN_CA = "en-CA"
     EN_GB = "en-GB"
+    EN_GH = "en-GH"
     EN_HK = "en-HK"
     EN_IE = "en-IE"
     EN_IN = "en-IN"
@@ -208,6 +209,7 @@ class Language(StrEnum):
     # Italian
     IT = "it"
     IT_IT = "it-IT"
+    IT_CH = "it-CH"
 
     # Inuktitut
     IU_CANS = "iu-Cans"

src/pipecat/transports/base_input.py

@@ -14,6 +14,8 @@ from pipecat.audio.vad.vad_analyzer import VADAnalyzer, VADState
 from pipecat.frames.frames import (
     BotInterruptionFrame,
     CancelFrame,
+    EmulateUserStartedSpeakingFrame,
+    EmulateUserStoppedSpeakingFrame,
     EndFrame,
     FilterUpdateSettingsFrame,
     Frame,
@@ -47,6 +49,13 @@ class BaseInputTransport(FrameProcessor):
         # if passthrough is enabled.
         self._audio_task = None
 
+    def enable_audio_in_stream_on_start(self, enabled: bool) -> None:
+        logger.debug(f"Enabling audio on start. {enabled}")
+        self._params.audio_in_stream_on_start = enabled
+
+    def start_audio_in_streaming(self):
+        pass
+
     @property
     def sample_rate(self) -> int:
         return self._sample_rate
@@ -105,9 +114,13 @@ class BaseInputTransport(FrameProcessor):
             await self.cancel(frame)
             await self.push_frame(frame, direction)
         elif isinstance(frame, BotInterruptionFrame):
-            logger.debug("Bot interruption")
-            await self._start_interruption()
-            await self.push_frame(StartInterruptionFrame())
+            await self._handle_bot_interruption(frame)
+        elif isinstance(frame, EmulateUserStartedSpeakingFrame):
+            logger.debug("Emulating user started speaking")
+            await self._handle_user_interruption(UserStartedSpeakingFrame())
+        elif isinstance(frame, EmulateUserStoppedSpeakingFrame):
+            logger.debug("Emulating user stopped speaking")
+            await self._handle_user_interruption(UserStoppedSpeakingFrame())
         # All other system frames
         elif isinstance(frame, SystemFrame):
             await self.push_frame(frame, direction)
@@ -130,7 +143,13 @@ class BaseInputTransport(FrameProcessor):
     # Handle interruptions
     #
 
-    async def _handle_interruptions(self, frame: Frame):
+    async def _handle_bot_interruption(self, frame: BotInterruptionFrame):
+        logger.debug("Bot interruption")
+        if self.interruptions_allowed:
+            await self._start_interruption()
+            await self.push_frame(StartInterruptionFrame())
+
+    async def _handle_user_interruption(self, frame: Frame):
         if isinstance(frame, UserStartedSpeakingFrame):
             logger.debug("User started speaking")
             # Make sure we notify about interruptions quickly out-of-band.
@@ -176,7 +195,7 @@ class BaseInputTransport(FrameProcessor):
                 frame = UserStoppedSpeakingFrame()
 
             if frame:
-                await self._handle_interruptions(frame)
+                await self._handle_user_interruption(frame)
 
             vad_state = new_vad_state
         return vad_state

src/pipecat/transports/base_output.py

@@ -14,7 +14,6 @@ from loguru import logger
 from PIL import Image
 
 from pipecat.audio.utils import create_default_resampler
-from pipecat.audio.vad.vad_analyzer import VAD_STOP_SECS
 from pipecat.frames.frames import (
     BotSpeakingFrame,
     BotStartedSpeakingFrame,
@@ -38,6 +37,8 @@ from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
 from pipecat.transports.base_transport import TransportParams
 from pipecat.utils.time import nanoseconds_to_seconds
 
+BOT_VAD_STOP_SECS = 0.3
+
 
 class BaseOutputTransport(FrameProcessor):
     def __init__(self, params: TransportParams, **kwargs):
@@ -169,6 +170,8 @@ class BaseOutputTransport(FrameProcessor):
         # TODO(aleix): Images and audio should support presentation timestamps.
         elif frame.pts:
             await self._sink_clock_queue.put((frame.pts, frame.id, frame))
+        elif direction == FrameDirection.UPSTREAM:
+            await self.push_frame(frame, direction)
         else:
             await self._sink_queue.put(frame)
 
@@ -321,15 +324,10 @@ class BaseOutputTransport(FrameProcessor):
                     )
                     yield frame
 
-        vad_stop_secs = (
-            self._params.vad_analyzer.params.stop_secs
-            if self._params.vad_analyzer
-            else VAD_STOP_SECS
-        )
         if self._params.audio_out_mixer:
-            return with_mixer(vad_stop_secs)
+            return with_mixer(BOT_VAD_STOP_SECS)
         else:
-            return without_mixer(vad_stop_secs)
+            return without_mixer(BOT_VAD_STOP_SECS)
 
     async def _sink_task_handler(self):
         async for frame in self._next_frame():

src/pipecat/transports/base_transport.py

@@ -39,6 +39,7 @@ class TransportParams(BaseModel):
     audio_in_sample_rate: Optional[int] = None
     audio_in_channels: int = 1
     audio_in_filter: Optional[BaseAudioFilter] = None
+    audio_in_stream_on_start: bool = True
     vad_enabled: bool = False
     vad_audio_passthrough: bool = False
     vad_analyzer: Optional[VADAnalyzer] = None

src/pipecat/transports/local/audio.py

@@ -26,17 +26,18 @@ except ModuleNotFoundError as e:
     raise Exception(f"Missing module: {e}")
 
 
-class LocalTransportParams(TransportParams):
-    input_device_index: int = 0
-    output_device_index: int = 0
+class LocalAudioTransportParams(TransportParams):
+    input_device_index: Optional[int] = None
+    output_device_index: Optional[int] = None
 
 
 class LocalAudioInputTransport(BaseInputTransport):
-    _params: LocalTransportParams
+    _params: LocalAudioTransportParams
 
-    def __init__(self, py_audio: pyaudio.PyAudio, params: LocalTransportParams):
+    def __init__(self, py_audio: pyaudio.PyAudio, params: LocalAudioTransportParams):
         super().__init__(params)
         self._py_audio = py_audio
+
         self._in_stream = None
         self._sample_rate = 0
 
@@ -77,11 +78,12 @@ class LocalAudioInputTransport(BaseInputTransport):
 
 
 class LocalAudioOutputTransport(BaseOutputTransport):
-    _params: LocalTransportParams
+    _params: LocalAudioTransportParams
 
-    def __init__(self, py_audio: pyaudio.PyAudio, params: TransportParams):
+    def __init__(self, py_audio: pyaudio.PyAudio, params: LocalAudioTransportParams):
         super().__init__(params)
         self._py_audio = py_audio
+
         self._out_stream = None
         self._sample_rate = 0
 
@@ -116,8 +118,8 @@ class LocalAudioOutputTransport(BaseOutputTransport):
             )
 
 
-class LocalAudioTransport(LocalTransportParams):
-    def __init__(self, params: LocalTransportParams):
+class LocalAudioTransport(BaseTransport):
+    def __init__(self, params: LocalAudioTransportParams):
         super().__init__()
         self._params = params
         self._pyaudio = pyaudio.PyAudio()

src/pipecat/transports/local/tk.py

@@ -34,8 +34,15 @@ except ModuleNotFoundError as e:
     raise Exception(f"Missing module: {e}")
 
 
+class TkTransportParams(TransportParams):
+    audio_input_device_index: Optional[int] = None
+    audio_output_device_index: Optional[int] = None
+
+
 class TkInputTransport(BaseInputTransport):
-    def __init__(self, py_audio: pyaudio.PyAudio, params: TransportParams):
+    _params: TkTransportParams
+
+    def __init__(self, py_audio: pyaudio.PyAudio, params: TkTransportParams):
         super().__init__(params)
         self._py_audio = py_audio
         self._in_stream = None
@@ -54,6 +61,7 @@ class TkInputTransport(BaseInputTransport):
             frames_per_buffer=num_frames,
             stream_callback=self._audio_in_callback,
             input=True,
+            input_device_index=self._params.audio_input_device_index,
         )
         self._in_stream.start_stream()
 
@@ -76,6 +84,8 @@ class TkInputTransport(BaseInputTransport):
 
 
 class TkOutputTransport(BaseOutputTransport):
+    _params: TkTransportParams
+
     def __init__(self, tk_root: tk.Tk, py_audio: pyaudio.PyAudio, params: TransportParams):
         super().__init__(params)
         self._py_audio = py_audio
@@ -103,6 +113,7 @@ class TkOutputTransport(BaseOutputTransport):
             channels=self._params.audio_out_channels,
             rate=self._sample_rate,
             output=True,
+            output_device_index=self._params.audio_output_device_index,
         )
         self._out_stream.start_stream()
 

src/pipecat/transports/network/fastapi_websocket.py

@@ -23,6 +23,8 @@ from pipecat.frames.frames import (
     OutputAudioRawFrame,
     StartFrame,
     StartInterruptionFrame,
+    TransportMessageFrame,
+    TransportMessageUrgentFrame,
 )
 from pipecat.processors.frame_processor import FrameDirection
 from pipecat.serializers.base_serializer import FrameSerializer, FrameSerializerType
@@ -139,6 +141,9 @@ class FastAPIWebsocketOutputTransport(BaseOutputTransport):
             await self._write_frame(frame)
             self._next_send_time = 0
 
+    async def send_message(self, frame: TransportMessageFrame | TransportMessageUrgentFrame):
+        await self._write_frame(frame)
+
     async def write_raw_audio_frames(self, frames: bytes):
         if self._websocket.client_state != WebSocketState.CONNECTED:
             # Simulate audio playback with a sleep.

src/pipecat/transports/network/websocket_server.py

@@ -6,6 +6,7 @@
 
 import asyncio
 import io
+import json
 import time
 import wave
 from typing import Awaitable, Callable, Optional
@@ -21,6 +22,9 @@ from pipecat.frames.frames import (
     OutputAudioRawFrame,
     StartFrame,
     StartInterruptionFrame,
+    TextFrame,
+    TransportMessageFrame,
+    TransportMessageUrgentFrame,
 )
 from pipecat.processors.frame_processor import FrameDirection
 from pipecat.serializers.base_serializer import FrameSerializer
@@ -46,6 +50,7 @@ class WebsocketServerCallbacks(BaseModel):
     on_client_connected: Callable[[websockets.WebSocketServerProtocol], Awaitable[None]]
     on_client_disconnected: Callable[[websockets.WebSocketServerProtocol], Awaitable[None]]
     on_session_timeout: Callable[[websockets.WebSocketServerProtocol], Awaitable[None]]
+    on_websocket_ready: Callable[[], Awaitable[None]]
 
 
 class WebsocketServerInputTransport(BaseInputTransport):
@@ -96,6 +101,7 @@ class WebsocketServerInputTransport(BaseInputTransport):
     async def _server_task_handler(self):
         logger.info(f"Starting websocket server on {self._host}:{self._port}")
         async with websockets.serve(self._client_handler, self._host, self._port) as server:
+            await self._callbacks.on_websocket_ready()
             await self._stop_server_event.wait()
 
     async def _client_handler(self, websocket: websockets.WebSocketServerProtocol, path):
@@ -186,6 +192,9 @@ class WebsocketServerOutputTransport(BaseOutputTransport):
             await self._write_frame(frame)
             self._next_send_time = 0
 
+    async def send_message(self, frame: TransportMessageFrame | TransportMessageUrgentFrame):
+        await self._write_frame(frame)
+
     async def write_raw_audio_frames(self, frames: bytes):
         if not self._websocket:
             # Simulate audio playback with a sleep.
@@ -254,6 +263,7 @@ class WebsocketServerTransport(BaseTransport):
             on_client_connected=self._on_client_connected,
             on_client_disconnected=self._on_client_disconnected,
             on_session_timeout=self._on_session_timeout,
+            on_websocket_ready=self._on_websocket_ready,
         )
         self._input: Optional[WebsocketServerInputTransport] = None
         self._output: Optional[WebsocketServerOutputTransport] = None
@@ -264,6 +274,7 @@ class WebsocketServerTransport(BaseTransport):
         self._register_event_handler("on_client_connected")
         self._register_event_handler("on_client_disconnected")
         self._register_event_handler("on_session_timeout")
+        self._register_event_handler("on_websocket_ready")
 
     def input(self) -> WebsocketServerInputTransport:
         if not self._input:
@@ -293,3 +304,6 @@ class WebsocketServerTransport(BaseTransport):
 
     async def _on_session_timeout(self, websocket):
         await self._call_event_handler("on_session_timeout", websocket)
+
+    async def _on_websocket_ready(self):
+        await self._call_event_handler("on_websocket_ready")

src/pipecat/transports/services/daily.py

@@ -13,9 +13,6 @@ from typing import Any, Awaitable, Callable, Mapping, Optional
 
 import aiohttp
 from daily import (
-    CallClient,
-    Daily,
-    EventHandler,
     VirtualCameraDevice,
     VirtualMicrophoneDevice,
     VirtualSpeakerDevice,
@@ -291,6 +288,7 @@ class DailyTransportClient(EventHandler):
         self._transcription_ids = []
         self._transcription_status = None
 
+        self._joining = False
         self._joined = False
         self._joined_event = asyncio.Event()
         self._leave_counter = 0
@@ -423,13 +421,14 @@ class DailyTransportClient(EventHandler):
             )
 
     async def join(self):
-        # Transport already joined, ignore.
-        if self._joined:
+        # Transport already joined or joining, ignore.
+        if self._joined or self._joining:
             # Increment leave counter if we already joined.
             self._leave_counter += 1
             return
 
         logger.info(f"Joining {self._room_url}")
+        self._joining = True
 
         # For performance reasons, never subscribe to video streams (unless a
         # video renderer is registered).
@@ -444,6 +443,7 @@ class DailyTransportClient(EventHandler):
 
             if not error:
                 self._joined = True
+                self._joining = False
                 # Increment leave counter if we successfully joined.
                 self._leave_counter += 1
 
@@ -462,6 +462,7 @@ class DailyTransportClient(EventHandler):
         except asyncio.TimeoutError:
             error_msg = f"Time out joining {self._room_url}"
             logger.error(error_msg)
+            self._joining = False
             await self._callbacks.on_error(error_msg)
 
     async def _start_transcription(self):
@@ -690,6 +691,13 @@ class DailyTransportClient(EventHandler):
         )
         await future
 
+    async def update_remote_participants(self, remote_participants: Mapping[str, Any] = None):
+        future = self._get_event_loop().create_future()
+        self._client.update_remote_participants(
+            remote_participants=remote_participants, completion=completion_callback(future)
+        )
+        await future
+
     #
     #
     # Daily (EventHandler)
@@ -830,6 +838,13 @@ class DailyInputTransport(BaseInputTransport):
     def vad_analyzer(self) -> Optional[VADAnalyzer]:
         return self._vad_analyzer
 
+    def start_audio_in_streaming(self):
+        # Create audio task. It reads audio frames from Daily and push them
+        # internally for VAD processing.
+        if self._params.audio_in_enabled or self._params.vad_enabled:
+            logger.debug(f"Start receiving audio")
+            self._audio_in_task = self.create_task(self._audio_in_task_handler())
+
     async def start(self, frame: StartFrame):
         # Parent start.
         await super().start(frame)
@@ -840,10 +855,8 @@ class DailyInputTransport(BaseInputTransport):
         # Inialize WebRTC VAD if needed.
         if self._params.vad_enabled and not self._params.vad_analyzer:
             self._vad_analyzer = WebRTCVADAnalyzer(sample_rate=self.sample_rate)
-        # Create audio task. It reads audio frames from Daily and push them
-        # internally for VAD processing.
-        if self._params.audio_in_enabled or self._params.vad_enabled:
-            self._audio_in_task = self.create_task(self._audio_in_task_handler())
+        if self._params.audio_in_stream_on_start:
+            self.start_audio_in_streaming()
 
     async def stop(self, frame: EndFrame):
         # Parent stop.
@@ -1167,6 +1180,9 @@ class DailyTransport(BaseTransport):
             participant_settings=participant_settings, profile_settings=profile_settings
         )
 
+    async def update_remote_participants(self, remote_participants: Mapping[str, Any] = None):
+        await self._client.update_remote_participants(remote_participants=remote_participants)
+
     async def _on_joined(self, data):
         await self._call_event_handler("on_joined", data)
 

src/pipecat/utils/string.py

@@ -13,8 +13,8 @@ ENDOFSENTENCE_PATTERN_STR = r"""
     (?<!Mr|Ms|Dr)    # Negative lookbehind: not preceded by Mr, Ms, Dr (combined bc. length is the same)
     (?<!Mrs)         # Negative lookbehind: not preceded by "Mrs"
     (?<!Prof)        # Negative lookbehind: not preceded by "Prof"
-    [\.\?\!:;]|      # Match a period, question mark, exclamation point, colon, or semicolon
-    [。？！：；।]       # the full-width version (mainly used in East Asian languages such as Chinese, Hindi)
+    [\.\?\!;]|       # Match a period, question mark, exclamation point, or semicolon
+    [。？！；।]        # the full-width version (mainly used in East Asian languages such as Chinese, Hindi)
     $                # End of string
 """
 ENDOFSENTENCE_PATTERN = re.compile(ENDOFSENTENCE_PATTERN_STR, re.VERBOSE)

test-requirements.txt

@@ -7,8 +7,10 @@ deepgram-sdk~=3.5.0
 fal-client~=0.4.1
 fastapi~=0.115.0
 faster-whisper~=1.0.3
-google-cloud-texttospeech~=2.21.1
-google-generativeai~=0.8.3
+google-cloud-speech~=2.31.0
+google-cloud-texttospeech~=2.25.0
+google-genai~=1.2.0
+google-generativeai~=0.8.4
 langchain~=0.2.14
 livekit~=0.13.1
 lmnt~=1.1.4
@@ -25,5 +27,6 @@ python-dotenv~=1.0.1
 silero-vad~=5.1
 soxr~=0.5.0
 together~=1.2.7
-transformers~=4.44.0
+transformers~=4.48.0
 websockets~=13.1
+Markdown~=3.7

tests/test_aggregators.py

@@ -29,12 +29,12 @@ class TestSentenceAggregator(unittest.IsolatedAsyncioTestCase):
         for word in sentence.split(" "):
             frames_to_send.append(TextFrame(text=word + " "))
 
-        expected_returned_frames = [TextFrame, TextFrame, TextFrame]
+        expected_down_frames = [TextFrame, TextFrame, TextFrame]
 
         (received_down, _) = await run_test(
             aggregator,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
         assert received_down[-3].text == "Hello, world. "
         assert received_down[-2].text == "How are you? "
@@ -59,7 +59,7 @@ class TestGatedAggregator(unittest.IsolatedAsyncioTestCase):
             LLMFullResponseEndFrame(),
         ]
 
-        expected_returned_frames = [
+        expected_down_frames = [
             OutputImageRawFrame,
             LLMFullResponseStartFrame,
             TextFrame,
@@ -72,5 +72,5 @@ class TestGatedAggregator(unittest.IsolatedAsyncioTestCase):
         (received_down, _) = await run_test(
             gated_aggregator,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )

tests/test_context_aggregators.py

@@ -0,0 +1,671 @@
+#
+# Copyright (c) 2024-2025 Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+import unittest
+
+import google.ai.generativelanguage as glm
+
+from pipecat.frames.frames import (
+    EmulateUserStartedSpeakingFrame,
+    EmulateUserStoppedSpeakingFrame,
+    InterimTranscriptionFrame,
+    LLMFullResponseEndFrame,
+    LLMFullResponseStartFrame,
+    OpenAILLMContextAssistantTimestampFrame,
+    StartInterruptionFrame,
+    TextFrame,
+    TranscriptionFrame,
+    UserStartedSpeakingFrame,
+    UserStoppedSpeakingFrame,
+)
+from pipecat.processors.aggregators.llm_response import (
+    LLMAssistantContextAggregator,
+    LLMUserContextAggregator,
+)
+from pipecat.processors.aggregators.openai_llm_context import (
+    OpenAILLMContext,
+    OpenAILLMContextFrame,
+)
+from pipecat.services.anthropic import (
+    AnthropicAssistantContextAggregator,
+    AnthropicLLMContext,
+    AnthropicUserContextAggregator,
+)
+from pipecat.services.google.google import (
+    GoogleAssistantContextAggregator,
+    GoogleLLMContext,
+    GoogleUserContextAggregator,
+)
+from pipecat.services.openai import OpenAIAssistantContextAggregator, OpenAIUserContextAggregator
+from pipecat.tests.utils import SleepFrame, run_test
+
+AGGREGATION_TIMEOUT = 0.1
+AGGREGATION_SLEEP = 0.15
+BOT_INTERRUPTION_TIMEOUT = 0.2
+BOT_INTERRUPTION_SLEEP = 0.25
+
+
+class BaseTestUserContextAggregator:
+    CONTEXT_CLASS = None  # To be set in subclasses
+    AGGREGATOR_CLASS = None  # To be set in subclasses
+    EXPECTED_CONTEXT_FRAMES = [OpenAILLMContextFrame]
+
+    def check_message_content(self, context: OpenAILLMContext, index: int, content: str):
+        assert context.messages[index]["content"] == content
+
+    def check_message_multi_content(
+        self, context: OpenAILLMContext, content_index: int, index: int, content: str
+    ):
+        assert context.messages[index]["content"] == content
+
+    async def test_se(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context)
+        frames_to_send = [UserStartedSpeakingFrame(), UserStoppedSpeakingFrame()]
+        expected_down_frames = [UserStartedSpeakingFrame, UserStoppedSpeakingFrame]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+    async def test_ste(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            TranscriptionFrame(text="Hello!", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello!")
+
+    async def test_site(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            InterimTranscriptionFrame(text="Hello", user_id="cat", timestamp=""),
+            TranscriptionFrame(text="Hello Pipecat!", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat!")
+
+    async def test_st1iest2e(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            TranscriptionFrame(text="Hello Pipecat!", user_id="cat", timestamp=""),
+            InterimTranscriptionFrame(text="How ", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+            UserStartedSpeakingFrame(),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat! How are you?")
+
+    async def test_siet(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            InterimTranscriptionFrame(text="How ", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "How are you?")
+
+    async def test_sieit(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            InterimTranscriptionFrame(text="How ", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+            InterimTranscriptionFrame(text="are you?", user_id="cat", timestamp=""),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "How are you?")
+
+    async def test_set(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "How are you?")
+
+    async def test_seit(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            InterimTranscriptionFrame(text="How ", user_id="cat", timestamp=""),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "How are you?")
+
+    async def test_st1et2(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            TranscriptionFrame(text="Hello Pipecat!", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_multi_content(context, 0, 0, "Hello Pipecat!")
+        self.check_message_multi_content(context, 0, 1, "How are you?")
+
+    async def test_set1t2(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            TranscriptionFrame(text="Hello Pipecat!", user_id="cat", timestamp=""),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat! How are you?")
+
+    async def test_siet1it2(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            InterimTranscriptionFrame(text="Hello ", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+            TranscriptionFrame(text="Hello Pipecat!", user_id="cat", timestamp=""),
+            InterimTranscriptionFrame(text="How ", user_id="cat", timestamp=""),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat! How are you?")
+
+    async def test_t(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            TranscriptionFrame(text="Hello!", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [*self.EXPECTED_CONTEXT_FRAMES]
+        expected_up_frames = [EmulateUserStartedSpeakingFrame, EmulateUserStoppedSpeakingFrame]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+            expected_up_frames=expected_up_frames,
+        )
+        self.check_message_content(context, 0, "Hello!")
+
+    async def test_it(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, aggregation_timeout=AGGREGATION_TIMEOUT)
+        frames_to_send = [
+            InterimTranscriptionFrame(text="Hello ", user_id="cat", timestamp=""),
+            SleepFrame(),
+            TranscriptionFrame(text="Hello Pipecat!", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [*self.EXPECTED_CONTEXT_FRAMES]
+        expected_up_frames = [EmulateUserStartedSpeakingFrame, EmulateUserStoppedSpeakingFrame]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+            expected_up_frames=expected_up_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat!")
+
+    async def test_sie_delay_it(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(
+            context,
+            aggregation_timeout=AGGREGATION_TIMEOUT,
+            bot_interruption_timeout=BOT_INTERRUPTION_TIMEOUT,
+        )
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            InterimTranscriptionFrame(text="How ", user_id="cat", timestamp=""),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+            SleepFrame(BOT_INTERRUPTION_SLEEP),
+            InterimTranscriptionFrame(text="are you?", user_id="cat", timestamp=""),
+            TranscriptionFrame(text="How are you?", user_id="cat", timestamp=""),
+            SleepFrame(sleep=AGGREGATION_SLEEP),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        expected_up_frames = [EmulateUserStartedSpeakingFrame, EmulateUserStoppedSpeakingFrame]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+            expected_up_frames=expected_up_frames,
+        )
+        self.check_message_content(context, 0, "How are you?")
+
+
+class BaseTestAssistantContextAggreagator:
+    CONTEXT_CLASS = None  # To be set in subclasses
+    AGGREGATOR_CLASS = None  # To be set in subclasses
+    EXPECTED_CONTEXT_FRAMES = [OpenAILLMContextFrame]
+
+    def check_message_content(self, context: OpenAILLMContext, index: int, content: str):
+        assert context.messages[index]["content"] == content
+
+    def check_message_multi_content(
+        self, context: OpenAILLMContext, content_index: int, index: int, content: str
+    ):
+        assert context.messages[index]["content"] == content
+
+    async def test_empty(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context)
+        frames_to_send = [LLMFullResponseStartFrame(), LLMFullResponseEndFrame()]
+        expected_down_frames = []
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+    async def test_single_text(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context)
+        frames_to_send = [
+            LLMFullResponseStartFrame(),
+            TextFrame(text="Hello Pipecat!"),
+            LLMFullResponseEndFrame(),
+        ]
+        expected_down_frames = [*self.EXPECTED_CONTEXT_FRAMES]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat!")
+
+    async def test_multiple_text(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, expect_stripped_words=False)
+        frames_to_send = [
+            LLMFullResponseStartFrame(),
+            TextFrame(text="Hello "),
+            TextFrame(text="Pipecat. "),
+            TextFrame(text="How are "),
+            TextFrame(text="you?"),
+            LLMFullResponseEndFrame(),
+        ]
+        expected_down_frames = [*self.EXPECTED_CONTEXT_FRAMES]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat. How are you?")
+
+    async def test_multiple_text_stripped(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context)
+        frames_to_send = [
+            LLMFullResponseStartFrame(),
+            TextFrame(text="Hello"),
+            TextFrame(text="Pipecat."),
+            TextFrame(text="How are"),
+            TextFrame(text="you?"),
+            LLMFullResponseEndFrame(),
+        ]
+        expected_down_frames = [*self.EXPECTED_CONTEXT_FRAMES]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_content(context, 0, "Hello Pipecat. How are you?")
+
+    async def test_multiple_llm_responses(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, expect_stripped_words=False)
+        frames_to_send = [
+            LLMFullResponseStartFrame(),
+            TextFrame(text="Hello "),
+            TextFrame(text="Pipecat."),
+            LLMFullResponseEndFrame(),
+            LLMFullResponseStartFrame(),
+            TextFrame(text="How are "),
+            TextFrame(text="you?"),
+            LLMFullResponseEndFrame(),
+        ]
+        expected_down_frames = [*self.EXPECTED_CONTEXT_FRAMES, *self.EXPECTED_CONTEXT_FRAMES]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_multi_content(context, 0, 0, "Hello Pipecat.")
+        self.check_message_multi_content(context, 0, 1, "How are you?")
+
+    async def test_multiple_llm_responses_interruption(self):
+        assert self.CONTEXT_CLASS is not None, "CONTEXT_CLASS must be set in a subclass"
+        assert self.AGGREGATOR_CLASS is not None, "AGGREGATOR_CLASS must be set in a subclass"
+
+        context = self.CONTEXT_CLASS()
+        aggregator = self.AGGREGATOR_CLASS(context, expect_stripped_words=False)
+        frames_to_send = [
+            LLMFullResponseStartFrame(),
+            TextFrame(text="Hello "),
+            TextFrame(text="Pipecat."),
+            LLMFullResponseEndFrame(),
+            SleepFrame(AGGREGATION_SLEEP),
+            StartInterruptionFrame(),
+            LLMFullResponseStartFrame(),
+            TextFrame(text="How are "),
+            TextFrame(text="you?"),
+            LLMFullResponseEndFrame(),
+        ]
+        expected_down_frames = [
+            *self.EXPECTED_CONTEXT_FRAMES,
+            StartInterruptionFrame,
+            *self.EXPECTED_CONTEXT_FRAMES,
+        ]
+        await run_test(
+            aggregator,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+        self.check_message_multi_content(context, 0, 0, "Hello Pipecat.")
+        self.check_message_multi_content(context, 0, 1, "How are you?")
+
+
+#
+# LLMUserContextAggregator, LLMAssistantContextAggregator
+#
+
+
+class TestLLMUserContextAggregator(BaseTestUserContextAggregator, unittest.IsolatedAsyncioTestCase):
+    CONTEXT_CLASS = OpenAILLMContext
+    AGGREGATOR_CLASS = LLMUserContextAggregator
+
+
+class TestLLMAssistantContextAggregator(
+    BaseTestAssistantContextAggreagator, unittest.IsolatedAsyncioTestCase
+):
+    CONTEXT_CLASS = OpenAILLMContext
+    AGGREGATOR_CLASS = LLMAssistantContextAggregator
+
+
+#
+# OpenAI
+#
+
+
+class TestOpenAIUserContextAggregator(
+    BaseTestUserContextAggregator, unittest.IsolatedAsyncioTestCase
+):
+    CONTEXT_CLASS = OpenAILLMContext
+    AGGREGATOR_CLASS = OpenAIUserContextAggregator
+
+
+class TestOpenAIAssistantContextAggregator(
+    BaseTestAssistantContextAggreagator, unittest.IsolatedAsyncioTestCase
+):
+    CONTEXT_CLASS = OpenAILLMContext
+    AGGREGATOR_CLASS = OpenAIAssistantContextAggregator
+    EXPECTED_CONTEXT_FRAMES = [OpenAILLMContextFrame, OpenAILLMContextAssistantTimestampFrame]
+
+
+#
+# Anthropic
+#
+
+
+class TestAnthropicUserContextAggregator(
+    BaseTestUserContextAggregator, unittest.IsolatedAsyncioTestCase
+):
+    CONTEXT_CLASS = AnthropicLLMContext
+    AGGREGATOR_CLASS = AnthropicUserContextAggregator
+
+    def check_message_multi_content(
+        self, context: OpenAILLMContext, content_index: int, index: int, content: str
+    ):
+        messages = context.messages[content_index]
+        assert messages["content"][index]["text"] == content
+
+
+class TestAnthropicAssistantContextAggregator(
+    BaseTestAssistantContextAggreagator, unittest.IsolatedAsyncioTestCase
+):
+    CONTEXT_CLASS = AnthropicLLMContext
+    AGGREGATOR_CLASS = AnthropicAssistantContextAggregator
+    EXPECTED_CONTEXT_FRAMES = [OpenAILLMContextFrame, OpenAILLMContextAssistantTimestampFrame]
+
+    def check_message_multi_content(
+        self, context: OpenAILLMContext, content_index: int, index: int, content: str
+    ):
+        messages = context.messages[content_index]
+        assert messages["content"][index]["text"] == content
+
+
+#
+# Google
+#
+
+
+class TestGoogleUserContextAggregator(
+    BaseTestUserContextAggregator, unittest.IsolatedAsyncioTestCase
+):
+    CONTEXT_CLASS = GoogleLLMContext
+    AGGREGATOR_CLASS = GoogleUserContextAggregator
+
+    def check_message_content(self, context: OpenAILLMContext, index: int, content: str):
+        obj = glm.Content.to_dict(context.messages[index])
+        assert obj["parts"][0]["text"] == content
+
+    def check_message_multi_content(
+        self, context: OpenAILLMContext, content_index: int, index: int, content: str
+    ):
+        obj = glm.Content.to_dict(context.messages[index])
+        assert obj["parts"][0]["text"] == content
+
+
+class TestGoogleAssistantContextAggregator(
+    BaseTestAssistantContextAggreagator, unittest.IsolatedAsyncioTestCase
+):
+    CONTEXT_CLASS = GoogleLLMContext
+    AGGREGATOR_CLASS = GoogleAssistantContextAggregator
+    EXPECTED_CONTEXT_FRAMES = [OpenAILLMContextFrame, OpenAILLMContextAssistantTimestampFrame]
+
+    def check_message_content(self, context: OpenAILLMContext, index: int, content: str):
+        obj = glm.Content.to_dict(context.messages[index])
+        assert obj["parts"][0]["text"] == content
+
+    def check_message_multi_content(
+        self, context: OpenAILLMContext, content_index: int, index: int, content: str
+    ):
+        obj = glm.Content.to_dict(context.messages[index])
+        assert obj["parts"][0]["text"] == content

tests/test_filters.py

@@ -25,11 +25,11 @@ class TestIdentifyFilter(unittest.IsolatedAsyncioTestCase):
     async def test_identity(self):
         filter = IdentityFilter()
         frames_to_send = [UserStartedSpeakingFrame(), UserStoppedSpeakingFrame()]
-        expected_returned_frames = [UserStartedSpeakingFrame, UserStoppedSpeakingFrame]
+        expected_down_frames = [UserStartedSpeakingFrame, UserStoppedSpeakingFrame]
         await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
 
@@ -37,32 +37,32 @@ class TestFrameFilter(unittest.IsolatedAsyncioTestCase):
     async def test_text_frame(self):
         filter = FrameFilter(types=(TextFrame,))
         frames_to_send = [TextFrame(text="Hello Pipecat!")]
-        expected_returned_frames = [TextFrame]
+        expected_down_frames = [TextFrame]
         await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
     async def test_end_frame(self):
         filter = FrameFilter(types=(EndFrame,))
         frames_to_send = [EndFrame()]
-        expected_returned_frames = [EndFrame]
+        expected_down_frames = [EndFrame]
         await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
             send_end_frame=False,
         )
 
     async def test_system_frame(self):
         filter = FrameFilter(types=())
         frames_to_send = [UserStartedSpeakingFrame()]
-        expected_returned_frames = [UserStartedSpeakingFrame]
+        expected_down_frames = [UserStartedSpeakingFrame]
         await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
 
@@ -73,11 +73,11 @@ class TestFunctionFilter(unittest.IsolatedAsyncioTestCase):
 
         filter = FunctionFilter(filter=passthrough)
         frames_to_send = [TextFrame(text="Hello Pipecat!")]
-        expected_returned_frames = [TextFrame]
+        expected_down_frames = [TextFrame]
         await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
     async def test_no_passthrough(self):
@@ -86,11 +86,11 @@ class TestFunctionFilter(unittest.IsolatedAsyncioTestCase):
 
         filter = FunctionFilter(filter=no_passthrough)
         frames_to_send = [TextFrame(text="Hello Pipecat!")]
-        expected_returned_frames = []
+        expected_down_frames = []
         await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
 
@@ -98,11 +98,11 @@ class TestWakeCheckFilter(unittest.IsolatedAsyncioTestCase):
     async def test_no_wake_word(self):
         filter = WakeCheckFilter(wake_phrases=["Hey, Pipecat"])
         frames_to_send = [TranscriptionFrame(user_id="test", text="Phrase 1", timestamp="")]
-        expected_returned_frames = []
+        expected_down_frames = []
         await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
     async def test_wake_word(self):
@@ -111,10 +111,10 @@ class TestWakeCheckFilter(unittest.IsolatedAsyncioTestCase):
             TranscriptionFrame(user_id="test", text="Hey, Pipecat", timestamp=""),
             TranscriptionFrame(user_id="test", text="Phrase 1", timestamp=""),
         ]
-        expected_returned_frames = [TranscriptionFrame, TranscriptionFrame]
+        expected_down_frames = [TranscriptionFrame, TranscriptionFrame]
         (received_down, _) = await run_test(
             filter,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
         assert received_down[-1].text == "Phrase 1"

tests/test_langchain.py

@@ -10,23 +10,22 @@ from langchain.prompts import ChatPromptTemplate
 from langchain_core.language_models import FakeStreamingListLLM
 
 from pipecat.frames.frames import (
-    EndFrame,
     LLMFullResponseEndFrame,
     LLMFullResponseStartFrame,
+    LLMMessagesFrame,
     TextFrame,
     TranscriptionFrame,
     UserStartedSpeakingFrame,
     UserStoppedSpeakingFrame,
 )
 from pipecat.pipeline.pipeline import Pipeline
-from pipecat.pipeline.runner import PipelineRunner
-from pipecat.pipeline.task import PipelineParams, PipelineTask
 from pipecat.processors.aggregators.llm_response import (
     LLMAssistantResponseAggregator,
     LLMUserResponseAggregator,
 )
 from pipecat.processors.frame_processor import FrameProcessor
 from pipecat.processors.frameworks.langchain import LangchainProcessor
+from pipecat.tests.utils import SleepFrame, run_test
 
 
 class TestLangchain(unittest.IsolatedAsyncioTestCase):
@@ -64,31 +63,26 @@ class TestLangchain(unittest.IsolatedAsyncioTestCase):
         self.mock_proc = self.MockProcessor("token_collector")
 
         tma_in = LLMUserResponseAggregator(messages)
-        tma_out = LLMAssistantResponseAggregator(messages)
+        tma_out = LLMAssistantResponseAggregator(messages, expect_stripped_words=False)
 
-        pipeline = Pipeline(
-            [
-                tma_in,
-                proc,
-                self.mock_proc,
-                tma_out,
-            ]
+        pipeline = Pipeline([tma_in, proc, self.mock_proc, tma_out])
+
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            TranscriptionFrame(text="Hi World", user_id="user", timestamp="now"),
+            SleepFrame(),
+            UserStoppedSpeakingFrame(),
+        ]
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            LLMMessagesFrame,
+        ]
+        await run_test(
+            pipeline,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
         )
 
-        task = PipelineTask(pipeline, PipelineParams(allow_interruptions=False))
-        await task.queue_frames(
-            [
-                UserStartedSpeakingFrame(),
-                TranscriptionFrame(text="Hi World", user_id="user", timestamp="now"),
-                UserStoppedSpeakingFrame(),
-                EndFrame(),
-            ]
-        )
-
-        runner = PipelineRunner()
-        await runner.run(task)
         self.assertEqual("".join(self.mock_proc.token), self.expected_response)
-        # TODO: Address this issue
-        # This next one would fail with:
-        # AssertionError: ' H e l l o   d e a r   h u m a n' != 'Hello dear human'
-        # self.assertEqual(tma_out.messages[-1]["content"], self.expected_response)
+        self.assertEqual(tma_out.messages[-1]["content"], self.expected_response)

tests/test_markdown_text_filter.py

@@ -0,0 +1,230 @@
+#
+# Copyright (c) 2024-2025 Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+
+import unittest
+
+from pipecat.utils.text.markdown_text_filter import MarkdownTextFilter
+
+
+class TestMarkdownTextFilter(unittest.IsolatedAsyncioTestCase):
+    def setUp(self):
+        self.filter = MarkdownTextFilter()
+
+    async def test_basic_markdown_removal(self):
+        """Test removal of basic Markdown formatting while preserving content."""
+        input_text = """
+            **Bold text** and *italic text*
+            1. Numbered list item
+            - Bullet point
+            Some `inline code` here
+        """
+
+        expected_text = """
+            Bold text and italic text
+            1. Numbered list item
+            - Bullet point
+            Some inline code here
+        """
+
+        result = self.filter.filter(input_text)
+        self.assertEqual(result.strip(), expected_text.strip())
+
+    async def test_space_preservation(self):
+        """Test preservation of leading and trailing spaces (for
+        word-by-word streaming in bot-tts-text).
+        """
+        input_text = [
+            "  Leading spaces",
+            "Trailing spaces  ",
+            "  Both ends  ",
+            "  Multiple  spaces  between  words  ",
+        ]
+
+        for text in input_text:
+            result = self.filter.filter(text)
+            self.assertEqual(
+                len(result), len(text), f"Space preservation failed for: '{text}'\nGot: '{result}'"
+            )
+            # Check if spaces are in the same positions
+            for i, char in enumerate(text):
+                if char == " ":
+                    self.assertEqual(
+                        result[i], " ", f"Space at position {i} was not preserved in: '{text}'"
+                    )
+
+    async def test_repeated_character_removal(self):
+        """Test removal of repeated character sequences (5 or more)."""
+        test_cases = {
+            "Hello!!!!!World": "HelloWorld",  # 5 exclamations removed
+            "Test####ing": "Test####ing",  # 4 hashes preserved
+            "Normal text": "Normal text",  # No repeated chars
+            "!!!!!": "",  # All repeated chars removed
+            "Mixed!!!!!...../////": "Mixed",  # Multiple repeated sequences
+            "Text^^^^test": "Text^^^^test",  # 4 carets preserved
+            "Text^^^^^test": "Texttest",  # 5 carets removed
+            "Dots....here": "Dots....here",  # 4 dots preserved
+            "Dots.....here": "Dotshere",  # 5 dots removed
+        }
+
+        for input_text, expected in test_cases.items():
+            result = self.filter.filter(input_text)
+            self.assertEqual(
+                result,
+                expected,
+                f"Failed to handle repeated characters in: '{input_text}'\nExpected: '{expected}'\nGot: '{result}'",
+            )
+
+    async def test_numbered_list_preservation(self):
+        """Test that numbered lists are preserved correctly."""
+        input_text = """1. First item
+        2. Second item
+        3. Third item with **bold**"""
+
+        expected = """1. First item
+        2. Second item
+        3. Third item with bold"""
+
+        result = self.filter.filter(input_text)
+        self.assertEqual(
+            result.strip(),
+            expected.strip(),
+            f"Numbered list preservation failed.\nExpected:\n{expected}\nGot:\n{result}",
+        )
+
+    async def test_html_entity_conversion(self):
+        """Test conversion of HTML entities to their plain text equivalents."""
+        test_cases = {
+            "This & that": "This & that",
+            "1 &lt; 2": "1 < 2",
+            "2 &gt; 1": "2 > 1",
+            "Line&nbsp;break": "Line break",
+            "Mixed &amp; &lt;entities&gt;": "Mixed & <entities>",
+        }
+
+        for input_text, expected in test_cases.items():
+            result = self.filter.filter(input_text)
+            self.assertEqual(result, expected, f"HTML entity conversion failed for: '{input_text}'")
+
+    async def test_asterisk_removal(self):
+        """Test removal of Markdown asterisk formatting."""
+        test_cases = {
+            "**bold text**": "bold text",  # Double asterisks
+            "*italic text*": "italic text",  # Single asterisks
+            "**bold** and *italic*": "bold and italic",  # Mixed
+            "multiple**bold**words": "multipleboldwords",  # No spaces
+            "edge**cases***here*": "edgecaseshere",  # Adjacent asterisks
+        }
+
+        for input_text, expected in test_cases.items():
+            result = self.filter.filter(input_text)
+            self.assertEqual(result, expected, f"Asterisk removal failed for: '{input_text}'")
+
+    async def test_newline_handling(self):
+        """Test handling of empty and whitespace-only lines."""
+        test_cases = {
+            "Line 1\n\nLine 2": "Line 1\n Line 2",  # Empty line becomes space
+            "Line 1\n   \nLine 2": "Line 1\n Line 2",  # Whitespace line becomes single space
+            "Text\n\n\nMore": "Text\n More",  # Multiple empty lines become spaces
+        }
+
+        for input_text, expected in test_cases.items():
+            result = self.filter.filter(input_text)
+            self.assertEqual(
+                result, expected, f"Newline handling failed for:\n{input_text}\nGot:\n{result}"
+            )
+
+    async def test_numbered_list_marker_handling(self):
+        """Test handling of numbered lists with the special §NUM§ marker."""
+        test_cases = {
+            "1. First\n2. Second": "1. First\n2. Second",  # Basic numbered list
+            "  1. Indented": "  1. Indented",  # Indented numbered list
+            "1. Item\nText\n2. Item": "1. Item\nText\n2. Item",  # Text between items
+            "1.No space": "1.No space",  # Not a list item (no space)
+            "12. Large number": "12. Large number",  # Multi-digit numbers
+        }
+
+        for input_text, expected in test_cases.items():
+            result = self.filter.filter(input_text)
+            self.assertEqual(
+                result,
+                expected,
+                f"Numbered list handling failed for:\n{input_text}\nGot:\n{result}",
+            )
+
+    async def test_inline_code_handling(self):
+        """Test handling of inline code with backticks."""
+        test_cases = {
+            "`code`": "code",  # Basic inline code
+            "Text `code` more": "Text code more",  # Inline code within text
+            "``nested`code``": "nested`code",  # Nested backticks
+            "`code1` and `code2`": "code1 and code2",  # Multiple inline codes
+            "No``space``between": "Nospacebetween",  # No spaces around backticks
+        }
+
+        for input_text, expected in test_cases.items():
+            result = self.filter.filter(input_text)
+            self.assertEqual(result, expected, f"Inline code handling failed for: '{input_text}'")
+
+    async def test_simple_table_removal(self):
+        """Test removal of a simple markdown table."""
+        filter = MarkdownTextFilter(params=MarkdownTextFilter.InputParams(filter_tables=True))
+
+        input_text = "| Column 1 | Column 2 |\n|----------|----------|\n| Cell 1   | Cell 2   |"
+
+        expected = ""
+
+        result = filter.filter(input_text)
+        self.assertEqual(
+            result.strip(),
+            expected.strip(),
+            f"Simple table removal failed.\nExpected:\n{expected}\nGot:\n{result}",
+        )
+
+    async def test_feature_toggles(self):
+        """Test enabling/disabling specific filter features."""
+        # Create a filter with all features disabled
+        filter = MarkdownTextFilter(
+            params=MarkdownTextFilter.InputParams(
+                enable_text_filter=False,
+                filter_code=False,
+                filter_tables=False,
+            )
+        )
+
+        # Test with text filtering disabled
+        text_with_markdown = "**bold** and *italic* with `code`"
+        self.assertEqual(
+            filter.filter(text_with_markdown),
+            text_with_markdown,
+            "Disabled filter should not modify text",
+        )
+
+        # Enable just text filtering
+        filter.update_settings({"enable_text_filter": True})
+        self.assertEqual(
+            filter.filter(text_with_markdown),
+            "bold and italic with code",
+            "Enabled filter should remove markdown",
+        )
+
+    async def test_settings_update(self):
+        """Test that filter settings can be updated at runtime."""
+        filter = MarkdownTextFilter()
+
+        # Initial state - formatting should be removed
+        input_text = "**bold** and *italic*"
+        self.assertEqual(filter.filter(input_text), "bold and italic")
+
+        # Disable text filtering
+        filter.update_settings({"enable_text_filter": False})
+        self.assertEqual(filter.filter(input_text), input_text, "Text filtering should be disabled")
+
+        # Re-enable text filtering
+        filter.update_settings({"enable_text_filter": True})
+        self.assertEqual(
+            filter.filter(input_text), "bold and italic", "Text filtering should be re-enabled"
+        )

tests/test_pipeline.py

@@ -21,11 +21,11 @@ class TestPipeline(unittest.IsolatedAsyncioTestCase):
         pipeline = Pipeline([IdentityFilter()])
 
         frames_to_send = [TextFrame(text="Hello from Pipecat!")]
-        expected_returned_frames = [TextFrame]
+        expected_down_frames = [TextFrame]
         await run_test(
             pipeline,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
     async def test_pipeline_multiple(self):
@@ -36,22 +36,22 @@ class TestPipeline(unittest.IsolatedAsyncioTestCase):
         pipeline = Pipeline([identity1, identity2, identity3])
 
         frames_to_send = [TextFrame(text="Hello from Pipecat!")]
-        expected_returned_frames = [TextFrame]
+        expected_down_frames = [TextFrame]
         await run_test(
             pipeline,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
     async def test_pipeline_start_metadata(self):
         pipeline = Pipeline([IdentityFilter()])
 
         frames_to_send = []
-        expected_returned_frames = [StartFrame]
+        expected_down_frames = [StartFrame]
         (received_down, _) = await run_test(
             pipeline,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
             ignore_start=False,
             start_metadata={"foo": "bar"},
         )
@@ -63,11 +63,11 @@ class TestParallelPipeline(unittest.IsolatedAsyncioTestCase):
         pipeline = ParallelPipeline([IdentityFilter()])
 
         frames_to_send = [TextFrame(text="Hello from Pipecat!")]
-        expected_returned_frames = [TextFrame]
+        expected_down_frames = [TextFrame]
         await run_test(
             pipeline,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
     async def test_parallel_multiple(self):
@@ -75,11 +75,11 @@ class TestParallelPipeline(unittest.IsolatedAsyncioTestCase):
         pipeline = ParallelPipeline([IdentityFilter()], [IdentityFilter()])
 
         frames_to_send = [TextFrame(text="Hello from Pipecat!")]
-        expected_returned_frames = [TextFrame]
+        expected_down_frames = [TextFrame]
         await run_test(
             pipeline,
             frames_to_send=frames_to_send,
-            expected_down_frames=expected_returned_frames,
+            expected_down_frames=expected_down_frames,
         )
 
 

tests/test_stt_mute_filter.py

@@ -0,0 +1,217 @@
+#
+# Copyright (c) 2024-2025 Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+import unittest
+
+from pipecat.frames.frames import (
+    BotStartedSpeakingFrame,
+    BotStoppedSpeakingFrame,
+    FunctionCallInProgressFrame,
+    FunctionCallResultFrame,
+    STTMuteFrame,
+    UserStartedSpeakingFrame,
+    UserStoppedSpeakingFrame,
+)
+from pipecat.processors.filters.stt_mute_filter import STTMuteConfig, STTMuteFilter, STTMuteStrategy
+from pipecat.tests.utils import run_test
+
+
+class TestSTTMuteFilter(unittest.IsolatedAsyncioTestCase):
+    async def test_first_speech_strategy(self):
+        filter = STTMuteFilter(config=STTMuteConfig(strategies={STTMuteStrategy.FIRST_SPEECH}))
+
+        frames_to_send = [
+            BotStartedSpeakingFrame(),  # First bot speech starts
+            UserStartedSpeakingFrame(),  # Should be suppressed
+            UserStoppedSpeakingFrame(),  # Should be suppressed
+            BotStoppedSpeakingFrame(),  # First bot speech ends
+            BotStartedSpeakingFrame(),  # Second bot speech
+            UserStartedSpeakingFrame(),  # Should pass through
+            UserStoppedSpeakingFrame(),  # Should pass through
+            BotStoppedSpeakingFrame(),
+        ]
+
+        expected_returned_frames = [
+            BotStartedSpeakingFrame,
+            STTMuteFrame,  # mute=True
+            BotStoppedSpeakingFrame,
+            STTMuteFrame,  # mute=False
+            BotStartedSpeakingFrame,
+            UserStartedSpeakingFrame,  # Now passes through
+            UserStoppedSpeakingFrame,  # Now passes through
+            BotStoppedSpeakingFrame,
+        ]
+
+        await run_test(
+            filter,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_returned_frames,
+        )
+
+    async def test_always_strategy(self):
+        filter = STTMuteFilter(config=STTMuteConfig(strategies={STTMuteStrategy.ALWAYS}))
+
+        frames_to_send = [
+            BotStartedSpeakingFrame(),  # First speech starts
+            UserStartedSpeakingFrame(),  # Should be suppressed
+            UserStoppedSpeakingFrame(),  # Should be suppressed
+            BotStoppedSpeakingFrame(),  # First speech ends
+            UserStartedSpeakingFrame(),  # Should pass through
+            UserStoppedSpeakingFrame(),  # Should pass through
+            BotStartedSpeakingFrame(),  # Second speech starts
+            UserStartedSpeakingFrame(),  # Should be suppressed again
+            UserStoppedSpeakingFrame(),  # Should be suppressed again
+            BotStoppedSpeakingFrame(),  # Second speech ends
+        ]
+
+        expected_returned_frames = [
+            BotStartedSpeakingFrame,
+            STTMuteFrame,  # mute=True
+            BotStoppedSpeakingFrame,
+            STTMuteFrame,  # mute=False
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            BotStartedSpeakingFrame,
+            STTMuteFrame,  # mute=True
+            BotStoppedSpeakingFrame,
+            STTMuteFrame,  # mute=False
+        ]
+
+        await run_test(
+            filter,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_returned_frames,
+        )
+
+    # TODO: Revisit once we figure out how to test SystemFrames and DataFrames
+    # async def test_function_call_strategy(self):
+    #     filter = STTMuteFilter(config=STTMuteConfig(strategies={STTMuteStrategy.FUNCTION_CALL}))
+
+    #     frames_to_send = [
+    #         UserStartedSpeakingFrame(),  # Should pass through initially
+    #         UserStoppedSpeakingFrame(),
+    #         FunctionCallInProgressFrame(
+    #             function_name="get_weather",
+    #             tool_call_id="call_123",
+    #             arguments='{"location": "San Francisco"}',
+    #         ),  # Start function call
+    #         UserStartedSpeakingFrame(),  # Should be suppressed
+    #         UserStoppedSpeakingFrame(),  # Should be suppressed
+    #         FunctionCallResultFrame(
+    #             function_name="get_weather",
+    #             tool_call_id="call_123",
+    #             arguments='{"location": "San Francisco"}',
+    #             result={"temperature": 22},
+    #         ),  # End function call
+    #         UserStartedSpeakingFrame(),  # Should pass through again
+    #         UserStoppedSpeakingFrame(),
+    #     ]
+
+    #     expected_returned_frames = [
+    #         UserStartedSpeakingFrame,
+    #         UserStoppedSpeakingFrame,
+    #         FunctionCallInProgressFrame,
+    #         STTMuteFrame,  # mute=True
+    #         FunctionCallResultFrame,
+    #         STTMuteFrame,  # mute=False
+    #         UserStartedSpeakingFrame,
+    #         UserStoppedSpeakingFrame,
+    #     ]
+
+    #     await run_test(
+    #         filter,
+    #         frames_to_send=frames_to_send,
+    #         expected_down_frames=expected_returned_frames,
+    #     )
+
+    async def test_mute_until_first_bot_complete_strategy(self):
+        filter = STTMuteFilter(
+            config=STTMuteConfig(strategies={STTMuteStrategy.MUTE_UNTIL_FIRST_BOT_COMPLETE})
+        )
+
+        frames_to_send = [
+            UserStartedSpeakingFrame(),  # Should be suppressed (starts muted)
+            UserStoppedSpeakingFrame(),  # Should be suppressed
+            BotStartedSpeakingFrame(),  # First bot speech
+            UserStartedSpeakingFrame(),  # Should be suppressed
+            UserStoppedSpeakingFrame(),  # Should be suppressed
+            BotStoppedSpeakingFrame(),  # First speech ends, unmutes
+            UserStartedSpeakingFrame(),  # Should pass through
+            UserStoppedSpeakingFrame(),  # Should pass through
+            BotStartedSpeakingFrame(),  # Second speech
+            UserStartedSpeakingFrame(),  # Should pass through
+            UserStoppedSpeakingFrame(),  # Should pass through
+            BotStoppedSpeakingFrame(),
+        ]
+
+        expected_returned_frames = [
+            STTMuteFrame,  # mute=True after first speech
+            BotStartedSpeakingFrame,
+            BotStoppedSpeakingFrame,
+            STTMuteFrame,  # mute=False after first speech
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            BotStartedSpeakingFrame,
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            BotStoppedSpeakingFrame,
+        ]
+
+        await run_test(
+            filter,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_returned_frames,
+        )
+
+    async def test_incompatible_strategies(self):
+        with self.assertRaises(ValueError):
+            STTMuteFilter(
+                config=STTMuteConfig(
+                    strategies={
+                        STTMuteStrategy.FIRST_SPEECH,
+                        STTMuteStrategy.MUTE_UNTIL_FIRST_BOT_COMPLETE,
+                    }
+                )
+            )
+
+    async def test_custom_strategy(self):
+        async def custom_mute_logic(processor: STTMuteFilter) -> bool:
+            return processor._bot_is_speaking
+
+        filter = STTMuteFilter(
+            config=STTMuteConfig(
+                strategies={STTMuteStrategy.CUSTOM},
+                should_mute_callback=custom_mute_logic,
+            )
+        )
+
+        frames_to_send = [
+            UserStartedSpeakingFrame(),  # Should pass through
+            UserStoppedSpeakingFrame(),  # Should pass through
+            BotStartedSpeakingFrame(),  # Bot starts speaking
+            UserStartedSpeakingFrame(),  # Should be suppressed
+            UserStoppedSpeakingFrame(),  # Should be suppressed
+            BotStoppedSpeakingFrame(),  # Bot stops speaking
+            UserStartedSpeakingFrame(),  # Should pass through
+            UserStoppedSpeakingFrame(),  # Should pass through
+        ]
+
+        expected_returned_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            BotStartedSpeakingFrame,
+            STTMuteFrame,  # mute=True
+            BotStoppedSpeakingFrame,
+            STTMuteFrame,  # mute=False
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+        ]
+
+        await run_test(
+            filter,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_returned_frames,
+        )

tests/test_transcript_processor.py

@@ -0,0 +1,428 @@
+#
+# Copyright (c) 2024-2025 Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+
+import asyncio
+import unittest
+from datetime import datetime, timezone
+from typing import List, Tuple, cast
+
+from pipecat.frames.frames import (
+    BotStartedSpeakingFrame,
+    BotStoppedSpeakingFrame,
+    CancelFrame,
+    StartInterruptionFrame,
+    TranscriptionFrame,
+    TranscriptionMessage,
+    TranscriptionUpdateFrame,
+    TTSTextFrame,
+)
+from pipecat.processors.transcript_processor import (
+    AssistantTranscriptProcessor,
+    UserTranscriptProcessor,
+)
+from pipecat.tests.utils import SleepFrame, run_test
+
+
+class TestUserTranscriptProcessor(unittest.IsolatedAsyncioTestCase):
+    """Tests for UserTranscriptProcessor"""
+
+    async def test_basic_transcription(self):
+        """Test basic transcription frame processing"""
+        # Create processor
+        processor = UserTranscriptProcessor()
+
+        # Create test timestamp
+        timestamp = datetime.now(timezone.utc).isoformat()
+
+        # Create frames to send
+        frames_to_send = [
+            TranscriptionFrame(text="Hello, world!", user_id="test_user", timestamp=timestamp)
+        ]
+
+        # Expected frames downstream - note the order:
+        # 1. TranscriptionUpdateFrame (processor emits the update first)
+        # 2. TranscriptionFrame (original frame is passed through)
+        expected_down_frames = [TranscriptionUpdateFrame, TranscriptionFrame]
+
+        # Run test
+        received_frames, _ = await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        # Verify the content of the TranscriptionUpdateFrame
+        update_frame = cast(
+            TranscriptionUpdateFrame, received_frames[0]
+        )  # Note: now checking first frame
+        self.assertIsInstance(update_frame, TranscriptionUpdateFrame)
+        self.assertEqual(len(update_frame.messages), 1)
+        message = update_frame.messages[0]
+        self.assertEqual(message.role, "user")
+        self.assertEqual(message.content, "Hello, world!")
+        self.assertEqual(message.timestamp, timestamp)
+
+    async def test_event_handler(self):
+        """Test that event handlers are called with transcript updates"""
+        # Create processor
+        processor = UserTranscriptProcessor()
+
+        # Track received updates
+        received_updates: List[TranscriptionMessage] = []
+
+        # Register event handler
+        @processor.event_handler("on_transcript_update")
+        async def handle_update(proc, frame: TranscriptionUpdateFrame):
+            received_updates.extend(frame.messages)
+
+        # Create test data
+        timestamp = datetime.now(timezone.utc).isoformat()
+        frames_to_send = [
+            TranscriptionFrame(text="First message", user_id="test_user", timestamp=timestamp),
+            TranscriptionFrame(text="Second message", user_id="test_user", timestamp=timestamp),
+        ]
+
+        expected_down_frames = [
+            TranscriptionUpdateFrame,
+            TranscriptionFrame,  # First message
+            TranscriptionUpdateFrame,
+            TranscriptionFrame,  # Second message
+        ]
+
+        # Run test
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        # Verify event handler received updates
+        self.assertEqual(len(received_updates), 2)
+
+        # Check first message
+        self.assertEqual(received_updates[0].role, "user")
+        self.assertEqual(received_updates[0].content, "First message")
+        self.assertEqual(received_updates[0].timestamp, timestamp)
+
+        # Check second message
+        self.assertEqual(received_updates[1].role, "user")
+        self.assertEqual(received_updates[1].content, "Second message")
+        self.assertEqual(received_updates[1].timestamp, timestamp)
+
+    async def test_text_aggregation(self):
+        """Test that TTSTextFrames are properly aggregated into a single message"""
+        # Create processor
+        processor = AssistantTranscriptProcessor()
+
+        # Track received updates
+        received_updates: List[TranscriptionUpdateFrame] = []
+
+        @processor.event_handler("on_transcript_update")
+        async def handle_update(proc, frame: TranscriptionUpdateFrame):
+            received_updates.append(frame)
+
+        # Create test frames simulating bot speaking multiple text chunks
+        frames_to_send = [
+            BotStartedSpeakingFrame(),
+            SleepFrame(sleep=0.1),  # Wait for StartedSpeaking to process
+            TTSTextFrame(text="Hello"),
+            TTSTextFrame(text="world!"),
+            TTSTextFrame(text="How"),
+            TTSTextFrame(text="are"),
+            TTSTextFrame(text="you?"),
+            SleepFrame(sleep=0.1),  # Wait for text frames to queue
+            BotStoppedSpeakingFrame(),
+        ]
+
+        # Expected order:
+        # 1. BotStartedSpeakingFrame (system frame, immediate)
+        # 2. All queued TTSTextFrames
+        # 3. BotStoppedSpeakingFrame (system frame, immediate)
+        # 4. TranscriptionUpdateFrame (after aggregation)
+        expected_down_frames = [
+            BotStartedSpeakingFrame,
+            TTSTextFrame,
+            TTSTextFrame,
+            TTSTextFrame,
+            TTSTextFrame,
+            TTSTextFrame,
+            BotStoppedSpeakingFrame,
+            TranscriptionUpdateFrame,
+        ]
+
+        # Run test
+        received_frames, _ = await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        # Verify update was received
+        self.assertEqual(len(received_updates), 1)
+
+        # Get the update frame
+        update_frame = received_updates[0]
+
+        # Should have one aggregated message
+        self.assertEqual(len(update_frame.messages), 1)
+
+        message = update_frame.messages[0]
+        self.assertEqual(message.role, "assistant")
+        self.assertEqual(message.content, "Hello world! How are you?")
+
+        # Verify timestamp exists
+        self.assertIsNotNone(message.timestamp)
+
+        # All frames should be passed through in order, with update at end
+        downstream_update = cast(TranscriptionUpdateFrame, received_frames[-1])
+        self.assertEqual(downstream_update.messages[0].content, "Hello world! How are you?")
+
+    async def test_empty_text_handling(self):
+        """Test that empty messages are not emitted"""
+        processor = AssistantTranscriptProcessor()
+
+        received_updates: List[TranscriptionUpdateFrame] = []
+
+        @processor.event_handler("on_transcript_update")
+        async def handle_update(proc, frame: TranscriptionUpdateFrame):
+            received_updates.append(frame)
+
+        frames_to_send = [
+            BotStartedSpeakingFrame(),
+            SleepFrame(sleep=0.1),
+            TTSTextFrame(text=""),  # Empty text
+            TTSTextFrame(text="   "),  # Just whitespace
+            TTSTextFrame(text="\n"),  # Just newline
+            BotStoppedSpeakingFrame(),
+            # Pipeline ends here; run_test will automatically send EndFrame
+        ]
+
+        # From our earlier tests, we know BotStoppedSpeakingFrame comes before TTSTextFrames
+        expected_down_frames = [
+            BotStartedSpeakingFrame,
+            BotStoppedSpeakingFrame,
+            TTSTextFrame,  # empty
+            TTSTextFrame,  # whitespace
+            TTSTextFrame,  # newline
+            # No TranscriptionUpdateFrame since content is empty after stripping
+        ]
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        self.assertEqual(len(received_updates), 0, "No updates should be emitted for empty content")
+
+    async def test_interruption_handling(self):
+        """Test that messages are properly captured when bot is interrupted"""
+        processor = AssistantTranscriptProcessor()
+
+        # Track received updates
+        received_updates: List[TranscriptionUpdateFrame] = []
+
+        @processor.event_handler("on_transcript_update")
+        async def handle_update(proc, frame: TranscriptionUpdateFrame):
+            received_updates.append(frame)
+
+        # Simulate bot being interrupted mid-sentence
+        frames_to_send = [
+            BotStartedSpeakingFrame(),
+            SleepFrame(sleep=0.1),
+            TTSTextFrame(text="Hello"),
+            TTSTextFrame(text="world"),
+            TTSTextFrame(text="!"),
+            SleepFrame(sleep=0.1),
+            StartInterruptionFrame(),  # User interrupts here
+            BotStartedSpeakingFrame(),
+            SleepFrame(sleep=0.1),
+            TTSTextFrame(text="New"),
+            TTSTextFrame(text="response"),
+            SleepFrame(sleep=0.1),
+            BotStoppedSpeakingFrame(),
+        ]
+
+        # Actual order of frames:
+        expected_down_frames = [
+            BotStartedSpeakingFrame,
+            TTSTextFrame,  # "Hello"
+            TTSTextFrame,  # "world"
+            TTSTextFrame,  # "!"
+            TranscriptionUpdateFrame,  # First message (emitted due to interruption)
+            StartInterruptionFrame,  # Interruption frame comes after the update
+            BotStartedSpeakingFrame,
+            TTSTextFrame,  # "New"
+            TTSTextFrame,  # "response"
+            BotStoppedSpeakingFrame,
+            TranscriptionUpdateFrame,  # Second message
+        ]
+
+        # Run test
+        received_frames, _ = await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        # Should have received two updates
+        self.assertEqual(len(received_updates), 2)
+
+        # First update should be interrupted message
+        first_message = received_updates[0].messages[0]
+        self.assertEqual(first_message.role, "assistant")
+        self.assertEqual(first_message.content, "Hello world !")
+        self.assertIsNotNone(first_message.timestamp)
+
+        # Second update should be new response
+        second_message = received_updates[1].messages[0]
+        self.assertEqual(second_message.role, "assistant")
+        self.assertEqual(second_message.content, "New response")
+        self.assertIsNotNone(second_message.timestamp)
+
+        # Verify timestamps are different
+        self.assertNotEqual(first_message.timestamp, second_message.timestamp)
+
+    async def test_end_frame_handling(self):
+        """Test that final messages are captured when pipeline ends normally"""
+        processor = AssistantTranscriptProcessor()
+
+        received_updates: List[TranscriptionUpdateFrame] = []
+
+        @processor.event_handler("on_transcript_update")
+        async def handle_update(proc, frame: TranscriptionUpdateFrame):
+            received_updates.append(frame)
+
+        frames_to_send = [
+            BotStartedSpeakingFrame(),
+            SleepFrame(sleep=0.1),
+            TTSTextFrame(text="Hello"),
+            TTSTextFrame(text="world"),
+            # Pipeline ends here; run_test will automatically send EndFrame
+        ]
+
+        expected_down_frames = [
+            BotStartedSpeakingFrame,
+            TTSTextFrame,
+            TTSTextFrame,
+            TranscriptionUpdateFrame,  # Final message emitted due to EndFrame
+        ]
+
+        # Run test - EndFrame will be sent automatically
+        received_frames, _ = await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        self.assertEqual(len(received_updates), 1)
+        message = received_updates[0].messages[0]
+        self.assertEqual(message.role, "assistant")
+        self.assertEqual(message.content, "Hello world")
+
+    async def test_cancel_frame_handling(self):
+        """Test that messages are properly captured when pipeline is cancelled"""
+        processor = AssistantTranscriptProcessor()
+
+        # Track updates with timestamps to verify order
+        received_updates: List[Tuple[str, float]] = []
+
+        @processor.event_handler("on_transcript_update")
+        async def handle_update(proc, frame: TranscriptionUpdateFrame):
+            # Record message content and time received
+            received_updates.append((frame.messages[0].content, asyncio.get_event_loop().time()))
+
+        frames_to_send = [
+            BotStartedSpeakingFrame(),
+            SleepFrame(sleep=0.1),
+            TTSTextFrame(text="Hello"),
+            TTSTextFrame(text="world"),
+            SleepFrame(sleep=0.1),  # Ensure messages are processed
+            CancelFrame(),
+        ]
+
+        # We don't need to verify frame order, just that CancelFrame triggers message emission
+        expected_down_frames = [
+            BotStartedSpeakingFrame,
+            TTSTextFrame,
+            TTSTextFrame,
+            CancelFrame,
+        ]
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+            send_end_frame=False,
+        )
+
+        # Verify that we received an update
+        self.assertEqual(len(received_updates), 1, "Should receive one update before cancellation")
+        content, _ = received_updates[0]
+        self.assertEqual(content, "Hello world")
+
+    async def test_transcript_processor_factory(self):
+        """Test that factory properly manages processors and event handlers"""
+        from pipecat.processors.transcript_processor import TranscriptProcessor
+
+        factory = TranscriptProcessor()
+        received_updates: List[TranscriptionMessage] = []
+
+        # Register handler with factory
+        @factory.event_handler("on_transcript_update")
+        async def handle_update(proc, frame: TranscriptionUpdateFrame):
+            received_updates.extend(frame.messages)
+
+        # Get processors and verify they're reused
+        user_proc1 = factory.user()
+        user_proc2 = factory.user()
+        self.assertIs(user_proc1, user_proc2, "User processor should be reused")
+
+        asst_proc1 = factory.assistant()
+        asst_proc2 = factory.assistant()
+        self.assertIs(asst_proc1, asst_proc2, "Assistant processor should be reused")
+
+        # Test user processor
+        timestamp = datetime.now(timezone.utc).isoformat()
+        frames_to_send = [
+            TranscriptionFrame(text="User message", user_id="user1", timestamp=timestamp)
+        ]
+
+        await run_test(
+            user_proc1,
+            frames_to_send=frames_to_send,
+            expected_down_frames=[TranscriptionUpdateFrame, TranscriptionFrame],
+        )
+
+        # Test assistant processor
+        frames_to_send = [
+            BotStartedSpeakingFrame(),
+            SleepFrame(sleep=0.1),
+            TTSTextFrame(text="Assistant"),
+            TTSTextFrame(text="message"),
+            BotStoppedSpeakingFrame(),
+        ]
+
+        # The actual order we see in the output:
+        await run_test(
+            asst_proc1,
+            frames_to_send=frames_to_send,
+            expected_down_frames=[
+                BotStartedSpeakingFrame,
+                BotStoppedSpeakingFrame,
+                TTSTextFrame,
+                TTSTextFrame,
+                TranscriptionUpdateFrame,
+            ],
+        )
+
+        # Verify both processors triggered the same handler
+        self.assertEqual(len(received_updates), 2)
+        self.assertEqual(received_updates[0].role, "user")
+        self.assertEqual(received_updates[0].content, "User message")
+        self.assertEqual(received_updates[1].role, "assistant")
+        self.assertEqual(received_updates[1].content, "Assistant message")

tests/test_user_idle_processor.py

@@ -0,0 +1,216 @@
+#
+# Copyright (c) 2024-2025 Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+import asyncio
+import unittest
+
+from pipecat.frames.frames import (
+    BotSpeakingFrame,
+    BotStartedSpeakingFrame,
+    BotStoppedSpeakingFrame,
+    UserStartedSpeakingFrame,
+    UserStoppedSpeakingFrame,
+)
+from pipecat.processors.user_idle_processor import UserIdleProcessor
+from pipecat.tests.utils import SleepFrame, run_test
+
+
+class TestUserIdleProcessor(unittest.IsolatedAsyncioTestCase):
+    async def test_basic_idle_detection(self):
+        """Test that idle callback is triggered after timeout when user stops speaking."""
+        callback_called = asyncio.Event()
+
+        async def idle_callback(processor: UserIdleProcessor) -> None:
+            callback_called.set()
+
+        # Create processor with a short timeout for testing
+        processor = UserIdleProcessor(callback=idle_callback, timeout=0.1)  # 100ms timeout
+
+        frames_to_send = [
+            # Start conversation
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            # Wait 200ms - double the idle timeout to ensure it triggers
+            SleepFrame(sleep=0.2),
+        ]
+
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+        ]
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        assert callback_called.is_set(), "Idle callback was not called"
+
+    async def test_active_listening_resets_idle(self):
+        """Test that bot speaking frames reset the idle timer because user is actively listening."""
+        callback_called = asyncio.Event()
+
+        async def idle_callback(processor: UserIdleProcessor) -> None:
+            callback_called.set()
+
+        processor = UserIdleProcessor(callback=idle_callback, timeout=0.2)
+
+        frames_to_send = [
+            # Start conversation
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            # Wait almost long enough for idle timeout
+            SleepFrame(sleep=0.1),
+            # Bot speaking frame should reset idle timer
+            BotSpeakingFrame(),
+            # Wait almost long enough for idle timeout again
+            SleepFrame(sleep=0.1),
+            # Another bot speaking frame resets timer again
+            BotSpeakingFrame(),
+        ]
+
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            BotSpeakingFrame,
+            BotSpeakingFrame,
+        ]
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        assert not callback_called.is_set(), (
+            "Idle callback was called even though bot speaking frames reset the timer"
+        )
+
+    async def test_idle_retry_callback(self):
+        """Test that retry count increases until user activity resets it."""
+        retry_counts = []
+
+        async def retry_callback(processor: UserIdleProcessor, retry_count: int) -> bool:
+            retry_counts.append(retry_count)
+            return True  # Keep monitoring for idle events
+
+        processor = UserIdleProcessor(callback=retry_callback, timeout=0.4)
+
+        frames_to_send = [
+            # Start conversation
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            # Wait for first idle timeout (count=1)
+            SleepFrame(sleep=0.5),
+            # Wait for second idle timeout (count=2)
+            SleepFrame(sleep=0.5),
+            # User activity resets the count
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            # Wait for new idle timeout (count should be 1 again)
+            SleepFrame(sleep=0.5),
+        ]
+
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+        ]
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        assert retry_counts == [1, 2, 1], f"Expected retry counts [1, 2, 1], got {retry_counts}"
+
+    async def test_idle_monitoring_stops_on_false_return(self):
+        """Test that idle monitoring stops when callback returns False."""
+        retry_counts = []
+
+        async def retry_callback(processor: UserIdleProcessor, retry_count: int) -> bool:
+            retry_counts.append(retry_count)
+            return retry_count < 2  # Stop after second retry
+
+        processor = UserIdleProcessor(callback=retry_callback, timeout=0.4)
+
+        frames_to_send = [
+            UserStartedSpeakingFrame(),
+            UserStoppedSpeakingFrame(),
+            SleepFrame(sleep=0.5),  # First retry (count=1, returns True)
+            SleepFrame(sleep=0.5),  # Second retry (count=2, returns False)
+            SleepFrame(sleep=0.5),  # Should not trigger callback
+        ]
+
+        expected_down_frames = [
+            UserStartedSpeakingFrame,
+            UserStoppedSpeakingFrame,
+        ]
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        assert retry_counts == [1, 2], f"Expected retry counts [1, 2], got {retry_counts}"
+
+    async def test_no_idle_before_conversation(self):
+        """Test that idle monitoring doesn't start before first conversation activity."""
+        callback_called = asyncio.Event()
+
+        async def idle_callback(processor: UserIdleProcessor) -> None:
+            callback_called.set()
+
+        processor = UserIdleProcessor(callback=idle_callback, timeout=0.1)
+
+        frames_to_send = [
+            SleepFrame(sleep=0.2),  # Should not trigger callback
+            # No conversation activity yet
+        ]
+
+        expected_down_frames = []
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        assert not callback_called.is_set(), "Idle callback was called before conversation started"
+
+    async def test_idle_starts_with_bot_speech(self):
+        """Test that monitoring starts with bot speaking frames, not just user speech."""
+        callback_called = asyncio.Event()
+
+        async def idle_callback(processor: UserIdleProcessor) -> None:
+            callback_called.set()
+
+        processor = UserIdleProcessor(callback=idle_callback, timeout=0.1)
+
+        frames_to_send = [
+            BotStartedSpeakingFrame(),
+            BotSpeakingFrame(),
+            BotStoppedSpeakingFrame(),
+            SleepFrame(sleep=0.2),
+        ]
+
+        expected_down_frames = [
+            BotStartedSpeakingFrame,
+            BotSpeakingFrame,
+            BotStoppedSpeakingFrame,
+        ]
+
+        await run_test(
+            processor,
+            frames_to_send=frames_to_send,
+            expected_down_frames=expected_down_frames,
+        )
+
+        assert callback_called.is_set(), "Idle callback not called after bot speech"

tests/test_utils_string.py

@@ -14,7 +14,6 @@ class TestUtilsString(unittest.IsolatedAsyncioTestCase):
         assert match_endofsentence("This is a sentence.")
         assert match_endofsentence("This is a sentence! ")
         assert match_endofsentence("This is a sentence?")
-        assert match_endofsentence("This is a sentence:")
         assert match_endofsentence("This is a sentence;")
         assert not match_endofsentence("This is not a sentence")
         assert not match_endofsentence("This is not a sentence,")
@@ -33,7 +32,6 @@ class TestUtilsString(unittest.IsolatedAsyncioTestCase):
             "你好！",
             "吃了吗？",
             "安全第一；",
-            "他说：",
         ]
         for i in chinese_sentences:
             assert match_endofsentence(i)