Merge pull request #3446 from pipecat-ai/mb/add-3392-changelog

Add PR 3392 to changelog, linting cleanup

CHANGELOG.md

@@ -24,39 +24,40 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   A list of strategies can be specified for both strategies; strategies are
   evaluated in order until one evaluates to true.
 
-    Available user turn start strategies:
+  Available user turn start strategies:
-      - VADUserTurnStartStrategy
-      - TranscriptionUserTurnStartStrategy
-      - MinWordsUserTurnStartStrategy
-      - ExternalUserTurnStartStrategy
 
-    Available user turn stop strategies:
+  - VADUserTurnStartStrategy
-      - TranscriptionUserTurnStopStrategy
+  - TranscriptionUserTurnStartStrategy
-      - TurnAnalyzerUserTurnStopStrategy
+  - MinWordsUserTurnStartStrategy
-      - ExternalUserTurnStopStrategy
+  - ExternalUserTurnStartStrategy
 
-    The default strategies are:
+  Available user turn stop strategies:
 
-      - start: [VADUserTurnStartStrategy, TranscriptionUserTurnStartStrategy]
+  - TranscriptionUserTurnStopStrategy
-      - stop: [TranscriptionUserTurnStopStrategy]
+  - TurnAnalyzerUserTurnStopStrategy
+  - ExternalUserTurnStopStrategy
 
-  urn strategies are configured when setting up `LLMContextAggregatorPair`.
+  The default strategies are:
+
+  - start: [VADUserTurnStartStrategy, TranscriptionUserTurnStartStrategy]
+  - stop: [TranscriptionUserTurnStopStrategy]
+
+  Turn strategies are configured when setting up `LLMContextAggregatorPair`.
   For example:
 
-    ```python
+  ```python
-    context_aggregator = LLMContextAggregatorPair(
+  context_aggregator = LLMContextAggregatorPair(
-        context,
+      context,
-        user_params=LLMUserAggregatorParams(
+      user_params=LLMUserAggregatorParams(
-            user_turn_strategies=UserTurnStrategies(
+          user_turn_strategies=UserTurnStrategies(
-                stop=[
+              stop=[
-                    TurnAnalyzerUserTurnStopStrategy(
+                  TurnAnalyzerUserTurnStopStrategy(turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
-turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
+                  )
-                    )
+              ],
-                ],
+          )
-            )
+      ),
-        ),
+  )
-    )
+  ```
-    ```
 
   In order to use the user turn strategies you must update to the new
   universal `LLMContext` and `LLMContextAggregatorPair`.
@@ -69,13 +70,13 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
 - Added `GrokRealtimeLLMService` for xAI's Grok Voice Agent API with real-time
   voice conversations:
 
-    - Support for real-time audio streaming with WebSocket connection
+  - Support for real-time audio streaming with WebSocket connection
-    - Built-in server-side VAD (Voice Activity Detection)
+  - Built-in server-side VAD (Voice Activity Detection)
-    - Multiple voice options: Ara, Rex, Sal, Eve, Leo
+  - Multiple voice options: Ara, Rex, Sal, Eve, Leo
-    - Built-in tools support: web_search, x_search, file_search
+  - Built-in tools support: web_search, x_search, file_search
-    - Custom function calling with standard Pipecat tools schema
+  - Custom function calling with standard Pipecat tools schema
-    - Configurable audio formats (PCM at 8kHz-48kHz)
+  - Configurable audio formats (PCM at 8kHz-48kHz)
-  (PR [#3267](https://github.com/pipecat-ai/pipecat/pull/3267))
+    (PR [#3267](https://github.com/pipecat-ai/pipecat/pull/3267))
 
 - Added an approximation of TTFB for Ultravox.
   (PR [#3268](https://github.com/pipecat-ai/pipecat/pull/3268))
@@ -86,11 +87,12 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
   (PR [#3289](https://github.com/pipecat-ai/pipecat/pull/3289))
 
 - `LLMUserAggregator` now exposes the following events:
-     - `on_user_turn_started`: triggered when a user turn starts
+
-     - `on_user_turn_stopped`: triggered when a user turn ends
+  - `on_user_turn_started`: triggered when a user turn starts
-     - `on_user_turn_stop_timeout`: triggered when a user turn does not stop
+  - `on_user_turn_stopped`: triggered when a user turn ends
-        and times out
+  - `on_user_turn_stop_timeout`: triggered when a user turn does not stop
-  (PR [#3291](https://github.com/pipecat-ai/pipecat/pull/3291))
+    and times out
+    (PR [#3291](https://github.com/pipecat-ai/pipecat/pull/3291))
 
 - Introducing user mute strategies. User mute strategies indicate when user
   input should be muted based on the current system state.
@@ -104,12 +106,12 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
   frame is muted if any of the configured strategies indicates it should be
   muted.
 
-    Available user mute strategies:
+  Available user mute strategies:
 
-    * `FirstSpeechUserMuteStrategy`
+  - `FirstSpeechUserMuteStrategy`
-    * `MuteUntilFirstBotCompleteUserMuteStrategy`
+  - `MuteUntilFirstBotCompleteUserMuteStrategy`
-    * `AlwaysUserMuteStrategy`
+  - `AlwaysUserMuteStrategy`
-    * `FunctionCallUserMuteStrategy`
+  - `FunctionCallUserMuteStrategy`
 
   User mute strategies replace the legacy `STTMuteFilter` and provide a more
   flexible and composable approach to muting user input.
@@ -117,16 +119,16 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
   User mute strategies are configured when setting up the
   `LLMContextAggregatorPair`. For example:
 
-    ```python
+  ```python
-    context_aggregator = LLMContextAggregatorPair(
+  context_aggregator = LLMContextAggregatorPair(
-        context,
+      context,
-        user_params=LLMUserAggregatorParams(
+      user_params=LLMUserAggregatorParams(
-            user_mute_strategies=[
+          user_mute_strategies=[
-                FirstSpeechUserMuteStrategy(),
+              FirstSpeechUserMuteStrategy(),
-            ]
+          ]
-        ),
+      ),
-    )
+  )
-    ```
+  ```
 
   In order to use user mute strategies you should update to the new universal
   `LLMContext` and `LLMContextAggregatorPair`.
@@ -159,16 +161,17 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
   (PR [#3357](https://github.com/pipecat-ai/pipecat/pull/3357))
 
 - Added image support to `OpenAIRealtimeLLMService` via `InputImageRawFrame`:
-    - New `start_video_paused` parameter to control initial video input state
+
-    - New `video_frame_detail` parameter to set image processing quality
+  - New `start_video_paused` parameter to control initial video input state
-  ("auto",
+  - New `video_frame_detail` parameter to set image processing quality
-      "low", or "high"). This corresponds to OpenAI Realtime's `image_detail`
+    ("auto",
-      parameter.
+    "low", or "high"). This corresponds to OpenAI Realtime's `image_detail`
-    - `set_video_input_paused()` method to pause/resume video input at runtime
+    parameter.
-    - `set_video_frame_detail()` method to adjust video frame quality
+  - `set_video_input_paused()` method to pause/resume video input at runtime
-       dynamically
+  - `set_video_frame_detail()` method to adjust video frame quality
-    - Automatic rate limiting (1 frame per second) to prevent API overload
+    dynamically
-  (PR [#3360](https://github.com/pipecat-ai/pipecat/pull/3360))
+  - Automatic rate limiting (1 frame per second) to prevent API overload
+    (PR [#3360](https://github.com/pipecat-ai/pipecat/pull/3360))
 
 - Added `UserTurnProcessor`, a frame processor built on `UserTurnController`
   that pushes `UserStartedSpeakingFrame` and `UserStoppedSpeakingFrame` frames
@@ -188,11 +191,12 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
   (PR [#3374](https://github.com/pipecat-ai/pipecat/pull/3374))
 
 - `LLMAssistantAggregator` now exposes the following events:
-     - `on_assistant_turn_started`: triggered when the assistant turn starts
+
-     - `on_assistant_turn_stopped`: triggered when the assistant turn ends
+  - `on_assistant_turn_started`: triggered when the assistant turn starts
-     - `on_assistant_thought`: triggered when there's an assistant thought
+  - `on_assistant_turn_stopped`: triggered when the assistant turn ends
-  available
+  - `on_assistant_thought`: triggered when there's an assistant thought
-  (PR [#3385](https://github.com/pipecat-ai/pipecat/pull/3385))
+    available
+    (PR [#3385](https://github.com/pipecat-ai/pipecat/pull/3385))
 
 - Added `KrispVivaTurn` analyzer for end of turn detection using the Krisp VIVA
   SDK (requires `krisp_audio`).
@@ -202,13 +206,14 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
   register custom pipeline task setup files by setting the
   `PIPECAT_SETUP_FILES` environment variable. This variable should contain a
   colon-separated list of Python files (e.g. `export
-  PIPECAT_SETUP_FILES="setup1.py:setup.py:..."`). Each file must define a
+PIPECAT_SETUP_FILES="setup1.py:setup.py:..."`). Each file must define a
   function with the following signature:
 
-    ```python
+  ```python
-    async def setup_pipeline_task(task: PipelineTask):
+  async def setup_pipeline_task(task: PipelineTask):
-        ...
+      ...
-    ```
+  ```
+
   (PR [#3397](https://github.com/pipecat-ai/pipecat/pull/3397))
 
 - Added a keepalive task for `InworldTTSService` to keep the service connected
@@ -238,12 +243,14 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
 
 - Updated `ElevenLabsRealtimeSTTService` to accept the
   `include_language_detection` parameter to detect language.
-    ```python
+
-      stt = ElevenLabsRealtimeSTTService(
+  ```python
-          api_key=os.getenv("ELEVENLABS_API_KEY"),
+    stt = ElevenLabsRealtimeSTTService(
-          include_language_detection=True
+        api_key=os.getenv("ELEVENLABS_API_KEY"),
-      )
+        include_language_detection=True
-    ```
+    )
+  ```
+
   (PR [#3216](https://github.com/pipecat-ai/pipecat/pull/3216))
 
 - Updated `SpeechmaticsSTTService` to use new Python Voice SDK with improved
@@ -251,16 +258,18 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
   without any impact on accuracy. Use the `turn_detection_mode` parameter to control
   the endpointing of speech, with `TurnDetectionMode.EXTERNAL` (default),
   `TurnDetectionMode.ADAPTIVE`, or `TurnDetectionMode.SMART_TURN`.
-    ```python
+
+  ```python
       stt = SpeechmaticsSTTService(
           api_key=os.getenv("SPEECHMATICS_API_KEY"),
           params=SpeechmaticsSTTService.InputParams(
               language=Language.EN,
-turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
+              turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
               speaker_active_format="<{speaker_id}>{text}</{speaker_id}>",
           ),
       )
-    ```
+  ```
+
   (PR [#3225](https://github.com/pipecat-ai/pipecat/pull/3225))
 
 - `daily-python` updated to 0.23.0.
@@ -273,10 +282,15 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
 
 - Updates to Inworld TTS services:
 
-    - Improved `InworldTTSService`'s websocket implementation to better flush
+  - Improved `InworldTTSService`'s websocket implementation to better flush
-      and close context to better handle long inputs.
+    and close context to better handle long inputs.
-    - Improved docstrings for `InworldTTSService` and `InworldHttpTTSService`.
+  - Improved docstrings for `InworldTTSService` and `InworldHttpTTSService`.
-  (PR [#3288](https://github.com/pipecat-ai/pipecat/pull/3288))
+    (PR [#3288](https://github.com/pipecat-ai/pipecat/pull/3288))
+
+- Improved the error handling and reconnection logic for `WebsocketServer` by
+  distinguishing between errors when disconnecting and websocket communication
+  errors.
+  (PR [#3392](https://github.com/pipecat-ai/pipecat/pull/3392))
 
 - Updated `DeepgramSTTService` to push user started/stopped speaking and
   interruption frames when `vad_enabled` is set to true. This centralizes the
@@ -308,7 +322,8 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
 - Smart Turn now takes into account `vad_start_seconds` when buffering audio,
   meaning that the start of the turn audio is not cut off. This improves
   accuracy for short utterances.
-    - The default value of `pre_speech_ms` is now set to 500ms for Smart Turn.
+
+- The default value of `pre_speech_ms` is now set to 500ms for Smart Turn.
   (PR [#3377](https://github.com/pipecat-ai/pipecat/pull/3377))
 
 - Improved Krisp SDK management to allow `KrispVivaTurn` and `KrispVivaFilter`
@@ -376,17 +391,18 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
   From the developer's point of view, switching to using `LLMContext`
   machinery will usually be a matter of going from this:
 
-    ```python
+  ```python
-    context = OpenAILLMContext(messages, tools)
+  context = OpenAILLMContext(messages, tools)
-    context_aggregator = llm.create_context_aggregator(context)
+  context_aggregator = llm.create_context_aggregator(context)
-    ```
+  ```
 
-    To this:
+  To this:
+
+  ```
+  context = LLMContext(messages, tools)
+  context_aggregator = LLMContextAggregatorPair(context)
+  ```
 
-    ```
-    context = LLMContext(messages, tools)
-    context_aggregator = LLMContextAggregatorPair(context)
-    ```
   (PR [#3263](https://github.com/pipecat-ai/pipecat/pull/3263))
 
 - `STTMuteFilter` is deprecated and will be removed in a future version. Use
@@ -401,16 +417,17 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
   `LLMUserAggregator`'s new parameter `user_turn_strategies` instead. For
   example, to disable interruptions but still get user turns you can do:
 
-    ```python
+  ```python
-    context_aggregator = LLMContextAggregatorPair(
+  context_aggregator = LLMContextAggregatorPair(
-        context,
+      context,
-        user_params=LLMUserAggregatorParams(
+      user_params=LLMUserAggregatorParams(
-            user_turn_strategies=UserTurnStrategies(
+          user_turn_strategies=UserTurnStrategies(
-start=[TranscriptionUserTurnStartStrategy(enable_interruptions=False)],
+              start=[TranscriptionUserTurnStartStrategy(enable_interruptions=False)],
-            ),
+          ),
-        ),
+      ),
-    )
+  )
-    ```
+  ```
+
   (PR [#3297](https://github.com/pipecat-ai/pipecat/pull/3297))
 
 - `TranscriptProcessor` and related data classes and frames
@@ -433,7 +450,8 @@ start=[TranscriptionUserTurnStartStrategy(enable_interruptions=False)],
 ### Fixed
 
 - Improved error handling in `ElevenLabsRealtimeSTTService`
-  - Fixed an issue in `ElevenLabsRealtimeSTTService` causing an infinite loop
+
+- Fixed an issue in `ElevenLabsRealtimeSTTService` causing an infinite loop
   that blocks the process if the websocket disconnects due to an error
   (PR [#3233](https://github.com/pipecat-ai/pipecat/pull/3233))
 
@@ -446,13 +464,14 @@ start=[TranscriptionUserTurnStartStrategy(enable_interruptions=False)],
   (PR [#3322](https://github.com/pipecat-ai/pipecat/pull/3322))
 
 - Updated `SpeechmaticsSTTService` for version `0.0.99+`:
-    - Fixed `SpeechmaticsSTTService` to listen for `VADUserStoppedSpeakingFrame`
+
-      in order to finalize transcription.
+  - Fixed `SpeechmaticsSTTService` to listen for `VADUserStoppedSpeakingFrame`
-    - Default to `TurnDetectionMode.FIXED` for Pipecat-controlled end of turn
+    in order to finalize transcription.
-      detection.
+  - Default to `TurnDetectionMode.FIXED` for Pipecat-controlled end of turn
-    - Only emit VAD + interruption frames if VAD is enabled within the plugin
+    detection.
-      (modes other than `TurnDetectionMode.FIXED` or `TurnDetectionMode.EXTERNAL`).
+  - Only emit VAD + interruption frames if VAD is enabled within the plugin
-  (PR [#3328](https://github.com/pipecat-ai/pipecat/pull/3328))
+    (modes other than `TurnDetectionMode.FIXED` or `TurnDetectionMode.EXTERNAL`).
+    (PR [#3328](https://github.com/pipecat-ai/pipecat/pull/3328))
 
 - Fixed an issue with function calling where a handler failing to invoke its
   result callback could leave the context stuck in IN_PROGRESS, causing LLM