Merge pull request #3446 from pipecat-ai/mb/add-3392-changelog
Add PR 3392 to changelog, linting cleanup
This commit is contained in:
33
CHANGELOG.md
33
CHANGELOG.md
@@ -25,12 +25,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
evaluated in order until one evaluates to true.
|
evaluated in order until one evaluates to true.
|
||||||
|
|
||||||
Available user turn start strategies:
|
Available user turn start strategies:
|
||||||
|
|
||||||
- VADUserTurnStartStrategy
|
- VADUserTurnStartStrategy
|
||||||
- TranscriptionUserTurnStartStrategy
|
- TranscriptionUserTurnStartStrategy
|
||||||
- MinWordsUserTurnStartStrategy
|
- MinWordsUserTurnStartStrategy
|
||||||
- ExternalUserTurnStartStrategy
|
- ExternalUserTurnStartStrategy
|
||||||
|
|
||||||
Available user turn stop strategies:
|
Available user turn stop strategies:
|
||||||
|
|
||||||
- TranscriptionUserTurnStopStrategy
|
- TranscriptionUserTurnStopStrategy
|
||||||
- TurnAnalyzerUserTurnStopStrategy
|
- TurnAnalyzerUserTurnStopStrategy
|
||||||
- ExternalUserTurnStopStrategy
|
- ExternalUserTurnStopStrategy
|
||||||
@@ -40,7 +42,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- start: [VADUserTurnStartStrategy, TranscriptionUserTurnStartStrategy]
|
- start: [VADUserTurnStartStrategy, TranscriptionUserTurnStartStrategy]
|
||||||
- stop: [TranscriptionUserTurnStopStrategy]
|
- stop: [TranscriptionUserTurnStopStrategy]
|
||||||
|
|
||||||
urn strategies are configured when setting up `LLMContextAggregatorPair`.
|
Turn strategies are configured when setting up `LLMContextAggregatorPair`.
|
||||||
For example:
|
For example:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@@ -49,8 +51,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
user_params=LLMUserAggregatorParams(
|
user_params=LLMUserAggregatorParams(
|
||||||
user_turn_strategies=UserTurnStrategies(
|
user_turn_strategies=UserTurnStrategies(
|
||||||
stop=[
|
stop=[
|
||||||
TurnAnalyzerUserTurnStopStrategy(
|
TurnAnalyzerUserTurnStopStrategy(turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
||||||
turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|
||||||
)
|
)
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
@@ -86,6 +87,7 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|||||||
(PR [#3289](https://github.com/pipecat-ai/pipecat/pull/3289))
|
(PR [#3289](https://github.com/pipecat-ai/pipecat/pull/3289))
|
||||||
|
|
||||||
- `LLMUserAggregator` now exposes the following events:
|
- `LLMUserAggregator` now exposes the following events:
|
||||||
|
|
||||||
- `on_user_turn_started`: triggered when a user turn starts
|
- `on_user_turn_started`: triggered when a user turn starts
|
||||||
- `on_user_turn_stopped`: triggered when a user turn ends
|
- `on_user_turn_stopped`: triggered when a user turn ends
|
||||||
- `on_user_turn_stop_timeout`: triggered when a user turn does not stop
|
- `on_user_turn_stop_timeout`: triggered when a user turn does not stop
|
||||||
@@ -106,10 +108,10 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|||||||
|
|
||||||
Available user mute strategies:
|
Available user mute strategies:
|
||||||
|
|
||||||
* `FirstSpeechUserMuteStrategy`
|
- `FirstSpeechUserMuteStrategy`
|
||||||
* `MuteUntilFirstBotCompleteUserMuteStrategy`
|
- `MuteUntilFirstBotCompleteUserMuteStrategy`
|
||||||
* `AlwaysUserMuteStrategy`
|
- `AlwaysUserMuteStrategy`
|
||||||
* `FunctionCallUserMuteStrategy`
|
- `FunctionCallUserMuteStrategy`
|
||||||
|
|
||||||
User mute strategies replace the legacy `STTMuteFilter` and provide a more
|
User mute strategies replace the legacy `STTMuteFilter` and provide a more
|
||||||
flexible and composable approach to muting user input.
|
flexible and composable approach to muting user input.
|
||||||
@@ -159,6 +161,7 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|||||||
(PR [#3357](https://github.com/pipecat-ai/pipecat/pull/3357))
|
(PR [#3357](https://github.com/pipecat-ai/pipecat/pull/3357))
|
||||||
|
|
||||||
- Added image support to `OpenAIRealtimeLLMService` via `InputImageRawFrame`:
|
- Added image support to `OpenAIRealtimeLLMService` via `InputImageRawFrame`:
|
||||||
|
|
||||||
- New `start_video_paused` parameter to control initial video input state
|
- New `start_video_paused` parameter to control initial video input state
|
||||||
- New `video_frame_detail` parameter to set image processing quality
|
- New `video_frame_detail` parameter to set image processing quality
|
||||||
("auto",
|
("auto",
|
||||||
@@ -188,6 +191,7 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|||||||
(PR [#3374](https://github.com/pipecat-ai/pipecat/pull/3374))
|
(PR [#3374](https://github.com/pipecat-ai/pipecat/pull/3374))
|
||||||
|
|
||||||
- `LLMAssistantAggregator` now exposes the following events:
|
- `LLMAssistantAggregator` now exposes the following events:
|
||||||
|
|
||||||
- `on_assistant_turn_started`: triggered when the assistant turn starts
|
- `on_assistant_turn_started`: triggered when the assistant turn starts
|
||||||
- `on_assistant_turn_stopped`: triggered when the assistant turn ends
|
- `on_assistant_turn_stopped`: triggered when the assistant turn ends
|
||||||
- `on_assistant_thought`: triggered when there's an assistant thought
|
- `on_assistant_thought`: triggered when there's an assistant thought
|
||||||
@@ -209,6 +213,7 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|||||||
async def setup_pipeline_task(task: PipelineTask):
|
async def setup_pipeline_task(task: PipelineTask):
|
||||||
...
|
...
|
||||||
```
|
```
|
||||||
|
|
||||||
(PR [#3397](https://github.com/pipecat-ai/pipecat/pull/3397))
|
(PR [#3397](https://github.com/pipecat-ai/pipecat/pull/3397))
|
||||||
|
|
||||||
- Added a keepalive task for `InworldTTSService` to keep the service connected
|
- Added a keepalive task for `InworldTTSService` to keep the service connected
|
||||||
@@ -238,12 +243,14 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|||||||
|
|
||||||
- Updated `ElevenLabsRealtimeSTTService` to accept the
|
- Updated `ElevenLabsRealtimeSTTService` to accept the
|
||||||
`include_language_detection` parameter to detect language.
|
`include_language_detection` parameter to detect language.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
stt = ElevenLabsRealtimeSTTService(
|
stt = ElevenLabsRealtimeSTTService(
|
||||||
api_key=os.getenv("ELEVENLABS_API_KEY"),
|
api_key=os.getenv("ELEVENLABS_API_KEY"),
|
||||||
include_language_detection=True
|
include_language_detection=True
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
(PR [#3216](https://github.com/pipecat-ai/pipecat/pull/3216))
|
(PR [#3216](https://github.com/pipecat-ai/pipecat/pull/3216))
|
||||||
|
|
||||||
- Updated `SpeechmaticsSTTService` to use new Python Voice SDK with improved
|
- Updated `SpeechmaticsSTTService` to use new Python Voice SDK with improved
|
||||||
@@ -251,6 +258,7 @@ turn_analyzer=LocalSmartTurnAnalyzerV3(params=SmartTurnParams())
|
|||||||
without any impact on accuracy. Use the `turn_detection_mode` parameter to control
|
without any impact on accuracy. Use the `turn_detection_mode` parameter to control
|
||||||
the endpointing of speech, with `TurnDetectionMode.EXTERNAL` (default),
|
the endpointing of speech, with `TurnDetectionMode.EXTERNAL` (default),
|
||||||
`TurnDetectionMode.ADAPTIVE`, or `TurnDetectionMode.SMART_TURN`.
|
`TurnDetectionMode.ADAPTIVE`, or `TurnDetectionMode.SMART_TURN`.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
stt = SpeechmaticsSTTService(
|
stt = SpeechmaticsSTTService(
|
||||||
api_key=os.getenv("SPEECHMATICS_API_KEY"),
|
api_key=os.getenv("SPEECHMATICS_API_KEY"),
|
||||||
@@ -261,6 +269,7 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
(PR [#3225](https://github.com/pipecat-ai/pipecat/pull/3225))
|
(PR [#3225](https://github.com/pipecat-ai/pipecat/pull/3225))
|
||||||
|
|
||||||
- `daily-python` updated to 0.23.0.
|
- `daily-python` updated to 0.23.0.
|
||||||
@@ -278,6 +287,11 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
|
|||||||
- Improved docstrings for `InworldTTSService` and `InworldHttpTTSService`.
|
- Improved docstrings for `InworldTTSService` and `InworldHttpTTSService`.
|
||||||
(PR [#3288](https://github.com/pipecat-ai/pipecat/pull/3288))
|
(PR [#3288](https://github.com/pipecat-ai/pipecat/pull/3288))
|
||||||
|
|
||||||
|
- Improved the error handling and reconnection logic for `WebsocketServer` by
|
||||||
|
distinguishing between errors when disconnecting and websocket communication
|
||||||
|
errors.
|
||||||
|
(PR [#3392](https://github.com/pipecat-ai/pipecat/pull/3392))
|
||||||
|
|
||||||
- Updated `DeepgramSTTService` to push user started/stopped speaking and
|
- Updated `DeepgramSTTService` to push user started/stopped speaking and
|
||||||
interruption frames when `vad_enabled` is set to true. This centralizes the
|
interruption frames when `vad_enabled` is set to true. This centralizes the
|
||||||
frames into the service, removing the need to have your application code
|
frames into the service, removing the need to have your application code
|
||||||
@@ -308,6 +322,7 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
|
|||||||
- Smart Turn now takes into account `vad_start_seconds` when buffering audio,
|
- Smart Turn now takes into account `vad_start_seconds` when buffering audio,
|
||||||
meaning that the start of the turn audio is not cut off. This improves
|
meaning that the start of the turn audio is not cut off. This improves
|
||||||
accuracy for short utterances.
|
accuracy for short utterances.
|
||||||
|
|
||||||
- The default value of `pre_speech_ms` is now set to 500ms for Smart Turn.
|
- The default value of `pre_speech_ms` is now set to 500ms for Smart Turn.
|
||||||
(PR [#3377](https://github.com/pipecat-ai/pipecat/pull/3377))
|
(PR [#3377](https://github.com/pipecat-ai/pipecat/pull/3377))
|
||||||
|
|
||||||
@@ -387,6 +402,7 @@ turn_detection_mode=SpeechmaticsSTTService.TurnDetectionMode.ADAPTIVE,
|
|||||||
context = LLMContext(messages, tools)
|
context = LLMContext(messages, tools)
|
||||||
context_aggregator = LLMContextAggregatorPair(context)
|
context_aggregator = LLMContextAggregatorPair(context)
|
||||||
```
|
```
|
||||||
|
|
||||||
(PR [#3263](https://github.com/pipecat-ai/pipecat/pull/3263))
|
(PR [#3263](https://github.com/pipecat-ai/pipecat/pull/3263))
|
||||||
|
|
||||||
- `STTMuteFilter` is deprecated and will be removed in a future version. Use
|
- `STTMuteFilter` is deprecated and will be removed in a future version. Use
|
||||||
@@ -411,6 +427,7 @@ start=[TranscriptionUserTurnStartStrategy(enable_interruptions=False)],
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
(PR [#3297](https://github.com/pipecat-ai/pipecat/pull/3297))
|
(PR [#3297](https://github.com/pipecat-ai/pipecat/pull/3297))
|
||||||
|
|
||||||
- `TranscriptProcessor` and related data classes and frames
|
- `TranscriptProcessor` and related data classes and frames
|
||||||
@@ -433,6 +450,7 @@ start=[TranscriptionUserTurnStartStrategy(enable_interruptions=False)],
|
|||||||
### Fixed
|
### Fixed
|
||||||
|
|
||||||
- Improved error handling in `ElevenLabsRealtimeSTTService`
|
- Improved error handling in `ElevenLabsRealtimeSTTService`
|
||||||
|
|
||||||
- Fixed an issue in `ElevenLabsRealtimeSTTService` causing an infinite loop
|
- Fixed an issue in `ElevenLabsRealtimeSTTService` causing an infinite loop
|
||||||
that blocks the process if the websocket disconnects due to an error
|
that blocks the process if the websocket disconnects due to an error
|
||||||
(PR [#3233](https://github.com/pipecat-ai/pipecat/pull/3233))
|
(PR [#3233](https://github.com/pipecat-ai/pipecat/pull/3233))
|
||||||
@@ -446,6 +464,7 @@ start=[TranscriptionUserTurnStartStrategy(enable_interruptions=False)],
|
|||||||
(PR [#3322](https://github.com/pipecat-ai/pipecat/pull/3322))
|
(PR [#3322](https://github.com/pipecat-ai/pipecat/pull/3322))
|
||||||
|
|
||||||
- Updated `SpeechmaticsSTTService` for version `0.0.99+`:
|
- Updated `SpeechmaticsSTTService` for version `0.0.99+`:
|
||||||
|
|
||||||
- Fixed `SpeechmaticsSTTService` to listen for `VADUserStoppedSpeakingFrame`
|
- Fixed `SpeechmaticsSTTService` to listen for `VADUserStoppedSpeakingFrame`
|
||||||
in order to finalize transcription.
|
in order to finalize transcription.
|
||||||
- Default to `TurnDetectionMode.FIXED` for Pipecat-controlled end of turn
|
- Default to `TurnDetectionMode.FIXED` for Pipecat-controlled end of turn
|
||||||
|
|||||||
Reference in New Issue
Block a user