wx44wx/pipecat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #3119 from pipecat-ai/aleix/changelog-formatting

Browse Source 
format CHANGELOG
This commit is contained in:
Mark Backman
2025-11-24 08:05:11 -05:00
committed by GitHub
parent 3458b74fc9 d0ff43134a
commit 861567bc59
1 changed files with 194 additions and 124 deletions
Download Patch File Download Diff File Expand all files Collapse all files

318
CHANGELOG.md
View File

@@ -7,156 +7,235 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Fixed
- Fixed an issue in `ElevenLabsRealtimeSTTService` where dynamic language updates were not working.
### Added ### Added
- Added optional speaking rate control to `InworldTTSService`. - Added optional speaking rate control to `InworldTTSService`.
- Introduced a new `AggregatedTextFrame` type to support passing text along with an - Introduced a new `AggregatedTextFrame` type to support passing text along with
`aggregated_by` field to describe the type of text included. `TTSTextFrame`s now an `aggregated_by` field to describe the type of text
inherit from `AggregatedTextFrame`. With this inheritance, an observer can watch for included. `TTSTextFrame`s now inherit from `AggregatedTextFrame`. With this
`AggregatedTextFrame`s to accumlate the perceived output and determine whether or not inheritance, an observer can watch for `AggregatedTextFrame`s to accumlate the
the text was spoken based on if that frame is also a `TTSTextFrame`. perceived output and determine whether or not the text was spoken based on if
that frame is also a `TTSTextFrame`.
With this frame, the llm token stream can be transformed into custom composable With this frame, the llm token stream can be transformed into custom
chunks, allowing for aggregation outside the TTS service. This makes it possible to composable chunks, allowing for aggregation outside the TTS service. This
listen for or handle those aggregations and sets the stage for doing things like makes it possible to listen for or handle those aggregations and sets the
composing a best effort of the perceived llm output in a more digestable form and stage for doing things like composing a best effort of the perceived llm
to do so whether or not it is processed by a TTS or if even a TTS exists. output in a more digestable form and to do so whether or not it is processed
by a TTS or if even a TTS exists.
- Introduced `LLMTextProcessor`: A new processor meant to allow customization for how - Introduced `LLMTextProcessor`: A new processor meant to allow customization
LLMTextFrames should be aggregated and considered. It's purpose is to turn for how LLMTextFrames should be aggregated and considered. It's purpose is to
`LLMTextFrame`s into `AggregatedTextFrame`s. By default, a TTSService will still turn `LLMTextFrame`s into `AggregatedTextFrame`s. By default, a TTSService
aggregate `LLMTextFrame`s by sentence for the service to consume. However, if you will still aggregate `LLMTextFrame`s by sentence for the service to
wish to override how the llm text is aggregated, you should no longer override the consume. However, if you wish to override how the llm text is aggregated, you
TTS's internal text_aggregator, but instead, insert this processor between your LLM should no longer override the TTS's internal text_aggregator, but instead,
and TTS in the pipeline. insert this processor between your LLM and TTS in the pipeline.
- New `bot-output` RTVI message to represent what the bot actually "says". - New `bot-output` RTVI message to represent what the bot actually "says".
- The `RTVIObserver` now emits `bot-output` messages based off the new `AggregatedTextFrame`s
(`bot-tts-text` and `bot-llm-text` are still supported and generated, but `bot-transcript` is - The `RTVIObserver` now emits `bot-output` messages based off the new
now deprecated in lieu of this new, more thorough, message). `AggregatedTextFrame`s (`bot-tts-text` and `bot-llm-text` are still
supported and generated, but `bot-transcript` is now deprecated in lieu of
this new, more thorough, message).
- The new `RTVIBotOutputMessage` includes the fields: - The new `RTVIBotOutputMessage` includes the fields:
- `spoken`: A boolean indicating whether the text was spoken by TTS - `spoken`: A boolean indicating whether the text was spoken by TTS
- `aggregated_by`: A string representing how the text was aggregated ("sentence", "word",
"my custom aggregation") - `aggregated_by`: A string representing how the text was aggregated
- Introduced new fields to `RTVIObserver` to support the new `bot-output` messaging: ("sentence", "word", "my custom aggregation")
- `bot_output_enabled`: Defaults to True. Set to false to disable bot-output messages.
- `skip_aggregator_types`: Defaults to `None`. Set to a list of strings that match - Introduced new fields to `RTVIObserver` to support the new `bot-output`
aggregation types that should not be included in bot-output messages. (Ex. `credit_card`) messaging:
- Introduced new methods, `add_text_transformer()` and `remove_text_transformer()`, to
`RTVIObserver` to support providing (and subsequently removing) callbacks for various types of - `bot_output_enabled`: Defaults to True. Set to false to disable bot-output
aggregations (or all aggregations with `*`) that can modify the text before being sent as a messages.
`bot-output` or `tts-text` message. (Think obscuring the credit card or inserting extra detail
the client might want that the context doesn't need.) - `skip_aggregator_types`: Defaults to `None`. Set to a list of strings that
match aggregation types that should not be included in bot-output
messages. (Ex. `credit_card`)
- Introduced new methods, `add_text_transformer()` and
`remove_text_transformer()`, to `RTVIObserver` to support providing (and
subsequently removing) callbacks for various types of aggregations (or all
aggregations with `*`) that can modify the text before being sent as a
`bot-output` or `tts-text` message. (Think obscuring the credit card or
inserting extra detail the client might want that the context doesn't need.)
- In `MiniMaxHttpTTSService`:
- Added support for speech-2.6-hd and speech-2.6-turbo models
- Added languages: Afrikaans, Bulgarian, Catalan, Danish, Persian, Filipino,
Hebrew, Croatian, Hungarian, Malay, Norwegian, Nynorsk, Slovak, Slovenian,
Swedish, and Tamil
- Added new emotions: calm and fluent
### Changed ### Changed
- Updated `daily-python` to 0.22.0. - Updated `daily-python` to 0.22.0.
- `BaseTextAggregator` changes: - `BaseTextAggregator` changes:
Modified the BaseTextAggregator type so that when text gets aggregated, metadata can
be associated with it. Currently, that just means a `type`, so that the aggregation Modified the BaseTextAggregator type so that when text gets aggregated,
can be classified or described. Changes made to support this: metadata can be associated with it. Currently, that just means a `type`, so
- ⚠️ IMPORTANT: Aggregators are now expected to strip leading/trailing white space that the aggregation can be classified or described. Changes made to support
characters before returning their aggregation from `aggregation()` or `.text`. This this:
way all aggregators have a consistent contract allowing downstream use to know how
to stitch aggregations back together. - ⚠️ IMPORTANT: Aggregators are now expected to strip leading/trailing white
- Introduced a new `Aggregation` dataclass to represent both the aggregated `text` and space characters before returning their aggregation from `aggregation()` or
a string identifying the `type` of aggregation (ex. "sentence", "word", "my custom `.text`. This way all aggregators have a consistent contract allowing
aggregation") downstream use to know how to stitch aggregations back together.
- ⚠️ Breaking change: `BaseTextAggregator.text` now returns an `Aggregation` (instead of `str`).
To update: `aggregated_text = myAggregator.text` -> `aggregated_text = myAggregator.text.text` - Introduced a new `Aggregation` dataclass to represent both the aggregated
- ⚠️ Breaking change: `BaseTextAggregator.aggregate()` now returns `Optional[Aggregation]` `text` and a string identifying the `type` of aggregation (ex. "sentence",
(instead of `Optional[str]`). To update: "word", "my custom aggregation")
```
aggregation = myAggregator.aggregate(text) - ⚠️ Breaking change: `BaseTextAggregator.text` now returns an `Aggregation`
if (aggregation): (instead of `str`).
print(f"successfully aggregated text: {aggregation.text}") // instead of {aggregation}
``` Before:
- `SimpleTextAggregator`, `SkipTagsAggregator`, `PatternPairAggregator` updated to
produce/consume `Aggregation` objects. ```python
aggregated_text = myAggregator.text
```
Now:
```python
aggregated_text = myAggregator.text.text
```
- ⚠️ Breaking change: `BaseTextAggregator.aggregate()` now returns
`Optional[Aggregation]` (instead of `Optional[str]`).
Before:
```python
aggregation = myAggregator.aggregate(text)
print(f"successfully aggregated text: {aggregation}")
```
Now:
```python
aggregation = myAggregator.aggregate(text)
if aggregation:
print(f"successfully aggregated text: {aggregation.text}")
```
- `SimpleTextAggregator`, `SkipTagsAggregator`, `PatternPairAggregator`
updated to produce/consume `Aggregation` objects.
- All uses of the above Aggregators have been updated accordingly. - All uses of the above Aggregators have been updated accordingly.
- Augmented the `PatternPairAggregator` so that matched patterns can be treated as their own - Augmented the `PatternPairAggregator` so that matched patterns can be treated
aggregation, taking advantage of the new. To that end: as their own aggregation, taking advantage of the new. To that end:
- Introduced a new, preferred version of `add_pattern` to support a new option for treating a
match as a separate aggregation returned from `aggregate()`. This replaces the now
deprecated `add_pattern_pair` method and you provide a `MatchAction` in lieu of the `remove_match` field.
- `MatchAction` enum: `REMOVE`, `KEEP`, `AGGREGATE`, allowing customization for how
a match should be handled.
- `REMOVE`: The text along with its delimiters will be removed from the streaming text.
Sentence aggregation will continue on as if this text did not exist.
- `KEEP`: The delimiters will be removed, but the content between them will be kept.
Sentence aggregation will continue on with the internal text included.
- `AGGREGATE`: The delimiters will be removed and the content between will be treated
as a separate aggregation. Any text before the start of the pattern will be
returned early, whether or not a complete sentence was found. Then the pattern
will be returned. Then the aggregation will continue on sentence matching after
the closing delimiter is found. The content between the delimiters is not
aggregated by sentence. It is aggregated as one single block of text.
- `PatternMatch` now extends `Aggregation` and provides richer info to handlers.
- ⚠️ Breaking change: The `PatternMatch` type returned to handlers registered via `on_pattern_match`
has been updated to subclass from the new `Aggregation` type, which means that `content`
has been replaced with `text` and `pattern_id` has been replaced with `type`:
```
async dev on_match_tag(match: PatternMatch):
pattern = match.type # instead of match.pattern_id
text = match.text # instead of match.content
```
- `TextFrame` now includes the field `append_to_context` to support setting whether or not the - Introduced a new, preferred version of `add_pattern` to support a new option
encompassing text should be added to the LLM context (by the LLM assistant aggregator). It for treating a match as a separate aggregation returned from
defaults to `True`. `aggregate()`. This replaces the now deprecated `add_pattern_pair` method
and you provide a `MatchAction` in lieu of the `remove_match` field.
- `MatchAction` enum: `REMOVE`, `KEEP`, `AGGREGATE`, allowing customization
for how a match should be handled.
- `REMOVE`: The text along with its delimiters will be removed from the
streaming text. Sentence aggregation will continue on as if this text
did not exist.
- `KEEP`: The delimiters will be removed, but the content between them
will be kept. Sentence aggregation will continue on with the internal
text included.
- `AGGREGATE`: The delimiters will be removed and the content between will
be treated as a separate aggregation. Any text before the start of the
pattern will be returned early, whether or not a complete sentence was
found. Then the pattern will be returned. Then the aggregation will
continue on sentence matching after the closing delimiter is found. The
content between the delimiters is not aggregated by sentence. It is
aggregated as one single block of text.
- `PatternMatch` now extends `Aggregation` and provides richer info to
handlers.
- ⚠️ Breaking change: The `PatternMatch` type returned to handlers registered
via `on_pattern_match` has been updated to subclass from the new
`Aggregation` type, which means that `content` has been replaced with
`text` and `pattern_id` has been replaced with `type`:
```python
async dev on_match_tag(match: PatternMatch):
pattern = match.type # instead of match.pattern_id
text = match.text # instead of match.content
```
- `TextFrame` now includes the field `append_to_context` to support setting
whether or not the encompassing text should be added to the LLM context (by
the LLM assistant aggregator). It defaults to `True`.
- `TTSService` base class updates: - `TTSService` base class updates:
- `TTSService`s now accept a new `skip_aggregator_types` to avoid speaking certain aggregation
types (now determined/returned by the aggregator) - `TTSService`s now accept a new `skip_aggregator_types` to avoid speaking
- Introduced the ability to do a just-in-time transform of text before it gets sent to the certain aggregation types (now determined/returned by the aggregator)
TTS service via callbacks you can set up via a new init field, `text_transforms` or a new
method `add_text_transformer()`. This makes it possible to do things like introduce - Introduced the ability to do a just-in-time transform of text before it gets
TTS-specific tags for spelling or emotion or change the pronunciation of something on the sent to the TTS service via callbacks you can set up via a new init field,
fly. `remove_text_transformer` has also been added to support removing a registered `text_transforms` or a new method `add_text_transformer()`. This makes it
transform callback. possible to do things like introduce TTS-specific tags for spelling or
- TTS services push `AggregatedTextFrame` in addition to `TTSTextFrame`s when either an emotion or change the pronunciation of something on the
aggregation occurs that should not be spoken or when the TTS service supports word-by-word fly. `remove_text_transformer` has also been added to support removing a
timestamping. In the latter case, the `TTSService` preliminarily generates an registered transform callback.
`AggregatedTextFrame`, aggregated by sentence to generate the full sentence content as early
as possible. - TTS services push `AggregatedTextFrame` in addition to `TTSTextFrame`s when
either an aggregation occurs that should not be spoken or when the TTS
service supports word-by-word timestamping. In the latter case, the
`TTSService` preliminarily generates an `AggregatedTextFrame`, aggregated by
sentence to generate the full sentence content as early as possible.
- Updated `CartesiaTTSService`: - Updated `CartesiaTTSService`:
- Modified use of custom default text_aggregator to avoid deprecation warnings and push users
towards use of transformers or the `LLMTextProcessor` - Modified use of custom default text_aggregator to avoid deprecation warnings
- Added convenience methods for taking advantage of Cartesia's SSML tags: spell, emotion, and push users towards use of transformers or the `LLMTextProcessor`
pauses, volume, and speed.
- Added convenience methods for taking advantage of Cartesia's SSML tags:
spell, emotion, pauses, volume, and speed.
- Updated `RimeTTSService`: - Updated `RimeTTSService`:
- Modified use of custom default text_aggregator to avoid deprecation warnings and push users
towards use of transformers or the `LLMTextProcessor` - Modified use of custom default text_aggregator to avoid deprecation warnings
- Added convenience methods for taking advantage of Rime's customization options: spell, and push users towards use of transformers or the `LLMTextProcessor`
pauses, pronunciations, and inline speed control.
- Added convenience methods for taking advantage of Rime's customization
options: spell, pauses, pronunciations, and inline speed control.
### Deprecated ### Deprecated
- The TTS constructor field, `text_aggregator` is deprecated in favor of the new - The TTS constructor field, `text_aggregator` is deprecated in favor of the new
`LLMTextProcessor`. TTSServices still have an internal aggregator for support of default `LLMTextProcessor`. TTSServices still have an internal aggregator for support
behavior, but if you want to override the aggregation behavior, you should use the new of default behavior, but if you want to override the aggregation behavior, you
processor. should use the new processor.
- The RTVI `bot-transcription` event is deprecated in favor of the new `bot-output` - The RTVI `bot-transcription` event is deprecated in favor of the new
message which is the canonical representation of bot output (spoken or not). The code `bot-output` message which is the canonical representation of bot output
still emits a transcription message for backwards compatibility while transition occurs. (spoken or not). The code still emits a transcription message for backwards
compatibility while transition occurs.
- Deprecated `add_pattern_pair` in the `PatternPairAggregator` which takes a `pattern_id` - Deprecated `add_pattern_pair` in the `PatternPairAggregator` which takes a
and `remove_match` field in favor of the new `add_pattern` method which takes a `type` and an `pattern_id` and `remove_match` field in favor of the new `add_pattern` method
`action` which takes a `type` and an `action`
- `english_normalization` input parameter for `MiniMaxHttpTTSService` is
deprecated, use `test_normalization` instead.
### Fixed ### Fixed
- Fixed an issue in `ElevenLabsRealtimeSTTService` where dynamic language
updates were not working.
- Fixed `InworldTTSService` audio config payload to use camelCase keys expected - Fixed `InworldTTSService` audio config payload to use camelCase keys expected
by the Inworld API. by the Inworld API.
@@ -218,20 +297,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Updated language mappings for the Google and Gemini TTS services to match - Updated language mappings for the Google and Gemini TTS services to match
official documentation. official documentation.
- In `MiniMaxHttpTTSService`:
-- Added support for speech-2.6-hd and speech-2.6-turbo models
-- Added languages: Afrikaans, Bulgarian, Catalan, Danish, Persian, Filipino, Hebrew,
Croatian, Hungarian, Malay, Norwegian, Nynorsk, Slovak, Slovenian, Swedish, and Tamil
-- Added new emotions: calm and fluent
### Deprecated ### Deprecated
- The `api_key` parameter in `GeminiTTSService` is deprecated. Use - The `api_key` parameter in `GeminiTTSService` is deprecated. Use
`credentials` or `credentials_path` instead for Google Cloud authentication. `credentials` or `credentials_path` instead for Google Cloud authentication.
- `english_normalization` input parameter for `MiniMaxHttpTTSService` is deprecated,
use `test_normalization` instead.
### Fixed ### Fixed
- Fixed a `SimliVideoService` connection issue. - Fixed a `SimliVideoService` connection issue.