Update Changelog

CHANGELOG.md

@@ -16,6 +16,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   services that subclass `TTSService` can indicate whether the text in the
   `TTSTextFrame`s they push already contain any necessary inter-frame spaces.
 
+- New bot-output RTVI message to represent what the bot actually "says".
+  - RTVIBotOutputMessage / RTVIBotOutputMessageData — includes:
+    - spoken: bool — whether the text was spoken by TTS
+    - aggregated_by: Optional[str|\"word\"|\"sentence\"] — how the text was aggregated
+  - RTVIObserver now emits bot-output messages (bot-tts-text and bot-llm-text are still
+    supported and generated. bot-transcript is now deprecated in lieu of this new, more
+    thorough, message).
+- Introduced new `AggregatedLLMTextFrame` type to support representing effective llm
+  output whether or not it is processed by the TTS. This new frame type includes the
+  field `aggregated_by` to represent the conceptual formaty by which the given text
+  is aggregated. `TTSTextFrame`s now inherit from `AggregatedLLMTextFrame`.
+
+- Updated the base aggregator type:
+  - Introduced a new `Aggregation` dataclass to represent both the aggregated `text` and
+    a string identifying the `type` of aggregation (ex. "sentence", "word", "my custom
+    aggregation")
+  - `BaseTextAggregator.text` now returns an `Aggregation` (instead of `str`).
+  - `BaseTextAggregator.aggregate()` now returns `Optional[Aggregation]`
+    (instead of `Optional[str]`)
+  - `SimpleTextAggregator`, `SkipTagsAggregator`, `PatternPairAggregator` updated to
+    produce/consume `Aggregation` objects.
+
+- Augmented the `PatterPairAggregator`:
+  - `PatternPairAggregator` now supports `type` and `action` per pattern.
+  - New `MatchAction` enum: `REMOVE`, `KEEP`, `AGGREGATE`, allowing customization for how
+    a match should be handled.
+    - `REMOVE`: The text along with its delimiters will be removed from the streaming text.
+                Sentence aggregation will continue on as if this text did not exist.
+    - `KEEP`: The delimiters will be removed, but the content between them will be kept.
+              Sentence aggregation will continue on with the internal text included.
+    - `AGGREGATE`: The delimiters will be removed and the content between will be treated
+              as a separate aggregation. Any text before the start of the pattern will be
+              returned early, whether or not a complete sentence was found. Then the pattern
+              will be returned. Then the aggregation will continue on sentence matching after
+              the closing delimeter is found. The content between the delimeters is not
+              aggregated by sentence. It is aggregated as one single block of text.
+    - `PatternMatch` now extends `Aggregation` and provides richer info to handlers.
+
 ### Changed
 
 - Updated all STT and TTS services to use consistent error handling pattern with
@@ -33,11 +71,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Updated language mappings for the Google and Gemini TTS services to match
   official documentation.
 
+- TTS flow respects aggregation metadata
+  - `TTSService` accepts a new `skip_aggregator_types` to avoid speaking certain aggregation types (as
+    now determined/returned by the aggregator)
+  - TTS services push `AggregatedLLMTextFrame` in addition to `TTSTextFrame`s when either an aggregation
+    occurs that should not be spoken or when the TTS service supports word-by-word timestamping. In the
+    latter case, the `TTSService` preliminarily generates an `AggregatedLLMTextFrame`, aggregated by sentence
+    to generate the full sentence content as early as possible.
+
 ### Deprecated
 
 - The `api_key` parameter in `GeminiTTSService` is deprecated. Use
   `credentials` or `credentials_path` instead for Google Cloud authentication.
 
+- The RTVI `bot-transcription` event is deprecated in favor of the new `bot-output` message which is the canonical representation of bot output (spoken or not). The code still emits a transcription message for backwards compatibility while transition occurs.
+
 ### Fixed
 
 - Fixed subtle issue of assistant context messages ending up with double spaces

src/pipecat/utils/text/pattern_pair_aggregator.py

@@ -8,7 +8,7 @@
 
 This module provides an aggregator that identifies and processes content between
 pattern pairs (like XML tags or custom delimiters) in streaming text, with
-support for custom handlers and configurable pattern removal.
+support for custom handlers and configurable actions for when a pattern is found.
 """
 
 import re
@@ -72,13 +72,18 @@ class PatternPairAggregator(BaseTextAggregator):
 
     This aggregator buffers text until it can identify complete pattern pairs
     (defined by start and end patterns), processes the content between these
-    patterns using registered handlers, and returns text at sentence boundaries.
-    It's particularly useful for processing structured content in streaming text,
-    such as XML tags, markdown formatting, or custom delimiters.
+    patterns using registered handlers. By default, its aggregation method
+    returns text at sentence boundaries, and remove the content found between
+    any matched patterns. However, matched patterns can also be configured to
+    returned as a separate aggregation object containing the content between
+    their start and end patterns or left in, so that only the delimiters are
+    removed and a callback can be triggered.
+
+    This aggregator is particularly useful for processing structured content in
+    streaming text, such as XML tags, markdown formatting, or custom delimiters.
 
     The aggregator ensures that patterns spanning multiple text chunks are
-    correctly identified and handles cases where patterns contain sentence
-    boundaries.
+    correctly identified.
     """
 
     def __init__(self, **kwargs):