Add backwards compatibility for add_pattern_pair

This commit is contained in:
mattie ruth backman
2025-11-13 12:53:56 -05:00
parent 4c698777f3
commit 3f269f9834
5 changed files with 88 additions and 45 deletions

View File

@@ -54,8 +54,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Introduced a new `Aggregation` dataclass to represent both the aggregated `text` and - Introduced a new `Aggregation` dataclass to represent both the aggregated `text` and
a string identifying the `type` of aggregation (ex. "sentence", "word", "my custom a string identifying the `type` of aggregation (ex. "sentence", "word", "my custom
aggregation") aggregation")
# MRKB TODO -- don't break. leave pattern_id as-is and remove 'type', so that the old
remove param can remain
- **BREAKING**: `BaseTextAggregator.text` now returns an `Aggregation` (instead of `str`). - **BREAKING**: `BaseTextAggregator.text` now returns an `Aggregation` (instead of `str`).
To update: `aggregated_text = myAggregator.text` -> `aggregated_text = myAggregator.text.text` To update: `aggregated_text = myAggregator.text` -> `aggregated_text = myAggregator.text.text`
- **BREAKING**: `BaseTextAggregator.aggregate()` now returns `Optional[Aggregation]` - **BREAKING**: `BaseTextAggregator.aggregate()` now returns `Optional[Aggregation]`
@@ -68,36 +66,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `SimpleTextAggregator`, `SkipTagsAggregator`, `PatternPairAggregator` updated to - `SimpleTextAggregator`, `SkipTagsAggregator`, `PatternPairAggregator` updated to
produce/consume `Aggregation` objects. produce/consume `Aggregation` objects.
- Augmented the `PatterPairAggregator`: - Augmented the `PatternPairAggregator`:
- **BREAKING CHANGES**: Support for the items below resulted in two breaking changes to the - Introduced a new, preferred version of `add_pattern` to support a new option for treating a
`PatternPairAggregator` methods match as a separate aggregation returned from `aggregate()`. This replaces the now
1. The `add_pattern_pair` method arguments have changed: deprecated `add_pattern_pair` method and you provide a `MatchAction` in lieu of the `remove_match` field.
- The `pattern_id` argument is now `type` - `MatchAction` enum: `REMOVE`, `KEEP`, `AGGREGATE`, allowing customization for how
- The `remove_match` argument has been replaced with the `action` argument. To update, a match should be handled.
change `remove_match: True` to `action: MatchAction.REMOVE` or `remove_match: False` to - `REMOVE`: The text along with its delimiters will be removed from the streaming text.
`action: MatchAction.KEEP` Sentence aggregation will continue on as if this text did not exist.
2. The `PatternMatch` type returned to handlers registered via `on_pattern_match` has been - `KEEP`: The delimiters will be removed, but the content between them will be kept.
updated to subclass from the new `Aggregation` type, which means that `content` has been Sentence aggregation will continue on with the internal text included.
replaced with `text` and `pattern_id` has been replaced with `type`: - `AGGREGATE`: The delimiters will be removed and the content between will be treated
``` as a separate aggregation. Any text before the start of the pattern will be
async dev on_match_tag(match: PatternMatch): returned early, whether or not a complete sentence was found. Then the pattern
pattern = match.type # instead of match.pattern_id will be returned. Then the aggregation will continue on sentence matching after
text = match.text # instead of match.content the closing delimiter is found. The content between the delimiters is not
``` aggregated by sentence. It is aggregated as one single block of text.
- `PatternPairAggregator` now supports `type` and `action` per pattern. - `PatternMatch` now extends `Aggregation` and provides richer info to handlers.
- New `MatchAction` enum: `REMOVE`, `KEEP`, `AGGREGATE`, allowing customization for how - **BREAKING**: The `PatternMatch` type returned to handlers registered via `on_pattern_match`
a match should be handled. has been updated to subclass from the new `Aggregation` type, which means that `content`
- `REMOVE`: The text along with its delimiters will be removed from the streaming text. has been replaced with `text` and `pattern_id` has been replaced with `type`:
Sentence aggregation will continue on as if this text did not exist. ```
- `KEEP`: The delimiters will be removed, but the content between them will be kept. async dev on_match_tag(match: PatternMatch):
Sentence aggregation will continue on with the internal text included. pattern = match.type # instead of match.pattern_id
- `AGGREGATE`: The delimiters will be removed and the content between will be treated text = match.text # instead of match.content
as a separate aggregation. Any text before the start of the pattern will be ```
returned early, whether or not a complete sentence was found. Then the pattern
will be returned. Then the aggregation will continue on sentence matching after
the closing delimiter is found. The content between the delimiters is not
aggregated by sentence. It is aggregated as one single block of text.
- `PatternMatch` now extends `Aggregation` and provides richer info to handlers.
### Changed ### Changed
@@ -148,6 +141,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
behavior, but if you want to override the aggregation behavior, you should use the new behavior, but if you want to override the aggregation behavior, you should use the new
processor. processor.
- Deprecated `add_pattern_pair` in the `PatternPairAggregator` which takes a `pattern_id`
and `remove_match` field in favor of the new `add_pattern` method which takes a `type` and an
`action`
### Fixed ### Fixed
- Fixed subtle issue of assistant context messages ending up with double spaces - Fixed subtle issue of assistant context messages ending up with double spaces

View File

@@ -110,7 +110,7 @@ async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
pattern_aggregator = PatternPairAggregator() pattern_aggregator = PatternPairAggregator()
# Add pattern for voice switching # Add pattern for voice switching
pattern_aggregator.add_pattern_pair( pattern_aggregator.add_pattern(
type="voice", type="voice",
start_pattern="<voice>", start_pattern="<voice>",
end_pattern="</voice>", end_pattern="</voice>",

View File

@@ -118,15 +118,15 @@ class IVRProcessor(FrameProcessor):
def _setup_xml_patterns(self): def _setup_xml_patterns(self):
"""Set up XML pattern detection and handlers.""" """Set up XML pattern detection and handlers."""
# Register DTMF pattern # Register DTMF pattern
self._aggregator.add_pattern_pair("dtmf", "<dtmf>", "</dtmf>", action=MatchAction.REMOVE) self._aggregator.add_pattern("dtmf", "<dtmf>", "</dtmf>", action=MatchAction.REMOVE)
self._aggregator.on_pattern_match("dtmf", self._handle_dtmf_action) self._aggregator.on_pattern_match("dtmf", self._handle_dtmf_action)
# Register mode pattern # Register mode pattern
self._aggregator.add_pattern_pair("mode", "<mode>", "</mode>", action=MatchAction.REMOVE) self._aggregator.add_pattern("mode", "<mode>", "</mode>", action=MatchAction.REMOVE)
self._aggregator.on_pattern_match("mode", self._handle_mode_action) self._aggregator.on_pattern_match("mode", self._handle_mode_action)
# Register IVR pattern # Register IVR pattern
self._aggregator.add_pattern_pair("ivr", "<ivr>", "</ivr>", action=MatchAction.REMOVE) self._aggregator.add_pattern("ivr", "<ivr>", "</ivr>", action=MatchAction.REMOVE)
self._aggregator.on_pattern_match("ivr", self._handle_ivr_action) self._aggregator.on_pattern_match("ivr", self._handle_ivr_action)
async def process_frame(self, frame: Frame, direction: FrameDirection): async def process_frame(self, frame: Frame, direction: FrameDirection):

View File

@@ -25,9 +25,16 @@ class MatchAction(Enum):
"""Actions to take when a pattern pair is matched. """Actions to take when a pattern pair is matched.
Parameters: Parameters:
REMOVE: Remove the matched pattern from the text. REMOVE: The text along with its delimiters will be removed from the streaming text.
KEEP: Keep the matched pattern in the text as normal text. Sentence aggregation will continue on as if this text did not exist.
AGGREGATE: Return the matched pattern as a separate aggregation object. KEEP: The delimiters will be removed, but the content between them will be kept.
Sentence aggregation will continue on with the internal text included.
AGGREGATE: The delimiters will be removed and the content between will be treated
as a separate aggregation. Any text before the start of the pattern will be
returned early, whether or not a complete sentence was found. Then the pattern
will be returned. Then the aggregation will continue on sentence matching after
the closing delimiter is found. The content between the delimiters is not
aggregated by sentence. It is aggregated as one single block of text.
""" """
REMOVE = "remove" REMOVE = "remove"
@@ -106,7 +113,7 @@ class PatternPairAggregator(BaseTextAggregator):
return Aggregation(self._text, pattern_start[1].get("type", "sentence")) return Aggregation(self._text, pattern_start[1].get("type", "sentence"))
return Aggregation(self._text, "sentence") return Aggregation(self._text, "sentence")
def add_pattern_pair( def add_pattern(
self, self,
type: str, type: str,
start_pattern: str, start_pattern: str,
@@ -148,6 +155,46 @@ class PatternPairAggregator(BaseTextAggregator):
} }
return self return self
def add_pattern_pair(
self, pattern_id: str, start_pattern: str, end_pattern: str, remove_match: bool = True
):
"""Add a pattern pair to detect in the text.
.. deprecated:: 0.0.95
This function is deprecated and will be removed in a future version.
Use `add_pattern` with a type and MatchAction instead.
This method calls `add_pattern` setting type with the provided pattern_id and action
to either MatchAction.REMOVE or MatchAction.KEEP based on `remove_match`.
Args:
pattern_id: Identifier for this pattern pair. Should be unique and ideally descriptive.
(e.g., 'code', 'speaker', 'custom'). pattern_id can not be 'sentence' as that is
reserved for the default behavior.
start_pattern: Pattern that marks the beginning of content.
end_pattern: Pattern that marks the end of content.
remove_match: If True, the matched pattern will be removed from the text. (Same as MatchAction.REMOVE)
If False, it will be kept and treated as normal text. (Same as MatchAction.KEEP)
"""
import warnings
with warnings.catch_warnings():
warnings.simplefilter("once")
warnings.warn(
"add_pattern_pair with a pattern_id or remove_match is deprecated and will be"
" removed in a future version. Use add_pattern with a type and MatchAction instead",
DeprecationWarning,
stacklevel=2,
)
action = MatchAction.REMOVE if remove_match else MatchAction.KEEP
return self.add_pattern(
type=pattern_id,
start_pattern=start_pattern,
end_pattern=end_pattern,
action=action,
)
def on_pattern_match( def on_pattern_match(
self, type: str, handler: Callable[[PatternMatch], Awaitable[None]] self, type: str, handler: Callable[[PatternMatch], Awaitable[None]]
) -> "PatternPairAggregator": ) -> "PatternPairAggregator":

View File

@@ -22,12 +22,11 @@ class TestPatternPairAggregator(unittest.IsolatedAsyncioTestCase):
# Add a test pattern # Add a test pattern
self.aggregator.add_pattern_pair( self.aggregator.add_pattern_pair(
type="test_pattern", pattern_id="test_pattern",
start_pattern="<test>", start_pattern="<test>",
end_pattern="</test>", end_pattern="</test>",
action=MatchAction.REMOVE,
) )
self.aggregator.add_pattern_pair( self.aggregator.add_pattern(
type="code_pattern", type="code_pattern",
start_pattern="<code>", start_pattern="<code>",
end_pattern="</code>", end_pattern="</code>",
@@ -119,14 +118,14 @@ class TestPatternPairAggregator(unittest.IsolatedAsyncioTestCase):
voice_handler = AsyncMock() voice_handler = AsyncMock()
emphasis_handler = AsyncMock() emphasis_handler = AsyncMock()
self.aggregator.add_pattern_pair( self.aggregator.add_pattern(
type="voice", type="voice",
start_pattern="<voice>", start_pattern="<voice>",
end_pattern="</voice>", end_pattern="</voice>",
action=MatchAction.REMOVE, action=MatchAction.REMOVE,
) )
self.aggregator.add_pattern_pair( self.aggregator.add_pattern(
type="emphasis", type="emphasis",
start_pattern="<em>", start_pattern="<em>",
end_pattern="</em>", end_pattern="</em>",