Update match_endofsentence to use NLTK sentence tokenizer

2025-07-21 23:04:52 -04:00
parent c33dfe8309
commit 26c937af87
4 changed files with 228 additions and 46 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -20,6 +20,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Updated `MiniMaxHttpTTSService` with a `base_url` arg where you can specify
  the Global endpoint (default) or Mainland China.

+- Replaced regex-based sentence detection in `match_endofsentence` with NLTK's
+  punkt_tab tokenizer for more reliable sentence boundary detection.
+
 - Changed the `livekit` optional dependency for `tenacity` to
  `tenacity>=8.2.3,<10.0.0` in order to support the `google-genai` package.