Split user-turn-stop into inference-triggered and finalized events
Fixes a real bug: with `filter_incomplete_user_turns` enabled, the
smart-turn detector's tentative stop was firing `on_user_turn_stopped`
before the LLM had a chance to veto it. Observers, transcript
appenders and UI indicators received an early — and sometimes
duplicated — signal.
Decomposes the single stop concern into two events:
- `on_user_turn_inference_triggered` fires when a stop strategy has
enough signal to start LLM inference. The aggregator pushes the
context here, kicking off the LLM call.
- `on_user_turn_stopped` fires only when the user turn is semantically
final. Built-in strategies fire both events at the same call site,
preserving today's behavior for the common case.
Adds `LLMTurnCompletionUserTurnStopStrategy`, which gates
finalization on a `UserTurnCompletedFrame` (a fieldless system frame
emitted by any component judging turn completeness — currently the
`UserTurnCompletionLLMServiceMixin` on `✓`).
Adds `deferred(strategy)` / `DeferredUserTurnStopStrategy`, a thin
wrapper that forwards an inner strategy's events except
`on_user_turn_stopped`. Use this to install a stop strategy as an
inference trigger only, leaving finalization to a peer (e.g. the LLM
completion strategy).
Adds `llm_completion_user_turn_stop_strategies()` for the common
case:
UserTurnStrategies(
stop=llm_completion_user_turn_stop_strategies(),
)
Deprecates `LLMUserAggregatorParams.filter_incomplete_user_turns`.
The aggregator emits a `DeprecationWarning`, wraps existing stop
strategies with `deferred(...)`, and appends
`LLMTurnCompletionUserTurnStopStrategy` automatically.
This commit is contained in:
@@ -19,7 +19,7 @@ from pipecat.turns.user_start import VADUserTurnStartStrategy
|
||||
from pipecat.turns.user_start.min_words_user_turn_start_strategy import (
|
||||
MinWordsUserTurnStartStrategy,
|
||||
)
|
||||
from pipecat.turns.user_stop import SpeechTimeoutUserTurnStopStrategy
|
||||
from pipecat.turns.user_stop import SpeechTimeoutUserTurnStopStrategy, deferred
|
||||
from pipecat.turns.user_turn_controller import UserTurnController
|
||||
from pipecat.turns.user_turn_strategies import ExternalUserTurnStrategies, UserTurnStrategies
|
||||
from pipecat.utils.asyncio.task_manager import TaskManager, TaskManagerParams
|
||||
@@ -71,6 +71,66 @@ class TestUserTurnController(unittest.IsolatedAsyncioTestCase):
|
||||
await asyncio.sleep(TRANSCRIPTION_TIMEOUT + 0.1)
|
||||
self.assertTrue(should_stop)
|
||||
|
||||
async def test_inference_triggered_fires_alongside_stopped(self):
|
||||
"""Default strategies fire both inference-triggered and stopped, in order."""
|
||||
controller = UserTurnController(
|
||||
user_turn_strategies=UserTurnStrategies(
|
||||
stop=[SpeechTimeoutUserTurnStopStrategy(user_speech_timeout=TRANSCRIPTION_TIMEOUT)],
|
||||
)
|
||||
)
|
||||
|
||||
await controller.setup(self.task_manager)
|
||||
|
||||
events: list[str] = []
|
||||
|
||||
@controller.event_handler("on_user_turn_inference_triggered")
|
||||
async def on_user_turn_inference_triggered(controller, strategy):
|
||||
events.append("inference_triggered")
|
||||
|
||||
@controller.event_handler("on_user_turn_stopped")
|
||||
async def on_user_turn_stopped(controller, strategy, params):
|
||||
events.append("stopped")
|
||||
|
||||
await controller.process_frame(VADUserStartedSpeakingFrame())
|
||||
await controller.process_frame(
|
||||
TranscriptionFrame(text="Hello!", user_id="", timestamp="now")
|
||||
)
|
||||
await controller.process_frame(VADUserStoppedSpeakingFrame())
|
||||
await asyncio.sleep(TRANSCRIPTION_TIMEOUT + 0.1)
|
||||
|
||||
self.assertEqual(events, ["inference_triggered", "stopped"])
|
||||
|
||||
async def test_deferred_wrapper_skips_stopped(self):
|
||||
"""A deferred() wrapper drops the inner strategy's on_user_turn_stopped event."""
|
||||
wrapped = deferred(
|
||||
SpeechTimeoutUserTurnStopStrategy(user_speech_timeout=TRANSCRIPTION_TIMEOUT)
|
||||
)
|
||||
controller = UserTurnController(user_turn_strategies=UserTurnStrategies(stop=[wrapped]))
|
||||
|
||||
await controller.setup(self.task_manager)
|
||||
|
||||
events: list[str] = []
|
||||
|
||||
@controller.event_handler("on_user_turn_inference_triggered")
|
||||
async def on_user_turn_inference_triggered(controller, strategy):
|
||||
events.append("inference_triggered")
|
||||
|
||||
@controller.event_handler("on_user_turn_stopped")
|
||||
async def on_user_turn_stopped(controller, strategy, params):
|
||||
events.append("stopped")
|
||||
|
||||
await controller.process_frame(VADUserStartedSpeakingFrame())
|
||||
await controller.process_frame(
|
||||
TranscriptionFrame(text="Hello!", user_id="", timestamp="now")
|
||||
)
|
||||
await controller.process_frame(VADUserStoppedSpeakingFrame())
|
||||
await asyncio.sleep(TRANSCRIPTION_TIMEOUT + 0.1)
|
||||
|
||||
# The inner strategy fires inference-triggered (forwarded by the
|
||||
# wrapper). Finalization is suppressed, but the controller's
|
||||
# stop watchdog eventually fires `stopped`.
|
||||
self.assertEqual(events[0], "inference_triggered")
|
||||
|
||||
async def test_user_turn_start_reset(self):
|
||||
controller = UserTurnController(
|
||||
user_turn_strategies=UserTurnStrategies(
|
||||
|
||||
Reference in New Issue
Block a user