47e2f7a037
Add the configuration surface to drive a realtime service like Gemini Live from local turn detection without paying user-transcript latency. Cascaded pipelines wait for a transcript before ending the user's turn because the downstream LLM needs the user's words recorded in context — but that wait is pure latency in pipelines using local turn detection to drive a realtime service, which consumes user audio directly. Set `wait_for_transcript_to_end_user_turn=False` on `LLMUserAggregatorParams` to turn this on. With that single flag the aggregator: - drops `TranscriptionUserTurnStartStrategy` from the start strategies (so late-arriving realtime transcripts don't trigger new turns), - sets `wait_for_transcript=False` on any stop strategy that supports it (so the turn ends on the audible end of the turn, without waiting for a transcript), - fires `on_user_turn_stopped` on the audible end of the turn with empty `content` (since the transcript hasn't arrived), and - defers the context flush until the transcript arrives or a backstop timer fires. A new `on_user_turn_message_finalized` event fires when the user's message has been written to context. In the default mode it coincides with `on_user_turn_stopped`; in the delayed-transcript mode it fires later. Consumers that want the populated transcript should subscribe to `on_user_turn_message_finalized` — it's the event that always carries the user message, regardless of mode. Strategy mutations are logged: loudly when the user passed their own strategies (we're overwriting parts of their config), quietly otherwise. The strategy-level `wait_for_transcript` parameter on `TurnAnalyzerUserTurnStopStrategy` and `SpeechTimeoutUserTurnStopStrategy` remains exposed for advanced cases. The example `realtime-gemini-live-local-vad.py` demonstrates the full pattern.
1.3 KiB
1.3 KiB
- Added the configuration surface to drive a realtime service like Gemini Live from local turn detection without paying user-transcript latency. Cascaded pipelines wait for a transcript before ending the user's turn because the downstream LLM needs the user's words recorded in context — but that wait is pure latency in pipelines using local turn detection to drive a realtime service, which consumes user audio directly. Set
wait_for_transcript_to_end_user_turn=FalseonLLMUserAggregatorParamsto turn this on: the aggregator dropsTranscriptionUserTurnStartStrategyfrom the start strategies (so late-arriving realtime transcripts don't trigger new turns), setswait_for_transcript=Falseon any stop strategies that support it (so the turn ends without waiting for a transcript), fireson_user_turn_stoppedon the audible end of the turn (with empty content, since the transcript hasn't arrived), and defers the context flush until the transcript arrives. A newon_user_turn_message_finalizedevent fires when the user's message has been written to context — in the default mode it coincides withon_user_turn_stopped; in the delayed-transcript mode it fires later. Consumers that want the populated transcript should subscribe toon_user_turn_message_finalized. Seeexamples/realtime/realtime-gemini-live-local-vad.pyfor the full pattern.