Inworld Realtime's session properties accept turn_detection=None to put
the service into manual mode (matching OpenAI Realtime's
turn_detection=False), but the Pipecat integration hardcoded
_handle_user_stopped_speaking and _handle_interruption to assume
server-side VAD: both were no-ops on the client side because Inworld's
server normally handles commit/cancel/response.create automatically. In
manual mode the server doesn't, so local-VAD-driven turns stalled —
the bot never responded after the user stopped speaking, and
interruptions left the in-flight response running.
Mirror the OpenAI Realtime pattern: on user-stopped-speaking in manual
mode, send InputAudioBufferCommitEvent + ResponseCreateEvent; on
interruption in manual mode, send InputAudioBufferClearEvent +
ResponseCancelEvent. Gate both on a new _is_manual_turn_detection()
helper.
Add examples/realtime/realtime-inworld-local-vad.py, the matching
*-local-vad.py variant for parity with the OpenAI Realtime and Grok
Realtime variants, and point the Inworld service docstring at it.
Grok Realtime supports manual mode (turn_detection=None) which disables
its server-side VAD and lets local VAD drive turn boundaries — same
pattern as OpenAI Realtime's turn_detection=False. Add the matching
*-local-vad.py variant for parity, and point the Grok service docstring
at it.
BaseOutputTransport only clears buffered audio mid-playback on
InterruptionFrame. Realtime services stream audio downstream as fast as
they produce it, and playback necessarily trails the buffer — so when the
user interrupts, the bot keeps talking past the interruption unless the
service surfaces the interruption to the pipeline.
Two realtime services were missing this signal:
- AWS Nova Sonic acknowledged the INTERRUPTED stop reason internally
(closing its own response state) but never broadcast InterruptionFrame.
- Ultravox's playback_clear_buffer message — the server's explicit
"drop buffered output audio" signal for interruptions — was not
handled at all.
In both cases the latent bug was masked by enabling local VAD on the
user aggregator, which produced UserStartedSpeakingFrame and triggered
the aggregator-side interruption path. The realtime context aggregator
work makes local VAD optional, so the underlying gap needs fixing first.
Wire broadcast_interruption() into both services on the server-side
interruption signal, firing before the response-end signal so the
assistant aggregator marks the message interrupted=True before
LLMFullResponseEndFrame closes the turn.
The keepalive could fire for a new turn's context before that context's
voice_settings context-init was sent, making the keepalive the context's
first message (no voice_settings) and causing ElevenLabs to reject the
later init with a 1008 policy violation. The keepalive now only targets a
context once its context-init has been sent (tracked in _context_init_sent).
- Replace custom LANGUAGE_MAP fallback in language_to_inworld_language with
resolve_language(language, LANGUAGE_MAP, use_base_code=False) to match the
pattern used by other services and restore the unverified-language warning
- Tighten delivery_mode type from str to Literal["STABLE", "BALANCED", "CREATIVE"]
- Update changelog entry to mention delivery_mode and language normalization
Same async-tool routing approach as #4441: detect async-tool messages in
the LLM context, deliver the final result via the formal tool-result
channel.
Caveat: as of this writing, Inworld Realtime doesn't appear to handle
the resulting delayed tool result reliably, so the routing is
best-effort and the service emits a one-time warning when async-tool
messages are seen. Streamed intermediate results remain unsupported.
Also adds function calling to the realtime-inworld.py example, and
softens the Inworld mention in the #4447 changelog now that the
exclusion is being closed.
