Mark Backman f1f5a986e8 Add UIWorker ...

UIWorker is an LLMContextWorker that observes and drives a client GUI over the
RTVI UI channel: it stores accessibility snapshots, auto-injects <ui_state> at
the start of each respond job, dispatches client events to @on_ui_event
handlers, sends UI commands back to the client, and surfaces fan-out work as
cancellable task cards via user_job_group(). The optional ReplyToolMixin exposes
a bundled reply tool.

The prompt_guide parameter auto-appends the UI wire-format guide to the LLM's
system instruction (default UI_STATE_PROMPT_GUIDE; override with a string or
disable with None), so the LLM can parse the injected <ui_state> / <ui_event>
messages without the app concatenating the guide by hand.

2026-05-21 23:20:40 -04:00

..

genesys

fix: always include parameters field in Genesys AudioHook messages

2026-03-20 16:37:53 -03:00

integration

Fix broken test_unified_function_calling_anthropic due to use of an unsupported/deprecated model.

2026-03-09 12:07:56 -04:00

__init__.py

tests: add some initial run_test() utilities

2025-01-20 17:41:21 -08:00

test_aggregated_frame_sequencer.py

test: add automated tests for word tracking, frame sequencing, and Cartesia TTS

2026-05-20 10:03:26 -03:00

test_aggregators.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_aic_filter.py

update test description.

2026-03-10 14:54:49 +01:00

test_aic_vad.py

add unit tests for AICVADAnalyzer and AICFilter.

2026-01-26 09:56:36 +01:00

test_app_resources.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_async_tool_messages.py

fix: restore cancel_on_interruption=False support in AWS Nova Sonic and OpenAI Realtime

2026-05-08 09:33:06 -04:00

test_audio_buffer_processor.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_aws_credentials.py

feat(aws): add shared credential resolver with boto3 chain fallback

2026-05-04 15:40:06 -04:00

test_base_worker.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_bridge_processor.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_bus_network.py

Skip pgmq/redis lazy-import tests when their extras are not installed

2026-05-21 10:13:21 -07:00

test_bus.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_cartesia_stt.py

Align websocket STT connection failures

2026-05-18 12:35:01 -04:00

test_cartesia_tts.py

test: add automated tests for word tracking, frame sequencing, and Cartesia TTS

2026-05-20 10:03:26 -03:00

test_context_aggregators_universal.py

Rename UserTurnCompletedFrame to UserTurnInferenceCompletedFrame

2026-05-07 17:47:41 -07:00

test_context_summarization.py

Fix context summarization failing with mid-conversation system messages

2026-04-14 11:48:50 -04:00

test_daily_transport_service.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_deepgram_stt.py

Add tests for send_media() exception handling in DeepgramSTTService

2026-03-26 09:20:58 -04:00

test_direct_functions.py

Modernize Python typing across the codebase

2026-04-16 09:28:23 -07:00

test_dtmf_aggregator.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_elevenlabs_stt.py

Add ElevenLabs STT keyterms support

2026-05-07 21:00:26 +00:00

test_elevenlabs_tts.py

Fix ElevenLabs keepalive racing context-init (1008 disconnects)

2026-05-20 08:59:01 -04:00

test_fastapi_websocket.py

Fix FastAPI WebSocket disconnect race condition causing pipeline hang

2026-03-28 00:01:25 -04:00

test_filters.py

Simplify ServiceSwitcher with closure-based filters

2026-02-09 14:12:33 -05:00

test_frame_adapters.py

Port multi-task unit tests from pipecat-subagents

2026-05-21 10:13:21 -07:00

test_frame_processor.py

Modernize Python typing across the codebase

2026-04-16 09:28:23 -07:00

test_function_calling_adapters.py

Merge pull request #4248 from omChauhanDev/add-openai-custom-tools-support

2026-04-10 10:27:28 -04:00

test_get_llm_invocation_params.py

Add system instruction conflict resolution tests for realtime adapters

2026-03-24 17:30:35 -04:00

test_google_utils.py

update examples and tests copyright and use a proper dash in 2024-2026

2026-01-07 19:32:22 -08:00

test_idle_frame_processor.py

Fix asyncio.Event race conditions in idle processors

2026-04-09 13:41:01 -07:00

test_inworld_tts_language.py

Fix Inworld TTS v2 request fields

2026-05-13 11:17:31 +03:00

test_ivr_navigation.py

Fix IVRNavigator to push AggregatedTextFrame when switching to conversation mode

2026-01-30 21:07:49 -05:00

test_job_group.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_krisp_ip_user_turn_start_strategy.py

test(krisp): scope importlib.metadata.version mock to imports only

2026-04-30 14:16:54 -04:00

test_krisp_sdk_manager.py

test(krisp): scope importlib.metadata.version mock to imports only

2026-04-30 14:16:54 -04:00

test_krisp_viva_filter.py

test(krisp): scope importlib.metadata.version mock to imports only

2026-04-30 14:16:54 -04:00

test_krisp_viva_vad.py

test(krisp): scope importlib.metadata.version mock to imports only

2026-04-30 14:16:54 -04:00

test_langchain.py

Fix langchain imports for langchain 1.x compatibility

2026-03-29 10:27:48 -04:00

test_livekit_transport.py

chore: install livekit as optional extra in CI instead of dev dep

2026-01-20 09:16:16 -05:00

test_llm_context_summarizer.py

Make max_context_tokens and max_unsummarized_messages independently optional

2026-03-03 20:08:22 -05:00

test_llm_context.py

Rename elide_large_values to truncate_large_values

2026-04-11 14:29:05 -04:00

test_llm_response.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_llm_service.py

Add LLMService.append_system_instruction()

2026-05-21 23:03:37 -04:00

test_llm_worker.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_markdown_text_filter.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_messages.py

Port multi-task unit tests from pipecat-subagents

2026-05-21 10:13:21 -07:00

test_novita_llm.py

tests: migrate LLM tests to Settings-based constructor API

2026-04-10 17:37:49 -07:00

test_openai_llm_timeout.py

tests: migrate LLM tests to Settings-based constructor API

2026-04-10 17:37:49 -07:00

test_openai_realtime_audio_frames.py

Handle gpt-realtime-2 multi-output-item audio responses

2026-05-12 10:34:50 -04:00

test_openai_realtime_reasoning.py

Add reasoning support to OpenAIRealtimeLLMService for gpt-realtime-2

2026-05-12 13:55:19 -04:00

test_openai_responses_websocket.py

Introduce WebsocketLLMService and refactor OpenAIResponsesLLMService to use it

2026-03-30 22:26:31 -04:00

test_pattern_pair_aggregator.py

Remove deprecated add_pattern_pair method from PatternPairAggregator

2026-04-02 10:28:01 -04:00

test_pgmq_backends.py

Skip pgmq/redis bus tests when their extras are not installed

2026-05-21 10:13:21 -07:00

test_pgmq_bus.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_pipeline_worker_ui_bridge.py

Add native RTVI⇄bus UI bridge to PipelineWorker

2026-05-21 23:03:37 -04:00

test_pipeline.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_piper_tts.py

Fixing Piper test.

2026-03-09 12:07:45 -03:00

test_producer_consumer.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_protobuf_serializer.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_redis_bus.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_registry.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_resampy_resampler.py

chore(audio): deprecate ResampyResampler in favor of SOXR resamplers

2026-05-06 09:40:13 -04:00

test_rnnoise_cancellation.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_rnnoise_filter.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_rnnoise_resampling.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_rtvi_processor.py

More RTVI version parsing improvements

2026-03-31 16:05:53 -04:00

test_rtvi_ui.py

feat(rtvi): type UI accessibility snapshots

2026-05-06 11:29:19 -04:00

test_run_inference.py

tests: migrate LLM tests to Settings-based constructor API

2026-04-10 17:37:49 -07:00

test_runner_downloads.py

test: drop webrtc-dependent test, remove webrtc extra from CI

2026-05-04 16:42:05 -04:00

test_runner_run.py

Use shared telephony route availability checks

2026-05-20 08:57:48 -04:00

test_runner_utils.py

Apply suggestions from code review

2026-02-03 16:42:13 +08:00

test_runner.py

Rename PipelineRunner.add_worker() to variadic add_workers(*workers)

2026-05-21 19:46:53 -07:00

test_sambanova_llm.py

tests: migrate LLM tests to Settings-based constructor API

2026-04-10 17:37:49 -07:00

test_serializers.py

Separate bus messages from pipeline frames

2026-05-21 19:07:13 -07:00

test_service_init.py

Address code review feedback

2026-03-07 07:42:42 -05:00

test_service_language.py

Modernize Python typing across the codebase

2026-04-16 09:28:23 -07:00

test_service_switcher.py

Fix ServiceSwitcher reacting to pass-through ErrorFrames from other pipeline stages

2026-03-25 22:53:04 -04:00

test_settings.py

Centralize OpenAI audio constants

2026-05-12 17:48:53 -04:00

test_simple_text_aggregator.py

Add text_aggregation_mode parameter to TTSService

2026-02-26 08:55:41 -05:00

test_skip_tags_aggregator.py

Add text_aggregation_mode parameter to TTSService

2026-02-26 08:55:41 -05:00

test_soniox_stt.py

Align websocket STT connection failures

2026-05-18 12:35:01 -04:00

test_startup_timing_observer.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_sync_parallel_pipeline.py

Add frame_order parameter to SyncParallelPipeline

2026-03-19 09:43:51 -04:00

test_tracing_context.py

Add tests for TracingContext and TurnTraceObserver

2026-02-11 23:27:35 -05:00

test_tts_frame_ordering.py

test: add automated tests for word tracking, frame sequencing, and Cartesia TTS

2026-05-20 10:03:26 -03:00

test_turn_trace_observer.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_turn_tracking_observer.py

update examples and tests copyright and use a proper dash in 2024-2026

2026-01-07 19:32:22 -08:00

test_ui_commands.py

Add UIWorker

2026-05-21 23:20:40 -04:00

test_ui_job_lifecycle.py

Add UIWorker

2026-05-21 23:20:40 -04:00

test_ui_tools.py

Add UIWorker

2026-05-21 23:20:40 -04:00

test_ui_worker.py

Add UIWorker

2026-05-21 23:20:40 -04:00

test_user_bot_latency_observer.py

Add chronological_events utility function to display UserBotLatencyObserver report

2026-03-02 19:23:42 -05:00

test_user_idle_controller.py

Make UserIdleController always-on with dynamic timeout updates

2026-02-14 09:54:30 -05:00

test_user_mute_strategy.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_user_turn_completion_mixin.py

Add LLMService.append_system_instruction()

2026-05-21 23:03:37 -04:00

test_user_turn_controller.py

Split user-turn-stop into inference-triggered and finalized events

2026-05-07 17:46:09 -07:00

test_user_turn_processor.py

Improve user turn stop timing by triggering timeout from VAD stop

2026-02-09 14:12:33 -05:00

test_user_turn_start_strategy.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_user_turn_stop_strategy.py

Split user-turn stop timeout into independent speech and STT timers

2026-04-20 11:55:09 -04:00

test_utils_network.py

tests: add unittest.main() call

2026-01-30 10:07:34 -08:00

test_utils_string.py

Fix sentence splitting for CJK and other non-Latin languages in TTS pipeline

2026-02-02 14:27:49 +08:00

test_vad_controller.py

Modernize Python typing across the codebase

2026-04-16 09:28:23 -07:00

test_vad_processor.py

Modernize Python typing across the codebase

2026-04-16 09:28:23 -07:00

test_vonage_video_connector.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_wake_phrase_user_turn_start_strategy.py

tests: yield in wake phrase strategy setup to let tasks start

2026-04-10 17:37:50 -07:00

test_websocket_proxy.py

Rename BaseTask → BaseWorker and reserve "task" for asyncio

2026-05-21 19:07:13 -07:00

test_websocket_service.py

Preserve websocket reconnect failure retries

2026-05-20 14:45:29 -04:00

test_websocket_transport.py

update examples and tests copyright and use a proper dash in 2024-2026

2026-01-07 19:32:22 -08:00

test_word_completion_tracker.py

test: add automated tests for word tracking, frame sequencing, and Cartesia TTS

2026-05-20 10:03:26 -03:00

test_word_timestamp_utils.py

test: add automated tests for word tracking, frame sequencing, and Cartesia TTS

2026-05-20 10:03:26 -03:00

test_xai_tts.py

Add XAITTSService for xAI streaming WebSocket TTS

2026-04-21 15:48:26 -04:00