Add pointing UIWorker example

The voice LLM delegates to a ReplyToolMixin UIWorker that scrolls offscreen
items into view and highlights the phones it names — exercising the scroll_to /
highlight UI commands and the [offscreen] state tag.

examples/multi-worker/ui-worker/pointing/README.md

@@ -0,0 +1,81 @@
+# pointing
+
+The UIWorker finds items on the page and points at them. A grid of
+phone listings tall enough that several rows sit below the fold; the
+user asks for one by name and the worker scrolls it into view and
+flashes it.
+
+## What it shows
+
+- The `scroll_to` and `highlight` UI commands round-tripping
+  end-to-end: the `UIWorker` emits them, the native bridge in
+  `PipelineWorker` translates them to RTVI frames, and the client
+  handler resolves the snapshot ref and acts on the live DOM.
+- `ReplyToolMixin`'s visual fields — `reply(answer, scroll_to=...,
+  highlight=[...])`. One tool call per turn; `answer` is required so
+  the model can't forget the spoken reply.
+- The `[offscreen]` state tag the client emits, and the LLM reading it
+  to decide whether a scroll is needed before highlighting.
+
+## What it adds vs. `hello-snapshot`
+
+`hello-snapshot` proved the worker can *read* the page. This one proves
+it can *act* on the page. Same skeleton (voice LLM in the main pipeline
+delegating to a `UIWorker` via a `respond` job); the new parts are the
+`scroll_to` / `highlight` commands and the client handlers for them.
+
+## Run
+
+Two terminals.
+
+**Terminal 1 — bot:**
+
+```bash
+cd examples/multi-worker/ui-worker/pointing
+uv run python bot.py
+```
+
+The bot starts on `http://localhost:7860`.
+
+**Terminal 2 — client:**
+
+```bash
+cd examples/multi-worker/ui-worker/pointing/client
+npm install            # one-time
+npm run dev
+```
+
+Open `http://localhost:5173` and click **Connect**.
+
+## What to try
+
+The page renders 20 phone cards in a responsive grid; the bottom rows
+usually land below the fold. Try:
+
+- _"Where's the iPhone 17?"_ — the worker scrolls the card into view and
+  flashes it.
+- _"Scroll to the Pixel 9 Pro."_ — same flow, different ref.
+- _"Which one is the Nothing phone?"_ — if it's already visible, the
+  worker just highlights without scrolling.
+- _"Which phones are from Google?"_ — a descriptive question; the worker
+  highlights each phone it names.
+- _"What's the cheapest one?"_ — the worker names and highlights it.
+
+Watch the bot logs: each turn shows the main LLM calling
+`answer_about_screen`, then the UIWorker's LLM emitting one `reply`
+(scroll/highlight + the spoken answer).
+
+## Requirements
+
+- `OPENAI_API_KEY`
+- `DEEPGRAM_API_KEY`
+- `CARTESIA_API_KEY`
+
+A `.env` in the example folder is the easiest way to set these (see
+`examples/multi-worker/env.example`).
+
+## What this example _doesn't_ show
+
+Form filling (see `form-fill/`), selection-based deixis (see `deixis/`),
+async task cards (see `async-tasks/`), or custom command handlers beyond
+the standard `scroll_to` / `highlight`.

examples/multi-worker/ui-worker/pointing/bot.py

@@ -0,0 +1,274 @@
+#
+# Copyright (c) 2026, Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+"""Pointing — the UIWorker acts on the page to direct the user's attention.
+
+The UIWorker composes ``ReplyToolMixin``, which exposes one bundled LLM
+tool: ``reply(answer, scroll_to=None, highlight=None, ...)``. One tool
+call per turn; the required ``answer`` argument is enforced by the API
+schema so the model cannot forget the spoken reply.
+
+When the user asks "where's the iPhone 17?", the UIWorker's LLM finds
+the matching ref in the snapshot and emits one ``reply`` call with
+``answer="Here's the iPhone 17."`` plus ``scroll_to`` and ``highlight``
+set to that ref. The mixin dispatches the UI commands and completes the
+job.
+
+Architecture::
+
+    Main worker (PipelineWorker, owns transport + RTVI):
+      transport.in → STT → user_agg → LLM → TTS → transport.out → assistant_agg
+        └── answer_about_screen(query) tool
+              └── params.pipeline_worker.job("ui", name="respond", payload={query})
+
+    PointingWorker (ReplyToolMixin + UIWorker):
+      └── inherited: reply(answer, scroll_to=None, highlight=None, ...)
+
+Run::
+
+    uv run python bot.py
+
+Then open the client at ``http://localhost:5173`` (see ``README.md``).
+
+Requirements:
+
+- OPENAI_API_KEY
+- DEEPGRAM_API_KEY
+- CARTESIA_API_KEY
+"""
+
+import os
+
+from dotenv import load_dotenv
+from loguru import logger
+
+from pipecat.adapters.schemas.tools_schema import ToolsSchema
+from pipecat.audio.vad.silero import SileroVADAnalyzer
+from pipecat.frames.frames import LLMRunFrame
+from pipecat.pipeline.job_context import JobError
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat.pipeline.runner import PipelineRunner
+from pipecat.pipeline.worker import PipelineParams, PipelineWorker
+from pipecat.processors.aggregators.llm_context import LLMContext
+from pipecat.processors.aggregators.llm_response_universal import (
+    LLMContextAggregatorPair,
+    LLMUserAggregatorParams,
+)
+from pipecat.runner.types import RunnerArguments
+from pipecat.runner.utils import create_transport
+from pipecat.services.cartesia.tts import CartesiaTTSService
+from pipecat.services.deepgram.stt import DeepgramSTTService
+from pipecat.services.llm_service import FunctionCallParams
+from pipecat.services.openai.llm import OpenAILLMService
+from pipecat.transports.base_transport import BaseTransport, TransportParams
+from pipecat.transports.daily.transport import DailyParams
+from pipecat.workers.ui import ReplyToolMixin, UIWorker
+
+load_dotenv(override=True)
+
+MAIN_NAME = "main"
+
+transport_params = {
+    "daily": lambda: DailyParams(audio_in_enabled=True, audio_out_enabled=True),
+    "webrtc": lambda: TransportParams(audio_in_enabled=True, audio_out_enabled=True),
+}
+
+
+VOICE_PROMPT = """\
+You are the voice layer of a screen-aware assistant. A separate UI \
+layer sees the page and writes the spoken reply.
+
+For every user utterance that could involve the page, call \
+``answer_about_screen`` with the user's request verbatim. The tool's \
+response is the spoken reply, already TTS-ready.
+
+Only respond directly for pure pleasantries (greetings, thanks, \
+goodbyes). Keep direct replies to one short spoken sentence."""
+
+
+# The UI wire-format guide (UI_STATE_PROMPT_GUIDE) is appended to the LLM's
+# system instruction automatically by UIWorker, so this prompt only needs the
+# app-specific behavior.
+UI_PROMPT = """\
+You help the user find and look at items on a long page of phone \
+listings. The current ``<ui_state>`` block is in your context.
+
+## Tool: reply
+
+Every turn calls ``reply`` exactly once. One tool call per turn, no \
+chaining.
+
+``reply(answer, scroll_to=None, highlight=None)``:
+
+- ``answer`` (REQUIRED): the spoken reply, plain language, one \
+short sentence. No markdown, no symbols, no specs read aloud.
+- ``scroll_to`` (OPTIONAL): a single snapshot ref like ``"e5"``. \
+Set this when at least one phone you want to point at is tagged \
+``[offscreen]`` in ``<ui_state>``. Pick the most relevant ref \
+(typically the first match).
+- ``highlight`` (OPTIONAL): a list of snapshot refs like ``["e5"]`` \
+or ``["e5", "e8", "e47"]``. Each ref pulses on screen \
+simultaneously. Use a single-element list for one phone, multi-element \
+for several.
+
+## Decision rules
+
+**Highlight every phone you name in your answer.** This is the most \
+reliable rule: whatever specific phones appear in the spoken text \
+should also pulse on screen. One phone named → \
+``highlight=["e5"]``. Three named → ``highlight=["e5", "e8", "e47"]``. \
+None named (a generic answer like "I don't see any matches") → \
+omit ``highlight``.
+
+When any highlighted phone is tagged ``[offscreen]`` in \
+``<ui_state>``, also set ``scroll_to`` to the ref of the most \
+relevant one (typically the first in the list, or the one the user \
+asked about most directly).
+
+## Examples
+
+- "Where's the iPhone 17?" (offscreen) → \
+``reply(answer="Here's the iPhone 17.", scroll_to="e5", highlight=["e5"])``
+- "Show me the Pixel 9 Pro." (offscreen) → \
+``reply(answer="Here's the Pixel 9 Pro.", scroll_to="e14", highlight=["e14"])``
+- "Tell me about the iPhone 17 Pro." (offscreen) → \
+``reply(answer="It's Apple's 2025 flagship with a 120Hz ProMotion display and periscope zoom.", scroll_to="e8", highlight=["e8"])``
+- "Which one is the Nothing phone?" (visible) → \
+``reply(answer="This one, the Nothing Phone 3.", highlight=["e29"])``
+- "Show me the Galaxy S25." (visible) → \
+``reply(answer="Here's the Galaxy S25.", highlight=["e17"])``
+- "Show me all the Apple phones." (all visible) → \
+``reply(answer="Here are the three Apple phones.", highlight=["e5", "e8", "e47"])``
+- "Highlight the Apple phones." (mix: e5 and e8 visible, e47 offscreen) → \
+``reply(answer="Highlighting the Apple phones now.", scroll_to="e47", highlight=["e5", "e8", "e47"])``
+- "Which phones are from Google?" → \
+``reply(answer="The Pixel 9, Pixel 9 Pro, and Pixel 9a are from Google.", highlight=["e11", "e14", "e50"])``
+- "What's the cheapest one?" (no specific phones named) → \
+``reply(answer="The iPhone 16e is the most budget-friendly option here.", highlight=["e47"])``"""
+
+
+class PointingWorker(ReplyToolMixin, UIWorker):
+    """UIWorker that points at items using the bundled ``reply`` tool.
+
+    Composes ``ReplyToolMixin``, which exposes a single
+    ``reply(answer, scroll_to=None, highlight=None, ...)`` LLM tool. One
+    tool call per turn; the required ``answer`` argument is enforced by
+    the API schema so the model cannot forget the spoken reply (the
+    failure mode chainable tools have with smaller models).
+
+    ``keep_history=False`` (the ``UIWorker`` default) clears the LLM
+    context at the start of every job, so each turn sees only the
+    current ``<ui_state>`` and the user's query — stale snapshots from
+    prior turns would otherwise contradict the current viewport.
+    """
+
+    def __init__(self):
+        llm = OpenAILLMService(
+            api_key=os.environ["OPENAI_API_KEY"],
+            settings=OpenAILLMService.Settings(system_instruction=UI_PROMPT),
+        )
+        super().__init__("ui", llm=llm)
+
+
+async def answer_about_screen(params: FunctionCallParams, query: str):
+    """Ask the screen-aware UI worker to point at and answer about the page.
+
+    Args:
+        query (str): The user's request, passed verbatim.
+    """
+    logger.info(f"answer_about_screen('{query}')")
+    try:
+        async with params.pipeline_worker.job(
+            "ui", name="respond", payload={"query": query}, timeout=10
+        ) as t:
+            pass
+    except JobError as e:
+        logger.warning(f"ui job failed: {e}")
+        await params.result_callback("Something went wrong on my side.")
+        return
+
+    speak = (t.response or {}).get("speak")
+    await params.result_callback(speak or "I'm not sure how to answer that.")
+
+
+async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
+    logger.info("Starting pointing bot")
+
+    runner = PipelineRunner(handle_sigint=runner_args.handle_sigint)
+
+    stt = DeepgramSTTService(api_key=os.environ["DEEPGRAM_API_KEY"])
+    tts = CartesiaTTSService(
+        api_key=os.environ["CARTESIA_API_KEY"],
+        settings=CartesiaTTSService.Settings(
+            voice=os.getenv("CARTESIA_VOICE_ID", "71a7ad14-091c-4e8e-a314-022ece01c121"),
+        ),
+    )
+    llm = OpenAILLMService(
+        api_key=os.environ["OPENAI_API_KEY"],
+        settings=OpenAILLMService.Settings(system_instruction=VOICE_PROMPT),
+    )
+    llm.register_direct_function(answer_about_screen, cancel_on_interruption=False, timeout_secs=30)
+
+    context = LLMContext(tools=ToolsSchema(standard_tools=[answer_about_screen]))
+    aggregators = LLMContextAggregatorPair(
+        context,
+        user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
+    )
+
+    pipeline = Pipeline(
+        [
+            transport.input(),
+            stt,
+            aggregators.user(),
+            llm,
+            tts,
+            transport.output(),
+            aggregators.assistant(),
+        ]
+    )
+
+    worker = PipelineWorker(
+        pipeline,
+        name=MAIN_NAME,
+        params=PipelineParams(enable_metrics=True, enable_usage_metrics=True),
+        idle_timeout_secs=runner_args.pipeline_idle_timeout_secs,
+    )
+
+    @transport.event_handler("on_client_connected")
+    async def on_client_connected(transport, client):
+        logger.info("Client connected")
+        context.add_message(
+            {
+                "role": "developer",
+                "content": (
+                    "Greet the user briefly. Tell them they can ask to find "
+                    "or scroll to any phone on the list. One short sentence."
+                ),
+            }
+        )
+        await worker.queue_frame(LLMRunFrame())
+
+    @transport.event_handler("on_client_disconnected")
+    async def on_client_disconnected(transport, client):
+        logger.info("Client disconnected")
+        await runner.cancel()
+
+    await runner.launch_worker(PointingWorker())
+    await runner.launch_worker(worker)
+
+    await runner.run()
+
+
+async def bot(runner_args: RunnerArguments):
+    """Main bot entry point compatible with Pipecat Cloud."""
+    transport = await create_transport(runner_args, transport_params)
+    await run_bot(transport, runner_args)
+
+
+if __name__ == "__main__":
+    from pipecat.runner.run import main
+
+    main()

examples/multi-worker/ui-worker/pointing/client/index.html

@@ -0,0 +1,123 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Pointing — UIAgent demo</title>
+    <link rel="stylesheet" href="./styles.css" />
+  </head>
+  <body>
+    <header>
+      <h1>Phone catalog</h1>
+      <button id="connect" type="button">Connect</button>
+    </header>
+
+    <main aria-label="Phones">
+      <div class="phone-card" id="iphone-17">
+        <h2>iPhone 17</h2>
+        <p class="brand">Apple · 2025</p>
+        <p class="desc">Titanium frame, A19 chip, 48MP triple camera.</p>
+      </div>
+      <div class="phone-card" id="iphone-17-pro">
+        <h2>iPhone 17 Pro</h2>
+        <p class="brand">Apple · 2025</p>
+        <p class="desc">ProMotion 120Hz, A19 Pro, periscope zoom.</p>
+      </div>
+      <div class="phone-card" id="pixel-9">
+        <h2>Pixel 9</h2>
+        <p class="brand">Google · 2024</p>
+        <p class="desc">Tensor G4, 6.3-inch OLED, magic eraser.</p>
+      </div>
+      <div class="phone-card" id="pixel-9-pro">
+        <h2>Pixel 9 Pro</h2>
+        <p class="brand">Google · 2024</p>
+        <p class="desc">Triple lens, 5x telephoto, Tensor G4.</p>
+      </div>
+      <div class="phone-card" id="galaxy-s25">
+        <h2>Galaxy S25</h2>
+        <p class="brand">Samsung · 2025</p>
+        <p class="desc">Snapdragon 8 Gen 4, 200MP main camera.</p>
+      </div>
+      <div class="phone-card" id="galaxy-s25-ultra">
+        <h2>Galaxy S25 Ultra</h2>
+        <p class="brand">Samsung · 2025</p>
+        <p class="desc">S Pen, 6.9-inch Dynamic AMOLED, titanium.</p>
+      </div>
+      <div class="phone-card" id="oneplus-13">
+        <h2>OnePlus 13</h2>
+        <p class="brand">OnePlus · 2025</p>
+        <p class="desc">Hasselblad cameras, 100W charging, 6000mAh.</p>
+      </div>
+      <div class="phone-card" id="xiaomi-15">
+        <h2>Xiaomi 15</h2>
+        <p class="brand">Xiaomi · 2024</p>
+        <p class="desc">Leica optics, Snapdragon 8 Elite.</p>
+      </div>
+      <div class="phone-card" id="nothing-3">
+        <h2>Nothing Phone 3</h2>
+        <p class="brand">Nothing · 2025</p>
+        <p class="desc">Glyph interface, transparent back, MediaTek 8300.</p>
+      </div>
+      <div class="phone-card" id="motorola-edge-60">
+        <h2>Motorola Edge 60</h2>
+        <p class="brand">Motorola · 2025</p>
+        <p class="desc">Curved pOLED, 125W charging, slim 7.6mm body.</p>
+      </div>
+      <div class="phone-card" id="oppo-find-x8">
+        <h2>OPPO Find X8</h2>
+        <p class="brand">OPPO · 2024</p>
+        <p class="desc">Hasselblad triple zoom, MediaTek Dimensity 9400.</p>
+      </div>
+      <div class="phone-card" id="vivo-x200">
+        <h2>Vivo X200</h2>
+        <p class="brand">Vivo · 2024</p>
+        <p class="desc">Zeiss optics, 6000mAh battery, AMOLED.</p>
+      </div>
+      <div class="phone-card" id="honor-magic-7">
+        <h2>Honor Magic 7</h2>
+        <p class="brand">Honor · 2025</p>
+        <p class="desc">AI-driven imaging, 6.8-inch quad-curve display.</p>
+      </div>
+      <div class="phone-card" id="zenfone-12">
+        <h2>Zenfone 12</h2>
+        <p class="brand">ASUS · 2025</p>
+        <p class="desc">Compact 5.9-inch flagship, Snapdragon 8 Elite.</p>
+      </div>
+      <div class="phone-card" id="iphone-16e">
+        <h2>iPhone 16e</h2>
+        <p class="brand">Apple · 2025</p>
+        <p class="desc">Budget A18 chip, single-lens 48MP, USB-C.</p>
+      </div>
+      <div class="phone-card" id="pixel-9a">
+        <h2>Pixel 9a</h2>
+        <p class="brand">Google · 2025</p>
+        <p class="desc">Mid-range Tensor G4, 6.1-inch OLED.</p>
+      </div>
+      <div class="phone-card" id="galaxy-a56">
+        <h2>Galaxy A56</h2>
+        <p class="brand">Samsung · 2025</p>
+        <p class="desc">5,000mAh battery, Exynos 1580, mid-tier flagship.</p>
+      </div>
+      <div class="phone-card" id="redmi-note-14-pro">
+        <h2>Redmi Note 14 Pro</h2>
+        <p class="brand">Xiaomi · 2024</p>
+        <p class="desc">200MP camera, Dimensity 7300, AMOLED.</p>
+      </div>
+      <div class="phone-card" id="oneplus-nord-4">
+        <h2>OnePlus Nord 4</h2>
+        <p class="brand">OnePlus · 2024</p>
+        <p class="desc">Metal unibody, Snapdragon 7 Gen 3, 100W charging.</p>
+      </div>
+      <div class="phone-card" id="poco-f7">
+        <h2>POCO F7</h2>
+        <p class="brand">Xiaomi · 2025</p>
+        <p class="desc">Flagship-grade Snapdragon, 120Hz AMOLED, gamer focus.</p>
+      </div>
+    </main>
+
+    <div id="status" aria-live="polite"></div>
+    <audio id="bot-audio" autoplay data-a11y-exclude></audio>
+
+    <script type="module" src="./main.js"></script>
+  </body>
+</html>

examples/multi-worker/ui-worker/pointing/client/main.js

@@ -0,0 +1,170 @@
+/**
+ * Pointing — vanilla JS client.
+ *
+ * Builds on the hello-snapshot wiring (PipecatClient +
+ * managed snapshot streaming + bot audio sink) and adds two command
+ * handlers: ``scroll_to`` and ``highlight``. Both resolve the target
+ * element via ``findElementByRef`` (the snapshot ref system the
+ * walker assigns) and act on the live DOM node.
+ *
+ * The React SDK ships ``useDefaultScrollToHandler`` and
+ * ``useDefaultHighlightHandler`` that do this same work in hooks.
+ * Vanilla apps subscribe to ``RTVIEvent.UICommand`` and filter by
+ * command name.
+ */
+
+import {
+  PipecatClient,
+  RTVIEvent,
+  findElementByRef,
+} from "@pipecat-ai/client-js";
+import { SmallWebRTCTransport } from "@pipecat-ai/small-webrtc-transport";
+
+const BOT_URL = "http://localhost:7860/api/offer";
+
+const connectButton = document.getElementById("connect");
+const status = document.getElementById("status");
+const botAudio = document.getElementById("bot-audio");
+
+let client;
+let unsubscribes = [];
+
+function setStatus(text, autoHideMs = 0) {
+  status.textContent = text;
+  status.dataset.show = text ? "1" : "0";
+  if (text && autoHideMs > 0) {
+    setTimeout(() => {
+      if (status.textContent === text) status.dataset.show = "0";
+    }, autoHideMs);
+  }
+}
+
+/**
+ * Resolve a payload that carries either ``ref`` (snapshot id) or
+ * ``target_id`` (DOM element id). Match what the standard React
+ * handlers do: prefer ref, fall back to target_id.
+ */
+function resolveTarget(payload) {
+  if (payload?.ref) {
+    const el = findElementByRef(payload.ref);
+    if (el) return el;
+  }
+  if (payload?.target_id) {
+    return document.getElementById(payload.target_id);
+  }
+  return null;
+}
+
+function handleScrollTo(payload) {
+  const el = resolveTarget(payload);
+  if (!el) return;
+  const behavior =
+    payload?.behavior === "instant" || payload?.behavior === "smooth"
+      ? payload.behavior
+      : "smooth";
+  el.scrollIntoView({ behavior, block: "center", inline: "nearest" });
+}
+
+function handleHighlight(payload) {
+  const el = resolveTarget(payload);
+  if (!el) return;
+  // The page CSS defines ``.ui-highlight`` as a keyframe pulse —
+  // scale + glow + tint, settling back. The animation duration is
+  // driven by the ``--highlight-duration`` CSS variable so the
+  // server-supplied ``duration_ms`` actually controls it.
+  const duration = payload?.duration_ms ?? 1500;
+  el.style.setProperty("--highlight-duration", `${duration}ms`);
+  // Re-trigger the animation cleanly if a previous highlight is
+  // still running on this element.
+  el.classList.remove("ui-highlight");
+  void el.offsetWidth; // force reflow so removing + re-adding restarts
+  el.classList.add("ui-highlight");
+  setTimeout(() => {
+    el.classList.remove("ui-highlight");
+    el.style.removeProperty("--highlight-duration");
+  }, duration);
+}
+
+function onUICommand(command, handler) {
+  const listener = (data) => {
+    if (data.command !== command) return;
+    handler(data.payload);
+  };
+  client.on(RTVIEvent.UICommand, listener);
+  return () => client.off(RTVIEvent.UICommand, listener);
+}
+
+async function connect() {
+  connectButton.disabled = true;
+  setStatus("Connecting…");
+
+  client = new PipecatClient({
+    transport: new SmallWebRTCTransport(),
+    enableMic: true,
+    enableCam: false,
+  });
+
+  client.on(RTVIEvent.BotConnected, () => setStatus("Bot connected", 1500));
+  client.on(RTVIEvent.Disconnected, () => {
+    setStatus("Disconnected", 2000);
+    connectButton.dataset.state = "";
+    connectButton.textContent = "Connect";
+    connectButton.disabled = false;
+    teardownUI();
+  });
+
+  // Pipe the bot's audio track into the <audio> sink.
+  client.on(RTVIEvent.TrackStarted, (track, participant) => {
+    if (track.kind !== "audio") return;
+    if (participant?.local) return;
+    botAudio.srcObject = new MediaStream([track]);
+  });
+
+  unsubscribes = [
+    onUICommand("scroll_to", handleScrollTo),
+    onUICommand("highlight", handleHighlight),
+  ];
+
+  try {
+    await client.connect({ webrtcUrl: BOT_URL });
+    client.startUISnapshotStream();
+    connectButton.dataset.state = "connected";
+    connectButton.textContent = "Disconnect";
+    connectButton.disabled = false;
+    setStatus("Connected. Try: 'where's the Pixel 9?'", 4000);
+  } catch (err) {
+    console.error("Connect failed:", err);
+    setStatus(`Connect failed: ${err.message ?? err}`, 4000);
+    teardownUI();
+    connectButton.disabled = false;
+  }
+}
+
+async function disconnect() {
+  connectButton.disabled = true;
+  setStatus("Disconnecting…");
+  try {
+    await client?.disconnect();
+  } finally {
+    teardownUI();
+    connectButton.dataset.state = "";
+    connectButton.textContent = "Connect";
+    connectButton.disabled = false;
+  }
+}
+
+function teardownUI() {
+  client?.stopUISnapshotStream();
+  unsubscribes.forEach((unsubscribe) => unsubscribe());
+  unsubscribes = [];
+  if (botAudio.srcObject) botAudio.srcObject = null;
+  client = undefined;
+}
+
+connectButton.addEventListener("click", () => {
+  if (connectButton.dataset.state === "connected") {
+    disconnect();
+  } else {
+    connect();
+  }
+});

examples/multi-worker/ui-worker/pointing/client/package.json

@@ -0,0 +1,18 @@
+{
+  "name": "pointing-client",
+  "private": true,
+  "version": "0.1.0",
+  "type": "module",
+  "scripts": {
+    "dev": "vite",
+    "build": "vite build",
+    "preview": "vite preview"
+  },
+  "dependencies": {
+    "@pipecat-ai/client-js": "1.9.0",
+    "@pipecat-ai/small-webrtc-transport": "^1.10.2"
+  },
+  "devDependencies": {
+    "vite": "^8"
+  }
+}

examples/multi-worker/ui-worker/pointing/client/styles.css

@@ -0,0 +1,151 @@
+:root {
+  color-scheme: light;
+  font-family:
+    system-ui,
+    -apple-system,
+    sans-serif;
+  --border: #d4d4d8;
+  --muted: #71717a;
+  --highlight: #fbbf24;
+}
+
+* {
+  box-sizing: border-box;
+}
+
+body {
+  margin: 0;
+  background: #fafafa;
+  color: #18181b;
+}
+
+header {
+  position: sticky;
+  top: 0;
+  z-index: 10;
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  padding: 1rem 1.5rem;
+  border-bottom: 1px solid var(--border);
+  background: #fff;
+}
+
+header h1 {
+  font-size: 1.25rem;
+  margin: 0;
+}
+
+#connect {
+  padding: 0.5rem 1rem;
+  border: 1px solid var(--border);
+  background: #fff;
+  border-radius: 6px;
+  cursor: pointer;
+  font-size: 0.875rem;
+}
+
+#connect:hover {
+  background: #f4f4f5;
+}
+
+#connect[data-state='connected'] {
+  background: #ef4444;
+  color: white;
+  border-color: #ef4444;
+}
+
+main {
+  display: grid;
+  grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
+  gap: 1rem;
+  padding: 1.5rem;
+  max-width: 1200px;
+  margin: 0 auto;
+}
+
+.phone-card {
+  background: #fff;
+  border: 1px solid var(--border);
+  border-radius: 8px;
+  padding: 1rem;
+  /* Tall enough that, combined with N rows, the bottom of the grid
+     scrolls below typical viewport heights — exercises offscreen
+     detection and scroll_to. */
+  min-height: 130px;
+  scroll-margin-top: 5rem;
+  /* Position relative + z-index so the lifted card during the
+     highlight pulse renders above its neighbors. */
+  position: relative;
+  z-index: 0;
+}
+
+.phone-card h2 {
+  margin: 0 0 0.25rem;
+  font-size: 1rem;
+}
+
+.phone-card .brand {
+  margin: 0 0 0.5rem;
+  font-size: 0.75rem;
+  color: var(--muted);
+  text-transform: uppercase;
+  letter-spacing: 0.04em;
+}
+
+.phone-card .desc {
+  margin: 0;
+  font-size: 0.875rem;
+  line-height: 1.45;
+  color: #3f3f46;
+}
+
+/* Highlight pulse: card scales up briefly, glows, tints, then
+   settles back. The animation duration is keyed off a CSS variable
+   so the handler can drive it from the server-supplied ``duration_ms``.
+   The visual restyle is entirely client-side — ``highlight`` is
+   just a command name, the page decides what "highlight" looks like. */
+@keyframes ui-highlight-pulse {
+  0% {
+    transform: scale(1);
+    box-shadow: 0 0 0 0 rgba(251, 191, 36, 0);
+    background: #fff;
+    border-color: var(--border);
+  }
+  35% {
+    transform: scale(1.04);
+    box-shadow: 0 0 36px 8px rgba(251, 191, 36, 0.55);
+    background: #fffbeb;
+    border-color: var(--highlight);
+  }
+  100% {
+    transform: scale(1);
+    box-shadow: 0 0 0 0 rgba(251, 191, 36, 0);
+    background: #fff;
+    border-color: var(--border);
+  }
+}
+
+.ui-highlight {
+  animation: ui-highlight-pulse var(--highlight-duration, 2500ms)
+    cubic-bezier(0.2, 0.8, 0.2, 1);
+  z-index: 2;
+}
+
+#status {
+  position: fixed;
+  bottom: 1rem;
+  right: 1rem;
+  padding: 0.5rem 0.75rem;
+  border-radius: 6px;
+  font-size: 0.8125rem;
+  background: #18181b;
+  color: white;
+  opacity: 0;
+  transition: opacity 0.2s;
+  pointer-events: none;
+}
+
+#status[data-show='1'] {
+  opacity: 1;
+}

examples/multi-worker/ui-worker/pointing/client/vite.config.js

@@ -0,0 +1,7 @@
+import { defineConfig } from "vite";
+
+export default defineConfig({
+  server: {
+    port: 5173,
+  },
+});