POC: Add MoQ transport

moq_prebuilt/PLAN-moq-transport-package.md

@@ -0,0 +1,115 @@
+# Plan: `@pipecat-ai/moq-transport`
+
+A future Pipecat Client SDK transport plugin for Media-over-QUIC. Lets any
+Pipecat JS UI — including the smallwebrtc playground — talk to a
+MOQ-backed bot with zero UI changes, just by swapping the transport.
+
+## Why this exists
+
+The Pipecat Client SDK (`@pipecat-ai/client-js`) is transport-agnostic. UIs
+talk to a `PipecatClient`, which delegates to a pluggable `Transport`. The
+existing transports are:
+
+- `@pipecat-ai/small-webrtc-transport` (WebRTC)
+- `@pipecat-ai/daily-transport` (Daily)
+- `@pipecat-ai/websocket-transport` (raw WebSocket / telephony)
+- `@pipecat-ai/gemini-live-websocket-transport`
+
+There is **no** `@pipecat-ai/moq-transport`. So MOQ bots today need a hand-
+rolled client (which is what `moq_prebuilt/client/` is). Publishing a
+proper transport plugin would:
+
+1. Replace `moq_prebuilt/client/` with the prebuilt smallwebrtc UI — same
+   conversation panel, metrics, device pickers, transcript overlay, etc.
+2. Let downstream Pipecat apps drop MOQ in as easily as WebRTC.
+3. Centralize the MOQ wire-protocol work in one package instead of every
+   demo reinventing it.
+
+## What we already have
+
+About 80% of the wire-protocol work is done in
+`moq_prebuilt/client/app.js` and `moq_prebuilt/client/moq-client.js`:
+
+- moq-lite-02 codec (CLIENT_SETUP, SERVER_SETUP, ANNOUNCE_PLEASE /
+  ANNOUNCE_INIT, SUBSCRIBE / SUBSCRIBE_OK, GROUP / FRAME).
+- WebTransport bring-up with certificate pinning.
+- Per-participant broadcast paths (`<namespace>/<participant_id>`) and
+  multi-track routing (`bot-audio`, `user-audio`, `transcript`).
+- Mic capture via AudioWorklet → 16 kHz PCM publish.
+- Bot-audio 24 kHz PCM playback via Web Audio.
+- RTVI message parsing on the transcript track (`bot-llm-*`,
+  `user-transcription`, etc.).
+
+The package's job is to **wrap that in the `Transport` interface** and
+emit the events `PipecatClient` expects.
+
+## Effort estimate
+
+### Usable v1 — 3–5 days
+
+For a single bot / single client dev setup that runs against the
+playground UI.
+
+| Day | Work |
+| --- | ---- |
+| 1 | TypeScript package skeleton (build, tsconfig, ESM/CJS exports, npm scripts). Port existing protocol logic into a class. |
+| 2 | Implement `Transport` interface: `connect()`, `disconnect()`, `sendMessage()`, `getDevices()`. Wire the RTVI message parsing on the transcript track into the event names `PipecatClient` expects (`connected`, `disconnected`, `trackStarted`, `userTranscript`, `botStartedSpeaking`, …). |
+| 3 | Device handling — mic picker, mute, sample-rate negotiation, AudioWorklet inlined as a string blob. |
+| 4 | Run against `ConsoleTemplate` from `@pipecat-ai/voice-ui-kit`. Fix lifecycle bugs (mid-call mute, reconnect, etc.). |
+| 5 | Docs, example app, README. |
+
+### Production-grade — ~2 weeks on top
+
+- Multi-participant discovery via outbound `ANNOUNCE_PLEASE` + reacting to
+  `ANNOUNCE_UPDATE` so multiple bots / multiple clients can share a namespace.
+- CA-signed cert path (not just self-signed `serverCertificateHashes`
+  pinning).
+- Reconnection / network-blip recovery.
+- Test suite (unit + at least one e2e against a real relay).
+- Cross-browser sanity (Chrome reference, Safari WebTransport just
+  shipped, Firefox still behind a flag).
+- npm publish workflow / CI.
+
+## Known unknowns (could blow the estimate)
+
+- **PipecatClient `Transport` surface area** — haven't done a full read of
+  `@pipecat-ai/client-js`. If the interface expects something MOQ can't
+  cleanly model (peer-connection introspection, ICE candidates, anything
+  WebRTC-specific that leaks into the abstraction), we need
+  workarounds/polyfills.
+- **Audio interop with the voice-ui-kit's audio visualizers** —
+  `<VoicePresence>`, level meters, etc. read from a `MediaStreamTrack`. MOQ
+  delivers raw PCM uni-streams, not a media-stream-track. We'd need to
+  synthesize a `MediaStreamTrack` from the decoded PCM so the rest of the
+  kit just works. Could be a half-day rabbit hole or could be quick — there
+  are off-the-shelf libraries (`AudioWorklet` + `MediaStreamDestination`) but
+  the integration details aren't trivial.
+
+## Suggested next step (cheap reconnaissance)
+
+Spend ~2 hours reading:
+- `@pipecat-ai/client-js/src/transport.ts` — the `Transport` abstract
+  interface.
+- `@pipecat-ai/small-webrtc-transport` — closest analogue, since both
+  open a session and stream media bidirectionally.
+- `@pipecat-ai/websocket-transport` — simpler reference; might be a closer
+  fit for MOQ since neither is WebRTC.
+
+Output: a concrete TypeScript API surface for `MoqTransport` (constructor
+options, method signatures, events emitted) + a sharper estimate. Two
+hours of investment to either commit to the project or pass on it
+informedly.
+
+## Open questions for whoever picks this up
+
+- One participant identity per `Transport` instance, or should the
+  transport manage multiple peers internally (matching how WebRTC SFUs do
+  it)?
+- How should `getDevices()` map to MOQ — pure browser device enumeration
+  (since there's no SDK-side device negotiation)?
+- Where does the per-call ID come from? Server-allocated via
+  `/start`, or client-generated UUID? (Today we hardcode `bot0` /
+  `client0`.)
+- Cert pinning UX — for prod, can we get away with assuming the relay has
+  a real CA cert, or do we need an explicit `serverCertificateHashes`
+  config knob?