# Omni-only Turn-Taking Experiment This project tests whether **Qwen2.5-Omni-3B can replace both the handcrafted audio/visual feature extractors and the downstream RNN/Transformer used by EgoSpeak** for egocentric turn-taking anticipation. The main experiment is deliberately simple: ```text causal audio + video up to time t | v Qwen2.5-Omni-3B Thinker | fused state q_t [2048] | v trainable prediction head | v future logits [10 horizons, 3 classes] ``` There is **no RNN after Qwen**. The Omni Thinker is the temporal model. The only supervised component in the first experiment is a small `10 x 3` classification head. ## Current status | File | Status | Purpose | |---|---|---| | `smoke_test.py` | Verified on one NVIDIA L20 | Text and 12-second EasyCom audio-video inference | | `easycom_manifest.py` | Implemented, awaiting dataset-side run | Discover one-minute clips, preserve session timecodes, and check 5 Hz targets | | `qwen_omni_features.py` | Implemented, not executed here | Load only the 3B Thinker and extract fused states from one AV window | | `streaming_turn_model.py` | Prototype implemented, not executed here | Produce one causal Omni state every 200 ms, with reference and KV-cache modes | | `TurnPredictionHead` | Defined, untrained | Map `[2048]` to `[10, 3]` logits | | Head training and EgoSpeak evaluation script | Not yet implemented | Align saved states with `target_perframe`, train the head, and reproduce Table 2 mAP | The official full model, Thinker input path, embedded audio, video decoding, and FlashAttention-2 have passed a 12-second EasyCom smoke test on one NVIDIA L20. Streaming extraction, cache equivalence, target alignment, training, and paper metrics remain unverified. ## Why Omni-only instead of Omni + RNN EgoSpeak separately extracts visual and audio features and then uses GRU, LSTR, or Mamba for temporal modeling: ```text ResNet50 video feature + wav2vec2 audio feature -> temporal model -> prediction ``` Qwen2.5-Omni already contains: - audio and visual encoders; - joint audio-video temporal positions; - a causal decoder-only Thinker with KV cache; - pretrained acoustic, visual, and semantic knowledge. Adding a GRU after a continuously processed Omni sequence would model history twice and would make it harder to attribute an improvement to Omni. The primary comparison is therefore: ```text EgoSpeak: ResNet50 + wav2vec2 + GRU Ours: Qwen2.5-Omni Thinker + small prediction head ``` An Omni-feature-plus-GRU model can be kept as an ablation, but it is not the main architecture. ## EgoSpeak contract that this project reproduces The inspected reference repository is `/Volumes/760P/Code/EgoSpeak`. It contains no Qwen or Omni implementation and no raw-media feature extraction scripts. It distributes precomputed arrays instead: ```text DATASET/ |- rgb_kinetics_resnet50/ # [T, 2048] |- audio_wav2vec2/ # [T, 5120] `- target_perframe/ # [T, 3] ``` The useful part for this project is the exact task interface. ### Time and classes Both EasyCom and the EgoSpeak `EGO4DSHUFFLE` split use **5 FPS**, so one task timestep is **0.2 seconds**. The three output columns are: ```text 0: background 1: camera wearer speaking 2: other people speaking ``` They are not strictly one-hot. Background is exclusive, but simultaneous camera-wearer and other-person speech is stored as `[0, 1, 1]`. The inspected EasyCom features contain 7,711 such overlap steps. Preserve the full multi-positive target rather than applying `argmax`. Relevant EgoSpeak files: - `LSTR/data/data_info.json`: FPS, classes, and session splits; - `MiniROAD/configs/EasyCom/miniroad_aa_easycom.yaml`: 128-step training windows, stride 4, three classes, and anticipation length 10; - `MiniROAD/datasets/dataset.py`: feature loading and future-label indexing; - `MiniROAD/criterions/loss.py`: anticipation loss; - `MiniROAD/trainer/eval.py`: per-horizon AP and mean anticipation mAP. ### Future target alignment For current task index `i`, the prediction target is: ```python target[i + 1 : i + 11] ``` Thus the ten output rows represent: ```text t+0.2, t+0.4, t+0.6, ..., t+2.0 seconds ``` If `states[i]` is verified to correspond to EgoSpeak task index `i`, the offline head-training tensors should be constructed as follows: ```python target = target_perframe # [T, 3], multi-positive usable = min(len(states), len(target)) - 10 x = states[:usable] # [N, 2048] y = stack( [target[h : h + usable] for h in range(1, 11)], axis=1, ) # [N, 10, 3] ``` Do not silently assume that the first saved Omni state matches label index zero. EgoSpeak does not publish the script that generated its original ResNet/wav2vec2 features, so the exact zero-time convention is not documented. Before training, inspect a short clip with known speech boundaries and test the candidate mappings `states[i] -> target[i]` and `states[i] -> target[i+1]`. Store the selected offset in every feature file/checkpoint. ### Splits Use the published split, not a random frame split. EasyCom: - train, preserving EgoSpeak's list: `Session_7`, `Session_11`, `Session_10`, `Session_6`, `Session_9`, `Session_4`, `Session_8`, `Session_5`, `Session_12`; - test, preserving EgoSpeak's list: `Session_3`, `Session_2`, `Session_1`. This is the **EgoSpeak comparison split**. The EasyCom dataset repository itself recommends a different train/validation/test split; do not substitute that split when reproducing EgoSpeak Table 2. Ego4D Audio-Visual Diarization: - use the `EGO4DSHUFFLE` key; - 346 training clips; - 87 test clips; - each clip is approximately five minutes. Never mix timesteps from the same session across training and validation. ## EasyCom raw-media layout and manifest The extracted release used here has this relevant structure: ```text extracted_easycom/ `- Main/ |- Video_Compressed/ | |- Session_1/ | | |- 00-00-000.mp4 | | |- 01-00-000.mp4 | | `- ... | `- Session_2 ... Session_12/ |- Glasses_Microphone_Array_Audio/ |- Close_Microphone_Audio/ |- Speech_Transcriptions/ `- Metadata/ ``` `Video_Compressed` is the AR-glasses wearer's egocentric stream with embedded stereo AAC audio. Each session is divided into approximately one-minute clips; the `mm-ss-SSS` filename is the clip's start time on the original recording timeline. Some intervals are absent because EasyCom redacted one-minute segments. These raw time gaps are expected. EgoSpeak does not preserve those empty intervals. Its appendix states that it crops every EasyCom clip to retain only the 1920-pixel-wide video region and then merges the retained clips into one dense video per session. Therefore the EgoSpeak-compatible label time for a clip is its cumulative position in the concatenation, not the `mm-ss-SSS` source time. The manifest stores both. Build a raw-video manifest and, when the EgoSpeak labels are available, check their duration at 5 Hz: ```bash python easycom_manifest.py \ --dataset-root /data1/yfzx/wangx/Datasets/easycom/extracted_easycom \ --target-root /path/to/EasyCom/target_perframe \ --output manifests/easycom.json ``` Omit `--target-root` for the first media-only inventory. The manifest reports raw-source discontinuities as informational and computes each clip's dense EgoSpeak concatenation offset and corresponding 5 Hz target index. For the audio-only ablation, use the same embedded glasses audio as A+V (extract it to mono WAV if needed); do not switch to close microphones, because that changes the sensor and makes the modality ablation unfair. Audit downloaded EgoSpeak arrays, and optionally compare every session length against the raw manifest: ```bash python audit_egospeak_features.py \ --feature-root /path/to/EgoSpeak/dataset/EasyCom \ --raw-manifest manifests/easycom.json \ --output reports/egospeak_easycom_audit.json ``` The inspected release contains 95,453 synchronized steps across all 12 sessions: 5.302944 hours at 5 Hz. Every session has identical RGB, flow, audio, and target length. Shapes are `[T,2048]`, `[T,2048]`, `[T,5120]`, and `[T,3]`. ## Recommended comparison matrix | Experiment | Visual sampling | Audio | Output rate | |---|---:|---|---:| | EgoSpeak official A+V | 5 Hz ResNet features | 5 Hz wav2vec2 features | 5 Hz | | Omni audio-only | none | raw glasses-wearer waveform | 5 Hz | | Omni A+V primary | 2 FPS | raw embedded waveform | 5 Hz | | Omni A+V FPS ablation | 5 FPS | raw embedded waveform | 5 Hz | Two FPS is the primary Omni setting because it matches Qwen's default video sampling and permits longer context. Five FPS is the visual-rate ablation. Input feature rates need not match EgoSpeak; the split, causal prediction times, three classes, ten target horizons, and evaluator must match exactly. ## Prepare an EgoSpeak-compatible EasyCom session Do not pass the raw one-minute files independently to Omni and do not preserve the gaps implied by their filenames. EgoSpeak concatenates the retained clips into a dense session timeline and crops the participant-ID banner on the left. `prepare_easycom_session.py` reproduces this policy and creates videos whose stored frames already lie on a fixed session-global 2 FPS or 5 FPS grid. This prevents Qwen's per-window video loader from choosing a new adaptive set of frames for every overlapping prefix. Session 1 has been checked against the local release: ```text 24 retained clips 27,768 source frames at 20 FPS / 4 frames per task step = 6,942 steps at 5 Hz = target_perframe/Session_1.npy length exactly ``` Preview the commands and validate frames against the target without encoding: ```bash python prepare_easycom_session.py \ --session-dir /Volumes/MX500B/Code/easycom/Main/Video_Compressed/Session_1 \ --output-dir prepared_easycom/Session_1 \ --target /Volumes/MX500B/Code/EgoSpeak/dataset/EasyCom/target_perframe/Session_1.npy \ --fps 2 5 \ --dry-run ``` Remove `--dry-run` to run FFmpeg. The outputs are: ```text Session_1_2fps.mp4 primary Omni A+V input Session_1_5fps.mp4 visual-rate ablation Session_1.json clip order, exact T, crop and sampling provenance Session_1.ffconcat reproducible dense concatenation order ``` The script fails rather than silently padding or truncating when `sum(source_frames) / 4 != target.shape[0]`. Each clip's decoded video and audio are trimmed to the exact duration implied by its video frame count before concatenation. This prevents AAC priming/tail padding from accumulating at the 24 clip boundaries and causing progressive audio/video drift. On the remote L20 machine, use its own dataset paths. For example: ```bash python prepare_easycom_session.py \ --session-dir /data1/yfzx/wangx/Datasets/easycom/extracted_easycom/Main/Video_Compressed/Session_1 \ --output-dir /data1/yfzx/wangx/Datasets/easycom/omni_prepared/Session_1 \ --target /path/to/EgoSpeak/dataset/EasyCom/target_perframe/Session_1.npy \ --fps 2 5 ``` This is the command that performs the potentially long FFmpeg processing; it has not been run by the code authoring workflow. If encoding completed but a later validation/metadata step failed, update the script and reuse the existing MP4 without encoding it again: ```bash python prepare_easycom_session.py ... --fps 2 --reuse-existing ``` ## What Qwen's native 2-second chunk means Qwen2.5-Omni is **not restricted to receiving audio once every two seconds**. It can receive an arbitrary-duration AV input. The two-second value describes internal processing: 1. the audio encoder divides acoustic features into approximately two-second attention blocks; 2. audio and video tokens are interleaved in two-second temporal groups before entering the Thinker. The audio encoder attention inside a block is non-causal. Therefore this is invalid for online evaluation: ```text encode the complete [0, 2.0 s] block -> read a purported state at 0.2 s ``` That early state can contain audio from 0.2--2.0 seconds. At prediction time `t`, the model must receive only audio and video with timestamps `<= t`. ## Causal 5 Hz extraction modes `streaming_turn_model.py` implements three modes. ### 1. `causal-prefix`: correctness and semantic baseline At every 200 ms, independently process the causal context ending at the current time: ```text [max(0, t-20 s), t] AV -> Omni -> q_t ``` Properties: - strict no-future input; - one continuous media window per prediction; - no cross-forward cache; - very expensive because overlapping context is recomputed; - preferred reference when checking semantics and causality. Example: ```bash python streaming_turn_model.py \ --video /path/to/easycom_session.mp4 \ --use-audio-in-video \ --mode causal-prefix \ --start 0 \ --end 20 \ --memory-seconds 20 \ --output features/easycom_20s_prefix.pt ``` ### 2. `commit-replay`: closed-turn compatibility baseline The extractor retains Thinker KV for completed two-second media blocks. During the unfinished block it branches from the same committed cache and evaluates only the available prefix: ```text committed historical KV + current block [0.0, 0.2] -> q_t committed historical KV + current block [0.0, 0.4] -> q_t ... committed historical KV + current block [0.0, 2.0] -> q_t -> commit media KV ``` The temporary readout/assistant query is never committed. Only the completed media block enters long-term cache. This avoids recomputing the full history, but still re-encodes at most the current two-second prefix. Example with numerical checking: ```bash python streaming_turn_model.py \ --video /path/to/easycom_session.mp4 \ --use-audio-in-video \ --mode commit-replay \ --start 0 \ --end 20 \ --memory-seconds 20 \ --verify-every 5 \ --verify-continuous-every 5 \ --output features/easycom_20s_stream.pt ``` `--verify-every 5` compares cached processing with a from-scratch replay of the same sequence of two-second media turns. The saved cosine similarity should be extremely close to 1. If it is not, the TMRoPE/cache path is invalid and must not be used for a paper result. `--verify-continuous-every 5` is a different check: it compares the cached sequence of two-second media turns with a single continuous causal window. This comparison need not be numerically identical, but it measures whether the input organization materially changes the representation. Save and report the two check types separately (`cache_replay` and `continuous_window_semantics`). There is an important experimental distinction: `commit-replay` represents history as a sequence of two-second media turns. Its numerical check proves cache equivalence to that same turn sequence, not equivalence to one continuous 20-second video placeholder. Compare it against `causal-prefix` on identical clips and report the representation if it changes mAP. ### 3. `open-stream-replay`: recommended efficient mode This mode keeps one user/media turn open across the whole bounded-memory segment. The first block contributes the chat and AV opening markers once; later completed two-second blocks contribute only interleaved audio/video payload tokens: ```text open AV prefix completed [0,2] payload -> commit KV current [2,2.2] payload -> temporary branch -> close/read q_t current [2,2.4] payload -> temporary branch -> close/read q_t ... completed [2,4] payload -> commit KV AV/chat closure -> never committed; readout branch only ``` The audio encoder accepts a shorter final chunk with padding and an actual-length mask. For the two-frame visual patch requirement, strict-grid mode duplicates only the most recent past frame when necessary. Run a four-second validation before using it for extraction: ```bash python streaming_turn_model.py \ --model-id /path/to/Qwen2.5-Omni-3B \ --video prepared_easycom/Session_1/Session_1_2fps.mp4 \ --use-audio-in-video \ --mode open-stream-replay \ --start 0 --end 4 \ --video-fps 2 \ --strict-fixed-video-grid \ --verify-every 1 \ --verify-continuous-every 1 \ --flash-attention-2 \ --output features/Session_1_open_first_4s.pt ``` `cache_replay` must be identical to replaying the same open stream from an empty cache. `continuous_window_semantics` compares it with one normal continuous Qwen AV input. Do not assume equivalence: the modality encoders are still invoked per tail block in the efficient path, so this numerical check is required even though artificial user-turn boundaries have been removed. ## Whole-session prediction and the 20-second limit Twenty seconds is a practical memory/context setting for the 3090, not the maximum video duration. The complete session is processed sequentially and a prediction is emitted every 0.2 seconds. The legacy `commit-replay` prototype resets Thinker KV at each non-overlapping 20-second boundary: ```text session: 0 -------- 20 -------- 40 -------- 60 ... cache: [segment 1] [segment 2] [segment 3] ... output: continuous predictions across the entire session ``` This preserves complete output coverage but limits available history after a reset. Report the memory length and additionally evaluate the first two seconds after each reset versus all other timesteps. Rolling KV eviction is a possible future optimization, but requires correct multimodal position rebasing. `open-stream-replay` instead uses blockwise rolling memory. With a 20-second budget it rebuilds the open cache from the latest nine completed blocks (18 seconds) when a new block starts, then adds the current 0--2 second tail. Its effective history therefore varies from 18 to 20 seconds and never drops to an empty 0.2-second context at a 20-second boundary. For the paper experiment, drive the output length from the preparation sidecar rather than a floating MP4 duration: ```bash python streaming_turn_model.py \ --session-metadata prepared_easycom/Session_1/Session_1.json \ --use-audio-in-video \ --mode open-stream-replay \ --memory-seconds 20 \ --video-fps 2 \ --strict-fixed-video-grid \ --verify-every 25 \ --verify-continuous-every 25 \ --output features/Session_1_omni_2fps.pt ``` The sidecar selects the matching fixed-FPS MP4 and sets `N` to its exact `num_steps`. Alternatively pass `--video ... --target ...` explicitly. A raw 20 FPS MP4 with `--video-fps 2` is rejected when `--strict-fixed-video-grid` is enabled. In strict A+V mode the extractor does not delegate temporal frame selection to the generic video utility. It reads the already-fixed MP4 grid explicitly and assigns both video frames and audio samples with half-open intervals `[start, end)`. Thus a frame exactly on a two-second boundary belongs only to the following block. It duplicates the most recent available frame when the visual tower needs an even frame count. Audio is loaded once at 16 kHz and sliced at the same causal boundary. The processor still receives `use_audio_in_video=True`, so Qwen's audio/video token interleaving is retained. Before a full session, run only the first four seconds and inspect both check types: ```bash python streaming_turn_model.py \ --model-id /data1/yfzx/wangx/Models/Qwen2.5-Omni-3B \ --video /data1/yfzx/wangx/Datasets/easycom/omni_prepared/Session_1/Session_1_2fps.mp4 \ --use-audio-in-video \ --mode open-stream-replay \ --start 0 --end 4 \ --video-fps 2 \ --strict-fixed-video-grid \ --verify-every 1 \ --verify-continuous-every 1 \ --flash-attention-2 \ --output features/Session_1_first_4s_check.pt ``` Do not launch all sessions unless `cache_replay` cosine is nearly 1. For `open-stream-replay`, a materially lower `continuous_window_semantics` cosine means that the custom payload stitching is not equivalent to one continuous Qwen input; it must not be dismissed as a media-turn-boundary effect because this mode intentionally keeps a single media turn open. Video can be sampled at 2 FPS while predictions remain at 5 Hz. Audio and the causal boundary update every 200 ms; consecutive prediction times may use the same most-recent visual frame. A frame with timestamp later than `t` must never be included. ## Saved streaming tensors Each `.pt` file contains: - `times`: `[N]`, prediction times in seconds; - `states`: `[N, 2048]`, causal fused Thinker states; - `label_indices`: `[0, ..., N-1]`, direct `target_perframe` row alignment; - `valid_10_horizon_steps`: `N-10`, because the last ten current rows do not have all future labels; - `mode`: `causal-prefix`, `commit-replay`, or `open-stream-replay`; - `stream_representation`: explicit input-organization metadata; - `config`: step, native block, memory, FPS, and pixel settings; - `verification`: cached-versus-exact numerical checks; - `input_video_fps` and `video_sampling_grid`: sampling provenance; - `media_interval_policy`: fixed-grid AV uses half-open `[start, end)` ownership; - `logits`: optional `[N, 10, 3]` output if a trained head is supplied; - `probabilities`: optional softmax probabilities. ## Training the `10 x 3` head The head must be trained. It is not part of pretrained Qwen. The first experiment should freeze all Qwen parameters and train only: ```text LayerNorm(2048) -> Linear(2048, 1024) -> GELU -> Dropout -> Linear(1024, 30) -> reshape [10, 3] ``` `streaming_turn_model.py` exposes: ```python from streaming_turn_model import ( TurnPredictionHead, egospeak_multihorizon_loss, ) ``` Recommended initial procedure: 1. Freeze Qwen and extract all training-session states offline. 2. Verify state/label time offset and truncate only the final ten invalid current positions. 3. Keep EgoSpeak targets as float multi-positive tensors `[N, 10, 3]`; never convert them with `argmax`. 4. Train with `egospeak_multihorizon_loss`. It exactly mirrors MiniROAD's anticipation objective: L2-normalize each three-class target, then compute its weighted negative log-softmax. 5. Select hyperparameters using only training sessions. For EasyCom, reserve a training session for development or use session-level cross-validation; never tune on Sessions 1--3. 6. After choosing the epoch/hyperparameters, optionally retrain on all nine EasyCom training sessions and evaluate the fixed model once on the test set. Class-balanced/focal loss, a learned turn-query token, and LoRA are later ablations. They should not replace the frozen-Omni linear/MLP-head baseline. ## EgoSpeak-compatible evaluation For every valid current timestep, produce probabilities shaped: ```text [number of timesteps, 10 future horizons, 3 classes] ``` Then reproduce EgoSpeak's Table 2 procedure: 1. for each horizon independently, concatenate test-session predictions and ground truth; 2. compute per-class frame-level average precision; 3. average the three class AP values to obtain horizon mAP; 4. report all ten horizon mAP values; 5. average the ten horizon mAP values for the final score. With EgoSpeak's `ignore_index=-100`, the released evaluator does not remove class index zero, so background is included. Reuse the same implementation or verify a reimplementation numerically. Reference Table 2 targets from the EgoSpeak paper: - EasyCom GRU A+V: **60.6 mean mAP**; - EasyCom Transformer A+V: **58.7 mean mAP**; - Ego4D Transformer audio: **69.2 mean mAP**; - Ego4D Transformer A+V: **69.0 mean mAP**; - Ego4D GRU audio: **69.2 mean mAP**; - Ego4D GRU A+V: **68.2 mean mAP**. Do not compare these numbers with EgoSpeak Table 3 target-speaker-only AP; Table 3 uses a different metric. ## Experiments that demonstrate Omni's semantic advantage The headline comparison alone cannot establish that improvements come from semantics. Run the following controlled experiments: 1. **Omni audio vs Omni audio+video**: measure whether visual grounding helps. 2. **Per-horizon curve**: semantic completion should be especially visible at longer horizons such as 1.2--2.0 seconds. 3. **Context length**: compare 4, 10, 15, and 20 seconds without adding an RNN. 4. **Layer ablation**: compare audio-encoder output and several Thinker layers, because an intermediate layer may be more discriminative than the final generation-oriented layer. 5. **Semantic-completion subset**: contrast completed utterances with hesitation/incomplete pauses while matching pause duration and basic prosody as closely as possible. 6. **Question/statement/backchannel subsets**: report target-speaker AP and premature-interruption rate on linguistically meaningful events. 7. **Causal transcript baseline**: compare Omni audio with causal ASR text plus a language model to isolate lexical/semantic contribution. The frozen-Omni head is the cleanest representation test. LoRA can be reported as a second-stage ceiling, not as the only result. ## Single-window feature inspection `qwen_omni_features.py` is useful for inspecting a short window or layer, but it is not the main whole-session experiment. If audio is embedded in the video: ```bash python qwen_omni_features.py \ --video /path/to/window.mp4 \ --use-audio-in-video \ --output features/window.pt ``` If audio and video are separate: ```bash python qwen_omni_features.py \ --video /path/to/window.mp4 \ --audio /path/to/window.wav \ --output features/window.pt ``` Do not provide the same audio both as a separate file and via `--use-audio-in-video`. Saved single-window fields are: - `token_features`: `[B, token_length, 2048]`; - `last_token`: `[B, 2048]`; - `tail_mean`: `[B, 2048]`; - `attention_mask`: valid-token mask. ## Installation Install `ffmpeg`, then: ```bash pip install -r requirements.txt ``` The implementation is pinned to `transformers==4.52.3` because the commit/replay path uses version-specific Thinker internals. On Linux, `qwen-omni-utils[decord]` is recommended for faster video decoding. On a compatible NVIDIA GPU, optionally install `flash-attn` and pass `--flash-attention-2`. ## Ego4D Audio-Visual Diarization download Do not download the complete Ego4D release for this experiment. Download only the compressed Audio-Visual Diarization benchmark clips and annotations: ```bash ego4d \ --output_directory=/path/to/ego4d \ --datasets clips annotations \ --benchmarks AV ``` The approximately 1 TB figure on the Ego4D download page refers to all benchmark clips combined, not only the AV subset. The annotated AVD subset has about 572 five-minute clips (roughly 50 hours). EgoSpeak uses a custom 346/87 train/test split, about 36.1 hours total. Reproducing its Table 2 requires the exact IDs in EgoSpeak's `EGO4DSHUFFLE` split. ## Required validation before a full run Do not launch full EasyCom/Ego4D extraction until all of these pass: - no input audio sample or video frame has timestamp later than prediction time `t`; - `states` and `target_perframe` lengths/time origins are reconciled and the chosen label offset is saved; - commit/replay cosine equivalence is near 1 on short clips; - commit/replay and causal-prefix predictions are compared on identical clips; - a 20-second run fits the 3090 with the selected pixel budget and attention implementation; - saved outputs contain exactly ten future horizons in the order 0.2--2.0 s; - the local metric reproduces EgoSpeak's evaluator on known predictions; - test session IDs never enter training, validation, normalization statistics, or hyperparameter selection. Once these checks pass, the first publishable experiment is: ```text frozen Qwen2.5-Omni-3B Thinker + trained 10x3 head + 20-second causal memory + full-session 5 Hz output + exact EgoSpeak split and Table 2 metric ```