3.8 KiB
Stage-one frozen EgoSpeak + Omni residual screen
Provenance and scope
These values were reported by the user on 2026-09-16, not recomputed on this
host. The preceding launcher command specifies seed 0, four fixed epochs on
EasyCom Sessions 4--12, and final evaluation on Sessions 1--3. The reported
run is results/easycom_egospeak_omni_residual_seed0/checkpoint_last.pt; its
actual checkpoint hash/config and sample counts were not supplied and remain
unverified. The base is the released miniroad_easycom_AV.pth checkpoint.
Implementation: stage-one runbook.
The original MiniROADA A+V path and Omni backbone remain frozen; only the
tail-phase-aware residual fusion is trained. Existing Omni states use the
8-s open-stream configuration and a fixed mean over the last-four stack.
Both full-timeline and causal-valid dense numbers are retained in
metrics.csv. All numbers in this record are percentages;
deltas are percentage points. Displayed values are rounded console output.
Findings
| Metric | Frozen EgoSpeak | Fused | Delta |
|---|---|---|---|
| Causal-valid 0.2-s dense mAP | 74.204 | 77.739 | +3.535 |
| Causal-valid 0.2-s wearer AP | 63.750 | 72.716 | +8.966 |
| Causal-valid ten-horizon dense mean mAP | 61.193 | 62.714 | +1.521 |
| Full-timeline ten-horizon dense mean mAP | 61.155 | 62.676 | +1.521 |
| Ten-horizon raw onset endpoint AP | 24.491 | 25.091 | +0.600 |
| Ten-horizon raw offset endpoint AP | 38.024 | 40.604 | +2.580 |
Offset AP improves at all ten horizons, by about 2.8--3.5 points at 0.4--1.2 s. Onset AP improves at nine horizons; its largest gain is +1.634 points at 0.8 s, while long-horizon differences are small. This is a single-seed positive screen, strongest for future wearer cessation. It is not a significance test or evidence of semantic floor-yield understanding. A parameter-matched EgoSpeak-only residual control and additional seeds remain necessary to attribute the gain specifically to Omni.
Raw anticipation definition
For each horizon h=0.2,...,2.0 s, use valid source/target pairs from the same session. Onset is evaluated only when wearer activity W(t)=0: positive if W(t+h)=1, negative if W(t+h)=0, with score p_wearer(t,h). Offset is evaluated only when W(t)=1: positive if W(t+h)=0, negative if W(t+h)=1, with score 1-p_wearer(t,h). AP ranks all eligible samples pooled over Sessions 1--3; the headline averages ten horizon APs. This is endpoint state-change anticipation, not one-event-one-vote timestamp detection. Intermediate transitions can be invisible when source and endpoint states agree.
The evaluator also computed exploratory/in-sample operating points, but only threshold-free endpoint APs are recorded here. They need no Session-4 threshold calibration. Do not use the accompanying in-sample rate columns as confirmatory test results.
Annotation decision and next stage
The user suspects issues with the provisional LLM-assisted semantic annotations. Opportunity-conditioned ENTRY/YIELD evaluation is therefore temporarily paused pending an annotation audit. Historical provisional metrics remain archived; they are neither human-reviewed ground truth nor discarded because of model performance. No new semantic result or semantic loss is introduced in this screen.
The next requested implementation is selective MiniROADA GRU/anticipation fine-tuning from the stage-one checkpoint, keeping original feature extractors and Omni frozen. This is an exploratory continuation of the previously documented route, not passage of the still-unfinished P0 semantic evaluation gate. Do not tune windows, loss weights or stopping on Sessions 1--3; compare fixed budgets and preserve the frozen stage-one/original base as references. A four-epoch continuation has eight total adaptation epochs and is not a matched-budget comparison against the four-epoch first stage.