Record EgoSpeak Omni residual dense and raw anticipation screen

This commit is contained in:
Xin Wang
2026-09-16 16:53:27 +08:00
parent 8c8f835a77
commit eff7a5ad29
5 changed files with 126 additions and 1 deletions

View File

@@ -135,7 +135,16 @@ result.
described as upstream Stage-3 code because that entry point remains described as upstream Stage-3 code because that entry point remains
unpublished. See the unpublished. See the
[`runbook`](docs/easycom_muvap_stage3_baseline.md). [`runbook`](docs/easycom_muvap_stage3_baseline.md).
- `egospeak-omni-zero-residual` is implemented but unmeasured. It strictly - `egospeak-omni-zero-residual` has a user-reported seed-0 screen. Causal-valid
dense mean mAP increases from `61.193` to `62.714`; raw onset/offset mean
AP increases from `24.491/38.024` to `25.091/40.604`. Offset improves at all
ten horizons; onset gains are modest. Actual checkpoint provenance and
counts were not supplied for independent verification. Opportunity-based
semantic evaluation is temporarily paused because the user suspects
provisional annotation issues; historical metrics remain archived and P0
remains unfinished. See the
[`result record`](results/easycom_egospeak_omni_residual_2026-09-16/README.md).
The implementation strictly
loads and freezes the released MiniROADA A+V/GRU/anticipation path, reuses loads and freezes the released MiniROADA A+V/GRU/anticipation path, reuses
existing causal 5 Hz Omni readouts, adds an explicit ten-bin partial-tail existing causal 5 Hz Omni readouts, adds an explicit ten-bin partial-tail
phase embedding, and learns only a zero-output residual over the original phase embedding, and learns only a zero-output residual over the original

View File

@@ -6,6 +6,7 @@ readers to the right comparison.
| Result record | Question answered | | Result record | Question answered |
|---|---| |---|---|
| [`easycom_egospeak_omni_residual_2026-09-16/`](easycom_egospeak_omni_residual_2026-09-16/README.md) | Single-seed frozen EgoSpeak + Omni residual dense/raw endpoint gains; semantic annotation evaluation temporarily paused |
| [`easycom_omni_floor_screen_2026-09-16/`](easycom_omni_floor_screen_2026-09-16/README.md) | Whether G+A, pooled-V, or current-frame F1 improve raw boundaries and provisional semantic opportunities over EgoSpeak | | [`easycom_omni_floor_screen_2026-09-16/`](easycom_omni_floor_screen_2026-09-16/README.md) | Whether G+A, pooled-V, or current-frame F1 improve raw boundaries and provisional semantic opportunities over EgoSpeak |
| [`easycom_egospeak_three_level_eval_2026-09-16/`](easycom_egospeak_three_level_eval_2026-09-16/README.md) | How dense forecasting, raw boundary anticipation, and provisional opportunity-conditioned ENTRY/YIELD differ for the EgoSpeak baseline | | [`easycom_egospeak_three_level_eval_2026-09-16/`](easycom_egospeak_three_level_eval_2026-09-16/README.md) | How dense forecasting, raw boundary anticipation, and provisional opportunity-conditioned ENTRY/YIELD differ for the EgoSpeak baseline |
| [easycom_oracle_pooled_wearer_audit_2026-09-10/](easycom_oracle_pooled_wearer_audit_2026-09-10/README.md) | Fixed-four-epoch three-seed oracle test results and the limited wearer gains of historical pooled AV | | [easycom_oracle_pooled_wearer_audit_2026-09-10/](easycom_oracle_pooled_wearer_audit_2026-09-10/README.md) | Fixed-four-epoch three-seed oracle test results and the limited wearer gains of historical pooled AV |

View File

@@ -0,0 +1,71 @@
# Stage-one frozen EgoSpeak + Omni residual screen
## Provenance and scope
These values were reported by the user on 2026-09-16, not recomputed on this
host. The preceding launcher command specifies seed 0, four fixed epochs on
EasyCom Sessions 4--12, and final evaluation on Sessions 1--3. The reported
run is `results/easycom_egospeak_omni_residual_seed0/checkpoint_last.pt`; its
actual checkpoint hash/config and sample counts were not supplied and remain
unverified. The base is the released `miniroad_easycom_AV.pth` checkpoint.
Implementation: [`stage-one runbook`](../../docs/easycom_egospeak_omni_residual.md).
The original MiniROADA A+V path and Omni backbone remain frozen; only the
tail-phase-aware residual fusion is trained. Existing Omni states use the
8-s open-stream configuration and a fixed mean over the last-four stack.
Both full-timeline and causal-valid dense numbers are retained in
[`metrics.csv`](metrics.csv). All numbers in this record are percentages;
deltas are percentage points. Displayed values are rounded console output.
## Findings
| Metric | Frozen EgoSpeak | Fused | Delta |
|---|---:|---:|---:|
| Causal-valid 0.2-s dense mAP | 74.204 | 77.739 | +3.535 |
| Causal-valid 0.2-s wearer AP | 63.750 | 72.716 | +8.966 |
| Causal-valid ten-horizon dense mean mAP | 61.193 | 62.714 | +1.521 |
| Full-timeline ten-horizon dense mean mAP | 61.155 | 62.676 | +1.521 |
| Ten-horizon raw onset endpoint AP | 24.491 | 25.091 | +0.600 |
| Ten-horizon raw offset endpoint AP | 38.024 | 40.604 | +2.580 |
Offset AP improves at all ten horizons, by about 2.8--3.5 points at
0.4--1.2 s. Onset AP improves at nine horizons; its largest gain is +1.634
points at 0.8 s, while long-horizon differences are small. This is a
single-seed positive screen, strongest for future wearer cessation. It is
not a significance test or evidence of semantic floor-yield understanding.
A parameter-matched EgoSpeak-only residual control and additional seeds
remain necessary to attribute the gain specifically to Omni.
## Raw anticipation definition
For each horizon h=0.2,...,2.0 s, use valid source/target pairs from the same
session. Onset is evaluated only when wearer activity W(t)=0: positive if
W(t+h)=1, negative if W(t+h)=0, with score p_wearer(t,h). Offset is evaluated
only when W(t)=1: positive if W(t+h)=0, negative if W(t+h)=1, with score
1-p_wearer(t,h). AP ranks all eligible samples pooled over Sessions 1--3;
the headline averages ten horizon APs. This is endpoint state-change
anticipation, not one-event-one-vote timestamp detection. Intermediate
transitions can be invisible when source and endpoint states agree.
The evaluator also computed exploratory/in-sample operating points, but
only threshold-free endpoint APs are recorded here. They need no Session-4
threshold calibration. Do not use the accompanying in-sample rate columns
as confirmatory test results.
## Annotation decision and next stage
The user suspects issues with the provisional LLM-assisted semantic
annotations. Opportunity-conditioned ENTRY/YIELD evaluation is therefore
temporarily paused pending an annotation audit. Historical provisional
metrics remain archived; they are neither human-reviewed ground truth nor
discarded because of model performance. No new semantic result or semantic
loss is introduced in this screen.
The next requested implementation is selective MiniROADA GRU/anticipation
fine-tuning from the stage-one checkpoint, keeping original feature
extractors and Omni frozen. This is an exploratory continuation of the
previously documented route, not passage of the still-unfinished P0 semantic
evaluation gate. Do not tune windows, loss weights or stopping on Sessions
1--3; compare fixed budgets and preserve the frozen stage-one/original base
as references. A four-epoch continuation has eight total adaptation epochs
and is not a matched-budget comparison against the four-epoch first stage.

View File

@@ -0,0 +1,21 @@
run,horizon_seconds,dense_full_map,dense_causal_map,dense_causal_background_ap,dense_causal_wearer_ap,dense_causal_others_ap,onset_endpoint_ap,offset_endpoint_ap
egospeak,0.2,74.124,74.204,68.189,63.750,90.673,5.139,9.696
egospeak,0.4,68.291,68.349,58.596,59.595,86.857,9.878,18.846
egospeak,0.6,64.267,64.313,52.373,56.619,83.947,15.389,27.730
egospeak,0.8,61.091,61.128,47.629,54.172,81.585,21.093,34.971
egospeak,1.0,59.734,59.768,46.172,52.916,80.216,27.225,40.646
egospeak,1.2,58.774,58.805,45.840,51.488,79.088,30.997,45.112
egospeak,1.4,57.471,57.498,43.587,50.765,78.143,33.563,47.880
egospeak,1.6,57.064,57.090,43.518,49.739,78.012,34.046,50.444
egospeak,1.8,55.933,55.954,42.227,48.605,77.031,34.030,52.051
egospeak,2.0,54.802,54.821,40.349,47.393,76.720,33.550,52.862
fused,0.2,77.658,77.739,69.105,72.716,91.396,5.722,11.756
fused,0.4,71.510,71.570,59.729,67.888,87.093,10.749,21.920
fused,0.6,66.855,66.903,53.297,63.553,83.858,16.613,31.248
fused,0.8,63.212,63.251,48.874,59.713,81.167,22.727,38.383
fused,1.0,60.961,60.996,46.574,56.953,79.460,27.946,43.741
fused,1.2,59.410,59.441,45.243,54.917,78.163,31.260,47.922
fused,1.4,58.375,58.404,43.983,53.446,77.782,33.502,50.269
fused,1.6,57.336,57.361,42.934,51.712,77.439,34.053,52.173
fused,1.8,56.222,56.244,41.284,50.607,76.840,34.202,53.814
fused,2.0,55.216,55.235,39.820,49.470,76.415,34.137,54.812
1 run horizon_seconds dense_full_map dense_causal_map dense_causal_background_ap dense_causal_wearer_ap dense_causal_others_ap onset_endpoint_ap offset_endpoint_ap
2 egospeak 0.2 74.124 74.204 68.189 63.750 90.673 5.139 9.696
3 egospeak 0.4 68.291 68.349 58.596 59.595 86.857 9.878 18.846
4 egospeak 0.6 64.267 64.313 52.373 56.619 83.947 15.389 27.730
5 egospeak 0.8 61.091 61.128 47.629 54.172 81.585 21.093 34.971
6 egospeak 1.0 59.734 59.768 46.172 52.916 80.216 27.225 40.646
7 egospeak 1.2 58.774 58.805 45.840 51.488 79.088 30.997 45.112
8 egospeak 1.4 57.471 57.498 43.587 50.765 78.143 33.563 47.880
9 egospeak 1.6 57.064 57.090 43.518 49.739 78.012 34.046 50.444
10 egospeak 1.8 55.933 55.954 42.227 48.605 77.031 34.030 52.051
11 egospeak 2.0 54.802 54.821 40.349 47.393 76.720 33.550 52.862
12 fused 0.2 77.658 77.739 69.105 72.716 91.396 5.722 11.756
13 fused 0.4 71.510 71.570 59.729 67.888 87.093 10.749 21.920
14 fused 0.6 66.855 66.903 53.297 63.553 83.858 16.613 31.248
15 fused 0.8 63.212 63.251 48.874 59.713 81.167 22.727 38.383
16 fused 1.0 60.961 60.996 46.574 56.953 79.460 27.946 43.741
17 fused 1.2 59.410 59.441 45.243 54.917 78.163 31.260 47.922
18 fused 1.4 58.375 58.404 43.983 53.446 77.782 33.502 50.269
19 fused 1.6 57.336 57.361 42.934 51.712 77.439 34.053 52.173
20 fused 1.8 56.222 56.244 41.284 50.607 76.840 34.202 53.814
21 fused 2.0 55.216 55.235 39.820 49.470 76.415 34.137 54.812

View File

@@ -0,0 +1,23 @@
{
"date": "2026-09-16",
"source": "user-reported console output; not locally recomputed",
"units": "percent; deltas are percentage points",
"split": {"train": [4, 5, 6, 7, 8, 9, 10, 11, 12], "test": [1, 2, 3]},
"seed": 0,
"epochs": 4,
"configuration_provenance": "preceding command; actual checkpoint/config not inspected",
"checkpoint": "results/easycom_egospeak_omni_residual_seed0/checkpoint_last.pt",
"checkpoint_sha256": null,
"base_checkpoint": "third_party/EgoSpeak/MiniROAD/checkpoints/miniroad_easycom_AV.pth",
"omni_memory_seconds": 8,
"omni_layer_fusion": "last-four fixed mean",
"num_seeds": 1,
"sample_counts": null,
"metrics": {
"egospeak": {"dense_causal_02_map": 74.204, "dense_causal_02_wearer_ap": 63.750, "dense_causal_mean_map": 61.193, "dense_full_mean_map": 61.155, "onset_mean_ap": 24.491, "offset_mean_ap": 38.024},
"fused": {"dense_causal_02_map": 77.739, "dense_causal_02_wearer_ap": 72.716, "dense_causal_mean_map": 62.714, "dense_full_mean_map": 62.676, "onset_mean_ap": 25.091, "offset_mean_ap": 40.604},
"fused_minus_egospeak": {"dense_causal_02_map": 3.535, "dense_causal_02_wearer_ap": 8.966, "dense_causal_mean_map": 1.521, "dense_full_mean_map": 1.521, "onset_mean_ap": 0.600, "offset_mean_ap": 2.580}
},
"semantic_evaluation": "paused: provisional annotation quality requires audit",
"claim_boundary": "single-seed dense/endpoint improvement; not semantic ENTRY/YIELD or Omni-specific causal attribution"
}