icassp2027-experiment/results/easycom_open_stream_8s_vs_20s_2026-08-23/README.md

EasyCom open-stream context: 8 seconds versus 20 seconds

Recorded on 2026-08-23. This is a single-run experiment snapshot, not a final multi-seed paper result.

Experiment contract

• Dataset: EasyCom with EgoSpeak's official split.
• Train sessions: 7, 11, 10, 6, 9, 4, 8, 5, and 12.
• Test sessions: 3, 2, and 1.
• Representation: frozen Qwen2.5-Omni-3B Thinker, open-stream replay.
• Head: MLP 2048 -> 1024 -> 30, reshaped to 10 x 3.
• Video sampling: 2 FPS.
• Prediction rate: 5 Hz.
• Context budgets: 8 seconds and 20 seconds.
• Targets: ten future horizons from 0.2 through 2.0 seconds.
• Loss/evaluation: EgoSpeak LSTR MCE and Table 2 frame AP/mAP protocol.

The heads were trained separately for their corresponding representations. A head trained on the 20-second states was not reused on the 8-second states.

Summary

All values are percentages.

Protocol	8 seconds	20 seconds	8s minus 20s
EgoSpeak full timeline	58.51937	58.316	+0.20337
Strict causal valid only	58.53015	58.326	+0.20415

The full-timeline per-horizon comparison is:

Horizon	Time	8s mAP	20s mAP	8s minus 20s
1	0.2s	71.99768	69.756	+2.24168
2	0.4s	66.74289	65.163	+1.57989
3	0.6s	62.38972	61.578	+0.81172
4	0.8s	59.12229	58.783	+0.33929
5	1.0s	56.71380	56.913	-0.19920
6	1.2s	55.36300	55.727	-0.36400
7	1.4s	54.21297	54.819	-0.60603
8	1.6s	53.43221	53.957	-0.52479
9	1.8s	52.84703	53.459	-0.61197
10	2.0s	52.37207	53.002	-0.62993
Mean	—	58.51937	58.316	+0.20337

For the full-timeline protocol, averaging AP over the ten horizons gives:

Class	8s mean AP	20s mean AP	8s minus 20s
Background	39.63094	40.49350	-0.86256
Camera wearer speaking	58.91206	58.59400	+0.31806
Other people speaking	77.01510	75.85970	+1.15540

Interpretation

The aggregate 8-second advantage is only 0.20 mAP, but it is not distributed uniformly. The 8-second representation is substantially stronger at 0.2--0.6 seconds, while the 20-second representation becomes stronger from 1.0 seconds onward. This looks like a short/long-horizon tradeoff rather than evidence that one context budget is uniformly better.

The class pattern is also mixed: 8 seconds improves the two speaking classes, especially other-person speech, but reduces background AP. Because the heads were trained once and the aggregate difference is small, training variance could plausibly reverse the mean comparison. Identical multi-seed runs are required before selecting the context budget for the paper.

Files and provenance

• raw_8s_console.txt preserves the reported 8-second console output.
• metrics.csv contains both contexts, protocols, horizons, and classes.
• summary.json contains the machine-readable aggregate comparison.
• The 20-second values come from the previously archived easycom_open_stream_vs_causal_prefix_2026-08-19 snapshot and therefore have only three-decimal precision.
• Aggregate differences use the separately reported aggregate means. Averaging differences reconstructed from the rounded 20-second horizon rows can vary slightly from those aggregate differences.
• The original remote metrics.json files have not yet been copied here.