Add causal person track GRU residual experiment

docs/easycom_track_gru_residual.md

@@ -0,0 +1,158 @@
+# EasyCom anonymous track-GRU residual
+
+## Decision
+
+The first track-temporal experiment keeps the strongest pooled local baseline
+as an exact, optionally frozen base and learns only a causal person-track
+correction:
+
+```text
+pooled base:  G_t + A_t + pooled-V_t -> concat-MLP -> base logits
+
+per visible person i:
+  past crop observations on the same temporary track
+      -> shared projection -> shared one-layer GRU -> H_t,i
+
+G_t + A_t -> masked attention over {NULL, H_t,1, ..., H_t,N}
+          -> zero-gated residual logits
+
+final logits = base logits + tanh(alpha) * person_valid * residual logits
+alpha starts at exactly zero
+```
+
+This is the smallest experiment that asks whether anonymous visual continuity
+adds information beyond `G+A+pooled-V`. It is not yet person-conditioned audio
+grounding and does not modify the Omni Thinker.
+
+## What was taken from EgoSpeak
+
+The official EgoSpeak repository is cloned for inspection at
+`third_party/EgoSpeak` and excluded from Git. The inspected revision is
+`b5bee35e796d8f2d6aa7da2bf1b07836270f8bfb`.
+
+Its `MiniROADA` model projects concatenated RGB/audio features, applies a
+single-layer GRU, expands the recurrent state into ten anticipation states,
+and uses a shared classifier. The EasyCom configuration uses a 128-step
+window, stride 4, 2048D embedding, 1024D GRU and ten future steps.
+
+We reuse the defensible ideas—causal recurrence, shared weights and a ten-step
+anticipation output—but not the implementation. Our GRU is shared across
+anonymous person tracks, reads only crop observations at or before the current
+cutoff, and corrects the already measured pooled Omni baseline. Consequently,
+this method must be described as an EgoSpeak-inspired track-temporal adapter,
+not an EgoSpeak GRU reproduction.
+
+## Causal and identity contract
+
+- A 5 Hz row with cutoff `(t+1)/5` reads the latest local video frame strictly
+  before that cutoff.
+- Track history contains at most `--person-history-frames` real 2 FPS crop
+  observations and is ordered oldest to newest.
+- Temporary IDs only link observations. They are never embedded or passed to
+  the model, so global ID renaming cannot change a prediction.
+- A source-clip boundary always resets a track. A time gap larger than
+  `--person-track-max-gap-seconds` also resets it.
+- Missing, `-1`, `None`, `null` and `unknown` IDs are isolated observations.
+- Duplicate known IDs in one frame fail instead of being silently merged.
+- Empty person sets produce exactly zero visual correction.
+
+The existing EasyCom local sidecars already contain crop features, frame times,
+source-clip indices and person IDs; this experiment does not require feature
+re-extraction.
+
+## Fair initialization
+
+`track-gru-residual` requires `--initialize-pooled-checkpoint`. The loader
+checks dataset, train/dev split, loss, layer fusion, dimensions and local
+feature contract before copying the pooled path. By default those copied
+parameters are frozen. The residual gate starts at zero, so before optimization
+the track model must reproduce the pooled logits exactly, element for element.
+
+The matched screening protocol is:
+
+- EasyCom Session 4--11: training;
+- Session 12: development/model selection;
+- Session 1--3: final test, run only after history/gap and the method are frozen;
+- 4 epochs, three seeds, identical optimizer/loss for pooled and track runs;
+- 16 observations at 2 FPS (up to 8 seconds), 1-second maximum link gap.
+
+This protocol intentionally re-runs the pooled base. Historical no-dev results
+cannot be used as the matched control.
+
+## Run
+
+After pulling on the L20 machine, run the focused CPU regression tests first:
+
+```bash
+python -m unittest \
+  test_local_turn_features.LocalTests.test_track_loader_builds_causal_histories_and_resets_tracks \
+  test_local_turn_features.LocalTests.test_track_gru_residual_starts_from_exact_pooled_logits \
+  test_local_turn_features.RoundTripTests.test_train_infer_easycom_and_ego4d
+```
+
+One-seed runtime smoke on an L20:
+
+```bash
+EPOCHS=1 \
+SESSIONS="4 5" \
+DEV_SESSIONS=5 \
+SCREEN_SESSIONS=5 \
+CHECKPOINT_ROOT=checkpoints/easycom_track_gru_smoke \
+RESULT_ROOT=results/easycom_track_gru_smoke \
+CUDA_VISIBLE_DEVICES=0 \
+bash run_easycom_track_gru_residual.sh all 0
+```
+
+Then use three GPUs in three foreground terminals. These runs report Session
+12 development scores, not the official test set:
+
+```bash
+SUMMARIZE=0 CUDA_VISIBLE_DEVICES=0 bash run_easycom_track_gru_residual.sh all 0
+SUMMARIZE=0 CUDA_VISIBLE_DEVICES=1 bash run_easycom_track_gru_residual.sh all 1
+SUMMARIZE=0 CUDA_VISIBLE_DEVICES=2 bash run_easycom_track_gru_residual.sh all 2
+```
+
+After all development jobs finish:
+
+```bash
+python summarize_local_baselines.py \
+  --root results/easycom_track_gru_residual \
+  --dataset easycom \
+  --protocol causal_valid_only \
+  --paired-deltas
+```
+
+Only after choosing and freezing the method, evaluate the saved checkpoints
+once on Sessions 1--3:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 bash run_easycom_track_gru_residual.sh test 0 1 2
+```
+
+This writes and summarizes `results/easycom_track_gru_residual_test` without
+retraining. A single GPU is sufficient because head inference is inexpensive.
+
+Paths and hyperparameters can be overridden with the environment variables at
+the top of `run_easycom_track_gru_residual.sh`. The launcher runs in the
+foreground and refuses to overwrite an existing checkpoint or result.
+
+## Interpretation and stopping rule
+
+The initial screen answers only whether track-temporal visual structure can
+improve the dense EgoSpeak-compatible metric. Advance this branch only if
+same-seed `track_gru - pooled` deltas are directionally consistent on Session
+12 without sacrificing short-horizon wearer AP. The official test is a final
+confirmation, not another model-selection stage.
+Because prior F1 variation was roughly a few tenths of a percentage point,
+tiny single-seed changes are not evidence.
+
+If the track branch wins, next run correct-track versus broken-track,
+past-shifted and no-ID temporal controls, followed by person-conditioned audio
+grounding. If it does not win, first compare 8/16/32 observations and optionally
+unfreeze the pooled base using Session 12 only; do not add deeper cross-attention
+merely to rescue a null result.
+
+The paper-level claim still requires the frozen competing-risk event evaluator
+and matched-recall false-entry analysis. Dense AP/mAP improvement alone is a
+representation result, not proof that the model understands multi-person floor
+behavior.