Add Ego4D AV assisted annotation review workflow

docs/ego4d_av_llm_annotation_review.md

@@ -0,0 +1,136 @@
+# Ego4D AV LLM-assisted floor-event annotation
+
+This workflow mirrors the EasyCom boundary-review pipeline while using the
+official Ego4D AV annotation schema. It creates semantic review data for
+wearer floor entry and interruption-driven yield; it does not modify Ego4D's
+voice activity annotations and does not turn LLM output into ground truth.
+
+## Data and claim boundary
+
+Expected dataset layout:
+
+```text
+DATA_ROOT/
+  v1/
+    annotations/av_train.json
+    annotations/av_val.json
+    clips/<clip_uid>.mp4
+```
+
+The generator reads clip-local `voice_segments`, `transcriptions`, and
+`social_segments_talking.is_at_me`. The social label is annotation-only
+evidence. It must never be passed to a forecasting model or used after a
+strictly causal prediction cutoff. Ego4D's voice annotation guideline joins
+sounds separated by less than one second, so a stored segment boundary is an
+activity boundary, not automatically a semantic turn boundary.
+
+The output split remains the split declared by Ego4D (`train` or `val`). Do not
+rename validation data to test or tune annotation rules after inspecting the
+final evaluation set.
+
+## 1. Generate deterministic candidates
+
+Start with one clip and inspect the JSONL before running the full set:
+
+```bash
+python3 prepare_ego4d_av_eot_int_review.py \
+  --annotations "$DATA_ROOT/v1/annotations/av_train.json" \
+  --output reports/ego4d_av_event_candidates/ego4d_av_train_candidates.jsonl \
+  --limit 1
+```
+
+Then generate the selected splits:
+
+```bash
+python3 prepare_ego4d_av_eot_int_review.py \
+  --annotations \
+    "$DATA_ROOT/v1/annotations/av_train.json" \
+    "$DATA_ROOT/v1/annotations/av_val.json" \
+  --output reports/ego4d_av_event_candidates/ego4d_av_candidates.jsonl
+```
+
+The command also writes `<output_stem>_timelines.json` beside the JSONL. Each
+raw wearer onset and offset is retained. Same-other pauses and other-to-other
+handoffs are added as independently reviewable hard-negative proposals.
+Timing rules only select and describe candidates; they never assign semantic
+positive labels.
+
+## 2. Inspect what will be uploaded
+
+The LLM client refuses to upload transcript excerpts without an explicit
+opt-in. First inspect the exact prompt locally:
+
+```bash
+python3 adjudicate_ego4d_av_eot_int_candidates.py \
+  --input reports/ego4d_av_event_candidates/ego4d_av_candidates.jsonl \
+  --output reports/ego4d_av_event_candidates/ego4d_av_candidates_ai.jsonl \
+  --max-events 8 \
+  --dry-run
+```
+
+The prompt uses only timestamps, participant roles, provided transcripts, and
+provided `is_at_me` facts. It does not upload video or audio.
+
+## 3. Run LLM pre-labeling
+
+After confirming that external transcript upload is authorized:
+
+```bash
+export DEEPSEEK_API_KEY=...
+python3 adjudicate_ego4d_av_eot_int_candidates.py \
+  --input reports/ego4d_av_event_candidates/ego4d_av_candidates.jsonl \
+  --output reports/ego4d_av_event_candidates/ego4d_av_candidates_ai.jsonl \
+  --passes 2 \
+  --batch-size 8 \
+  --allow-external-transcript-upload
+```
+
+The client checkpoints after every successful batch. Two passes must agree;
+disagreement becomes `EXCLUDE/INSUFFICIENT_EVIDENCE`. Malformed singleton
+responses are written to a pending-human JSONL rather than silently dropped.
+Every successful LLM decision has `needs_human_review=true`.
+
+## 4. Review in the browser
+
+```bash
+python3 review_ego4d_av_events.py \
+  --events reports/ego4d_av_event_candidates/ego4d_av_candidates_ai.jsonl \
+  --dataset-root "$DATA_ROOT" \
+  --host 127.0.0.1 \
+  --port 8765
+```
+
+Open `http://127.0.0.1:8765`. The UI shows the local video, clip-level speaker
+timeline, participant-aligned transcripts, LLM rationale, and the closed human
+label taxonomy. Reviews are atomically saved to
+`reports/ego4d_av_event_reviews/reviews.jsonl` with a CSV companion and can be
+resumed. The server rejects media paths outside `DATA_ROOT` and supports byte
+ranges for browser seeking.
+
+For an SSH host, keep the service on loopback and forward the port:
+
+```bash
+ssh -L 8765:127.0.0.1:8765 USER@HOST
+```
+
+## 5. Freeze and audit
+
+Before using these annotations in a paper or evaluator:
+
+1. Freeze candidate windows and the closed taxonomy on training data.
+2. Double-review a random sample plus every low-confidence/disagreement item.
+3. Report raw candidates, positives, hard negatives, exclusions, and exclusion
+   reasons separately for train and validation.
+4. Print and independently inspect example timelines for entry, yield,
+   backchannel, same-speaker pause, and other-to-other cases.
+5. Keep the final label named human-adjudicated semantic annotation. LLM output
+   alone is never ground truth.
+
+## Tests
+
+```bash
+python3 -m unittest -v \
+  test_prepare_ego4d_av_eot_int_review.py \
+  test_review_floor_events.py \
+  test_adjudicate_easycom_eot_int_candidates.py
+```