Add dense-cache evaluation for Omni LoRA

README.md

@@ -36,6 +36,7 @@ classification head.
 | `TurnPredictionHead` | Trained and evaluated | Map `[2048]` to `[10, 3]` logits |
 | `train_turn_head.py` | Verified on EasyCom | Train the frozen-Omni anticipation head with the official EasyCom split |
 | `infer_turn_head.py` | Verified on EasyCom | Save EgoSpeak-format predictions and reproduce per-horizon AP/mAP |
+| `eval_omni_turn_lora.py` | Implemented, remote validation pending | Evaluate Thinker LoRA from dense AV caches with the same AP/mAP protocol |
 
 The full Thinker input path, embedded audio, direct video decoding,
 FlashAttention-2, 5 Hz streaming extraction, cache replay, target alignment,
@@ -1030,6 +1031,47 @@ for example, change 8 GPUs with batch 2 and accumulation 1 to 4 GPUs with batch
 post-hoc continuation because a one-epoch cosine schedule has already reached
 zero.
 
+### Evaluate a Thinker-LoRA checkpoint
+
+Use `eval_omni_turn_lora.py` for the formal EasyCom Table 2 evaluation.  It
+loads the trained Thinker LoRA and `10 x 3` turn head, reconstructs every causal
+state from dense AV caches, and reports both the EgoSpeak full-timeline and
+strict causal-valid protocols.  Sparse training caches are rejected because
+they do not contain predictions at every 200 ms timestamp.
+
+The following command evaluates the official EasyCom test sessions on three
+GPUs, with one complete session assigned to each rank:
+
+```bash
+CHECKPOINT=/data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/omni_turn_lora_easycom_sparse_8s_ddp8_epoch1
+
+CUDA_VISIBLE_DEVICES=0,1,2 \
+OMP_NUM_THREADS=1 \
+MKL_NUM_THREADS=1 \
+OPENBLAS_NUM_THREADS=1 \
+PYTORCH_ALLOC_CONF=expandable_segments:True \
+TORCH_NCCL_ASYNC_ERROR_HANDLING=1 \
+torchrun --standalone --nproc_per_node=3 eval_omni_turn_lora.py \
+  --model-id /data1/yfzx/wangx/Models/Qwen2.5-Omni-3B \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_3 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_2 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_1 \
+  --adapter-checkpoint "${CHECKPOINT}/adapter" \
+  --head-checkpoint "${CHECKPOINT}/turn_head.pt" \
+  --output-dir results/omni_turn_lora_easycom_sparse_8s_ddp8_epoch1 \
+  --batch-size 2 \
+  --block-cache-size 32 \
+  --dtype bfloat16 \
+  --flash-attention-2 \
+  --ddp-timeout-minutes 60
+```
+
+The evaluator saves per-session probabilities and valid masks under `pred/`
+and `valid/`, and writes the aggregate metrics to `metrics.json`.  Add
+`--overwrite` only when intentionally replacing an existing result directory.
+Treat Sessions 1, 2, and 3 as the official test set: inspect their metrics only
+after the training recipe and hyperparameters have been fixed.
+
 For a speed-only alternative, `train_omni_turn_lora.py` supports a two-GPU
 producer-consumer benchmark.  The producer GPU runs the frozen AV towers and
 places current-tail embeddings in a bounded CPU queue while the consumer GPU