Add transition-sparse Omni AV caching

README.md

@@ -903,17 +903,20 @@ Recommended experimental procedure:
 Class-balanced/focal loss, a learned turn-query token, and LoRA are later
 ablations. They should not replace the frozen-Omni linear/MLP-head baseline.
 
-## Planned Thinker-LoRA sampling and cache policy
+## Thinker-LoRA sampling and cache policy
 
-This is a planned second-stage experiment, not a reported benchmark result.
+This is an implemented second-stage experiment, not yet a reported benchmark
+result.
 Dense 5 Hz partial-prefix caching is fast during training but does not fit the
 combined EasyCom and Ego4D storage budget.  The 24-second Session 7 pilot used
-0.115 GiB, which extrapolates to about 17.25 GiB per media hour.  Caching only
-complete 2-second history blocks and recomputing every current partial online
-reduced storage, but the single-L20 pilot slowed 200 LoRA updates from 83 s
-(2.39 update/s) to 254.085 s (0.787 update/s).  Online AV preparation used
-98.654 s for 163 partials and peak allocation was 9.56 GiB, so the bottleneck
-is repeated preprocessing/modality encoding rather than memory capacity.
+0.115 GiB, which extrapolates to about 17.25 GiB per media hour.  In the latest
+controlled single-L20 test without gradient checkpointing, 200 updates took
+57.718 s (3.465 update/s) with every partial cached, 130.151 s (1.537
+update/s) with current partials recomputed using processor defaults, and
+111.146 s (1.799 update/s) after CUDA log-Mel and bounded-audio-padding
+optimization.  The optimized online run spent 53.408 s in 163 frozen-AV calls;
+the cached run therefore remains the preferred training path when storage is
+available.
 
 The proposed training sampler is therefore:
 
@@ -934,6 +937,39 @@ The paper must report the sampling rule and compare uniform-only sampling with
 transition-enhanced sampling so that any gain is not attributed solely to
 additional transition examples.
 
+`cache_omni_av_pieces.py --anchor-policy transition-sparse` implements this
+policy without changing the trainer cache schema.  A transition is any change
+in the three-class EasyCom target vector.  The default two-second dense band is
+defined explicitly as one second before through one second after that target
+row.  Every complete two-second source block is still cached once as reusable
+history.  Ordinary selected anchors and transition-band anchors cache only
+their exact current causal phase.  Label information selects training examples
+offline but never changes the media endpoint supplied to Qwen.
+
+Inspect the selection counts for the 24-second pilot without loading media or
+the model:
+
+```bash
+python cache_omni_av_pieces.py \
+  --model-id /data1/yfzx/wangx/Models/Qwen2.5-Omni-3B \
+  --easycom-session-dir /data1/yfzx/wangx/Datasets/easycom/extracted_easycom/Main/Video_Compressed/Session_7 \
+  --target /data1/yfzx/wangx/Datasets/easycom/easycom_target_frame/Session_7.npy \
+  --output-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_sparse_8s/Session_7_24s \
+  --end 24 \
+  --anchor-policy transition-sparse \
+  --ordinary-anchor-hz 1 \
+  --transition-before-seconds 1 \
+  --transition-after-seconds 1 \
+  --selection-only
+```
+
+After auditing the counts, remove `--selection-only` and add the normal AV
+extraction options.  Use `--end 0` only after the 24-second cache and replay
+verification pass; zero selects the complete target timeline.  The resulting
+manifest contains only selected training anchors, so `train_omni_turn_lora.py`
+uses the sparse dataset automatically and must be run without
+`--online-current-tail`.
+
 For a speed-only alternative, `train_omni_turn_lora.py` supports a two-GPU
 producer-consumer benchmark.  The producer GPU runs the frozen AV towers and
 places current-tail embeddings in a bounded CPU queue while the consumer GPU