Add low-drift LoRA training controls

README.md

@@ -1077,6 +1077,83 @@ process uses its `LOCAL_RANK`.  DDP intentionally rejects
 `--online-current-tail`; the online producer-consumer path remains a separate
 benchmark.
 
+### Dense low-drift LoRA probe
+
+The first follow-up to the sparse-LoRA regression should remove sparse
+supervision as a variable and constrain adaptation near the Thinker output.
+Use dense 5-Hz caches, reserve `Session_12` as development data, and train on
+`Session_4` through `Session_11`.  `--lora-last-n-layers 8` discovers the
+language-layer indices and stores PEFT `layers_to_transform=[28,...,35]` for
+the 36-layer Qwen2.5-Omni-3B Thinker.  Combining it with
+`--lora-target-modules q_proj,v_proj,o_proj` reduces the adapter from all 252
+attention/MLP projections to 24 output-side attention projections.
+
+Start with the successful frozen 8-second head fixed.  `--freeze-head`
+requires `--head-checkpoint`, excludes the head from DDP and AdamW, but still
+backpropagates through it into the Thinker LoRA.  This isolates representation
+adaptation from head decision-boundary drift.  Do not continue from a regressed
+adapter: create this adapter from the original Thinker.
+
+For four GPUs with global batch 16, compute exactly one epoch from the dense
+manifest anchor counts and run:
+
+```bash
+cd /data1/yfzx/wangx/Code/icassp2027-experiment
+
+MAX_STEPS=$(python - <<'PY'
+import json
+import math
+from pathlib import Path
+
+root = Path("/data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s")
+sessions = (4, 5, 6, 7, 8, 9, 10, 11)
+examples = sum(len(json.loads((root / f"Session_{s}" / "manifest.json").read_text())["anchors"])
+               for s in sessions)
+print(math.ceil(examples / 16))
+PY
+)
+WARMUP_STEPS=$(( (MAX_STEPS + 19) / 20 ))
+
+CUDA_VISIBLE_DEVICES=0,1,2,3 \
+OMP_NUM_THREADS=2 \
+MKL_NUM_THREADS=2 \
+OPENBLAS_NUM_THREADS=2 \
+PYTORCH_ALLOC_CONF=expandable_segments:True \
+torchrun --standalone --nproc_per_node=4 train_omni_turn_lora.py \
+  --model-id /data1/yfzx/wangx/Models/Qwen2.5-Omni-3B \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_4 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_5 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_6 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_7 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_8 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_9 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_10 \
+  --cache-dir /data1/yfzx/wangx/Datasets/easycom/omni_lora_cache_dense_8s/Session_11 \
+  --head-checkpoint checkpoints/easycom_open_stream_2fps_8s_head_seed0.pth \
+  --freeze-head \
+  --lora-target-modules q_proj,v_proj,o_proj \
+  --lora-last-n-layers 8 \
+  --lora-rank 8 \
+  --lora-alpha 16 \
+  --learning-rate 5e-6 \
+  --output-dir checkpoints/omni_turn_lora_dense_8s_last8_qvo_frozen_head_epoch1 \
+  --max-steps "${MAX_STEPS}" \
+  --batch-size 2 \
+  --gradient-accumulation-steps 2 \
+  --num-workers 0 \
+  --block-cache-size 32 \
+  --warmup-steps "${WARMUP_STEPS}" \
+  --dtype bfloat16 \
+  --flash-attention-2 \
+  --log-every 20
+```
+
+Select this recipe on dense `Session_12` using 0.2-second mAP as the primary
+metric and mean mAP as the tie-breaker.  Only after the recipe is fixed should
+it be evaluated on official test `Sessions_1--3`.  A second controlled run may
+omit `--freeze-head` and use `--head-learning-rate 1e-5`; all other settings
+must remain identical.
+
 ### Frozen-Thinker sparse head-only control
 
 To determine whether the regression comes from transition-sparse sampling or