oracle: add fixed four-epoch final protocol

This commit is contained in:
Xin Wang
2026-09-10 09:53:33 +08:00
parent c6885d1aa5
commit 0abefd6f77
3 changed files with 87 additions and 15 deletions

View File

@@ -71,17 +71,39 @@ Sessions 1--3.
If preparation reports any transcript/target mismatch, stop and inspect it;
the preparation command has no mismatch-tolerance option.
## Locked four-epoch train-all protocol
The final protocol fixes the training budget at four epochs, trains on all
official EasyCom training sessions (Sessions 4--12), performs no development
checkpoint selection, and evaluates Sessions 1--3 once:
```bash
CUDA_VISIBLE_DEVICES=0 bash run_easycom_oracle_information.sh final 0 1 2
```
The `final` stage enforces `EPOCHS=4` and exactly Sessions 4--12, clears the
development-session list internally, and saves the last epoch. Its artifacts
use separate `_train4_12_fixed4` checkpoint/result roots by default, so they
cannot overwrite the historical Session-12 development screen. Override those
roots only with `FINAL_CHECKPOINT_ROOT` and `FINAL_RESULT_ROOT`.
mAP itself does not require threshold calibration. Nevertheless, the official
test results must not be used to choose a variant, epoch, loss, fusion design,
or any later hyperparameter. Run `final` only after those decisions are frozen;
repeatedly inspecting Sessions 1--3 and modifying the method would contaminate
the official test split.
## Follow-up
If `per-person` is consistently better on Session 12, add seeds 1 and 2 using
the already prepared sidecars:
To reproduce the historical Session-12 development screen for seeds 1 and 2,
use the already prepared sidecars:
```bash
CUDA_VISIBLE_DEVICES=0 bash run_easycom_oracle_information.sh screen 1 2
```
Only after freezing the method and event evaluator should selected variants be
run once on Sessions 1--3:
The older two-stage equivalent of final test inference is retained for
reproducibility, but the locked protocol above is preferred for new final runs:
```bash
VARIANTS="audio aggregate-visible per-person shuffled-id delayed" \