oracle: add fixed four-epoch final protocol
This commit is contained in:
@@ -71,17 +71,39 @@ Sessions 1--3.
|
||||
If preparation reports any transcript/target mismatch, stop and inspect it;
|
||||
the preparation command has no mismatch-tolerance option.
|
||||
|
||||
## Locked four-epoch train-all protocol
|
||||
|
||||
The final protocol fixes the training budget at four epochs, trains on all
|
||||
official EasyCom training sessions (Sessions 4--12), performs no development
|
||||
checkpoint selection, and evaluates Sessions 1--3 once:
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0 bash run_easycom_oracle_information.sh final 0 1 2
|
||||
```
|
||||
|
||||
The `final` stage enforces `EPOCHS=4` and exactly Sessions 4--12, clears the
|
||||
development-session list internally, and saves the last epoch. Its artifacts
|
||||
use separate `_train4_12_fixed4` checkpoint/result roots by default, so they
|
||||
cannot overwrite the historical Session-12 development screen. Override those
|
||||
roots only with `FINAL_CHECKPOINT_ROOT` and `FINAL_RESULT_ROOT`.
|
||||
|
||||
mAP itself does not require threshold calibration. Nevertheless, the official
|
||||
test results must not be used to choose a variant, epoch, loss, fusion design,
|
||||
or any later hyperparameter. Run `final` only after those decisions are frozen;
|
||||
repeatedly inspecting Sessions 1--3 and modifying the method would contaminate
|
||||
the official test split.
|
||||
|
||||
## Follow-up
|
||||
|
||||
If `per-person` is consistently better on Session 12, add seeds 1 and 2 using
|
||||
the already prepared sidecars:
|
||||
To reproduce the historical Session-12 development screen for seeds 1 and 2,
|
||||
use the already prepared sidecars:
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0 bash run_easycom_oracle_information.sh screen 1 2
|
||||
```
|
||||
|
||||
Only after freezing the method and event evaluator should selected variants be
|
||||
run once on Sessions 1--3:
|
||||
The older two-stage equivalent of final test inference is retained for
|
||||
reproducibility, but the locked protocol above is preferred for new final runs:
|
||||
|
||||
```bash
|
||||
VARIANTS="audio aggregate-visible per-person shuffled-id delayed" \
|
||||
|
||||
Reference in New Issue
Block a user