icassp2027-experiment/manuscript/main/REVIEWER_CHECKLIST.md

ICASSP 2027 completion checklist

This is an internal execution checklist. It is not part of the submitted PDF.

Reviewer verdict and frozen paper shape

The potentially publishable question is: can a general-purpose Omni readout be adapted into 200 ms, wearer-relative floor control by explicitly preserving short-window causal audio and anonymous visible-person evidence?

The contribution is not “the first audio-visual multiparty turn model.” VAP, EgoSpeak, SICNet, and MuVAP already occupy adjacent territory. The defensible gap is continuous, causal floor-action projection for an invisible camera wearer, combined with a controlled diagnosis of global omni states and a behavior-aligned multi-other event protocol.

Current Qwen2.5-Omni-3B results are exploratory motivation only. They neither prove a limitation shared by Omni models nor establish track-consistent fusion as the final method. Qwen2.5-Omni-3B is primary; Qwen2.5-Omni-7B is an optional capacity control. MiniCPM-o remains outside the core method and main comparison; it is permitted only as a diagnostic streaming/cache ablation with a matched Qwen refresh-rate control.

The four-page paper has only three contributions: temporal--role adaptation mismatch, one causal local floor adapter, and causal behavioral evidence. LoRA, the oracle, and the event protocol are respectively a baseline, motivation, and validation mechanism.

P0 — Freeze the task before confirmatory adaptation runs

• Define the exact 5 Hz causal cutoff and physical target interval for each of the ten 0.2 s horizons.
• Freeze wearer-onset/yield and multi-other exchange definitions.
• Implement and unit-test competing-risk rules: a response candidate ends before the next other onset and a yield candidate before the next other offset; right-censored and invalid windows are counted explicitly.
• Audit the competing-risk implementation on real timelines and freeze the timeout/source-boundary convention before confirmatory reporting.
• Specify overlap, minimum silence, maximum response delay, duplicate detections, source discontinuities, track visibility, offscreen speakers, and whether target-of-speech strata are diagnostic only.
• Produce train/test event-count tables, duration, class prevalence, and visibility coverage before examining model scores.
• Mark all already viewed Sessions 1–3 results as exploratory. Lock the protocol and select thresholds, checkpoints, and hyperparameters using a training-side fold or cross-fitting over Sessions 4–12.

Gate: two independent code paths or a manual sample audit must agree on at least 100 randomly selected events. Publish the number of unique other IDs and multi-other opportunity minutes before the main sweep.

P1 — Audit the primary backbone interface

For Qwen2.5-Omni-3B, and for 7B only if used as the scaling control, record:

• exact model/code revision, processor, dtype, and hardware;
• native audio/video sampling and streaming/chunk semantics;
• exact causal hidden-state readout and dimensionality;
• context limit, cache/reset behavior, and source-boundary handling;
• a cutoff/impulse test proving no future frame or sample enters the state;
• P50/P95 latency, throughput, and peak memory.

Do not force an offline or future-conditioned feature into the common table. If a model cannot expose a comparable causal state, report that protocol as unsupported. Raw scores are not a pure architecture ranking; emphasize within-backbone deltas.

For the diagnostic streaming/cache ablation:

• compare Qwen full causal-prefix recomputation and cache reuse at the same 5-Hz cutoffs, recording numerical agreement, throughput, and memory;
• map the native MiniCPM-o one-second output to 5-Hz rows by causal forward-hold only, never by backfilling the state into earlier rows;
• run a Qwen 1-Hz update-and-hold control with the same physical chunk boundaries and report the within-Qwen 5-Hz versus 1-Hz delta;
• freeze initialization before the first completed chunk and reset every modality/language-model cache at session and source-gap boundaries;
• before admitting a MiniCPM-o 5-Hz cached-branch row, compare it against full causal-prefix recomputation at sampled cutoffs under a frozen cosine and maximum-absolute-error tolerance;
• run future-media intervention tests at every tested cutoff and record throughput, peak memory, feature dimensionality, and exact code revision.

If the 5-Hz cached branch fails equivalence or leakage tests, omit that row and report only the native 1-Hz causal-hold diagnostic. Do not present unmatched Qwen--MiniCPM raw scores as a pure backbone comparison.

P2 — Establish the compact adaptation matrix

Run three seeds for the primary backbone with the same data, physical cutoffs, target, prediction-head capacity, optimizer exposure, and evaluator:

1. Frozen global causal state.
2. Pre-registered PEFT/LoRA budget.
3. Global state plus recent local audio.
4. Global state plus pooled local audio and vision.

Add a parameter- and history-matched temporal local branch before attributing gain to audiovisual organization. Use the same local adapter for the optional 7B scaling row.

Gate: baseline standard deviations are known; the evaluator reproduces identical scores from saved predictions; parameter counts and training budgets are reported. A single failed LoRA recipe is evidence about that recipe, not proof that the model cannot adapt.

P3 — Train the no-ID local floor adapter

The primary method is a short-window causal encoder over recent mixed audio and per-frame anonymous visible-person sets. It uses no participant-ID embedding or cross-time track state. Compare it with:

1. Recent local audio only.
2. Global/full-frame visual pooling.
3. A parameter- and history-matched temporal branch.
4. Time-shifted local vision.
5. Removed local vision.
6. A same-rate control if the local path runs faster than 5 Hz.

Keep the dense EgoSpeak wearer/others target as the primary task. Target of speech is analysis-only and never model input. Use ``dual-rate'' only if the local input/update rate differs explicitly and defeats the same-rate control.

Unit-test variable person count, empty sets, padding, person-set permutation, source/gap resets, and strict causal cutoffs.

P4 — Consider track continuity only after the no-ID method wins

Only if a track-aware candidate then beats the no-ID method on training-side data, evaluate one frozen checkpoint with:

• correct person tracks;
• clip-consistent global renaming, which must be invariant;
• deliberately broken cross-time identity with unchanged per-frame crops;
• time-shifted tracks with unchanged audio;
• removed visual features;
• fixed-rate ID switch, box dropout, and fragmentation;
• visibility-conditioned subsets.

The separately trained no-ID model belongs in the main ablation table. Do not retrain test-time counterfactuals.

P5 — Statistics and evidence

• Three seeds, mean ± sample standard deviation.
• Paired cluster bootstrap by session/conversation block for 95% intervals.
• Operating points at wearer-onset recall 0.7, 0.8, and 0.9; headline 0.8.
• Fixed-horizon entry/INT event AP, recall-matched hard-negative opportunity FPR, subtype FPR, and original ten-horizon dense AP/mAP.
• Treat one-to-one event matching and signed latency as secondary; report them only after freezing the online threshold, persistence, refractory period, matching interval, and competing-risk rule.
• Error strata: visibility, number of visible people, overlap, offscreen activity, and transition proxy. Add track age/fragmentation only if a track-aware method is selected.
• Ego4D AVD coverage audit on the exact EgoSpeak split before claiming the same multi-other analysis.
• Name the causal box source and its limitations. Add separate provided-track/online-tracker rows only if tracking enters the selected method.

Qualitative cases follow a frozen selection rule and include the largest paired improvement, a median case, a failure, and one preserved true wearer onset. Examples explain rates; they never replace them.

Paper assembly order

1. Freeze and audit the event protocol.
2. Audit the Qwen2.5-Omni-3B readout and matched PEFT baseline.
3. Complete the frozen/PEFT/audio/matched-temporal/local-AV matrix.
4. Run correct/time-shift/remove-vision controls and the optional 7B scaling row.
5. Run the Qwen refresh-rate control and the scoped MiniCPM-o streaming/cache diagnostic; admit the 5-Hz MiniCPM-o row only after equivalence and leakage audits.
6. Consider track continuity only if the no-ID method already passes.
7. Produce one confirmatory event table and one mechanism-ablation table.
8. Draw one compact final system figure; remove rejected methods from the main narrative.
9. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.

Stop/go decisions

• Claim boundary: with Qwen-only evidence, describe a Qwen2.5-Omni case study and the tested adaptation mechanism; do not generalize to all Omni models.
• Streaming boundary: call native MiniCPM-o 1-Hz plus causal hold a held-state baseline, not native 5-Hz inference; a 5-Hz cache result exists only after cached/full-prefix equivalence and future-input invariance pass.
• Audio-local reframe: local audio improves but all visual variants fail; claim a fine-grained acoustic readout bottleneck, not social visual reasoning.
• Track-aware method: correct continuity beats pooled vision, temporal capacity, no-ID/reset, broken continuity, and time-shifted controls at matched recall without worse onset/yield behavior.
• Stop the adapter: aggregate mAP rises but multi-other false entries do not fall, or legitimate onset/yield behavior degrades.