icassp2027-experiment/manuscript/main/REVIEWER_CHECKLIST.md

ICASSP 2027 completion checklist

This is an internal execution checklist. It is not part of the submitted PDF.

Reviewer verdict on the current draft

The potentially publishable question is now model-agnostic: do general-purpose omni representations expose the fine-grained evidence needed for 200 ms egocentric floor action, and can causal local evidence reduce false wearer entries at matched onset recall when frozen readout or PEFT is insufficient?

The contribution is not “the first audio-visual multiparty turn model.” VAP, EgoSpeak, SICNet, and MuVAP already occupy adjacent territory. The defensible gap is continuous, causal floor-action projection for an invisible camera wearer, combined with a controlled diagnosis of global omni states and a behavior-aligned multi-other event protocol.

Current Qwen2.5-Omni-3B results are exploratory motivation only. They neither prove a limitation shared by omni models nor establish track-consistent fusion as the final method. The planned initial backbone set is Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, and MiniCPM-o 4.5.

P0 — Freeze the task before the backbone sweep

• Define the exact 5 Hz causal cutoff and physical target interval for each of the ten 0.2 s horizons.
• Freeze wearer-onset/yield and multi-other exchange definitions.
• Add competing-risk rules: a candidate event ends at the first matched wearer action, incompatible floor change, timeout, or invalid source boundary.
• Specify overlap, minimum silence, maximum response delay, duplicate detections, source discontinuities, track visibility, offscreen speakers, and whether target-of-speech strata are diagnostic only.
• Produce train/test event-count tables, duration, class prevalence, and visibility coverage before examining model scores.
• Mark all already viewed Sessions 1–3 results as exploratory. Lock the protocol and select thresholds, checkpoints, and hyperparameters using a training-side fold or cross-fitting over Sessions 4–12.

Gate: two independent code paths or a manual sample audit must agree on at least 100 randomly selected events. Publish the number of unique other IDs and multi-other opportunity minutes before the main sweep.

P1 — Audit each backbone interface

For Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, and MiniCPM-o 4.5, record:

• exact model/code revision, processor, dtype, and hardware;
• native audio/video sampling and streaming/chunk semantics;
• exact causal hidden-state readout and dimensionality;
• context limit, cache/reset behavior, and source-boundary handling;
• a cutoff/impulse test proving no future frame or sample enters the state;
• P50/P95 latency, throughput, and peak memory.

Do not force an offline or future-conditioned feature into the common table. If a model cannot expose a comparable causal state, report that protocol as unsupported. Raw scores are not a pure architecture ranking; emphasize within-backbone deltas.

P2 — Establish the compact representation matrix

Run three seeds per supported backbone with the same data, physical cutoffs, target, prediction-head capacity, optimizer exposure, and evaluator:

1. Frozen global causal state.
2. Pre-registered PEFT/LoRA budget.
3. Global state plus recent local audio.
4. Global state plus pooled local audio and vision.

Use a common local encoder as the primary cross-backbone control. If native local encoders are tested, report them in a separate within-model panel. Do not mix common-encoder and native-encoder deltas in one claim.

Gate: baseline standard deviations are known; the evaluator reproduces identical scores from saved predictions; parameter counts and training budgets are reported. A single failed LoRA recipe is evidence about that recipe, not proof that the model cannot adapt.

P3 — Select local fusion only after P2

If local evidence gives a reliable event-level gain, compare candidates in increasing complexity:

1. Pooled local audio/vision.
2. Parameter-matched temporal branch.
3. Per-frame person set with no cross-time identity.
4. Shared track-wise temporal encoder.
5. Person–audio cross-attention.
6. Optional per-person active-speaker auxiliary loss, only after selecting a person-aware activity-only model.

Keep the dense EgoSpeak wearer/others target as the primary task. Target of speech is analysis-only and never model input. Describe only the train-side winner as the final method; move rejected candidates to ablations.

For a track-aware candidate, temporary IDs route hidden state only: no ID embedding, cross-recording identity, or fixed seating slot. Unit-test variable person count, empty sets, padding, slot permutation, global ID renaming, source/gap resets, and strict causal cutoffs.

P4 — Run visual counterfactuals conditionally

Only if a track-aware candidate is selected, evaluate one frozen checkpoint with:

• correct person tracks;
• clip-consistent global renaming, which must be invariant;
• deliberately broken cross-time identity with unchanged per-frame crops;
• time-shifted tracks with unchanged audio;
• removed visual features;
• fixed-rate ID switch, box dropout, and fragmentation;
• visibility-conditioned subsets.

The separately trained no-ID model belongs in the main ablation table. Do not retrain test-time counterfactuals.

P5 — Statistics and evidence

• Three seeds, mean ± sample standard deviation.
• Paired cluster bootstrap by session/conversation block for 95% intervals.
• Operating points at wearer-onset recall 0.7, 0.8, and 0.9; headline 0.8.
• False wearer entries per multi-other opportunity minute, onset latency, wearer-yield recall/latency, and original ten-horizon AP/mAP.
• Error strata: backbone, visibility, number of visible people, overlap, offscreen activity, track age/fragmentation, and transition proxy.
• Ego4D AVD coverage audit on the exact EgoSpeak split before claiming the same multi-other analysis.
• Provided-track and online-tracker conditions named separately.

Qualitative cases follow a frozen selection rule and include the largest paired improvement, a median case, a failure, and one preserved true wearer onset. Examples explain rates; they never replace them.

Paper assembly order

1. Freeze and audit the event protocol.
2. Implement and causally audit all backbone readouts.
3. Complete the compact frozen/PEFT/local matrix.
4. Decide whether the paper can support a cross-family or only model-specific representation claim.
5. Select a local fusion candidate using training-side event metrics.
6. Run conditional visual/track counterfactuals.
7. Produce the cross-backbone event table and paired intervals.
8. Draw one compact final system figure; remove rejected methods from the main narrative.
9. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.

Stop/go decisions

• General claim: at least two backbone families show the same frozen/PEFT weakness and a reliable local-evidence improvement.
• Model-specific claim: the effect appears only within Qwen; say so and do not generalize to all omni models.
• Audio-local reframe: local audio improves but all visual variants fail; claim a fine-grained acoustic readout bottleneck, not social visual reasoning.
• Track-aware method: correct continuity beats pooled vision, temporal capacity, no-ID/reset, broken continuity, and time-shifted controls at matched recall without worse onset/yield behavior.
• Stop the adapter: aggregate mAP rises but multi-other false entries do not fall, or legitimate onset/yield behavior degrades.