docs: add streaming cache ablation to manuscript

This commit is contained in:
Xin Wang
2026-09-14 21:43:33 +08:00
parent 7d9db4af58
commit ce5958036f
3 changed files with 90 additions and 15 deletions

View File

@@ -47,7 +47,10 @@ Its single candidate contribution is a causal local floor-evidence adapter:
- the primary local method uses an anonymous, no-ID visible-person set;
- track continuity is conditional and must defeat no-ID/shuffled/capacity
controls before entering the paper;
- MiniCPM-o and a cross-family sweep are removed from the core plan;
- MiniCPM-o remains outside the core method and main comparison; it appears
only as a scoped streaming/cache diagnostic with a matched Qwen 1-Hz
causal-hold control, and its 5-Hz cache row requires equivalence and leakage
audits before reporting;
- the oracle is analysis-only motivation, not a deployable method or headline;
- the publication gate is lower competing-risk-censored multi-other false
entry at matched wearer-onset recall without worse valid onset or yield.

View File

@@ -17,7 +17,9 @@ behavior-aligned multi-other event protocol.
Current Qwen2.5-Omni-3B results are exploratory motivation only. They neither
prove a limitation shared by Omni models nor establish track-consistent fusion
as the final method. Qwen2.5-Omni-3B is primary; Qwen2.5-Omni-7B is an optional
capacity control. MiniCPM-o and a cross-family sweep are outside the core paper.
capacity control. MiniCPM-o remains outside the core method and main comparison;
it is permitted only as a diagnostic streaming/cache ablation with a matched
Qwen refresh-rate control.
The four-page paper has only three contributions: temporal--role adaptation
mismatch, one causal local floor adapter, and causal behavioral evidence.
@@ -63,6 +65,26 @@ a model cannot expose a comparable causal state, report that protocol as
unsupported. Raw scores are not a pure architecture ranking; emphasize
within-backbone deltas.
For the diagnostic streaming/cache ablation:
- [ ] compare Qwen full causal-prefix recomputation and cache reuse at the
same 5-Hz cutoffs, recording numerical agreement, throughput, and memory;
- [ ] map the native MiniCPM-o one-second output to 5-Hz rows by causal
forward-hold only, never by backfilling the state into earlier rows;
- [ ] run a Qwen 1-Hz update-and-hold control with the same physical chunk
boundaries and report the within-Qwen 5-Hz versus 1-Hz delta;
- [ ] freeze initialization before the first completed chunk and reset every
modality/language-model cache at session and source-gap boundaries;
- [ ] before admitting a MiniCPM-o 5-Hz cached-branch row, compare it against
full causal-prefix recomputation at sampled cutoffs under a frozen cosine and
maximum-absolute-error tolerance;
- [ ] run future-media intervention tests at every tested cutoff and record
throughput, peak memory, feature dimensionality, and exact code revision.
If the 5-Hz cached branch fails equivalence or leakage tests, omit that row and
report only the native 1-Hz causal-hold diagnostic. Do not present unmatched
Qwen--MiniCPM raw scores as a pure backbone comparison.
## P2 — Establish the compact adaptation matrix
Run three seeds for the primary backbone with the same data, physical cutoffs,
@@ -144,17 +166,23 @@ Examples explain rates; they never replace them.
2. Audit the Qwen2.5-Omni-3B readout and matched PEFT baseline.
3. Complete the frozen/PEFT/audio/matched-temporal/local-AV matrix.
4. Run correct/time-shift/remove-vision controls and the optional 7B scaling row.
5. Consider track continuity only if the no-ID method already passes.
6. Produce one confirmatory event table and one mechanism-ablation table.
7. Draw one compact final system figure; remove rejected methods from the main
5. Run the Qwen refresh-rate control and the scoped MiniCPM-o streaming/cache
diagnostic; admit the 5-Hz MiniCPM-o row only after equivalence and leakage
audits.
6. Consider track continuity only if the no-ID method already passes.
7. Produce one confirmatory event table and one mechanism-ablation table.
8. Draw one compact final system figure; remove rejected methods from the main
narrative.
8. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.
9. Replace every visible TODO and verify the ICASSP 2027 kit/page limit.
## Stop/go decisions
- **Claim boundary:** with Qwen-only evidence, describe a Qwen2.5-Omni case
study and the tested adaptation mechanism; do not generalize to all Omni
models.
- **Streaming boundary:** call native MiniCPM-o 1-Hz plus causal hold a held-state
baseline, not native 5-Hz inference; a 5-Hz cache result exists only after
cached/full-prefix equivalence and future-input invariance pass.
- **Audio-local reframe:** local audio improves but all visual variants fail;
claim a fine-grained acoustic readout bottleneck, not social visual reasoning.
- **Track-aware method:** correct continuity beats pooled vision, temporal

View File

@@ -157,8 +157,9 @@ is not interpreted as a general limitation of LoRA or all Omni models.
\todo{Freeze the exact Qwen2.5-Omni-3B revision, processor, input rates,
streaming/cache semantics, readout layer, context, dtype, LoRA targets/rank,
trainable parameter budget, and cutoff audit. Add Qwen2.5-Omni-7B only as a
capacity-scaling control. Do not add another backbone family unless it is
needed to support and experimentally earns a cross-family claim.}
capacity-scaling control. MiniCPM-o 4.5 is restricted to the diagnostic
streaming/cache ablation below and does not enter the core method or support a
cross-family superiority claim.}
\subsection{Causal local floor adapter}
@@ -267,6 +268,17 @@ control. All learned variants use identical causal examples, prediction
horizons, prediction heads, and the class-balanced binary cross-entropy
objective in Eq.~\eqref{eq:loss}.
The diagnostic streaming ablation additionally uses MiniCPM-o 4.5
\cite{cui2026minicpmo45}. Its native cached stream is evaluated at its native
one-second update interval: a state becomes available to the 200-ms row whose
cutoff equals the chunk end and is then causally held until the next update. A
Qwen row subjected to the same 1-Hz update-and-hold schedule separates
refresh-rate effects from backbone effects. A MiniCPM-o
5-Hz cache row is admissible only if a cached branch at every cutoff agrees
with full causal-prefix recomputation within a frozen numerical tolerance and
passes future-input intervention tests. Thus the native 1-Hz condition is a
strictly causal held-state baseline, not a claim of native 5-Hz refresh.
Model selection is performed using training-side data only. Event operating
points are selected from session-level out-of-fold predictions over
Sessions~4--12; the final model is then retrained on all nine sessions before
@@ -392,27 +404,40 @@ numbers from the four-page version.}
\subsection{Mechanism Analysis}
To determine whether the gain arises from correctly aligned visual floor
evidence rather than additional parameters or generic visual context, we
evaluate the controls in Table~\ref{tab:ablation}.
evidence rather than additional parameters or generic visual context, and to
separate streaming refresh from representation family, we evaluate the
controls in Table~\ref{tab:ablation}.
\begin{table}[t]
\caption{Mechanism controls for the selected Omni-3B local adapter. All
variants use the same prediction head and comparable temporal capacity; FPRs
are measured at matched recall $R=0.8$.}
\caption{Mechanism and streaming/cache controls. The streaming panel uses
frozen global states and capacity-matched downstream heads; all rows share
targets and evaluation. FPRs are measured at matched recall $R=0.8$.
Cross-family raw scores are diagnostic.}
\label{tab:ablation}
\centering
\small
\resizebox{\columnwidth}{!}{%
\begin{tabular}{lcccc}
\hline
Local condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
Condition & Mean mAP $\uparrow$ & EOT FPR $\downarrow$
& O$\rightarrow$O FPR $\downarrow$ & INT FPR $\downarrow$\\
\hline
\multicolumn{5}{l}{\emph{Local-evidence controls (Qwen2.5-Omni-3B)}}\\
Remove vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
Time-shift vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
Full-frame vision & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
Correct local AV & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
\hline
\multicolumn{5}{l}{\emph{Streaming/cache controls}}\\
Qwen, full causal prefix at 5 Hz (no cache)
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
Qwen, cached 5-Hz update & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
Qwen, cached 1-Hz + causal hold & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
MiniCPM-o, native cached 1-Hz + causal hold
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
MiniCPM-o, audited cached 5-Hz branch
& \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
\hline
\end{tabular}}
\end{table}
@@ -430,7 +455,7 @@ evidence-routing claim. If dense mAP improves but event behavior does not, do
not claim improved floor control. If legitimate onset or yield degrades, do
not describe lower false entry as success.}
\subsection{Scaling Control}
\subsection{Scaling and Streaming Controls}
Increasing the frozen Omni backbone from 3B to 7B yields
\todo{$X$ points} in 0.2-s mAP and \todo{EOT/INT FPR of $Y/Z$} at matched
@@ -443,6 +468,25 @@ provides a complementary benefit to backbone scaling}.
Because both backbones belong to the Qwen2.5-Omni family, we do not claim that
this conclusion generalizes to all omni-modal models.
The streaming rows answer two different questions. First, full causal-prefix
recomputation and streaming cache reuse at the same 5-Hz Qwen cutoffs test
whether cache/chunk semantics change the representation; equivalent outputs
make this an efficiency control rather than a distinct predictive method.
Second, the matched Qwen 5-Hz and 1-Hz cached rows test whether an advantage is
explained by more frequent causal state refresh rather than the proposed local
evidence route. For native 1-Hz streams, the state emitted at the end of each
one-second chunk is never backfilled into earlier 200-ms rows; source gaps and
session boundaries reset all modality and language-model caches. MiniCPM-o is
a secondary cross-family diagnostic under the same 1-Hz schedule, not a
same-frequency architecture ranking. Its 5-Hz cached-branch row is omitted if
cached/full-prefix equivalence, cutoff invariance, or reset audits fail.
\todo{Freeze native chunk boundaries, initialization of rows before the first
complete chunk, cache-reset rules, the cached/full-prefix tolerance, and the
future-media intervention test. Report within-backbone 5-Hz versus 1-Hz
deltas plus cache/no-cache throughput and memory; do not interpret unmatched
Qwen--MiniCPM raw scores as a pure backbone effect.}
\subsection{Limitations and Qualitative Analysis}
The results support \todo{precise supported claim}, but several limitations