diff --git a/manuscript/targeted/Makefile b/manuscript/targeted/Makefile index e989da4..38bf904 100644 --- a/manuscript/targeted/Makefile +++ b/manuscript/targeted/Makefile @@ -2,15 +2,16 @@ LATEXMK ?= latexmk .PHONY: all clean figure -all: figures/system_overview.pdf +all: figures/system_overview_onecolumn.pdf figures/task_overview.pdf mkdir -p build $(LATEXMK) -pdf -interaction=nonstopmode -halt-on-error -outdir=build main.tex -figure: figures/system_overview.pdf +figure: figures/system_overview_onecolumn.pdf figures/task_overview.pdf -figures/system_overview.pdf: figures/system_overview.tex +figures/%.pdf: figures/%.tex $(LATEXMK) -pdf -interaction=nonstopmode -halt-on-error -outdir=figures $< clean: $(LATEXMK) -C -outdir=build main.tex - $(LATEXMK) -c -outdir=figures figures/system_overview.tex + $(LATEXMK) -c -outdir=figures figures/system_overview_onecolumn.tex + $(LATEXMK) -c -outdir=figures figures/task_overview.tex diff --git a/manuscript/targeted/README.md b/manuscript/targeted/README.md index f31115b..6544312 100644 --- a/manuscript/targeted/README.md +++ b/manuscript/targeted/README.md @@ -14,8 +14,21 @@ that is the strongest complete story supported by archived evidence: - a three-seed causal oracle shows that timely visible-participant activity is informative, while identity continuity is not currently necessary. +The draft also reflects two seed-0 exploratory screens reported in +`../main/main.tex`: no-box full-frame F3 (Table 1) and a global-only +chunk-construction comparison (Table 2, panel B). Neither supplies +matched visual-motion evidence, endpoint gains, or the pending T/C factorial. +The result directories cited by the main draft for F3 and its G+A recheck +are absent from this checkout, so their underlying logs have not been +independently audited here. + No semantic ENTRY/YIELD, addressee, online-detector, or real-time-throughput claim is made. The deployable matrix is explicitly single-seed and exploratory. +The six-group L/T/C/B/H-Q/S table remains in `main.tex` as an +**illustrative, unmeasured planning aid**, excluded from the page-limited +PDF. Its directional statements are hypotheses and falsifiers, not EasyCom +scores. Replace or remove every row after the corresponding experiment, +including results that contradict the working story. ## Build @@ -24,13 +37,25 @@ cd manuscript/targeted make ``` -The editable sources are `main.tex`, `references.bib`, and -`figures/system_overview.tex`. Generated files are written to `build/`. +The editable sources are `main.tex`, `references.bib`, +`figures/task_overview.tex`, and `figures/system_overview_onecolumn.tex`. +Generated LaTeX files are written to `build/`. -The build currently uses the repository's unmodified ICASSP 2026 style assets -as a compatibility preview. Replace the `template_dir` in `latexmkrc` with the -official ICASSP 2027 author kit before submission and repeat all page/layout -checks. +The preview is four technical pages plus a reference-only fifth page. Both +the task diagram (Fig. 1) and the system diagram (Fig. 2) occupy one column. +The current PDF reserves five numbered result tables: measured deployable +and oracle screens share Table 1; L controls extend its first panel. The +unmeasured T/C matrix and a separate measured chunk screen share Table 2; +B, H, and S occupy Tables 3--5. Pending cells are explicitly unmeasured. +Figure 3 is an H/Q case slot, not a selected or measured example. Its +cutoff-marked timelines must be inserted only after the slice protocol and +case-selection rule are frozen. The single-column system figure is generated +from figures/system_overview_onecolumn.tex; the earlier full-width source +is retained for comparison. + +The build uses the repository's ICASSP 2027 LaTeX assets in +`../ICASSP2027_Paper_Templates/`, selected by `latexmkrc`. The supplied +`spconf.sty` and `IEEEbib.bst` match the 2026 copies. ## Evidence sources diff --git a/manuscript/targeted/REVIEWER_REPORT.md b/manuscript/targeted/REVIEWER_REPORT.md index b434b1c..35d1fc5 100644 --- a/manuscript/targeted/REVIEWER_REPORT.md +++ b/manuscript/targeted/REVIEWER_REPORT.md @@ -78,10 +78,23 @@ the science. causal active-speaker/floor estimator and require correct-time evidence to beat delayed, removed, and capacity-matched controls. Otherwise retain the honest audio-local paper and remove visual-method language. -5. Rebuild with the official 2027 kit, keep technical content within four - pages, put only references on an optional fifth page, and retain the IEEE - AI-assisted drafting disclosure. +5. Recheck the four-page technical limit after inserting results. The current + build uses the 2027 kit and puts only references on page five; retain the + IEEE AI-assisted drafting disclosure. Passing gates 1--3 would support a solid ICASSP empirical paper. Passing gate 4 with a meaningful matched-recall behavior gain would support the stronger method paper originally envisioned. + +## Review 4: expanded five-page working preview + +The layout now uses four pages for technical content and a fifth page for +references only. The Introduction includes a causal task timeline; the +method figure separates deployable inputs from privileged audit labels. The +results presentation has measured main/oracle tables and a clearly labeled +six-group **illustrative scenario** with explicit falsifiers. This improves +readability and makes the intended experimental logic reviewable, but it does +not raise the scientific verdict above borderline: L/T/C/B/H-Q/S values and +full-pipeline efficiency remain unmeasured, and the scenario cannot be cited +as evidence. Re-review after gates 1--4 and replacement of the scenario table +with actual per-seed/per-session results. diff --git a/manuscript/targeted/figures/system_overview.pdf b/manuscript/targeted/figures/system_overview.pdf index 149823a..f2175cd 100644 Binary files a/manuscript/targeted/figures/system_overview.pdf and b/manuscript/targeted/figures/system_overview.pdf differ diff --git a/manuscript/targeted/figures/system_overview.tex b/manuscript/targeted/figures/system_overview.tex index 1ceed60..0765e16 100644 --- a/manuscript/targeted/figures/system_overview.tex +++ b/manuscript/targeted/figures/system_overview.tex @@ -1,92 +1,71 @@ \documentclass[tikz,border=2pt]{standalone} \usepackage{amsmath} -\usetikzlibrary{arrows.meta,calc,positioning} +\usetikzlibrary{arrows.meta,positioning,calc} \definecolor{navy}{HTML}{19324A} \definecolor{bluefill}{HTML}{EAF1F7} -\definecolor{teal}{HTML}{0B7A75} -\definecolor{tealfill}{HTML}{E5F4F2} -\definecolor{orange}{HTML}{C96A12} +\definecolor{teal}{HTML}{087F78} +\definecolor{tealfill}{HTML}{E6F4F1} +\definecolor{orange}{HTML}{B85D0F} \definecolor{orangefill}{HTML}{FFF0DF} -\definecolor{purple}{HTML}{6E4B8B} -\definecolor{purplefill}{HTML}{F1EBF6} -\definecolor{grayline}{HTML}{697782} +\definecolor{grayline}{HTML}{687782} \begin{document} \begin{tikzpicture}[ - font=\sffamily\scriptsize, >=Latex, - flow/.style={-{Latex[length=1.7mm]},line width=0.65pt,draw=navy}, + font=\sffamily\scriptsize,>=Latex, + flow/.style={-{Latex[length=1.6mm]},line width=.65pt,draw=navy}, localflow/.style={flow,draw=teal}, - auditflow/.style={flow,draw=purple,dashed}, - box/.style={draw=navy,rounded corners=1.5pt,line width=0.65pt, - fill=bluefill,minimum height=8mm,align=center,inner sep=2.2pt}, - localbox/.style={box,draw=teal,fill=tealfill}, - auditbox/.style={box,draw=purple,fill=purplefill,dashed}, - note/.style={font=\sffamily\tiny,text=grayline,align=center} -] + block/.style={draw=navy,fill=bluefill,rounded corners=1.6pt, + line width=.6pt,minimum height=8.5mm,align=center,inner sep=2pt}, + local/.style={block,draw=teal,fill=tealfill}, + hint/.style={font=\sffamily\tiny,text=grayline,align=center}] \node[anchor=west,font=\sffamily\scriptsize\bfseries,text=navy] - at (-0.1,3.2) {Prediction path: integrated history plus explicit recent evidence}; -\node[align=center,text=navy] (av) at (0.7,2.45) - {observed full-scene\\audio--video prefix}; -\node[box] (omni) at (3.25,2.45) {frozen Qwen2.5-Omni\\8-s maximum context}; -\node[box] (global) at (5.95,2.45) {last-four-layer\\global state $G_n$}; -\node[align=center,text=teal] (audio) at (0.7,1.23) - {most recent 1-s\\mixed audio}; -\node[localbox] (ae) at (3.25,1.23) {frozen audio encoder\\valid-token mean}; -\node[localbox] (ap) at (5.95,1.23) {local projection\\$\widetilde A_n$ + validity}; -\node[align=center,text=teal] (video) at (0.7,0.02) - {recent causal\\person crops}; -\node[localbox,dashed] (ve) at (3.25,0.02) - {frozen visual encoder\\crop / person pooling}; -\node[localbox,dashed] (vp) at (5.95,0.02) - {optional projection\\$\widetilde V_n$ + validity}; -\node[box,minimum width=21mm] (fusion) at (8.65,1.23) - {cutoff-aligned\\fusion head}; -\node[box,minimum width=24mm] (out) at (11.55,1.23) - {$10\!\times\!3$ activity forecast\\0.2--2.0 s}; -\draw[flow] (av) -- (omni); -\draw[flow] (omni) -- (global); + at (0,3.12) {Deployable prediction path at the same causal cutoff $t_n$}; +\node[block,minimum width=25mm] (history) at (1.35,2.26) + {observed AV history\\8-s maximum budget}; +\node[block,minimum width=25mm] (omni) at (4.15,2.26) + {frozen Omni Thinker\\last-four-layer state}; +\node[local,minimum width=25mm] (audio) at (1.35,1.13) + {last 1-s mixed audio\\cropped at $t_n$}; +\node[local,minimum width=25mm] (ae) at (4.15,1.13) + {frozen audio encoder\\valid-token mean}; +\node[local,dashed,minimum width=25mm] (video) at (1.35,0) + {optional recent frames\\provided person boxes}; +\node[local,dashed,minimum width=25mm] (ve) at (4.15,0) + {frozen visual encoder\\pooled person tokens}; +\node[block,minimum width=20mm] (head) at (7.15,1.13) + {normalized features\\+ validity $\to$ MLP}; +\node[block,minimum width=26mm] (out) at (10.4,1.13) + {$10\times3$ forecast\\0.2--2.0-s horizons}; +\draw[flow] (history) -- (omni); \draw[localflow] (audio) -- (ae); -\draw[localflow] (ae) -- (ap); \draw[localflow,dashed] (video) -- (ve); -\draw[localflow,dashed] (ve) -- (vp); -\draw[flow] (global.east) -- ++(0.30,0) |- (fusion.north); -\draw[localflow] (ap) -- (fusion); -\draw[localflow,dashed] (vp.east) -- ++(0.30,0) |- (fusion.south); -\draw[flow] (fusion) -- (out); -\node[note] at (11.45,0.18) - {background / invisible wearer / aggregate others}; - -\draw[grayline,line width=0.45pt] (-0.1,-0.62) -- (12.9,-0.62); +\draw[flow] (omni.east) -- ++(.38,0) |- (head.north); +\draw[localflow] (ae) -- (head); +\draw[localflow,dashed] (ve.east) -- ++(.38,0) |- (head.south); +\draw[flow] (head) -- (out); +\node[hint,text=navy] at (10.4,.17) + {background / wearer / aggregate others}; +\draw[draw=grayline,line width=.45pt] (0,-.62)--(11.85,-.62); \node[anchor=west,font=\sffamily\scriptsize\bfseries,text=navy] - at (-0.1,-0.94) {Strictly causal stream at cutoff $t_n$}; -\node[anchor=east,text=navy] at (2.05,-1.62) {Omni media}; -\foreach \a/\b in {2.3/3.85,3.85/5.4,5.4/6.95} { - \draw[fill=bluefill,draw=navy,line width=0.55pt] - (\a,-1.84) rectangle (\b,-1.40); - \node[text=navy,font=\sffamily\tiny] at ({(\a+\b)/2},-1.62) {completed 2 s}; + at (0,-.94) {Bounded streaming state: no media after $t_n$}; +\node[anchor=east,text=navy] at (1.9,-1.62) {Omni blocks}; +\foreach \a/\b in {2.12/3.55,3.55/4.98,4.98/6.41} { + \draw[draw=navy,fill=bluefill,line width=.55pt] + (\a,-1.84) rectangle (\b,-1.39); + \node[font=\sffamily\tiny,text=navy] at ({(\a+\b)/2},-1.615) {completed}; } -\draw[fill=orangefill,draw=orange,line width=0.65pt] - (6.95,-1.84) rectangle (8.45,-1.40); -\node[text=orange,font=\sffamily\tiny] at (7.70,-1.62) {observed tail}; -\draw[dashed,draw=orange,line width=0.75pt] - (8.45,-2.95) -- (8.45,-1.10); -\node[anchor=south,text=orange,font=\sffamily\tiny\bfseries] - at (8.45,-1.10) {cutoff $t_n$}; -\node[note,anchor=west] at (8.85,-1.62) - {replay completed payloads; recompute only\\the temporary tail; exclude future media}; -\node[anchor=east,text=teal] at (2.05,-2.31) {local evidence}; -\draw[draw=teal,line width=1.8pt] (7.65,-2.31) -- (8.45,-2.31); -\node[note,anchor=west,text=teal] at (8.85,-2.31) - {one-second causal window, refreshed every 200 ms}; -\node[anchor=east,text=navy] at (2.05,-2.86) {decision grid}; -\draw[draw=navy,line width=0.6pt] (2.3,-2.86) -- (8.45,-2.86); -\foreach \x in {2.42,2.58,...,8.34} { - \draw[draw=navy,line width=0.5pt] (\x,-2.95) -- (\x,-2.77); -} - -\node[auditbox,minimum width=42mm] (oracle) at (11.02,-2.78) - {analysis only: past visible-person activity\\aggregate / per-person / delayed / shuffled}; -\draw[auditflow] (oracle.north) -- ++(0,0.32) -| (fusion.south); -\node[note,text=purple] at (11.02,-3.35) - {privileged information audit; never a deployable model input}; +\draw[draw=orange,fill=orangefill,line width=.65pt] + (6.41,-1.84) rectangle (7.85,-1.39); +\node[font=\sffamily\tiny,text=orange] at (7.13,-1.615) {observed tail}; +\draw[draw=orange,dashed,line width=.7pt] (7.85,-2.43)--(7.85,-1.12); +\node[anchor=south,font=\sffamily\tiny\bfseries,text=orange] + at (7.85,-1.12) {$t_n$}; +\node[hint,anchor=west] at (8.16,-1.62) + {reuse completed payloads;\\recompute temporary tail only}; +\node[anchor=east,text=teal] at (1.9,-2.33) {local 1 s}; +\draw[draw=teal,line width=2pt] (7.14,-2.33)--(7.85,-2.33); +\node[hint,anchor=west,text=teal] at (8.16,-2.33) + {recent evidence refreshed every 200 ms}; +\node[hint,anchor=west] at (2.12,-2.74) + {Privileged past-activity labels are evaluated separately as an information audit; they never enter this prediction path.}; \end{tikzpicture} \end{document} diff --git a/manuscript/targeted/figures/system_overview_onecolumn.pdf b/manuscript/targeted/figures/system_overview_onecolumn.pdf new file mode 100644 index 0000000..32e431b Binary files /dev/null and b/manuscript/targeted/figures/system_overview_onecolumn.pdf differ diff --git a/manuscript/targeted/figures/system_overview_onecolumn.tex b/manuscript/targeted/figures/system_overview_onecolumn.tex new file mode 100644 index 0000000..ff4896a --- /dev/null +++ b/manuscript/targeted/figures/system_overview_onecolumn.tex @@ -0,0 +1,50 @@ +\documentclass[tikz,border=2pt]{standalone} +\usetikzlibrary{arrows.meta,positioning} +\definecolor{navy}{HTML}{19324A} +\definecolor{bluefill}{HTML}{EAF1F7} +\definecolor{teal}{HTML}{087F78} +\definecolor{tealfill}{HTML}{E6F4F1} +\definecolor{orange}{HTML}{B85D0F} +\definecolor{orangefill}{HTML}{FFF0DF} +\begin{document} +\begin{tikzpicture}[ + font=\sffamily\small, + >=Latex, + box/.style={draw=navy,fill=bluefill,rounded corners=1.5pt, + align=center,minimum height=7mm,inner sep=2pt}, + local/.style={box,draw=teal,fill=tealfill}, + flow/.style={-{Latex[length=1.6mm]},draw=navy,line width=.6pt}] +\node[box,text width=20mm] (global) at (1.12,3.9) + {Observed AV\\8-s maximum}; +\node[box,text width=22mm] (thinker) at (4.13,3.9) + {Frozen Thinker\\readout $G$}; +\node[local,text width=20mm] (audio) at (1.12,2.9) + {Recent 1-s\\mixed audio}; +\node[local,text width=22mm] (aenc) at (4.13,2.9) + {Frozen encoder\\local $A$}; +\node[local,dashed,text width=20mm] (video) at (1.12,1.9) + {Past person\\crops (optional)}; +\node[local,dashed,text width=22mm] (venc) at (4.13,1.9) + {Frozen encoder\\pooled $V$}; +\node[box,text width=18mm] (head) at (7.05,2.9) + {Validities +\\trained head}; +\draw[flow] (global) -- (thinker); +\draw[flow,draw=teal] (audio) -- (aenc); +\draw[flow,draw=teal,dashed] (video) -- (venc); +\draw[flow] (thinker.east) -- ++(.18,0) |- (head.north); +\draw[flow,draw=teal] (aenc) -- (head); +\draw[flow,draw=teal,dashed] (venc.east) -- ++(.18,0) |- (head.south); +\node[box,text width=32mm] (out) at (5.7,.88) + {$10\times3$ future activity\\0.2--2.0-s horizons}; +\draw[flow] (head.south) -- (out.north); +\draw[draw=navy,line width=.5pt] (0,.12) -- (8.25,.12); +\node[anchor=west,text=navy] at (0,-.28) {Completed blocks}; +\foreach \x in {0,1.04,2.08} { + \draw[draw=navy,fill=bluefill] (\x,-.93) rectangle ++(.96,.35); +} +\draw[draw=orange,fill=orangefill] (3.12,-.93) rectangle ++(.96,.35); +\node[anchor=west,text=orange] at (4.25,-.76) {Observed tail}; +\node[anchor=west,text=navy] at (0,-1.33) + {Reuse completed payloads; query a temporary tail at $t_n$.}; +\end{tikzpicture} +\end{document} diff --git a/manuscript/targeted/figures/task_overview.pdf b/manuscript/targeted/figures/task_overview.pdf new file mode 100644 index 0000000..d548879 Binary files /dev/null and b/manuscript/targeted/figures/task_overview.pdf differ diff --git a/manuscript/targeted/figures/task_overview.tex b/manuscript/targeted/figures/task_overview.tex new file mode 100644 index 0000000..c2ef209 --- /dev/null +++ b/manuscript/targeted/figures/task_overview.tex @@ -0,0 +1,39 @@ +\documentclass[tikz,border=2pt]{standalone} +\usepackage{amsmath} +\usetikzlibrary{arrows.meta,calc} +\definecolor{navy}{HTML}{19324A} +\definecolor{teal}{HTML}{087F78} +\definecolor{orange}{HTML}{B85D0F} +\definecolor{light}{HTML}{EAF1F7} +\definecolor{warm}{HTML}{FFF0DF} +\begin{document} +\begin{tikzpicture}[font=\sffamily\scriptsize,>=Latex] +\node[anchor=west,font=\sffamily\scriptsize\bfseries,text=navy] at (0,2.48) + {Predict the invisible wearer's future activity}; +\draw[draw=navy,fill=light,rounded corners=2pt] (0,1.25) rectangle (2.55,1.95); +\node[align=center,text=navy] at (1.275,1.60) + {camera + mixed audio\\visible people, unseen wearer}; +\draw[-{Latex[length=1.7mm]},draw=navy,line width=.7pt] + (2.65,1.60)--(3.25,1.60); +\draw[draw=navy,line width=.7pt] (3.32,1.28)--(3.32,1.94); +\node[anchor=south,font=\sffamily\tiny\bfseries,text=navy] at (3.32,1.97) + {decision $t_n$}; +\draw[draw=orange,fill=warm,rounded corners=2pt] (4.02,1.25) rectangle (7.6,1.95); +\node[align=center,text=orange] at (5.81,1.60) + {10 future states, every 0.2 s\\background / wearer / others}; +\draw[draw=navy,line width=.55pt] (.35,.66)--(7.55,.66); +\foreach \x in {.35,1.08,1.81,2.54,3.27,4.00,4.73,5.46,6.19,6.92,7.55} + {\draw[draw=navy,line width=.5pt] (\x,.57)--(\x,.75);} +\draw[draw=teal,line width=2pt] (.35,.94)--(3.27,.94); +\node[anchor=east,font=\sffamily\tiny,text=teal] at (.25,.94) {input}; +\draw[draw=orange,line width=2pt] (3.27,.94)--(7.55,.94); +\node[anchor=west,font=\sffamily\tiny,text=orange] at (7.65,.94) {target}; +\node[font=\sffamily\tiny,text=navy] at (3.27,.36) {$t_n$}; +\node[font=\sffamily\tiny,text=navy] at (4.0,.36) {$+.2$ s}; +\node[font=\sffamily\tiny,text=navy] at (7.55,.36) {$+2.0$ s}; +\node[anchor=west,font=\sffamily\tiny,text=navy] at (.35,.02) + {ENTRY-like: wearer silent $\to$ active}; +\node[anchor=west,font=\sffamily\tiny,text=navy] at (4.0,.02) + {YIELD-like: wearer active $\to$ silent}; +\end{tikzpicture} +\end{document} diff --git a/manuscript/targeted/latexmkrc b/manuscript/targeted/latexmkrc index 860d59d..d14d1dc 100644 --- a/manuscript/targeted/latexmkrc +++ b/manuscript/targeted/latexmkrc @@ -1,5 +1,5 @@ my $path_separator = $^O eq 'MSWin32' ? ';' : ':'; -my $template_dir = '../ICASSP2026_Paper_Templates'; +my $template_dir = '../ICASSP2027_Paper_Templates'; $ENV{'TEXINPUTS'} = $template_dir . $path_separator . ($ENV{'TEXINPUTS'} // ''); diff --git a/manuscript/targeted/main.tex b/manuscript/targeted/main.tex index 3f6ed95..e3fe993 100644 --- a/manuscript/targeted/main.tex +++ b/manuscript/targeted/main.tex @@ -1,42 +1,46 @@ % Acceptance-target ICASSP 2027 draft grounded only in archived measurements. -% The official 2027 author kit should replace the 2026-compatible style assets -% before submission. Four technical pages plus an optional reference-only page. +% Uses the repository's ICASSP 2027 author kit. +% Four technical pages plus an optional reference-only page. \documentclass{article} \usepackage{spconf} -\usepackage{amsmath,amssymb,graphicx,booktabs,multirow,hyperref,xcolor} +\usepackage{amsmath,amssymb,graphicx,booktabs,multirow,hyperref,xcolor,float} \hypersetup{hidelinks} \newcommand{\vect}[1]{\boldsymbol{#1}} \newcommand{\G}{\mathrm{G}} \newcommand{\A}{\mathrm{A}} \newcommand{\V}{\mathrm{V}} \setlength{\emergencystretch}{2em} +\setcounter{topnumber}{5} +\setcounter{dbltopnumber}{5} +\renewcommand{\topfraction}{0.95} +\renewcommand{\dbltopfraction}{0.95} +\renewcommand{\textfraction}{0.05} +\renewcommand{\floatpagefraction}{0.75} +\renewcommand{\dblfloatpagefraction}{0.75} \title{Does an Omni Model Know When the Wearer Will Speak?\\ A Causal Global--Local Audit of Egocentric Speech Anticipation} -\name{Authors withheld in this working draft} -\address{} +\name{Xin Wang, Dingtian Yan, Shengxiang Hu, Changdong Liu, +Xiaoju Dai, Li Zhang, and Guorun Chen} +\address{School of Computer Science, Example University, Shanghai, China} \begin{document} \ninept \maketitle \begin{abstract} -An egocentric conversational agent must anticipate the future speech of an -invisible camera wearer from a causal multiparty audio--video stream. We ask -whether a frozen general-purpose Omni state exposes the sub-second evidence -needed by this task. Our cutoff-aligned model combines an 8-s Qwen2.5-Omni -context with the most recent 1-s audio and predicts three activity channels at -ten horizons from 0.2 to 2.0~s. In a single-seed EasyCom screen, this model reaches -81.72\% mAP at 0.2~s and 65.11\% averaged over horizons, compared with -74.20\% and 61.19\% for released EgoSpeak. The gain is larger for wearer -offset anticipation (+7.23 points) than onset anticipation (+1.47). Pooled -visual features add only 0.29 mean-mAP points. A separate three-seed, -strictly causal information audit explains this gap: privileged past activity -of visible participants adds 4.83 points at 0.2~s and 1.77 mean-mAP points, -whereas visibility, one-second-delayed activity, and identity shuffling show -that timeliness matters more than identity continuity. The results isolate an -evidence-access bottleneck: recent acoustic evidence is already useful, while -current generic visual pooling does not recover the available visible-floor -signal. +We test whether a frozen Omni readout exposes sub-second evidence for +forecasting an invisible camera wearer's speech from causal multiparty +audio--video. Our 5-Hz model combines 8-s Qwen2.5-Omni context and recent +1-s audio to predict three activity channels at ten 0.2--2.0-s horizons. +On EasyCom, a seed-0 screen reaches 81.72\% mAP at 0.2~s and 65.11\% +mean mAP, versus 74.20/61.19\% for released EgoSpeak. Wearer offset AP +improves by 7.23 points, onset AP by 1.47. Pooled vision adds only 0.29 +mean-mAP points. A three-seed causal information audit finds that privileged +past visible-person activity adds 4.83 points at 0.2~s and 1.77 mean-mAP +points; visibility and delayed activity add little. A separate no-box +full-frame screen reaches 82.38/65.54\%, but unmatched training and head +capacity prevent visual attribution. Timely evidence access, not semantic +floor understanding, remains the supported conclusion. \end{abstract} \begin{keywords} @@ -51,8 +55,18 @@ Conversational agents must decide not only \emph{what} to say, but also \emph{when} to participate. This decision is particularly asymmetric in an egocentric stream: the camera wearer is normally invisible, yet the model must forecast that wearer's behavior from mixed audio and observations of other -people. A useful predictor must update before a turn boundary is known and -must not use media from after its decision cutoff. +people. As Fig.~\ref{fig:task} illustrates, prediction precedes any known +turn boundary and excludes media after the decision cutoff. + +\begin{figure}[H] +\centering +\includegraphics[width=\columnwidth]{figures/task_overview.pdf} +\caption{Task, not an observed event example. At each 5-Hz decision cutoff, +only the past audio--video prefix is input; ten future activity states are +targets. ENTRY-like and YIELD-like refer to wearer activity transitions, +not semantic floor ownership.} +\label{fig:task} +\end{figure} Voice Activity Projection (VAP) casts turn taking as continuous prediction of future activity \cite{ekstedt2022vap}. EgoSpeak brings this objective to @@ -74,28 +88,22 @@ state $\G$ from explicit recent evidence $\A$ and optional $\V$, and treat their value as an empirical question rather than assuming that a larger backbone solves temporal access. -This paper makes three contributions. First, we formulate a strictly causal -global--local audit for invisible-wearer activity forecasting, with a common -5-Hz cutoff and ten future horizons. Second, we realize the audit with a -bounded Omni stream that combines completed history with a temporary observed -tail and a separate one-second audio path. Third, we distinguish deployable -feature gains from an analysis-only visible-activity oracle. This reveals a -specific negative result---generic pooled or current-frame person features do -not recover most of the available local-floor information---and identifies -timely activity estimation, rather than identity continuity, as the next -modeling target. +We contribute a cutoff-aligned global--local audit, a bounded Omni stream +with observed-tail refresh and recent audio, and a separation of deployable +features from an analysis-only visible-activity oracle. Generic pooled or +current-frame person features recover little of the available activity signal; +timely activity estimation is the next modeling target. -\begin{figure*}[t] +\begin{figure}[t] \centering -\includegraphics[width=0.97\textwidth]{figures/system_overview.pdf} -\caption{Global--local causal audit. At each 200-ms cutoff, the prediction -path fuses an integrated Omni state with explicit recent audio and optional -person-crop vision. Completed 2-s blocks are reused while only the observed -partial tail is temporarily recomputed. The purple branch is a privileged -past-activity audit used to test information value; it is never a deployable -input and contains no future activity.} +\includegraphics[width=\columnwidth]{figures/system_overview_onecolumn.pdf} +\caption{Deployable global--local path and bounded stream. At each 200-ms +cutoff, the head fuses the frozen integrated Omni state with explicit recent +audio and optional pooled person-crop vision. Completed 2-s payloads are +reused; only the observed partial tail is temporarily recomputed. Privileged +past-activity audit labels are kept outside this graph and are not inputs.} \label{fig:overview} -\end{figure*} +\end{figure} \section{Method} \label{sec:method} @@ -137,16 +145,13 @@ is frozen-backbone adaptation rather than end-to-end training. \subsection{Cutoff-Aligned Streaming} -The global stream uses source-aligned 2-s blocks and an 8-s maximum -presentation budget. After warm-up, three completed blocks and the observed -part of the current block are available; the exact span varies with block -phase. Completed modality payloads are reused when the bounded language-model -cache is rebuilt. At every cutoff, the incomplete current block is cropped at -$t_n$, appended to a temporary cache, and queried for $\vect g_n$. Neither -the temporary tail nor the query is committed. At a block boundary the -completed payload is committed once and a new tail begins. Sessions and -source discontinuities reset all caches. Thus the logical state is refreshed -at 5~Hz without backfilling a completed block into earlier rows. +Figure~\ref{fig:overview} uses source-aligned 2-s blocks under an 8-s +presentation cap. After warm-up, three completed blocks and the observed +current tail are available; span varies with block phase. Completed +modality payloads are reused. At each cutoff, the tail is cropped at $t_n$ +and queried in a temporary cache, without committing it. Only completed +blocks are committed; session or source discontinuities reset caches. This +refreshes the logical state at 5~Hz without backfilling earlier rows. \subsection{Visible-Activity Information Audit} @@ -161,6 +166,148 @@ second, and (v) one OR-pooled aggregate-visible history. These conditions never expose future activity, but their speech labels make them privileged diagnostics, not deployable features or upper bounds. +\begin{table*}[t] +\caption{Measured EasyCom screens (\%). Panel A: deployable models; Panel B: +three-seed privileged past-activity audit, which is not deployable. Mean is +ten-horizon dense mAP; On/Off are raw wearer endpoint AP, not semantic events. +F3 is a no-box, two-frame causal full-frame screen; it and the historical +deployable rows are exploratory seed-0 runs with unmatched head/training +details. F1 has three seeds. Dashes are unavailable, not zero.} +\label{tab:main} +\centering +\normalsize +\renewcommand{\arraystretch}{0.87} +\setlength{\tabcolsep}{5.0pt} +\begin{tabular}{@{}lccccc@{}} +\toprule +Method & $\mathrm{mAP}_{.2}\uparrow$ & $\mathrm{WAP}_{.2}\uparrow$ +& Mean$\uparrow$ & On$\uparrow$ & Off$\uparrow$ \\\\ +\midrule +EgoSpeak MiniROADA A+V & 74.204 & 63.750 & 61.193 & 24.491 & 38.024 \\\\ +Recent Qwen audio $\A$ & 79.692 & 84.100 & 60.508 & -- & -- \\\\ +Omni global $\G$ & 73.446 & 77.867 & 60.849 & -- & -- \\\\ +Omni $\G+\A$ & 81.723 & 85.066 & 65.109 & 25.960 & 45.251 \\\\ +Omni $\G+\A+$pooled-$\V$ & 82.027 & 85.086 +& 65.400 & 26.019 & 45.371 \\\\ +No-box full-frame F3 & 82.380 & -- & 65.540 & -- & -- \\\\ +Current-frame person set F1 & $81.777{\pm}0.156$ & -- +& $64.951{\pm}0.095$ & -- & -- \\\\ +Matched-width $\G$ (L2) & -- & -- & -- & -- & -- \\\\ +Past-delayed $\A$ (L3) & -- & -- & -- & -- & -- \\\\ +Current-state persistence (L4) & -- & -- & -- & -- & -- \\\\ +\bottomrule +\end{tabular} +\par\smallskip +\emph{Panel B: privileged past-activity audit (mean $\pm$ sample SD).}\par\smallskip +\normalsize +\setlength{\tabcolsep}{3.2pt} +\begin{tabular}{@{}lccc@{}} +\toprule +Extra condition & $\mathrm{mAP}_{.2}$ & $\mathrm{WAP}_{.2}$ & Mean \\\\ +\midrule +None (audio baseline) & $81.869{\pm}.049$ & $85.264{\pm}.110$ & $65.177{\pm}.058$ \\\\ +Visibility only & $81.945{\pm}.098$ & $85.200{\pm}.144$ & $65.217{\pm}.048$ \\\\ +Activity delayed 1 s & $81.951{\pm}.086$ & $85.270{\pm}.137$ & $65.341{\pm}.051$ \\\\ +Shuffled person IDs & $85.917{\pm}.085$ & $85.577{\pm}.246$ & $66.460{\pm}.090$ \\\\ +Per-person activity & $86.266{\pm}.182$ & $85.608{\pm}.220$ & $66.660{\pm}.135$ \\\\ +Aggregate visible activity & $\mathbf{86.696{\pm}.085}$ & $\mathbf{85.836{\pm}.147}$ +& $\mathbf{66.951{\pm}.022}$ \\\\ +\bottomrule +\end{tabular} +\end{table*} + +\begin{table*}[t] +\caption{Chunk and context audit. Panel A is an unmeasured matched T/C +matrix: each row pairs $\G$ and $\G+\A$ on common valid cutoffs. Panel B is +a separate exploratory seed-0 global-only screen (10 epochs, 8-s cap, +2-FPS video); its rows are not entries in Panel A. Dashes are pending.} +\label{tab:stream_context} +\centering\normalsize +\renewcommand{\arraystretch}{0.87} +\setlength{\tabcolsep}{4pt} +\begin{tabular}{@{}llccccc@{}} +\toprule +Group & Refresh / cap & $\G$ mAP & $\G+\A$ mAP & $\Delta$ mAP +& $\Delta$ On & $\Delta$ Off \\ +\midrule +T & Completed-only, 8 s & -- & -- & -- & -- & -- \\ +T & Observed-tail, 8 s & -- & -- & -- & -- & -- \\ +\midrule +C & Tail-aware, 2 s & -- & -- & -- & -- & -- \\ +C & Tail-aware, 4 s & -- & -- & -- & -- & -- \\ +C & Tail-aware, 6 s & -- & -- & -- & -- & -- \\ +C & Tail-aware, 8 s & -- & -- & -- & -- & -- \\ +C & Tail-aware, 10 s & -- & -- & -- & -- & -- \\ +C & Tail-aware, 12 s & -- & -- & -- & -- & -- \\ +\bottomrule +\end{tabular} +\par\smallskip +\emph{Panel B: reported 200-ms chunk screen; dense mAP (\%).}\par\smallskip +\begin{tabular}{@{}lccc@{}} +\toprule +Global feature construction & 0.2-s mAP & Mean mAP & Processor calls \\ +\midrule +Native 2-s completed hold & 59.62 & 55.19 & 9,539 \\ +Non-native 200-ms commits & 67.05 & 57.84 & 95,453 \\ +Completed blocks + observed tail & 70.85 & 59.17 & -- \\ +\bottomrule +\end{tabular} +\end{table*} + +\begin{table}[t] +\caption{B result slots: paired within-backbone local-audio gains (unmeasured). +MiniCPM requires a matched readout/refresh/history audit before comparison.} +\label{tab:backbone} +\centering\normalsize +\setlength{\tabcolsep}{3pt} +\begin{tabular}{@{}lccc@{}} +\toprule +Backbone & $\G$ & $\G+\A$ & $\Delta$ \\ +\midrule +Qwen Omni 3B & -- & -- & -- \\ +Qwen Omni 7B & -- & -- & -- \\ +MiniCPM-o 4.5 & -- & -- & -- \\ +\bottomrule +\end{tabular} +\end{table} + +\begin{table}[t] +\caption{H result slots: predeclared activity slices (unmeasured). Report +positive/negative counts and per-session scores; AP is undefined for a +pure-negative slice, which instead uses calibrated FPR.} +\label{tab:hard_slices} +\centering\normalsize +\setlength{\tabcolsep}{2.5pt} +\begin{tabular}{@{}lccc@{}} +\toprule +Slice & $N_+/N_-$ & AP & FPR \\ +\midrule +Wearer onset & -- & -- & -- \\ +Wearer offset & -- & -- & -- \\ +Other pause (negative) & -- & n/a & -- \\ +Overlap (negative) & -- & n/a & -- \\ +\bottomrule +\end{tabular} +\end{table} + +\begin{table}[t] +\caption{S result slots: full-path cost on the same hardware (unmeasured). +Include I/O and block transitions; logical 5 Hz is not measured throughput.} +\label{tab:efficiency} +\centering\normalsize +\setlength{\tabcolsep}{3pt} +\begin{tabular}{@{}lccc@{}} +\toprule +Stage & P50 ms & P95 ms & Peak GB \\ +\midrule +Completed block & -- & -- & -- \\ +Observed tail & -- & -- & -- \\ +Local path + head & -- & -- & -- \\ +Full path & -- & -- & -- \\ +\bottomrule +\end{tabular} +\end{table} + \section{Experiments} \label{sec:experiments} @@ -191,36 +338,16 @@ already sufficient. Because that screen used Session~12 model selection whereas the historical pooled row did not, their difference is diagnostic, not a matched causal attribution. +\textbf{Six pre-specified control families.} +We freeze six pending control families: (L) capacity and recency, (T) tail +refresh crossed with local audio, (C) six context caps, (B) paired backbone +transfer, (H/Q) error slices and timelines, and (S) full-path efficiency. +They share the official split and physical cutoffs; the 8-s T/C row reuses +one run. The source retains unmeasured hypotheses and falsifiers. + \section{Results} \label{sec:results} -\begin{table*}[t] -\caption{EasyCom Sessions~1--3 results (\%). $\mathrm{mAP}_{.2}$ and -$\mathrm{WAP}_{.2}$ are dense 0.2-s metrics; Mean averages dense mAP over ten -horizons. On/Off are raw wearer endpoint AP, not semantic event scores. -Deployable rows are archived single-seed screens; $\pm$ is shown only for the -three-seed F1 screen. Dashes denote unavailable measurements.} -\label{tab:main} -\centering -\scriptsize -\setlength{\tabcolsep}{5.0pt} -\begin{tabular}{@{}lccccc@{}} -\toprule -Method & $\mathrm{mAP}_{.2}\uparrow$ & $\mathrm{WAP}_{.2}\uparrow$ -& Mean$\uparrow$ & On$\uparrow$ & Off$\uparrow$ \\\\ -\midrule -EgoSpeak MiniROADA A+V & 74.204 & 63.750 & 61.193 & 24.491 & 38.024 \\\\ -Recent Qwen audio $\A$ & 79.692 & 84.100 & 60.508 & -- & -- \\\\ -Omni global $\G$ & 73.446 & 77.867 & 60.849 & -- & -- \\\\ -Omni $\G+\A$ & 81.723 & 85.066 & 65.109 & 25.960 & 45.251 \\\\ -Omni $\G+\A+$pooled-$\V$ & \textbf{82.027} & \textbf{85.086} -& \textbf{65.400} & \textbf{26.019} & 45.371 \\\\ -Current-frame person set F1 & $81.777{\pm}0.156$ & -- -& $64.951{\pm}0.095$ & -- & -- \\\\ -\bottomrule -\end{tabular} -\end{table*} - \subsection{Global Context and Recent Audio Are Complementary} Table~\ref{tab:main} shows that $\A$ is strong at 0.2~s, while $\G$ is @@ -238,6 +365,70 @@ the local-only head, so this comparison demonstrates predictive complementarity, not that semantic reasoning alone causes the gain. A wider history-matched control is required for a representation-level causal claim. +% Retained as a planning aid; illustrative, unmeasured patterns do not belong +% in the page-limited submission preview. +\iffalse +\begin{table*}[t] +\caption{Six-group \textbf{illustrative scenario, not measured results}. +These are conditional, pre-run interpretations for the pending matrix, not +EasyCom scores. $\Delta$ denotes paired mean-mAP change on common valid +cutoffs; signs and shape must be replaced by measured values even if contrary +to the story. L/T/C/B require seeds 0--2 and per-session reporting.} +\label{tab:scenario} +\centering +\normalsize +\renewcommand{\arraystretch}{1.12} +\setlength{\tabcolsep}{4.2pt} +\begin{tabular}{@{}p{0.095\textwidth}p{0.29\textwidth}p{0.29\textwidth}p{0.235\textwidth}@{}} +\toprule +Group & Frozen comparison & Illustrative pattern \emph{if observed} & Falsifier / interpretation limit \\ +\midrule +L1--4 & $\A,\G,\G+\A,\G+\A+\V$; matched-width $\G$; +past-delayed local cues; current-state persistence +& $\Delta(\G+\A-\G)>0$ after width matching; delayed $\A$ loses +near-horizon benefit; persistence does not recover the gain +& Width-only or persistence matches fusion, or delay does not hurt: +do not attribute gain to timely distinct evidence. \\ +\addlinespace[2.5pt] +T1--2 & 2-s completed-only vs 200-ms observed-tail refresh, +crossed with $\A$ on/off; causal replay and phase audit +& Tail helps most without $\A$; local $\A$ still adds at each refresh; +future-only media never changes prior readouts +& No independent local gain under tail refresh implies redundancy; +replay/leakage failure invalidates timing claims. \\ +\addlinespace[2.5pt] +C1--2 & Genuine 2/4/6/8/10/12-s feature extraction; fixed 1-s $\A$; +add $\G$-only at each cap for a window-stability claim +& Fused performance plateaus around mid-range caps while +$\G$-only remains more cap-sensitive +& If long history substitutes for $\A$, local-evidence claim narrows; +without C2, no robustness comparison is justified. \\ +\addlinespace[2.5pt] +B0--2 & Paired $\G$ and $\G+\A$ on Qwen 3B/7B and, after causal +interface audit, MiniCPM-o 4.5 +& Positive within-backbone paired $\Delta$ on all admissible rows; +absolute backbone ranking is not inferred +& Failed causal/history match is unsupported, not zero gain; +one negative paired row defeats universal transfer. \\ +\addlinespace[2.5pt] +H1/Q1 & Predefined onset/offset and other-pause/overlap slices; +same-horizon, cutoff-marked case selection +& Offset gains coexist with hard wearer-onset negatives; examples +explain, rather than replace, counted slice statistics +& Pure-negative slice AP is undefined; selected examples cannot +establish reduced false-entry rate. \\ +\addlinespace[2.5pt] +S1 & End-to-end p50/p95 latency and peak memory, including I/O, +encoding, tail recomputation, local path, and boundaries +& A feasible throughput--quality tradeoff can be plotted +against 200-ms output cadence +& If p95 exceeds cadence, describe logical 5-Hz prediction only, +not real-time deployment. \\ +\bottomrule +\end{tabular} +\end{table*} +\fi + \subsection{Generic Vision Does Not Recover the Available Signal} Adding pooled $\V$ to $\G+\A$ produces only +0.304 points at 0.2~s and @@ -245,32 +436,15 @@ Adding pooled $\V$ to $\G+\A$ produces only +0.304 points at 0.2~s and averaged over horizons; onset/offset changes are +0.059/+0.120. A richer current-frame person-set model is stable across seeds but reaches lower mean mAP than the historical pooled row under a non-matched selection protocol. -These results do not support a claim of learned speaker grounding. +These results do not support a claim of learned speaker grounding. A separate +seed-0 no-box full-frame F3 screen reaches 82.38\% at 0.2~s and 65.54\% +mean mAP, versus 81.72/65.11\% for the existing $\G+\A$ row under the same +causal-valid dense protocol. F3 adds a two-frame 2-FPS token history and a +shared GRU, while head capacity and training batch differ; raw boundary +metrics and repeat seeds are unavailable. Thus the +0.66/+0.43-point +difference does not isolate visual motion or floor behavior. -\begin{table}[t] -\caption{Three-seed privileged-information audit (mean $\pm$ sample standard -deviation, \%). Every row includes $\G+\A$; activity conditions use only -past labels. Bold marks the best diagnostic, not a deployable model.} -\label{tab:oracle} -\centering -\scriptsize -\setlength{\tabcolsep}{3.2pt} -\begin{tabular}{@{}lccc@{}} -\toprule -Extra condition & $\mathrm{mAP}_{.2}$ & $\mathrm{WAP}_{.2}$ & Mean \\\\ -\midrule -None (audio baseline) & $81.869{\pm}.049$ & $85.264{\pm}.110$ & $65.177{\pm}.058$ \\\\ -Visibility only & $81.945{\pm}.098$ & $85.200{\pm}.144$ & $65.217{\pm}.048$ \\\\ -Activity delayed 1 s & $81.951{\pm}.086$ & $85.270{\pm}.137$ & $65.341{\pm}.051$ \\\\ -Shuffled person IDs & $85.917{\pm}.085$ & $85.577{\pm}.246$ & $66.460{\pm}.090$ \\\\ -Per-person activity & $86.266{\pm}.182$ & $85.608{\pm}.220$ & $66.660{\pm}.135$ \\\\ -Aggregate visible activity & $\mathbf{86.696{\pm}.085}$ & $\mathbf{85.836{\pm}.147}$ -& $\mathbf{66.951{\pm}.022}$ \\\\ -\bottomrule -\end{tabular} -\end{table} - -Table~\ref{tab:oracle} locates the missing information. Aggregate visible +Table~\ref{tab:main}, panel B, locates the missing information. Aggregate visible activity improves over the audio baseline by 4.826 points at 0.2~s and 1.775 mean-mAP points. Visibility alone changes mean mAP by only +0.040, and one-second-delayed activity by +0.164. Thus availability or extra parameters @@ -282,45 +456,73 @@ The majority of the oracle's 0.2-s macro gain comes from background and others; wearer AP rises by only 0.572. This is why neither the oracle nor a dense mAP gain establishes fewer false wearer entries. -\subsection{What the Audit Establishes} +\begin{figure}[t] +\centering +\fbox{\begin{minipage}[c][18mm][c]{0.94\columnwidth} +\centering +\textbf{Q1 case slots: examples pending}\\[2pt] +Onset / other-pause hard negative \quad|\quad Offset / overlap\\ +Show source cutoff, observed activity and same-horizon wearer scores. +\end{minipage}} +\caption{Reserved H/Q case figure. Select examples by a frozen rule after +the slice counts in Table~\ref{tab:hard_slices} are available. No case or +behavioral improvement is asserted by this placeholder.} +\label{fig:hq_cases} +\end{figure} -Three conclusions survive the controls. First, a general-purpose global -state and a one-second acoustic readout provide different useful timescales. -Second, generic visual pooling has not converted visible people into reliable -local floor evidence. Third, perfect timely aggregate activity contains -recoverable information, whereas identity contributes little under the -current target. A dedicated causal active-speaker front end---designed for -mouth motion, blur, and short audio--visual synchrony as in egocentric ASD -\cite{huh2025egoasd}---is consequently a better-motivated next step than a -larger identity-aware fusion module. +\subsection{Pending Controls and Limits} -The evidence remains bounded. EasyCom has only three official test sessions; -the deployable matrix is single seed and has been examined repeatedly. The -oracle uses labels and is not a performance upper bound. Provided boxes omit -online detection cost, and logical 5-Hz output is not a measured throughput -claim. Finally, aggregate others cannot supply addressee or semantic -interruption ground truth. Confirmatory behavioral claims require a frozen -training-side operating point, competing-risk event attribution, and -multi-seed matched controls. +Tables~\ref{tab:stream_context}--\ref{tab:efficiency} reserve result slots +for T/C, B, H, and S; Fig.~\ref{fig:hq_cases} reserves Q cases. Panel B of +Table~\ref{tab:stream_context} separately reports a global-only chunk screen: +the 200-ms commits beat 2-s hold after startup exclusion but require about +ten times as many processor calls across 12 sessions. Freshness and chunk +construction both change, and the older tail bank is not extraction-matched; +these values do not fill the pending T/C factorial or establish real-time +throughput. L requires a +trained width-matched global control and past-delayed features on identical +valid pairs; T crosses tail refresh with local audio; C re-extracts each +causal context cap and pairs it with a global-only run. B compares the +within-backbone audio increment only after readout and history audits. +H/Q reports slice counts and cutoff-marked examples; S measures full-path +latency and memory. The oracle uses privileged labels, provided boxes omit +online detection, and aggregate others cannot establish addressee or +semantic floor ownership. + +\subsection{Decision Rules and Reproducibility} + +The unit of comparison is the same physical cutoff and future target time, +not an arbitrary feature row. For every new condition we will archive +input-source timestamps, common valid-pair counts, seeds, per-session AP, +trainable parameter counts, processor/model revisions, checkpoint hashes, +and prediction scores. Frames and horizons are correlated, not independent +replicates. + +We retain the fixed four-epoch recipe for the matched matrix. Any loss +weight, threshold, context selection, or case-selection rule is frozen +using training-side evidence before looking at Sessions~1--3. Because +these test sessions have already informed this research program, even +new seed runs remain exploratory rather than restoring an unseen test set. +The raw endpoint AP in Table~\ref{tab:main} may motivate a behavioral +hypothesis, but an ENTRY/YIELD error-rate claim additionally requires +one-event-one-vote attribution with response windows censored at the next +other onset and yield windows at the next other offset. Until that +competing-risk evaluator passes real-timeline audit, we report only dense +forecasting and endpoint anticipation. \section{Conclusion} -We presented a strictly causal global--local audit of egocentric wearer speech -anticipation. Frozen Omni context and recent audio are strongly complementary, -especially for future wearer cessation. In contrast, pooled or current-frame -person features do not recover the local-floor signal revealed by a causal -visible-activity oracle. The actionable conclusion is deliberately narrow: -the next bottleneck is timely, learnable local activity evidence, not simply -more multimodal capacity or persistent identity. This distinction provides a -clearer target for future egocentric conversational agents and prevents dense -forecasting gains from being mistaken for semantic floor control. +In this strictly causal audit, frozen Omni context and recent audio improve +wearer activity forecasting, especially future speech cessation. Generic +pooled vision does not recover the timely visible-activity signal exposed by +the analysis-only oracle. Matched controls and censored event evaluation +remain necessary before claiming improved floor behavior. \section{Acknowledgment} -OpenAI Codex assisted with substantive drafting and figure layout in the -abstract, introduction, and presentation of the archived analysis; the -authors remain responsible for verification, experimental claims, and the -final manuscript. +OpenAI Codex assisted with drafting and figure layout; the authors remain +responsible for all experimental claims and final content. +\clearpage \bibliographystyle{IEEEbib} \bibliography{references} \end{document}