Update manuscript

This commit is contained in:
Xin Wang
2026-09-14 13:40:27 +08:00
parent 4bfea907b8
commit f16107d38b
4 changed files with 57 additions and 51 deletions

View File

@@ -1,8 +1,4 @@
LATEXMK ?= latexmk
TEMPLATE_DIR := $(abspath ../ICASSP2026_Paper_Templates)
export TEXINPUTS := $(TEMPLATE_DIR):
export BSTINPUTS := $(TEMPLATE_DIR):
.PHONY: all clean

View File

@@ -3,7 +3,7 @@
This is the editable paper directory. It uses the unmodified official ICASSP
2026 LaTeX assets stored in `../ICASSP2026_Paper_Templates/`. When the ICASSP
2027 author kit is released, replace or add the official template and update
`TEMPLATE_DIR` in the Makefile only after checking the new instructions.
`template_dir` in `latexmkrc` only after checking the new instructions.
Compile from this directory:
@@ -14,12 +14,19 @@ make
The PDF and all intermediate files are written to `build/`; only source files
belong in Git.
VS Code's LaTeX Workshop extension is supported when the repository is opened
as the workspace. Open `main.tex` and run **LaTeX Workshop: Build LaTeX
project**. The workspace setting writes output to `build/`, while `latexmkrc`
provides the template and bibliography-style search paths to both LaTeX
Workshop and command-line builds.
Current source layout:
```text
main/
main.tex
references.bib
latexmkrc
Makefile
README.md
build/ # generated, ignored

View File

@@ -0,0 +1,9 @@
my $path_separator = $^O eq 'MSWin32' ? ';' : ':';
my $template_dir = '../ICASSP2026_Paper_Templates';
$ENV{'TEXINPUTS'} =
$template_dir . $path_separator . ($ENV{'TEXINPUTS'} // '');
$ENV{'BSTINPUTS'} =
$template_dir . $path_separator . ($ENV{'BSTINPUTS'} // '');
$out_dir = 'build';

View File

@@ -169,7 +169,7 @@ $\mathcal P_\tau=\{(\vect x_{\tau,i},\vect b_{\tau,i})\}_{i=1}^{N_\tau}$,
a shared visual encoder and masked set operator produce
\begin{equation}
\vect v_\tau=\mathrm{SetPool}
\left(\{E_v(\vect x_{\tau,i},\vect b_{\tau,i})}_{i=1}^{N_\tau}
\left(\{E_v(\vect x_{\tau,i},\vect b_{\tau,i})\}_{i=1}^{N_\tau}
\cup\{\vect v_\varnothing\}\right),
\label{eq:set}
\end{equation}
@@ -219,16 +219,44 @@ its label/input boundary is fully reported.}
\section{Experiments}
\label{sec:experiments}
\textbf{Data and split.}
We follow the EgoSpeak EasyCom split: Sessions 4--12 for training-side model
development and Sessions 1--3 for final reporting \cite{donley2021easycom}.
Thresholds, windows, checkpoints, loss weights, and early stopping are selected
without using the official test sessions. Already observed test results are
treated as exploratory internal evidence and are not included in the paper.
\todo{Report duration, frame/event counts, class prevalence, visibility and
box coverage, source discontinuities, and the frozen calibration fold.}
\subsection{Dataset and Data Preparation}
\textbf{Baselines.}
We evaluate on EasyCom following the EgoSpeak split, using Sessions 4--12 for training and development and Sessions 1--3 for final evaluation \cite{donley2021easycom}. All hyperparameters, event windows, operating thresholds, and checkpoints are selected without access to the test sessions. The input stream is converted to strictly causal examples at 5~Hz. Each decision point is paired with ten future 0.2-s activity bins for background, camera wearer, and aggregate non-wearer speech. Provided person boxes are used only to construct the visible-person inputs; participant identity, close-mic audio, transcripts, and future observations are not available to the model. \todo{Report dataset duration, number of training/test examples and events, and the frozen development/calibration split.}
\subsection{Model Configuration and Training}
Our main backbone is Qwen2.5-Omni-3B. We compare a frozen global Omni readout, matched PEFT/LoRA adaptation, global features with recent local audio, a parameter- and history-matched temporal local branch, and the proposed causal local audio--visual adapter. Qwen2.5-Omni-7B is used only as a capacity-scaling control. All learned variants use the same causal context, prediction head, training examples, and class-balanced binary cross-entropy objective in Eq.~\eqref{eq:loss}. Unless otherwise stated, model selection is performed on the development data and final numbers are averaged over three random seeds. \todo{Freeze context length, local window/rates, optimizer, epochs, LoRA configuration, trainable parameter counts, and inference latency.}
\subsection{Evaluation Metrics}
We evaluate both dense future-activity prediction and event-level floor
control. For comparability with EgoSpeak, we report per-frame AP at
$0.2$--$2.0$~s horizons and their mean mAP.
For event-level evaluation, a \emph{wearer entry} is a transition from wearer
silence to speech, while a \emph{wearer yield} is a transition from speech to
silence. An entry prediction is considered correct if it is committed within
the predefined response window of a ground-truth wearer onset. We report entry
recall, false entries per minute of eligible wearer-silent time, and signed
onset latency. Yield events are evaluated analogously using recall and latency.
Multiparty interaction introduces a competing-risk condition. After another
speaker releases the floor, the current wearer-entry opportunity is terminated
if another non-wearer participant starts speaking before the wearer does.
Predictions after this competing onset are therefore not matched to the
previous opportunity. This prevents an other-to-other handover from being
treated as an open floor for the wearer.
Operating thresholds are selected on the development split and frozen for test
evaluation. Our primary behavioral metric is false wearer entries per minute at
a matched wearer-entry recall of $0.8$; dense mAP and wearer-yield performance
are reported as complementary measures.
\section{Results and Analysis}
\label{sec:results}
\subsection{Baselines}
The confirmatory comparison contains: an EgoSpeak reproduction; the frozen
Qwen2.5-Omni-3B global state $G$; matched LoRA/PEFT; $G$ plus recent local
audio; a parameter- and history-matched temporal local branch; and the proposed
@@ -237,40 +265,6 @@ not a second paper storyline. \todo{Freeze all baseline implementations and
ensure the EgoSpeak comparison uses identical targets, split, causal validity,
and evaluation.}
\textbf{Event protocol.}
Dense ten-horizon AP/mAP is retained for EgoSpeak comparability. The implemented
competing-risk evaluator counts each eligible aggregate-other boundary once,
ends a response window before the next other onset, and ends a yield window
before the next other offset. Pending the remaining P0 real-data audit and
scenario freeze, the intended primary behavior metric is false wearer entries per
minute of multi-other opportunity at a training-side selected wearer-onset
recall, with $R=.8$ as the headline operating point. The intended analysis also
reports onset latency and wearer-yield recall/latency, following the
event-oriented emphasis on recall, false alarms, and latency
\cite{jiang2026turnbench}. The protocol will censor a response window after an
other offset at the next other onset, and a yield window after an other onset
at the next other offset.
\todo{P0 blocker: run the tested competing-risk evaluator on real predictions,
manually audit sampled timelines, and freeze pairing before treating any event
score as a paper result. Specify overlap,
minimum duration, timeouts, duplicate detections, source boundaries, offscreen
cases, and event counts. Until this is complete, call existing values
timing-derived proxies rather than final event results.}
\textbf{Controls and statistics.}
The main intervention compares correctly aligned local evidence with
time-shifted evidence, removed vision, global-frame visual pooling, and the
capacity-matched temporal branch. A same-rate control is required for a
dual-rate claim. Track/no-ID/broken-continuity controls are added only if a
track-aware variant first wins on training-side data. We report three seeds,
mean$\pm$sample standard deviation, and paired cluster-bootstrap intervals over
sessions or conversation blocks. \todo{Freeze seeds, bootstrap unit/count,
optimizer, epochs, hardware, throughput, P50/P95 latency, and trainable
parameters.}
\section{Results and Analysis}
\label{sec:results}
\begin{table}[t]
\caption{Confirmatory EasyCom results. Lower multi-other false entries are