|
|
|
|
@@ -7,11 +7,11 @@
|
|
|
|
|
\usepackage{amsmath,amssymb,graphicx,hyperref,xcolor}
|
|
|
|
|
|
|
|
|
|
\newcommand{\todo}[1]{\textcolor{red}{\textbf{[TODO: #1]}}}
|
|
|
|
|
\newcommand{\method}{WearerSet}
|
|
|
|
|
\newcommand{\vect}[1]{\boldsymbol{#1}}
|
|
|
|
|
\setlength{\emergencystretch}{2em}
|
|
|
|
|
|
|
|
|
|
\title{NOT EVERY OPEN FLOOR IS MINE: ADDRESSEE-CONDITIONED FLOOR-ENTRY
|
|
|
|
|
PROJECTION IN EGOCENTRIC MULTIPARTY CONVERSATION}
|
|
|
|
|
\title{WHEN SHOULD AN EGOCENTRIC AGENT SPEAK? DIAGNOSING AND ADAPTING
|
|
|
|
|
OMNI REPRESENTATIONS FOR MULTIPARTY FLOOR ACTION}
|
|
|
|
|
|
|
|
|
|
% TODO: Insert the final author list and affiliations after the ICASSP 2027
|
|
|
|
|
% review policy and author order are finalized.
|
|
|
|
|
@@ -23,55 +23,59 @@ PROJECTION IN EGOCENTRIC MULTIPARTY CONVERSATION}
|
|
|
|
|
\maketitle
|
|
|
|
|
|
|
|
|
|
\begin{abstract}
|
|
|
|
|
An egocentric conversational agent must decide not only when a speaking
|
|
|
|
|
opportunity arises, but also whether that opportunity belongs to it. Existing
|
|
|
|
|
speech-activity projection collapses all non-wearer speakers, allowing strong
|
|
|
|
|
frame-level accuracy to coexist with intrusive entries during conversations
|
|
|
|
|
between bystanders. We formulate addressee-conditioned floor-entry projection:
|
|
|
|
|
dense causal forecasts of wearer and non-wearer activity at 200-ms resolution,
|
|
|
|
|
conditioned on a variable-cardinality set of visible participants. Our model
|
|
|
|
|
augments a frozen streaming omni-model with person-aligned audio--visual
|
|
|
|
|
representations and permutation-invariant role queries for floor state,
|
|
|
|
|
wearer address, and third-party interaction. We further introduce an EasyCom
|
|
|
|
|
event protocol that separates addressed-to-wearer responses from third-party
|
|
|
|
|
exchanges and reports false entries at matched response recall and decision
|
|
|
|
|
latency. Correct-, wrong-person, time-shifted, and removed-video controls test
|
|
|
|
|
whether a gain follows social grounding rather than model capacity.
|
|
|
|
|
\todo{Replace this sentence with the main result: FP/min reduction at 80\%
|
|
|
|
|
response recall, response latency/recall non-degradation, and 95\% confidence
|
|
|
|
|
interval.}
|
|
|
|
|
Before generating a response, an egocentric conversational agent must decide
|
|
|
|
|
whether to speak or keep listening in a scene that may contain several other
|
|
|
|
|
people. General-purpose omni models offer broad streaming audio--visual
|
|
|
|
|
context, but it is unclear whether their global representations expose the
|
|
|
|
|
fine-grained local cues required for a decision every 200~ms. We study this
|
|
|
|
|
question across representative omni backbones and adaptation regimes: frozen
|
|
|
|
|
global readout, parameter-efficient tuning, and causal augmentation with
|
|
|
|
|
recent audio and visible-person histories. The human camera wearer serves as a
|
|
|
|
|
behavioral proxy for the agent, while the dense EgoSpeak target remains the
|
|
|
|
|
common output. An EasyCom event protocol reports false wearer entries during
|
|
|
|
|
multi-other exchanges at matched wearer-onset recall, and track/alignment
|
|
|
|
|
controls test whether visual gains follow person continuity rather than added
|
|
|
|
|
capacity. Exploratory Qwen2.5-Omni-3B results motivate the study but do not yet
|
|
|
|
|
establish a limitation shared by omni models or determine the final adapter.
|
|
|
|
|
\todo{After the train-side study is locked, name the selected backbones and
|
|
|
|
|
adapter and insert: frozen/PEFT/local deltas, multi-other false-entry reduction
|
|
|
|
|
at 80\% onset recall, onset/yield non-degradation, and 95\% intervals.}
|
|
|
|
|
\end{abstract}
|
|
|
|
|
|
|
|
|
|
\begin{keywords}
|
|
|
|
|
turn-taking, speech initiation, addressee detection, egocentric video,
|
|
|
|
|
audio-visual learning
|
|
|
|
|
turn-taking, speech initiation, egocentric video, omni models,
|
|
|
|
|
audio-visual adaptation
|
|
|
|
|
\end{keywords}
|
|
|
|
|
|
|
|
|
|
\section{Introduction}
|
|
|
|
|
\label{sec:intro}
|
|
|
|
|
|
|
|
|
|
Conversational agents must continuously decide whether to speak or keep
|
|
|
|
|
listening. In a dyadic exchange, a pause or an apparent end of turn is often
|
|
|
|
|
treated as an invitation to respond. This assumption fails in multiparty
|
|
|
|
|
interaction: one participant may finish an utterance addressed to another
|
|
|
|
|
participant, and an egocentric agent that reacts only to acoustic availability
|
|
|
|
|
will enter a floor that was never offered to it. Such an error is socially more
|
|
|
|
|
disruptive than a small framewise prediction error. A useful agent must
|
|
|
|
|
therefore distinguish \emph{when the floor may change} from \emph{whether the
|
|
|
|
|
change is directed to the camera wearer}.
|
|
|
|
|
Consider an embodied conversational agent that observes the scene through an
|
|
|
|
|
egocentric camera and hears mixed environmental audio. Before response
|
|
|
|
|
generation and speech synthesis, a causal floor controller must decide every
|
|
|
|
|
200~ms whether the agent should continue listening, begin speaking, or yield.
|
|
|
|
|
In a dyadic exchange, a pause or an apparent end of turn may be an invitation
|
|
|
|
|
to respond. This assumption fails in multiparty interaction: one participant
|
|
|
|
|
may hand the floor to another visible participant, and an agent that reacts
|
|
|
|
|
only to acoustic availability may interrupt an exchange that does not involve
|
|
|
|
|
it. We study this floor-action controller; response content generation and
|
|
|
|
|
closed-loop conversational effects are outside our scope. The observed human
|
|
|
|
|
camera wearer is a behavioral proxy rather than an equivalent autonomous
|
|
|
|
|
agent: its activity supplies human demonstrations of floor timing under the
|
|
|
|
|
same first-person observation geometry.
|
|
|
|
|
|
|
|
|
|
Voice Activity Projection (VAP) models future joint speech activity and
|
|
|
|
|
derives turn-taking events without manually labeling each event
|
|
|
|
|
\cite{ekstedt2022vap}. EgoSpeak extends future activity prediction to causal,
|
|
|
|
|
untrimmed egocentric video, forecasting wearer and aggregate non-wearer
|
|
|
|
|
activity over the next two seconds \cite{kim2025egospeak}. Its role-relative
|
|
|
|
|
output is attractive for a wearable agent, but an ``others'' channel cannot
|
|
|
|
|
distinguish an utterance addressed to the wearer from an exchange between
|
|
|
|
|
bystanders. Conventional per-frame average precision can consequently be
|
|
|
|
|
dominated by ordinary activity while hiding precisely the false-entry behavior
|
|
|
|
|
that visual social cues should prevent. Event-oriented evaluation likewise
|
|
|
|
|
emphasizes recall, false alarms, and latency rather than a single frame-level
|
|
|
|
|
score \cite{jiang2026turnbench}.
|
|
|
|
|
output is attractive for a wearable agent, but an ``others'' channel discards
|
|
|
|
|
whether the same or different visible people produced the recent activity and
|
|
|
|
|
how their behavior evolved across time. Conventional per-frame average
|
|
|
|
|
precision can consequently be dominated by ordinary activity while hiding
|
|
|
|
|
precisely the false-entry behavior that person-consistent visual cues may
|
|
|
|
|
prevent. Event-oriented evaluation likewise emphasizes recall, false alarms,
|
|
|
|
|
and latency rather than a single frame-level score \cite{jiang2026turnbench}.
|
|
|
|
|
|
|
|
|
|
Related work supplies complementary parts of the problem. Egocentric
|
|
|
|
|
talking-to-me detection predicts whether a completed utterance addresses the
|
|
|
|
|
@@ -79,46 +83,44 @@ camera wearer and benefits from long-term social context
|
|
|
|
|
\cite{kong2024sicnet}. Multiparty VAP grounds future activity in visible face
|
|
|
|
|
tracks to predict floor shifts and next visible speakers \cite{qi2026muvap}.
|
|
|
|
|
The former is generally utterance-level rather than continuous anticipation;
|
|
|
|
|
the latter does not ask whether an invisible camera wearer should enter. Our
|
|
|
|
|
gap is therefore narrower than ``audio--visual multiparty turn taking'': we
|
|
|
|
|
study continuous, wearer-relative floor action when the candidate responder is
|
|
|
|
|
behind the camera and the number of visible participants changes over time.
|
|
|
|
|
the latter predicts visible participants rather than the floor action of an
|
|
|
|
|
invisible camera wearer. Our gap is therefore narrower than ``audio--visual
|
|
|
|
|
multiparty turn taking'': we study whether short-term anonymous participant
|
|
|
|
|
continuity helps direct, continuous wearer-action projection when the candidate
|
|
|
|
|
responder is behind the camera and the visible participant set changes.
|
|
|
|
|
|
|
|
|
|
We call this task \emph{addressee-conditioned egocentric floor-entry
|
|
|
|
|
projection}. A global streaming audio--video representation supplies broad
|
|
|
|
|
context, while a set branch preserves individual visual histories and their
|
|
|
|
|
alignment to recent mixed audio. Fixed semantic role queries compress a
|
|
|
|
|
variable participant set into wearer-address and third-party evidence without
|
|
|
|
|
assuming a fixed seating order. EasyCom is suitable because it provides
|
|
|
|
|
synchronized egocentric audio--video, per-person tracks, voice activity, and
|
|
|
|
|
target-of-speech annotations \cite{donley2021easycom}. Target annotations are
|
|
|
|
|
used only for training-side supervision and evaluation strata, never as input
|
|
|
|
|
at test time.
|
|
|
|
|
We call this task \emph{egocentric floor-action projection}. The study asks
|
|
|
|
|
four questions: (RQ1) how much wearer activity is linearly or shallowly
|
|
|
|
|
readable from a frozen global omni state; (RQ2) whether standard
|
|
|
|
|
parameter-efficient tuning recovers missing performance; (RQ3) whether causal
|
|
|
|
|
local audio and person-level visual evidence provide complementary information;
|
|
|
|
|
and (RQ4) whether any aggregate gain reduces false agent entries without
|
|
|
|
|
making the controller globally passive. EasyCom is suitable because it
|
|
|
|
|
provides synchronized egocentric audio--video, per-person tracks, and voice
|
|
|
|
|
activity \cite{donley2021easycom}. Target-of-speech annotations, when used, are
|
|
|
|
|
analysis strata only and are never model input or required output.
|
|
|
|
|
|
|
|
|
|
Our contributions are threefold. (1) We formulate continuous speech
|
|
|
|
|
initiation as recipient-aware floor-entry projection for an invisible camera
|
|
|
|
|
wearer. (2) We propose a causal set-based fusion module that grounds mixed
|
|
|
|
|
audio in a variable number of visible participants and preserves the original
|
|
|
|
|
dense EgoSpeak target. (3) We introduce an event evaluation that reports
|
|
|
|
|
third-party false entries at matched addressed-response recall, accompanied by
|
|
|
|
|
latency and visual counterfactual controls. This design makes the scientific
|
|
|
|
|
claim falsifiable: visual fusion is useful only if \emph{correctly aligned}
|
|
|
|
|
vision improves the operating trade-off without suppressing legitimate wearer
|
|
|
|
|
responses.
|
|
|
|
|
The intended contributions are therefore a controlled cross-backbone
|
|
|
|
|
representation diagnosis, a lightweight local-evidence adaptation selected
|
|
|
|
|
from pre-registered candidates, and a behavior-aligned event evaluation. The
|
|
|
|
|
current Qwen2.5-Omni observations are motivation, not a claim that all omni
|
|
|
|
|
models intrinsically fail. Likewise, track-consistent cross-attention becomes
|
|
|
|
|
the final method only if it beats pooled, no-ID, and capacity-matched controls.
|
|
|
|
|
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\centering
|
|
|
|
|
\fbox{\parbox[c][1.05in][c]{0.94\columnwidth}{\centering
|
|
|
|
|
\todo{Draw the final method figure: causal omni state $\vect g_t$; recent
|
|
|
|
|
mixed-audio tokens; a masked set of face-track histories; shared per-person
|
|
|
|
|
audio--visual grounding; three role queries; zero-initialized residual; and
|
|
|
|
|
the $10\times3$ activity head. Mark all causal cutoffs and masks.}}}
|
|
|
|
|
\caption{Proposed wearer-relative set fusion. The number and ordering of
|
|
|
|
|
visible participants may change, while the output remains fixed-size.}
|
|
|
|
|
\todo{After model selection, draw the final system figure. For now it must show
|
|
|
|
|
the shared experiment interface: raw causal AV, backbone-specific global
|
|
|
|
|
readout, frozen/PEFT/local-adapter branches, optional recent audio and anonymous
|
|
|
|
|
person histories, and the common $10\times3$ head. Mark causal cutoffs,
|
|
|
|
|
trainable modules, parameter budgets, and source-boundary resets.}}}
|
|
|
|
|
\caption{Planned model-agnostic floor-action study. The local adapter remains a
|
|
|
|
|
candidate family until the controlled train-side comparison is complete.}
|
|
|
|
|
\label{fig:method}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
\section{Task and Method}
|
|
|
|
|
\section{Methods}
|
|
|
|
|
\label{sec:method}
|
|
|
|
|
|
|
|
|
|
\subsection{Wearer-relative activity projection}
|
|
|
|
|
@@ -141,74 +143,109 @@ windows; elsewhere it remains a future-activity probability. This retains the
|
|
|
|
|
EgoSpeak target and evaluation while exposing a deployable 5-Hz decision
|
|
|
|
|
signal.
|
|
|
|
|
|
|
|
|
|
\subsection{Frozen global stream and local participant set}
|
|
|
|
|
\subsection{Omni backbones and representation protocols}
|
|
|
|
|
|
|
|
|
|
We obtain a causal global state $\vect{g}_t$ from the streaming Thinker of
|
|
|
|
|
Qwen2.5-Omni \cite{xu2025qwenomni}. The backbone is frozen in the main
|
|
|
|
|
experiment because preliminary LoRA adaptation caused negative transfer
|
|
|
|
|
(Sec.~\ref{sec:results}). From the most recent $L_a$ seconds, a shared audio
|
|
|
|
|
encoder produces token sequence $\vect{A}_t$. For each visible track
|
|
|
|
|
$i\in\{1,\ldots,N_t\}$, a shared temporal visual encoder produces
|
|
|
|
|
$\vect{v}_{t,i}$ from its recent face history. Person-aligned evidence is
|
|
|
|
|
Let $m$ index an omni backbone and let $\vect{g}^{(m)}_t$ be a
|
|
|
|
|
backbone-specific causal global readout at the same physical cutoff $t$.
|
|
|
|
|
Qwen2.5-Omni \cite{xu2025qwenomni} is the currently implemented family;
|
|
|
|
|
Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, and MiniCPM-o~4.5
|
|
|
|
|
\cite{cui2026minicpmo45} form the planned initial comparison. These models have
|
|
|
|
|
different native streaming APIs, temporal tokenization, hidden dimensions, and
|
|
|
|
|
adaptation interfaces. We therefore emphasize within-backbone deltas rather
|
|
|
|
|
than treating raw cross-model scores as a pure architecture ranking.
|
|
|
|
|
|
|
|
|
|
Every supported backbone is evaluated under three representation protocols:
|
|
|
|
|
(i) a frozen global readout with a dimension-matched prediction head;
|
|
|
|
|
(ii) a pre-registered parameter-efficient adaptation budget; and (iii) the
|
|
|
|
|
same global readout augmented by causal local evidence. Input cutoffs, EgoSpeak
|
|
|
|
|
targets, context duration, training examples, head capacity, and evaluation
|
|
|
|
|
are held fixed whenever the backbone permits it. A failure of one LoRA recipe
|
|
|
|
|
is reported as that recipe's result, not as proof that the backbone cannot
|
|
|
|
|
adapt.
|
|
|
|
|
|
|
|
|
|
\todo{Implement and audit one adapter for each planned backbone. For every
|
|
|
|
|
model, document model revision, processor, native audio/video rates, exact
|
|
|
|
|
hidden-state readout, causal/streaming semantics, context limit, dtype,
|
|
|
|
|
trainable LoRA targets/rank, parameter count, peak memory, and P50/P95 latency.
|
|
|
|
|
Verify no future frames or samples enter $\vect g^{(m)}_t$. If a model cannot
|
|
|
|
|
expose a comparable causal state, report it as unsupported rather than inventing
|
|
|
|
|
a non-equivalent readout. Add future backbones only before the experiment
|
|
|
|
|
protocol is locked.}
|
|
|
|
|
|
|
|
|
|
\subsection{Candidate local-evidence adaptation}
|
|
|
|
|
|
|
|
|
|
The current hypothesis is that global omni states preserve broad semantic
|
|
|
|
|
context but may make recent prosody, activity boundaries, and individual
|
|
|
|
|
visual motion difficult for a shallow floor head to recover. From the most
|
|
|
|
|
recent $L_a$ seconds, a local encoder produces audio evidence $\vect A_t$ and,
|
|
|
|
|
optionally, an anonymous visible-person set $\mathcal P_t$. A generic candidate
|
|
|
|
|
adapter is
|
|
|
|
|
\begin{equation}
|
|
|
|
|
\vect{p}_{t,i}=\vect{v}_{t,i}+
|
|
|
|
|
\mathrm{CA}(\vect{v}_{t,i},\vect{A}_t,\vect{A}_t),
|
|
|
|
|
\label{eq:person}
|
|
|
|
|
\vect z^{(m)}_t=\vect g^{(m)}_t+
|
|
|
|
|
\gamma_t H_\phi(\vect g^{(m)}_t,\vect A_t,\mathcal P_t),
|
|
|
|
|
\qquad \gamma_t\vert_{\mathrm{init}}=0,
|
|
|
|
|
\label{eq:adapter}
|
|
|
|
|
\end{equation}
|
|
|
|
|
so every participant queries the same mixed audio rather than requiring
|
|
|
|
|
unavailable separated microphones. A masked set encoder models interactions
|
|
|
|
|
among $\mathcal{P}_t=\{\vect{p}_{t,i}\}_{i=1}^{N_t}$ and is permutation
|
|
|
|
|
equivariant \cite{lee2019settransformer}. Missing detections use a learned
|
|
|
|
|
\textsc{offscreen/null} element. In minibatches, sets are padded to
|
|
|
|
|
$N_{\max}$ with an attention mask; no slot-ID embedding is used, and track
|
|
|
|
|
slots are randomly permuted during training.
|
|
|
|
|
followed by the common prediction head. Local evidence is derived from the
|
|
|
|
|
same causal raw media, not close microphones, transcripts, speaker labels, or
|
|
|
|
|
future frames.
|
|
|
|
|
|
|
|
|
|
\todo{Implement and verify the shared track encoder, mixed-audio
|
|
|
|
|
cross-attention, OFFSCREEN token, padding mask, and numerical invariance under
|
|
|
|
|
track-slot permutation. Freeze $L_a$, visual crop history, $N_{\max}$, and
|
|
|
|
|
face-track confidence filtering from train-side experiments only.}
|
|
|
|
|
We evaluate candidates in increasing structural complexity: pooled recent
|
|
|
|
|
audio/vision; a capacity-matched temporal branch; a per-frame person set with
|
|
|
|
|
no cross-time identity; a shared track-wise temporal encoder; and person--audio
|
|
|
|
|
cross-attention. Only a candidate that improves the pre-registered event metric
|
|
|
|
|
over simpler and capacity-matched controls becomes the final method. If
|
|
|
|
|
person continuity is selected, temporary IDs route hidden state only---there
|
|
|
|
|
is no ID embedding, name recognition, or cross-recording identity---and the set
|
|
|
|
|
operator is permutation equivariant \cite{lee2019settransformer}.
|
|
|
|
|
|
|
|
|
|
\subsection{Role-query fusion}
|
|
|
|
|
|
|
|
|
|
Mean pooling can erase which participant is speaking to whom. Instead, three
|
|
|
|
|
learned semantic queries represent floor state, address-to-wearer evidence,
|
|
|
|
|
and third-party interaction. Each query is conditioned on $\vect{g}_t$ and
|
|
|
|
|
cross-attends to the participant set:
|
|
|
|
|
\begin{align}
|
|
|
|
|
\vect{Q}_t &= [\vect{q}_{\mathrm{floor}};
|
|
|
|
|
\vect{q}_{\mathrm{addr}};
|
|
|
|
|
\vect{q}_{\mathrm{third}}]+W_g\vect{g}_t,\\
|
|
|
|
|
\vect{R}_t &= \mathrm{MHA}(\vect{Q}_t,\widetilde{\mathcal P}_t,
|
|
|
|
|
\widetilde{\mathcal P}_t),\\
|
|
|
|
|
\vect{z}_t &= \vect{g}_t+\tanh(\alpha)W_r\mathrm{vec}(\vect{R}_t),
|
|
|
|
|
\label{eq:fusion}
|
|
|
|
|
\end{align}
|
|
|
|
|
where $\alpha$ is initialized to zero. The zero-initialized residual makes the
|
|
|
|
|
starting model identical to the frozen global baseline and allows the visual
|
|
|
|
|
branch to be removed exactly. A two-layer prediction head maps $\vect{z}_t$ to
|
|
|
|
|
Eq.~\eqref{eq:task}.
|
|
|
|
|
\todo{Run the candidate ladder before fixing $H_\phi$. Match trainable
|
|
|
|
|
parameters and effective local history. Compare a shared local encoder across
|
|
|
|
|
backbones (cross-model control) with each backbone's native local encoder
|
|
|
|
|
(within-model control); do not mix these protocols in one claimed delta.
|
|
|
|
|
For track candidates, specify visibility/age masks, gap/source resets,
|
|
|
|
|
OFFSCREEN handling, and global-rename invariance. Describe only the selected
|
|
|
|
|
architecture as the final method; move rejected candidates to ablations.}
|
|
|
|
|
|
|
|
|
|
\subsection{Training objectives}
|
|
|
|
|
|
|
|
|
|
The main loss is class-balanced binary cross entropy over all horizons and
|
|
|
|
|
activity channels. When annotations are available, auxiliary heads predict
|
|
|
|
|
per-track active-speaker state and whether the current source addresses the
|
|
|
|
|
wearer:
|
|
|
|
|
The common main loss is class-balanced binary cross entropy over all horizons
|
|
|
|
|
and activity channels. This dense EgoSpeak target is the only supervision
|
|
|
|
|
required by the task. If a person-aware candidate survives the activity-only
|
|
|
|
|
comparison, an optional auxiliary head predicts current active-speaker state:
|
|
|
|
|
\begin{equation}
|
|
|
|
|
\mathcal{L}=\mathcal{L}_{\mathrm{act}}+
|
|
|
|
|
\lambda_{\mathrm{asd}}\mathcal{L}_{\mathrm{asd}}+
|
|
|
|
|
\lambda_{\mathrm{addr}}\mathcal{L}_{\mathrm{addr}}.
|
|
|
|
|
\lambda_{\mathrm{asd}}\mathcal{L}_{\mathrm{asd}}.
|
|
|
|
|
\label{eq:loss}
|
|
|
|
|
\end{equation}
|
|
|
|
|
Ambiguous, group-addressed, and offscreen-target samples are masked from
|
|
|
|
|
$\mathcal{L}_{\mathrm{addr}}$, but remain in activity training. No language
|
|
|
|
|
model or pseudo-label is used to define primary test events.
|
|
|
|
|
Participant ID, target-of-speech, transcript, close-microphone audio, and
|
|
|
|
|
future frames are never model inputs. Target-of-speech may stratify qualitative
|
|
|
|
|
analysis but is not a predicted variable or a requirement for the primary
|
|
|
|
|
multi-other event definition.
|
|
|
|
|
|
|
|
|
|
\todo{Run the staged ablation: activity only; +ASD; +addressee. Select
|
|
|
|
|
$\lambda$ values and early stopping on a training-side calibration fold. Do
|
|
|
|
|
not inspect Sessions 1--3 during selection.}
|
|
|
|
|
\todo{First compare all representation/adaptation protocols with activity loss
|
|
|
|
|
only. Then ablate +ASD for the selected person-aware candidate. Add an
|
|
|
|
|
event-specific wearer onset/yield loss only as a separately reported extension.
|
|
|
|
|
Select all coefficients and early stopping on a training-side calibration
|
|
|
|
|
fold; do not inspect Sessions 1--3 during selection.}
|
|
|
|
|
|
|
|
|
|
\section{Experimental Setup}
|
|
|
|
|
\subsection{Track source and deployment contract}
|
|
|
|
|
|
|
|
|
|
Person-aware candidates first use dataset-provided boxes and anonymous IDs to
|
|
|
|
|
isolate the value of local visual structure. A deployable version instead
|
|
|
|
|
obtains causal tracklets from a face/head detector followed by an online
|
|
|
|
|
multi-object tracker; no future-frame smoothing or cross-session recognition
|
|
|
|
|
is permitted. EasyCom IDs are continued only across genuinely adjacent source
|
|
|
|
|
segments and reset across redacted gaps. Ego4D IDs are local to a clip.
|
|
|
|
|
|
|
|
|
|
\todo{If a person-aware candidate is selected, state precisely whether face or
|
|
|
|
|
head boxes are used. Report provided-track
|
|
|
|
|
coverage, mean/maximum people, occlusion gaps, track fragments, and source-gap
|
|
|
|
|
resets for both datasets. Add either one online detector/tracker experiment or
|
|
|
|
|
controlled box jitter, dropout, fragmentation, and ID-switch curves; otherwise
|
|
|
|
|
limit every deployment claim to ``given causal participant tracklets.''}
|
|
|
|
|
|
|
|
|
|
\section{Experiments}
|
|
|
|
|
\label{sec:setup}
|
|
|
|
|
|
|
|
|
|
\textbf{Data and splits.}
|
|
|
|
|
@@ -216,45 +253,83 @@ We use the EgoSpeak EasyCom protocol: Sessions 4--12 for model development and
|
|
|
|
|
Sessions 1--3 for final reporting. All architecture, checkpoint, threshold,
|
|
|
|
|
and hyperparameter choices use training-side folds only. Inputs contain the
|
|
|
|
|
camera stream and shared array audio available to the wearer; close-talking
|
|
|
|
|
microphones and target-of-speech labels are supervision only. We also report aggregate
|
|
|
|
|
activity forecasting on the EgoSpeak Ego4D split \cite{grauman2022ego4d}, but
|
|
|
|
|
do not claim addressee grounding there because comparable target annotations
|
|
|
|
|
are absent.
|
|
|
|
|
microphones, per-person voice labels, and target-of-speech labels are not model
|
|
|
|
|
inputs. We also report aggregate activity forecasting on the EgoSpeak Ego4D
|
|
|
|
|
split \cite{grauman2022ego4d}. Ego4D Audio-Visual Diarization supplies
|
|
|
|
|
clip-local anonymous participant tracks; its per-person voice coverage on the
|
|
|
|
|
433-clip EgoSpeak split must be audited before reporting the same multi-other
|
|
|
|
|
event analysis.
|
|
|
|
|
|
|
|
|
|
\textbf{Backbone comparison.}
|
|
|
|
|
The initial study covers Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, and
|
|
|
|
|
MiniCPM-o~4.5. For each backbone $m$, we first compare its frozen causal state
|
|
|
|
|
$G_m$, a parameter-efficiently adapted state $P_m$, and local-evidence
|
|
|
|
|
augmentation $G_m+L$. The primary diagnostic is the within-backbone change;
|
|
|
|
|
cross-backbone absolute scores are secondary because native tokenization,
|
|
|
|
|
pretraining, streaming interfaces, and hidden-state semantics differ. Common
|
|
|
|
|
local encoders test whether adaptation transfers across global backbones,
|
|
|
|
|
whereas native local encoders are a separate, model-specific protocol. All
|
|
|
|
|
rows share physical input cutoffs, targets, data splits, and evaluation.
|
|
|
|
|
Unsupported causal readouts are disclosed rather than replaced by offline or
|
|
|
|
|
future-conditioned features.
|
|
|
|
|
|
|
|
|
|
\todo{Implement the three backbone adapters and validate state timestamps with
|
|
|
|
|
an impulse/cutoff test. Pre-register one matched PEFT budget and one common
|
|
|
|
|
local branch. Report native-vs-common local encoders in separate table panels.
|
|
|
|
|
Keep future model additions outside the confirmatory comparison once this
|
|
|
|
|
matrix is locked.}
|
|
|
|
|
|
|
|
|
|
\textbf{Event construction.}
|
|
|
|
|
An \emph{addressed response} starts from a pure other-to-wearer utterance and
|
|
|
|
|
requires a wearer onset before an unrelated other onset. A \emph{third-party
|
|
|
|
|
exchange} starts from a pure other-to-other target while the wearer remains
|
|
|
|
|
silent. Group, none, multiple, and unknown targets form a separate diagnostic
|
|
|
|
|
stratum. Threshold $\tau_r$ is selected on the development split to obtain
|
|
|
|
|
addressed-response recall $r\in\{0.7,0.8,0.9\}$ and is then frozen. The primary
|
|
|
|
|
metric is false wearer entries per minute of third-party opportunity at
|
|
|
|
|
matched recall; the headline operating point is $r=0.8$. We also report
|
|
|
|
|
addressed-response recall, signed detection latency, wearer yield recall, and
|
|
|
|
|
the original ten-horizon per-frame mAP.
|
|
|
|
|
We derive \emph{wearer onset} events directly from the wearer activity target.
|
|
|
|
|
A \emph{multi-other exchange} is a wearer-silent opportunity interval in which
|
|
|
|
|
at least two distinct non-wearer participant IDs contribute temporally ordered
|
|
|
|
|
speech activity. It is an observable interaction proxy, not a claim about the
|
|
|
|
|
speakers' mental intent or addressee. Threshold $\tau_r$ is selected on the
|
|
|
|
|
development split to obtain wearer-onset recall
|
|
|
|
|
$r\in\{0.7,0.8,0.9\}$ and is then frozen. The primary metric is false wearer
|
|
|
|
|
entries per minute of multi-other opportunity at matched onset recall; the
|
|
|
|
|
headline operating point is $r=0.8$. We also report signed onset latency,
|
|
|
|
|
wearer-yield recall/latency, and the original ten-horizon per-frame mAP.
|
|
|
|
|
|
|
|
|
|
\todo{Freeze the event pairing and competing-risk rules, including pre/post
|
|
|
|
|
windows, minimum silence, maximum response delay, overlap handling, and one
|
|
|
|
|
detection per event. Publish counts for train/test positive and negative
|
|
|
|
|
events, visibility, duration, and excluded target types. Treat all previously
|
|
|
|
|
viewed Sessions 1--3 scores as exploratory; lock the final protocol and model
|
|
|
|
|
before the confirmatory three-seed run.}
|
|
|
|
|
\todo{Freeze the event pairing and competing-risk rules, including the
|
|
|
|
|
multi-other alternation window, minimum speech/silence, overlap, maximum onset
|
|
|
|
|
delay, duplicate detections, one detection per event, track visibility, source
|
|
|
|
|
boundaries, and group/offscreen cases. Publish train/test onset counts,
|
|
|
|
|
multi-other events/minutes, number of distinct visible speakers, and exclusion
|
|
|
|
|
counts. Manually audit at least 100 events. Treat all previously viewed
|
|
|
|
|
Sessions 1--3 scores as exploratory; lock the protocol and model before the
|
|
|
|
|
confirmatory three-seed run.}
|
|
|
|
|
|
|
|
|
|
\textbf{Baselines and controls.}
|
|
|
|
|
We compare: global omni state (G); G plus recent audio (G+A); G+A plus pooled
|
|
|
|
|
visual context (G+A+V-pool); a parameter-matched temporal branch without
|
|
|
|
|
person identity; and the proposed set branch. Required counterfactuals replace
|
|
|
|
|
correct tracks with (i) wrong participants from the same scene, (ii)
|
|
|
|
|
time-shifted tracks, and (iii) zeros. These controls share parameter count,
|
|
|
|
|
audio history, and optimization budget.
|
|
|
|
|
For each backbone we compare: frozen global state $G_m$; PEFT $P_m$; $G_m$
|
|
|
|
|
plus recent local audio; and $G_m$ plus pooled local audio--visual evidence.
|
|
|
|
|
This compact matrix tests the representation hypothesis before investing in a
|
|
|
|
|
complex fusion module. If local vision survives three seeds and the event
|
|
|
|
|
gate, we then compare a parameter-matched temporal branch without person
|
|
|
|
|
continuity, a separately trained per-frame/no-ID set branch, and a
|
|
|
|
|
track-consistent branch. On a frozen track-aware checkpoint, a clip-consistent
|
|
|
|
|
global renaming must leave outputs unchanged, whereas \emph{broken
|
|
|
|
|
continuity} periodically remaps IDs while preserving every crop, timestamp,
|
|
|
|
|
audio sample, and target. Time-shifted and removed vision provide additional
|
|
|
|
|
alignment controls.
|
|
|
|
|
|
|
|
|
|
\todo{Define the broken-continuity corruption before evaluation: remap only at
|
|
|
|
|
pre-registered 1-s boundaries or report a fixed ID-switch-rate curve. Do not
|
|
|
|
|
call a clip-consistent slot permutation ``broken ID.'' Verify unchanged crops,
|
|
|
|
|
frame counts, audio, labels, and visibility masks. Train the no-ID/reset
|
|
|
|
|
baseline from scratch with matched parameters; evaluate all test-time
|
|
|
|
|
counterfactuals using one frozen checkpoint and one frozen threshold.}
|
|
|
|
|
|
|
|
|
|
\textbf{Implementation and statistics.}
|
|
|
|
|
All models run at 5~Hz with an 8-s causal memory. The 3B Thinker is frozen and
|
|
|
|
|
local audio/face histories are encoded causally. We report three seeds and
|
|
|
|
|
mean$\pm$standard deviation. Confidence intervals for event metrics use paired
|
|
|
|
|
cluster bootstrap over sessions/conversation blocks rather than treating
|
|
|
|
|
adjacent frames as independent. Runtime includes P50/P95 decision latency,
|
|
|
|
|
peak memory, and trainable parameter count.
|
|
|
|
|
All prediction heads run at 5~Hz and use an 8-s causal observation window;
|
|
|
|
|
model-specific deviations required by a native streaming interface are stated
|
|
|
|
|
explicitly. In the frozen protocol the omni backbone does not update. In PEFT
|
|
|
|
|
and local protocols, trainable parameters and optimizer exposure are reported
|
|
|
|
|
separately. Every temporal state resets at non-contiguous source boundaries.
|
|
|
|
|
We report three seeds and mean$\pm$standard deviation. Confidence intervals for
|
|
|
|
|
event metrics use paired cluster bootstrap over sessions/conversation blocks
|
|
|
|
|
rather than treating adjacent frames as independent. Runtime includes P50/P95
|
|
|
|
|
decision latency, throughput, peak memory, and trainable parameter count.
|
|
|
|
|
|
|
|
|
|
\todo{Fill optimizer, learning rate, batch size, epochs, feature dimensions,
|
|
|
|
|
$L_a$, crop rate/resolution, number of heads/layers, $N_{\max}$, parameter
|
|
|
|
|
@@ -264,95 +339,138 @@ count, hardware, and real-time latency after the architecture is frozen.}
|
|
|
|
|
\label{sec:results}
|
|
|
|
|
|
|
|
|
|
\begin{table}[t]
|
|
|
|
|
\caption{Current aggregate evidence on EasyCom test Sessions 1--3. Values are
|
|
|
|
|
percentages from one seed and are motivation, not the final main result.}
|
|
|
|
|
\caption{Exploratory Qwen2.5-Omni-3B local-branch evidence on EasyCom test
|
|
|
|
|
Sessions 1--3. Values are percentages from one seed, not a cross-backbone or
|
|
|
|
|
confirmatory result.}
|
|
|
|
|
\label{tab:preliminary}
|
|
|
|
|
\centering
|
|
|
|
|
\small
|
|
|
|
|
\resizebox{\columnwidth}{!}{%
|
|
|
|
|
\begin{tabular}{lccc}
|
|
|
|
|
\hline
|
|
|
|
|
Model & 0.2-s mAP & 0.2-s wearer AP & Mean mAP\\
|
|
|
|
|
\hline
|
|
|
|
|
G+A & 81.72 & 85.07 & 65.11\\
|
|
|
|
|
G+A+V-pool & 82.03 & 85.09 & 65.40\\
|
|
|
|
|
Qwen-3B $G$+local A & 81.72 & 85.07 & 65.11\\
|
|
|
|
|
Qwen-3B $G$+local AV-pool & 82.03 & 85.09 & 65.40\\
|
|
|
|
|
\hline
|
|
|
|
|
$\Delta$ visual & +0.30 & +0.02 & +0.29\\
|
|
|
|
|
\hline
|
|
|
|
|
\end{tabular}
|
|
|
|
|
\end{tabular}}
|
|
|
|
|
\end{table}
|
|
|
|
|
|
|
|
|
|
Table~\ref{tab:preliminary} shows why aggregate mAP alone is insufficient.
|
|
|
|
|
Pooled vision gives a small gain, while immediate wearer AP is nearly
|
|
|
|
|
unchanged. This is compatible with either weak but useful social grounding or
|
|
|
|
|
generic visual regularization; only matched-recall event evaluation and
|
|
|
|
|
counterfactual identity controls can distinguish them.
|
|
|
|
|
track-continuity controls can distinguish them.
|
|
|
|
|
A separate Qwen-3B dense-cache diagnostic gave 58.16 mean mAP for its frozen
|
|
|
|
|
readout and 56.20 for one LoRA recipe. Because that diagnostic used a different
|
|
|
|
|
feature protocol, it is not combined numerically with
|
|
|
|
|
Table~\ref{tab:preliminary}; it only shows why a fair, protocol-matched PEFT
|
|
|
|
|
study remains necessary.
|
|
|
|
|
|
|
|
|
|
\begin{table}[t]
|
|
|
|
|
\caption{Primary EasyCom event evaluation. Lower $\mathrm{FP}_{\mathrm{3p}}$
|
|
|
|
|
is better; all rows use the same development-selected operating protocol.}
|
|
|
|
|
\caption{Planned cross-backbone EasyCom event evaluation. Lower
|
|
|
|
|
$\mathrm{FP}_{\mathrm{mo}}$ is better; all rows use the same
|
|
|
|
|
development-selected operating protocol. ``Local'' denotes the train-side
|
|
|
|
|
selected common local-evidence candidate.}
|
|
|
|
|
\label{tab:main}
|
|
|
|
|
\centering
|
|
|
|
|
\small
|
|
|
|
|
\resizebox{\columnwidth}{!}{%
|
|
|
|
|
\begin{tabular}{lccccc}
|
|
|
|
|
\hline
|
|
|
|
|
Model & $\mathrm{FP}_{\mathrm{3p}}$/min@$R=.8$ & $R_{\mathrm{addr}}$ &
|
|
|
|
|
Latency & Yield R & Mean mAP\\
|
|
|
|
|
Backbone/protocol & $\mathrm{FP}_{\mathrm{mo}}$/min@$R=.8$ & Onset R &
|
|
|
|
|
Onset latency & Yield R & Mean mAP\\
|
|
|
|
|
\hline
|
|
|
|
|
G & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
G+A & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
G+A+V-pool & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Temporal control & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
\method{} & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
\method{}+ASD & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
\method{}+Addr & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Qwen-3B frozen & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Qwen-3B PEFT & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Qwen-3B + local & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Qwen-7B frozen & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Qwen-7B PEFT/local & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
MiniCPM-o frozen & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
MiniCPM-o PEFT/local & \todo{--} & \todo{--} & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
\hline
|
|
|
|
|
\end{tabular}}
|
|
|
|
|
\end{table}
|
|
|
|
|
|
|
|
|
|
\todo{Report Table~\ref{tab:main} as mean$\pm$standard deviation over three
|
|
|
|
|
seeds, plus 95\% paired cluster-bootstrap confidence intervals for the
|
|
|
|
|
proposed-vs-G+A difference. The paper's main claim is supported only if
|
|
|
|
|
third-party FP/min decreases at matched recall without worse response latency
|
|
|
|
|
or wearer-yield recall.}
|
|
|
|
|
seeds, plus 95\% paired cluster-bootstrap confidence intervals for frozen vs
|
|
|
|
|
PEFT and frozen vs local within each backbone. A general omni-representation
|
|
|
|
|
claim requires a consistent pattern across at least two backbone families; an
|
|
|
|
|
effect confined to Qwen must be stated as Qwen-specific. The behavior claim is
|
|
|
|
|
supported only if multi-other FP/min decreases at matched wearer-onset recall
|
|
|
|
|
without worse onset latency or wearer-yield recall. If the selected local
|
|
|
|
|
candidate uses tracks, also report its paired intervals against no-ID and
|
|
|
|
|
temporal-capacity controls.}
|
|
|
|
|
|
|
|
|
|
\begin{table}[t]
|
|
|
|
|
\caption{Visual grounding counterfactual on the frozen best model.}
|
|
|
|
|
\caption{Conditional track-continuity counterfactual on the best track-aware
|
|
|
|
|
model. This table is retained only if train-side selection chooses such a
|
|
|
|
|
candidate; all rows reuse one frozen checkpoint.}
|
|
|
|
|
\label{tab:counterfactual}
|
|
|
|
|
\centering
|
|
|
|
|
\small
|
|
|
|
|
\resizebox{\columnwidth}{!}{%
|
|
|
|
|
\begin{tabular}{lccc}
|
|
|
|
|
\hline
|
|
|
|
|
Visual evidence & $\mathrm{FP}_{\mathrm{3p}}$/min & Addr. recall & 0.2-s mAP\\
|
|
|
|
|
Track/visual evidence & $\mathrm{FP}_{\mathrm{mo}}$/min & Onset recall & 0.2-s mAP\\
|
|
|
|
|
\hline
|
|
|
|
|
Correct tracks & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Wrong person & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Global ID rename & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Broken continuity & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Time shifted & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
Removed & \todo{--} & \todo{--} & \todo{--}\\
|
|
|
|
|
\hline
|
|
|
|
|
\end{tabular}
|
|
|
|
|
\end{tabular}}
|
|
|
|
|
\end{table}
|
|
|
|
|
|
|
|
|
|
Table~\ref{tab:counterfactual} is the identity test. A true social-grounding
|
|
|
|
|
effect should be largest for correct tracks and should collapse for wrong or
|
|
|
|
|
time-shifted people. If every visual variant improves equally, the defensible
|
|
|
|
|
conclusion is additional capacity/history, not addressee grounding.
|
|
|
|
|
If a track-aware candidate is selected, Table~\ref{tab:counterfactual} is the
|
|
|
|
|
continuity test. Global renaming must be
|
|
|
|
|
numerically invariant because IDs carry no semantics. If the model exploits
|
|
|
|
|
person history, correct tracks should outperform broken continuity even though
|
|
|
|
|
both contain exactly the same per-frame people. If broken, time-shifted, and
|
|
|
|
|
removed variants perform similarly to correct tracks, the defensible
|
|
|
|
|
conclusion is additional capacity or generic visual context, not useful
|
|
|
|
|
person-continuous reasoning.
|
|
|
|
|
|
|
|
|
|
\todo{Add one compact error analysis split by target visibility, number of
|
|
|
|
|
visible people, overlap, and backchannel/interruption proxy. Include event
|
|
|
|
|
counts with every rate. Move negative LoRA/7B/ROMA diagnostics to supplementary
|
|
|
|
|
material unless they directly explain the final method.}
|
|
|
|
|
\todo{Add one compact error analysis split by model family, track visibility,
|
|
|
|
|
number of visible people, overlap, track age/fragmentation, and
|
|
|
|
|
backchannel/interruption proxy. Include event counts with every rate. Move
|
|
|
|
|
rejected PEFT recipes and fusion candidates to supplementary material unless
|
|
|
|
|
they directly explain the selected method.}
|
|
|
|
|
|
|
|
|
|
\textbf{Qualitative evidence.}
|
|
|
|
|
For a multi-other exchange, visualize per-person speech activity and track IDs,
|
|
|
|
|
wearer ground truth, the frozen-global and selected-local wearer-entry scores,
|
|
|
|
|
and the single frozen threshold on one physical time axis. If the selected
|
|
|
|
|
candidate is track-aware, also show its broken-continuity score. A useful
|
|
|
|
|
example shows the frozen baseline rising at an other-speaker boundary while
|
|
|
|
|
the selected adapter remains below threshold; a companion positive example
|
|
|
|
|
must show that a real wearer onset is not suppressed.
|
|
|
|
|
|
|
|
|
|
\todo{Do not hand-pick only successful clips. Before viewing the final test
|
|
|
|
|
plots, freeze a deterministic selection rule and show: largest paired
|
|
|
|
|
improvement, near-median improvement, one failure, and one correctly retained
|
|
|
|
|
wearer onset. Include full local timelines and source frame times, not cropped
|
|
|
|
|
screenshots without context. State that examples illustrate the aggregate
|
|
|
|
|
event result and cannot establish it alone.}
|
|
|
|
|
|
|
|
|
|
\section{Conclusion}
|
|
|
|
|
\label{sec:conclusion}
|
|
|
|
|
|
|
|
|
|
We study when an egocentric agent should enter a multiparty floor, not merely
|
|
|
|
|
whether someone will speak. The proposed formulation retains dense future
|
|
|
|
|
activity prediction while grounding wearer entry in a variable set of visible
|
|
|
|
|
participants. Its event protocol tests the intended behavior directly:
|
|
|
|
|
reducing intrusions into third-party exchanges at matched legitimate-response
|
|
|
|
|
recall. \todo{Replace with one quantitative conclusion and one limitation;
|
|
|
|
|
do not claim success from aggregate mAP alone.}
|
|
|
|
|
We study when an egocentric agent should enter or yield a multiparty floor, not
|
|
|
|
|
merely whether someone will speak. The central question is model-agnostic:
|
|
|
|
|
whether frozen or parameter-efficiently adapted omni representations expose
|
|
|
|
|
the fine-grained floor cues required by a 5-Hz controller, and whether causal
|
|
|
|
|
local evidence closes any gap. Dense activity forecasting supplies a common
|
|
|
|
|
target, while matched-recall event evaluation tests whether an apparent gain
|
|
|
|
|
actually prevents intrusions during multi-other exchanges. Current Qwen-3B
|
|
|
|
|
results are exploratory motivation only. \todo{After the controlled study,
|
|
|
|
|
state which backbone/adapter was selected, whether the representation pattern
|
|
|
|
|
generalized beyond Qwen, the quantitative false-entry and onset/yield results,
|
|
|
|
|
and the main deployment limitation. Do not infer success from aggregate mAP or
|
|
|
|
|
selected examples alone.}
|
|
|
|
|
|
|
|
|
|
\bibliographystyle{IEEEbib}
|
|
|
|
|
\bibliography{references}
|
|
|
|
|
|
Xin Wang