From f4a7c01552b974250f1b487be1909b0188fa4045 Mon Sep 17 00:00:00 2001
From: Xin Wang <wangx@macmini.com>
Date: Mon, 14 Sep 2026 16:04:49 +0800
Subject: [PATCH] Update evaluate protocol

---
 manuscript/main/main.tex | 46 +++++++++++++++++++++++-----------------
 1 file changed, 27 insertions(+), 19 deletions(-)

diff --git a/manuscript/main/main.tex b/manuscript/main/main.tex
index 7be85f4..8bbf4e2 100644
--- a/manuscript/main/main.tex
+++ b/manuscript/main/main.tex
@@ -231,25 +231,31 @@ camera-wearer, and aggregate non-wearer activity. Second, the original
 participant timeline is retained to derive committed interaction events for
 behavioral evaluation.
 
-We consider two wearer-relative events. An \emph{EOT-response} denotes a
-non-wearer floor release followed by the wearer taking the floor, whereas an
-\emph{INT-yield} denotes a non-wearer takeover while the wearer is speaking,
-followed by the wearer relinquishing the floor. Candidate events are generated
-automatically from participant-level onset/offset annotations. Mid-turn pauses,
-other-to-other handovers, backchannels, and non-takeover overlaps are retained
-as hard-negative regions rather than positive events. Annotation-corrupted or
-unresolvable boundaries are excluded from scoring. Ambiguous candidates are
-manually adjudicated.
+An \emph{EOT-response} is an observed wearer take-turn after a non-wearer floor
+release; an \emph{INT-yield} is an observed wearer floor release after a
+non-wearer takeover during wearer speech.
+
+\textbf{Hard-negative region definition.}
+Committed positive events and hard-negative regions are derived separately
+from the participant-resolved speech timeline. Positive EOT-response and
+INT-yield events are anchored by observed wearer behavior. In contrast,
+hard-negative regions do not require any wearer transition. For example, a
+mid-turn pause followed by the same non-wearer resuming speech, or an
+other-to-other floor transfer with the wearer remaining silent throughout,
+defines an EOT hard-negative region: a take-turn commitment within this region
+is counted as a false positive. Likewise, brief backchannels or non-takeover
+overlaps during wearer speech define INT hard-negative regions. Ambiguous or
+annotation-corrupted regions are excluded from scoring.
 
 Participant identity is used only to construct these evaluation annotations
 and is never provided to the prediction model. Provided person boxes are used
 only for the causal visible-person input; transcripts, target-of-speech labels,
 close-microphone audio, and future observations are unavailable to the model.
 \todo{Report duration and event statistics: EOT positives/hard negatives,
-INT positives/hard negatives, excluded candidates, and manual-adjudication
-count for train/development and test. Freeze the event-construction and
-adjudication protocol before reporting semantic EOT/INT results; until then,
-retain the timing-derived proxy terminology.}
+INT positives/hard negatives, excluded candidates, and audited-region counts
+for train/development and test. Freeze the event-construction and region-audit
+protocol before reporting semantic EOT/INT results; until then, retain the
+timing-derived proxy terminology.}
 
 \subsection{Model Configuration and Training}
 
@@ -276,12 +282,14 @@ EgoSpeak, reporting AP at the 0.2--2.0~s horizons and their mean mAP.
 
 Behavioral evaluation follows a committed-event protocol. For both
 EOT-response and INT-yield, each gold event is matched to at most one committed
-prediction and is scored by recall and signed detection latency. A prediction
-inside a task-specific hard-negative region is counted as a false positive.
-EOT hard negatives include mid-turn continuations and competing
-other-to-other floor transfers; INT hard negatives include backchannels and
-overlaps in which the wearer retains the floor. Regions excluded during
-adjudication do not contribute to either the positive or negative denominator.
+prediction and is scored by recall and signed detection latency.
+
+\textbf{False-positive scoring rule.}
+A false positive is defined by the model committing inside a labeled
+hard-negative region, regardless of whether the wearer subsequently exhibits
+an onset or offset. Thus, false-positive evaluation includes conversational
+situations in which the correct wearer behavior is to remain silent or retain
+the floor.
 
 We report the recall--false-positive trade-off and a common operating point at
 matched recall $R=0.8$. The primary behavioral quantities are therefore EOT