wx44wx/icassp2027-experiment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Delegate wearer boundary semantics to review

Browse Source
This commit is contained in:
Xin Wang
2026-09-15 02:12:18 +08:00
parent 71dfd8808a
commit 3a127a3f9f
12 changed files with 596 additions and 404 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
PROJECT_CONTEXT.md
View File

@@ -304,7 +304,7 @@ Do not reorganize them physically without compatibility wrappers.
| `derive_easycom_interaction_events.py` | Deterministic transcript/timing proxies |
| `adjudicate_easycom_interaction_events.py` | Optional ambiguous-event adjudication |
| `audit_easycom_floor_events.py` | Count EOT/takeover/backchannel candidates |
| `prepare_easycom_eot_int_review.py` | Group short non-competed wearer resumptions into floor episodes; retain every resulting wearer onset as an entry candidate with optional preceding-release metadata, require `wearer_floor_start < other_onset < wearer_offset` plus boundary crossing for INT, preserve ineligible raw boundaries as controls, and scan independent hard negatives; semantic labels remain AI/human review outputs |
| `prepare_easycom_eot_int_review.py` | Retain every raw wearer onset and offset as an unresolved semantic candidate; attach nearby participant boundaries, activity relations, transcripts, and target-of-speech facts without entry/takeover gating; also scan independent hard-negative regions; semantic labels remain AI/human review outputs |
| `adjudicate_easycom_eot_int_candidates.py` | Add optional DeepSeek transcript/timing suggestions; never human ground truth |
| `review_floor_events.py` | Serve the local video/transcript workspace and persist human event reviews |
| `prepare_easycom_oracle_features.py` | Build explicit past-only oracle person-activity sidecars and analysis-only addressee controls |

53
adjudicate_easycom_eot_int_candidates.py
View File

@@ -23,10 +23,11 @@ from typing import Any, Sequence, TextIO
import urllib.error
import urllib.request
from easycom_event_taxonomy import EXCLUDE_REASONS, exclude_reason_group
from prepare_easycom_eot_int_review import requires_semantic_adjudication
PROMPT_VERSION = "easycom-wearer-entry-int-prelabel-v12"
PROMPT_VERSION = "easycom-wearer-boundary-semantic-prelabel-v14"
DEFAULT_MODEL = "deepseek-v4-flash"
DEFAULT_BASE_URL = "https://api.deepseek.com"
VERDICTS = {"POSITIVE", "HARD_NEGATIVE", "EXCLUDE"}
@@ -40,8 +41,6 @@ HARD_NEGATIVE_TYPES = {
"INT_WEARER_HOLDS_FLOOR",
"OTHER_HARD_NEGATIVE",
}
def _progress_line(
completed: int, total: int, *, label: str, elapsed_seconds: float, width: int = 26
) -> str:
@@ -91,10 +90,10 @@ SYSTEM_PROMPT = """You conservatively pre-label candidate regions for a scientif
Evidence rules:
1. Reconstruct the event chronologically before assigning semantics. Use exact start/end timestamps to identify (a) the proposed release or competing onset, (b) the wearer behavior anchor, (c) which same participant continues across the anchor, and (d) who produces the next substantive turn. Then use transcript content and Target_of_Speech, when supplied, to decide conversational function. Timing has priority over speculative transcript interpretation, but duration alone never determines floor function.
2. Never alter or propose an event timestamp. Do not infer prosody, gaze, intention, or acoustic cues that were not supplied.
3. A raw wearer onset is not automatically a floor entry; a wearer offset is not automatically a yield; an other onset is not automatically a takeover. A pause is not automatically a release, overlap is not automatically interruption, and words such as "yeah" are not automatically backchannels. Determine what the utterance does in context, not only how long it lasts.
3. Every supplied WEARER_ONSET and WEARER_OFFSET is a raw observed wearer activity boundary. Candidate generation has made no floor-entry, within-turn-resumption, interruption, or takeover decision. A raw wearer onset is not automatically a floor entry; a wearer offset is not automatically a yield; an other onset is not automatically a takeover. A pause is not automatically a release, overlap is not automatically interruption, and words such as "yeah" are not automatically backchannels. Determine what the utterance does in context, not only how long it lasts.
4. EXCLUDE is the sole non-scoring outcome. Use it both for insufficient/unresolvable evidence and for clear out-of-scope non-events that should not enter evaluation, including a normal wearer completion that merely coincides with laughter or feedback. Never force such a region into HARD_NEGATIVE for coverage.
5. Every supplied WEARER_ONSET is an observed floor-level onset after deterministic within-turn pause merging. Decide whether the wearer genuinely enters or re-enters the conversational floor. A POSITIVE WEARER_ENTRY does not require a preceding non-wearer release: it may occur after a release, from silence, or while another speaker is active. temporal_evidence.opportunity_options and entry_context_timing are context metadata only. If one supplied release clearly prompted the entry, selected_opportunity_uid may contain its uid; otherwise return null. Never invent a trigger or timestamp. A direct, semantically complete answer can be a genuine entry even when short. Do not call it a backchannel merely because it lasts less than one second or because another participant speaks later. For example, "Nope." beginning 0.15 s after an addressed question ends is normally WEARER_ENTRY, not ENTRY_WEARER_BACKCHANNEL. Later conversation does not retroactively cancel a completed entry.
6. Every supplied WEARER_OFFSET has already passed both deterministic INT prerequisites: a new non-wearer onset occurs strictly inside the current wearer floor episode (t_w < t_o < t_y), and that same non-wearer interval crosses the wearer yield boundary within the recorded annotation tolerance. This is only a retrieval gate. Verify that the same incoming non-wearer actually takes and sustains the next floor after the wearer offset. A different participant's later substantive turn is not evidence that the incoming speaker took over. Do not label INT_WEARER_HOLDS_FLOOR merely because the wearer speaks again later: if the competing other establishes an intervening substantive floor and the wearer has an intervening silent interval, the later wearer onset is a new WEARER_ENTRY and does not invalidate the earlier INT_YIELD. If the wearer reaches a semantically complete natural ending while the overlapping contribution is only [L] or feedback, and that contributor does not take the next substantive floor, return EXCLUDE: out-of-scope normal completion, not INT_YIELD and not a scored hard negative. Pre-existing other activity, post-completion other onset, within-turn wearer pauses, and completion-to-silence are controls outside this queue.
5. For every WEARER_ONSET, decide whether the wearer genuinely enters or re-enters the conversational floor. First inspect the previous wearer offset and silence gap, gap_other_intervals, nearby other activity, transcript, and Target_of_Speech. A short within-turn pause followed by continuation of the same wearer contribution is not a new entry; use EXCLUDE unless the region is a valid named false-positive opportunity. A POSITIVE WEARER_ENTRY does not require a preceding non-wearer release: it may occur after a release, from silence, or while another speaker is active. temporal_evidence.opportunity_options contains nearby preceding other offsets with factual relation flags, not pre-approved causes. If one supplied release clearly prompted the entry, selected_opportunity_uid may contain its uid; otherwise return null. Never invent a trigger or timestamp. A direct, semantically complete answer can be a genuine entry even when short. Do not call it a backchannel merely because it lasts less than one second or because another participant speaks later. For example, "Nope." beginning 0.15 s after an addressed question ends is normally WEARER_ENTRY, not ENTRY_WEARER_BACKCHANNEL. Later conversation does not retroactively cancel a completed entry.
6. For every WEARER_OFFSET, answer the central question: did the wearer stop because a non-wearer genuinely took over the conversational floor? Reconstruct the wearer activity-run start and offset, every nearby other onset and offset, which others are active immediately before and after the offset, whether each other is pre-existing or starts during wearer speech, whether it crosses the offset, overlap duration, who speaks next, transcript content, and Target_of_Speech. All temporal booleans are descriptive evidence, not prerequisite labels. A valid INT_YIELD may involve a newly entering or pre-existing competitor if the chronology and semantics establish genuine takeover and wearer relinquishment. Conversely, crossing the offset or satisfying t_w < t_o < t_y never proves takeover by itself. Verify that the relevant non-wearer actually takes and sustains the next floor. A different participant's later substantive turn is not evidence that the proposed competitor took over. Do not label INT_WEARER_HOLDS_FLOOR merely because the wearer speaks again later: if a competitor establishes an intervening substantive floor and the wearer has an intervening silent interval, the later wearer onset is a new WEARER_ENTRY and does not invalidate the earlier INT_YIELD. Natural completion, completion-to-silence, a within-turn wearer pause, or a post-completion other onset that could not have caused the stop is normally EXCLUDE unless the region is a valid named hard-negative opportunity. If the wearer reaches a semantically complete natural ending while overlapping activity is only [L] or feedback and that contributor does not take the next substantive floor, return EXCLUDE.
7. Transcript event markers are evidence, not lexical turns. In EasyCom, a transcription consisting of [L] denotes laughter-only vocalization. Do not treat [L] as a sustained floor entry or interruption takeover merely because its activity interval creates an onset, offset, or overlap candidate. Specifically:
- wearer [L] during a non-wearer's retained turn is normally HARD_NEGATIVE / ENTRY_WEARER_BACKCHANNEL, not WEARER_ENTRY;
- non-wearer [L] overlapping a wearer's retained turn is normally HARD_NEGATIVE / INT_OTHER_BACKCHANNEL, not INT_YIELD;
@@ -106,6 +105,19 @@ Top-level verdicts:
- HARD_NEGATIVE: a deliberately retained, valid false-positive opportunity matching one of the behavioral counterexamples below; it enters evaluation as a negative.
- EXCLUDE: a non-scoring region. This includes clear out-of-scope/non-target opportunities such as natural wearer completion coincident with feedback, as well as insufficient evidence, annotation corruption, a recording boundary, missing context, a non-speech artifact, timestamp corruption, or another unresolvable defect.
EXCLUDE requires exactly one exclude_reason from this closed taxonomy:
- OUT_OF_SCOPE_NORMAL_COMPLETION: the wearer naturally completes; no interruption-yield target occurs.
- OUT_OF_SCOPE_WITHIN_TURN_CONTINUATION: a raw boundary is only a pause/resumption within the same wearer contribution.
- OUT_OF_SCOPE_POST_COMPLETION_ACTIVITY: another contribution begins only after wearer completion and cannot have caused the boundary.
- OUT_OF_SCOPE_NON_EVENT: another clear, interpretable non-target boundary not covered above.
- INSUFFICIENT_EVIDENCE: the supplied valid evidence cannot resolve conversational function.
- DATA_ANNOTATION_ERROR: participant, speech, transcript, or target annotation is evidently erroneous or contradictory.
- DATA_RECORDING_BOUNDARY: the recording/clip boundary prevents reliable interpretation.
- DATA_MISSING_CONTEXT: required context is absent from the supplied annotation window or source.
- DATA_NON_SPEECH_ARTIFACT: the boundary is caused by an artifact rather than an annotated conversational vocalization.
- DATA_TIMESTAMP_CORRUPTION: timestamps are corrupted or irreconcilably misaligned.
The OUT_OF_SCOPE_* reasons describe valid data outside the evaluation target. DATA_* reasons describe bad or unusable data. Never substitute one family for the other.
POSITIVE event_type must be exactly WEARER_ENTRY or INT_YIELD. WEARER_ENTRY means the wearer genuinely enters or re-enters the floor with a contextually complete response or substantive contribution; no prior release is required and the contribution need not be long. INT_YIELD means a non-wearer genuinely takes over during wearer speech and the wearer relinquishes the floor. A later WEARER_ENTRY after an intervening other floor is a new event and must not retroactively invalidate the preceding INT_YIELD.
HARD_NEGATIVE subtypes are behavioral counterexamples, not data-quality labels:
@@ -124,12 +136,12 @@ Subtype precedence when descriptions overlap:
4. Use OTHER_HARD_NEGATIVE only after the named subtypes have been ruled out. Never use it merely because confidence is low.
Return exactly one JSON object with this shape and no markdown:
{"items":[{"uid":"...","verdict":"EXCLUDE","event_type":null,"subtype":null,"selected_opportunity_uid":null,"confidence":0.25,"reason":"brief evidence-based sentence"}]}
{"items":[{"uid":"...","verdict":"EXCLUDE","event_type":null,"subtype":null,"exclude_reason":"INSUFFICIENT_EVIDENCE","selected_opportunity_uid":null,"confidence":0.25,"reason":"brief evidence-based sentence"}]}
Field combinations are strict:
- POSITIVE / WEARER_ENTRY: subtype=null; selected_opportunity_uid is one supplied opportunity uid or null.
- POSITIVE / INT_YIELD: subtype=null; selected_opportunity_uid=null.
- HARD_NEGATIVE: event_type=null; selected_opportunity_uid=null; subtype is one valid subtype from the candidate's ENTRY or INT family.
- EXCLUDE: event_type=null; subtype=null; selected_opportunity_uid=null.
- POSITIVE / WEARER_ENTRY: subtype=null; exclude_reason=null; selected_opportunity_uid is one supplied opportunity uid or null.
- POSITIVE / INT_YIELD: subtype=null; exclude_reason=null; selected_opportunity_uid=null.
- HARD_NEGATIVE: event_type=null; exclude_reason=null; selected_opportunity_uid=null; subtype is one valid subtype from the candidate's ENTRY or INT family.
- EXCLUDE: event_type=null; subtype=null; selected_opportunity_uid=null; exclude_reason is exactly one taxonomy value above.
Return every supplied uid exactly once. Copy uid byte-for-byte and never add a backslash before an underscore. confidence must be a number from 0 to 1."""
@@ -245,6 +257,7 @@ def validate_decisions(
verdict = item.get("verdict")
event_type = item.get("event_type")
subtype = item.get("subtype")
exclude_reason = item.get("exclude_reason")
selected_opportunity_uid = item.get("selected_opportunity_uid")
confidence = item.get("confidence")
reason = item.get("reason")
@@ -260,6 +273,10 @@ def validate_decisions(
raise ValueError(f"HARD_NEGATIVE requires a valid subtype for {uid}")
if verdict != "HARD_NEGATIVE" and subtype is not None:
raise ValueError(f"Only HARD_NEGATIVE may have subtype for {uid}")
if verdict == "EXCLUDE" and exclude_reason not in EXCLUDE_REASONS:
raise ValueError(f"EXCLUDE requires a valid exclude_reason for {uid}")
if verdict != "EXCLUDE" and exclude_reason is not None:
raise ValueError(f"Only EXCLUDE may have exclude_reason for {uid}")
family = None if expected_families is None else expected_families.get(str(uid))
if verdict == "POSITIVE" and family in {"ENTRY", "INT"}:
required_event_type = "WEARER_ENTRY" if family == "ENTRY" else "INT_YIELD"
@@ -287,6 +304,8 @@ def validate_decisions(
"verdict": str(verdict),
"event_type": event_type,
"subtype": subtype,
"exclude_reason": exclude_reason,
"exclude_reason_group": exclude_reason_group(exclude_reason),
"selected_opportunity_uid": selected_opportunity_uid,
"confidence": float(confidence),
"reason": reason.strip(),
@@ -437,12 +456,12 @@ def add_suggestions(
for source in rows:
decisions = by_uid[source["uid"]]
labels = {
(d["verdict"], d["event_type"], d["subtype"])
(d["verdict"], d["event_type"], d["subtype"], d["exclude_reason"])
for d in decisions
}
unanimous = len(labels) == 1
if unanimous:
verdict, event_type, subtype = next(iter(labels))
verdict, event_type, subtype, exclude_reason = next(iter(labels))
selected_options = {d["selected_opportunity_uid"] for d in decisions}
selected_opportunity_uid = (
next(iter(selected_options)) if len(selected_options) == 1 else None
@@ -451,7 +470,9 @@ def add_suggestions(
reason = " | ".join(item["reason"] for item in decisions)
needs_human_review = any(item["needs_human_review"] for item in decisions)
else:
verdict, event_type, subtype, selected_opportunity_uid = "EXCLUDE", None, None, None
verdict, event_type, subtype, exclude_reason, selected_opportunity_uid = (
"EXCLUDE", None, None, "INSUFFICIENT_EVIDENCE", None
)
confidence = min(float(item["confidence"]) for item in decisions)
reason = "The independent DeepSeek passes disagreed."
needs_human_review = True
@@ -474,6 +495,8 @@ def add_suggestions(
"verdict": verdict,
"event_type": event_type,
"subtype": subtype,
"exclude_reason": exclude_reason,
"exclude_reason_group": exclude_reason_group(exclude_reason),
"selected_opportunity_uid": selected_opportunity_uid,
"selected_opportunity_seconds": selected_opportunity_seconds,
"valid_prediction_interval_seconds": (
@@ -490,6 +513,8 @@ def add_suggestions(
ai_verdict=verdict,
ai_event_type=event_type,
ai_subtype=subtype,
ai_exclude_reason=exclude_reason,
ai_exclude_reason_group=exclude_reason_group(exclude_reason),
ai_selected_opportunity_uid=selected_opportunity_uid,
ai_selected_opportunity_seconds=selected_opportunity_seconds,
ai_confidence=confidence,

102
docs/easycom_event_review_tool.md
View File

@@ -6,7 +6,7 @@ ground truth.
```text
EasyCom participant speech intervals
-> floor-episode grouping + rule candidates + structured timing evidence
-> all wearer boundaries + independent rule regions + structured factual evidence
-> DeepSeek semantic pre-label (optional but recommended)
-> human Accept / Relabel / Exclude
```
@@ -25,47 +25,37 @@ python3 prepare_easycom_eot_int_review.py \
The generator emits two independent files:
- `easycom_eot_int_candidates.jsonl` (legacy-compatible filename) contains every
floor-level wearer `0->1` behavior anchor (`WEARER_ONSET`,
`WEARER_ENTRY_CANDIDATE`), only episode-ending wearer
`1->0` boundaries where a new non-wearer onset satisfies
`wearer_floor_start < other_onset < wearer_offset` and the same activity
crosses the offset (`WEARER_OFFSET`, `YIELD_CANDIDATE`), and hard-negative
raw wearer `0->1` boundary (`WEARER_ONSET`, `WEARER_ENTRY_CANDIDATE`), every
raw wearer `1->0` boundary (`WEARER_OFFSET`, `YIELD_CANDIDATE`), and hard-negative
regions that need no wearer transition:
`SAME_SPEAKER_PAUSE`, `OTHER_TO_OTHER`, `OTHER_BACKCHANNEL`, and
`NON_TAKEOVER_OVERLAP`;
- `easycom_eot_int_controls.jsonl` contains short within-turn raw onsets
(`WEARER_RESUME`) and wearer offsets that fail
the strict INT ordering/crossing prerequisite (`WEARER_OFFSET_CONTROL`). They
are retained for audits, but are not sent to DeepSeek or placed in the
primary human-review queue.
- `easycom_eot_int_controls.jsonl` is retained for file-format compatibility
and legacy inputs. The current boundary protocol does not deterministically
demote wearer onsets or offsets into this file.
A raw wearer onset is merged into its preceding floor episode only when its
preceding silence is at most 0.8 seconds and no plausible sustained non-wearer
takeover occurs in that gap. Short feedback-like lexical or nonverbal activity
is recorded but does not by itself split the episode. A non-feedback other
contribution that occupies at least `continuation_min_seconds` of the gap or
continues across the resumed onset conservatively preserves a new wearer-entry
candidate. Every decision is stored in `temporal_evidence`, including the raw
gap, feedback IDs, competing IDs, threshold, and `floor_episode_merged` flag.
No short-gap merge or takeover heuristic controls wearer-boundary admission.
For each raw onset, `temporal_evidence` stores the previous wearer offset,
silence duration, every other-speaker interval intersecting that gap, interval
coverage, whether it continues across the onset, transcript, and
`Target_of_Speech`. DeepSeek and the human reviewer decide whether the onset is
a genuine entry, within-turn resumption, backchannel, or excluded non-event.
Entry construction retains every floor-level wearer onset, then searches backward
for preceding non-wearer releases as optional context. A release is an admissible option only when the
wearer stays silent and no new non-wearer onset or already-active other floor
invalidates the interval before the wearer onset. Silence length alone does not
invalidate an option. Each wearer onset produces exactly one primary row. A
missing release does not demote an entry to a control: an entry may occur from
silence or while another speaker is active. `entry_context_timing` records
`AFTER_OTHER_RELEASE`, `OTHER_ACTIVE`, or `FROM_SILENCE_OR_UNRESOLVED` as
metadata. When present, `opportunity_options` contains the only release anchors
that DeepSeek or a human may optionally select; neither may invent a timestamp.
Entry construction searches backward for nearby non-wearer releases as optional
context. Every option carries factual flags for intervening wearer activity,
intervening other onsets, and activity at the wearer onset; the generator does
not declare one a valid conversational cause. Each wearer onset produces exactly
one primary row. When present, `opportunity_options` contains the only release
anchors that DeepSeek or a human may optionally select; neither may invent a
timestamp.
Floor-level wearer onsets and strictly qualified offsets carry the rule-stage value
`candidate`; timing alone is not a semantic positive label. Deterministic rows
outside the primary queue carry `control`, never a semantic verdict. The INT prerequisite is
`wearer_floor_start < other.start < wearer_offset < other.end`, with a
configurable one-frame (50 ms) tolerance only at the offset boundary.
Pre-existing other activity and raw offsets inside a merged wearer episode stay
in the controls file. Hard-negative scans are rule hints and still require AI
All wearer onsets and offsets carry the rule-stage value `candidate`; timing
alone is not a semantic positive label. For each offset, the JSON records the
wearer activity-run start, nearby other intervals, pre-existing and new-other
flags, activity immediately before/after the boundary, strict and configurable
50-ms-tolerant crossing, overlap duration, and subsequent speech onsets.
`wearer_start < other.start < wearer_offset < other.end` is evidence rather than
an INT prerequisite. Hard-negative scans are rule hints and still require AI
and human review. The JSONL retains candidate source, original event time,
participant-level activity, raw durations/gaps/overlap evidence, transcript,
and `Target_of_Speech` when present. `Target_of_Speech` is annotation assistance
@@ -80,7 +70,6 @@ All heuristic durations are configurable rather than frozen paper rules:
--short-vocalization-max-seconds
--continuation-min-seconds
--int-offset-tolerance-seconds
--wearer-resume-merge-max-seconds
```
Use `--max-candidates 1` for a one-event smoke test. `summary.json` records the
@@ -90,7 +79,7 @@ timeline; it is not sent to DeepSeek.
## 2. Add DeepSeek pre-labels
Prompt v11 first reconstructs the exact participant-resolved chronology and
Prompt v14 first reconstructs the exact participant-resolved chronology and
then uses transcript content and `Target_of_Speech` to determine conversational
function. It treats the EasyCom transcript marker `[L]` as laughter-only evidence,
not as a lexical floor take. Wearer laughter during another participant's
@@ -98,8 +87,8 @@ retained turn is directed toward `ENTRY_WEARER_BACKCHANNEL`; non-wearer laughter
overlapping a retained wearer turn is directed toward `INT_OTHER_BACKCHANNEL`.
Ambiguous laughter falls back to `EXCLUDE`, and `[L]` alone is neither evidence
of a floor take nor evidence of a visible smile. Preceding release options are
optional entry metadata, whereas strict INT ordering remains a deterministic
retrieval prerequisite. A positive `WEARER_ENTRY` may optionally select one
optional entry metadata. No entry/within-turn or INT/takeover semantic decision
is made before the prompt. A positive `WEARER_ENTRY` may optionally select one
supplied release; entries from silence or during other activity use `null`.
`needs_human_review` is not requested from DeepSeek: every pre-label enters the
@@ -135,10 +124,14 @@ transcript together. It returns one of `POSITIVE`, `HARD_NEGATIVE`, or
`WEARER_ENTRY` or `INT_YIELD`, and hard negatives receive a named subtype. It
cannot modify timestamps. The output preserves the original rule candidate,
the selected preceding-release UID when applicable, and the complete AI
decision/provenance. `EXCLUDE` is the sole non-scoring outcome. It covers
out-of-scope non-events (including a normal wearer completion that merely
coincides with feedback), insufficient evidence, unresolvable data quality, or
disagreement between the two model passes.
decision/provenance. `EXCLUDE` is the sole non-scoring outcome, but it is never
an untyped bucket. Every AI and human exclusion stores one closed-vocabulary
`exclude_reason` and the derived `exclude_reason_group`. `OUT_OF_SCOPE_*` means
the annotations are valid but the behavior is outside the scored target;
`INSUFFICIENT_EVIDENCE` means the supplied valid material cannot resolve
conversational function; and `DATA_*` identifies unusable annotation,
recording, context, artifact, or timestamp quality. Model-pass disagreement is
recorded as `INSUFFICIENT_EVIDENCE`, never as data corruption.
Hard negatives use behavior-specific subtypes. ENTRY distinguishes a same-other
mid-turn pause, a different-other transition, and a wearer backchannel that
@@ -156,8 +149,9 @@ all candidate/pass work, including the current pass, batch, percentage, and
elapsed time. Redirected logs emit periodic progress lines instead of carriage
return animation.
As defense in depth, the adjudicator also detects and skips non-crossing
`WEARER_OFFSET` rows from older v2 candidate files before making any API call.
Every current `WEARER_ONSET` and `WEARER_OFFSET` row is sent for semantic
adjudication regardless of its timing pattern. Explicit control rows from older
protocol files remain excluded unless controls are intentionally reviewed.
## 3. Human review
@@ -192,22 +186,18 @@ decision, relabel flag, exclusion state, confidence, and provenance. The current
UI does not ask for a reviewer name; the export retains an empty `reviewer` field
only for compatibility with older review files.
The primary review queue also filters legacy non-crossing `WEARER_OFFSET` rows
and explicit `WEARER_RESUME` controls. To audit the controls intentionally,
The primary review queue filters only explicit legacy control rows. To audit
those controls intentionally,
pass `--events .../easycom_eot_int_controls.jsonl --include-controls` and use a
separate review output file. The old `--include-offset-controls` spelling
remains an alias.
Protocol wording for the paper appendix:
> Raw wearer speech onsets are first grouped into floor-level speech episodes
> by merging short within-turn resumptions that are not separated by a
> competing floor takeover. Every resulting floor-level 0→1 transition is
> proposed as a wearer-entry candidate; a preceding non-wearer release is
> retained as context rather than required for positive status.
> The original speech intervals are preserved unchanged for dense forecasting
> and auditability; this grouping is used only for behavioral event annotation.
> Every observed wearer speech onset and offset is retained as a semantic review
> candidate. Deterministic construction records only participant-resolved timing,
> transcript, and target-of-speech facts; it does not infer within-turn
> resumption, floor entry, interruption, or takeover.
## Freeze discipline

44
easycom_event_taxonomy.py Normal file
View File

@@ -0,0 +1,44 @@
"""Closed vocabularies shared by EasyCom event pre-label and review tools."""
from __future__ import annotations
EXCLUDE_REASONS = {
"OUT_OF_SCOPE_NORMAL_COMPLETION",
"OUT_OF_SCOPE_WITHIN_TURN_CONTINUATION",
"OUT_OF_SCOPE_POST_COMPLETION_ACTIVITY",
"OUT_OF_SCOPE_NON_EVENT",
"INSUFFICIENT_EVIDENCE",
"DATA_ANNOTATION_ERROR",
"DATA_RECORDING_BOUNDARY",
"DATA_MISSING_CONTEXT",
"DATA_NON_SPEECH_ARTIFACT",
"DATA_TIMESTAMP_CORRUPTION",
}
# Accepted only when loading or editing old review records. New DeepSeek and UI
# outputs use the prefixed taxonomy above so scope and data-quality counts are
# never silently combined.
LEGACY_EXCLUDE_REASONS = {
"ANNOTATION_ERROR",
"RECORDING_BOUNDARY",
"MISSING_CONTEXT",
"NON_SPEECH_ARTIFACT",
"TIMESTAMP_CORRUPTION",
"UNRESOLVABLE",
}
def exclude_reason_group(reason: str | None) -> str | None:
"""Map a leaf reason to a stable aggregation group."""
if reason is None:
return None
if reason.startswith("OUT_OF_SCOPE_"):
return "OUT_OF_SCOPE"
if reason == "INSUFFICIENT_EVIDENCE":
return "INSUFFICIENT_EVIDENCE"
if reason.startswith("DATA_"):
return "DATA_QUALITY"
if reason in LEGACY_EXCLUDE_REASONS:
return "LEGACY_UNSPECIFIED"
return "UNKNOWN"

23
floor_event_review_web/app.js
View File

@@ -57,13 +57,23 @@ const decisionMeanings = {
INT_WEARER_HOLDS_FLOOR: "Other 不是单纯 backchannel，且 wearer 在重叠后明确继续或恢复原 turn。",
OTHER_HARD_NEGATIVE: "这是可解释的困难负例，但不符合任何命名子类型；不能用于替代 Exclude。",
EXCLUDE: "该区域是无需参评的非目标事件，或存在证据/数据问题，不进入评分。",
OUT_OF_SCOPE_NORMAL_COMPLETION: "有效语境中的自然结束，不是 interruption yield。",
OUT_OF_SCOPE_WITHIN_TURN_CONTINUATION: "这是同一 wearer turn 内的暂停或继续，不是新的 floor event。",
OUT_OF_SCOPE_POST_COMPLETION_ACTIVITY: "Other 在 wearer 已结束后才开始，不能构成导致停止的 takeover。",
OUT_OF_SCOPE_NON_EVENT: "数据有效且可解释，但该边界不属于目标行为。",
INSUFFICIENT_EVIDENCE: "数据未显示明确损坏，但现有证据不足以可靠确定 conversational function。",
DATA_ANNOTATION_ERROR: "参与者、语音、转写或 target 标注明显错误或互相矛盾。",
DATA_RECORDING_BOUNDARY: "录制或 clip 边界使该事件无法可靠解释。",
DATA_MISSING_CONTEXT: "判断所需上下文未被提供。",
DATA_NON_SPEECH_ARTIFACT: "候选由非语音伪影造成。",
DATA_TIMESTAMP_CORRUPTION: "时间戳损坏或无法对齐。",
};
const currentAiVerdicts = new Set(["POSITIVE", "HARD_NEGATIVE", "EXCLUDE"]);
function suggestionView(suggestion) {
const verdict = suggestion.verdict || suggestion.protocol_disposition || "—";
const label = suggestion.event_type || suggestion.subtype || suggestion.hard_negative_type || verdict;
const label = suggestion.event_type || suggestion.subtype || suggestion.exclude_reason || suggestion.hard_negative_type || verdict;
const rawConfidence = suggestion.confidence ?? suggestion.final_confidence;
const numericConfidence = Number(rawConfidence);
const confidence = Number.isFinite(numericConfidence)
@@ -170,7 +180,7 @@ function applyFilters({ keepCurrent = true } = {}) {
(elements.splitFilter.value === "all" || item.split === elements.splitFilter.value) &&
(elements.familyFilter.value === "all" || item.family === elements.familyFilter.value) &&
(status === "all" || (status === "reviewed" ? reviewed : !reviewed)) &&
(!search || `${item.uid} ${item.candidate_source || ""} ${item.rule_candidate_type || ""} ${item.ai_verdict || ""} ${item.ai_event_type || ""} ${item.ai_subtype || ""} ${item.session}`.toLowerCase().includes(search))
(!search || `${item.uid} ${item.candidate_source || ""} ${item.rule_candidate_type || ""} ${item.ai_verdict || ""} ${item.ai_event_type || ""} ${item.ai_subtype || ""} ${item.ai_exclude_reason || ""} ${item.session}`.toLowerCase().includes(search))
);
});
renderQueue();
@@ -206,7 +216,7 @@ function renderQueue() {
const label = document.createElement("div");
label.className = "queue-label";
label.textContent = item.ai_verdict
? `AI ${item.ai_verdict} · ${item.ai_event_type || item.ai_subtype || "—"}`
? `AI ${item.ai_verdict} · ${item.ai_event_type || item.ai_subtype || item.ai_exclude_reason || "—"}`
: `${item.rule_candidate_type || item.automatic_label} · 待 AI 粗标`;
button.append(line, meta, label);
button.addEventListener("click", () => selectCandidate(item.candidate_id));
@@ -292,7 +302,8 @@ function renderDetail(detail) {
const familyPrefix = detail.family === "entry" ? "ENTRY_" : detail.family === "int" ? "INT_" : null;
const canAccept = currentAiVerdicts.has(view.verdict)
&& (view.verdict !== "POSITIVE" || !familyEvent || suggestion.event_type === familyEvent)
&& (view.verdict !== "HARD_NEGATIVE" || !familyPrefix || suggestion.subtype === "OTHER_HARD_NEGATIVE" || String(suggestion.subtype || "").startsWith(familyPrefix));
&& (view.verdict !== "HARD_NEGATIVE" || !familyPrefix || suggestion.subtype === "OTHER_HARD_NEGATIVE" || String(suggestion.subtype || "").startsWith(familyPrefix))
&& (view.verdict !== "EXCLUDE" || Boolean(suggestion.exclude_reason));
elements.acceptRadio.disabled = !canAccept;
elements.acceptTitle.textContent = canAccept
? `接受 ${view.verdict.replaceAll("_", " ")}`
@@ -302,7 +313,9 @@ function renderDetail(detail) {
const anchorText = selectedOption ? ` · trigger ${formatTime(selectedOption.anchor_seconds)} P${selectedOption.participant_id}` : "";
elements.acceptSummary.textContent = canAccept
? `${view.label}${anchorText} · ${view.meaning}`
: "该标签已移除，请改标为正例/硬负例，或选择 Exclude。";
: (view.verdict === "EXCLUDE"
? "旧 AI EXCLUDE 没有 reason taxonomy，请选择 Exclude 并补充原因。"
: "该标签已移除，请改标为正例/硬负例，或选择 Exclude。");
} else {
elements.acceptRadio.disabled = true;
elements.acceptTitle.textContent = "没有 AI 结论";

2
floor_event_review_web/index.html
View File

@@ -88,7 +88,7 @@
<label id="subtypeField" hidden><span>Hard-negative subtype</span><select name="human_subtype"><option value="">请选择</option><option>ENTRY_OTHER_MIDTURN_PAUSE</option><option>ENTRY_OTHER_TO_OTHER</option><option>ENTRY_WEARER_BACKCHANNEL</option><option>INT_OTHER_BACKCHANNEL</option><option>INT_NON_TAKEOVER_OVERLAP</option><option>INT_WEARER_HOLDS_FLOOR</option><option>OTHER_HARD_NEGATIVE</option></select></label>
</section>
<section id="excludeFields" class="conditional-fields" hidden><label><span>排除原因</span><select name="exclude_reason"><option value="">请选择</option><option value="OUT_OF_SCOPE_NORMAL_COMPLETION">OUT_OF_SCOPE_NORMAL_COMPLETION — wearer 正常说完</option><option value="OUT_OF_SCOPE_NON_EVENT">OUT_OF_SCOPE_NON_EVENT — 其他无需参评事件</option><option>ANNOTATION_ERROR</option><option>RECORDING_BOUNDARY</option><option>MISSING_CONTEXT</option><option>NON_SPEECH_ARTIFACT</option><option>TIMESTAMP_CORRUPTION</option><option>UNRESOLVABLE</option></select></label></section>
<section id="excludeFields" class="conditional-fields" hidden><label><span>排除原因（分类统计）</span><select name="exclude_reason"><option value="">请选择</option><optgroup label="OUT OF SCOPE — 数据有效但不是目标行为"><option value="OUT_OF_SCOPE_NORMAL_COMPLETION">正常说完，没有 INT yield</option><option value="OUT_OF_SCOPE_WITHIN_TURN_CONTINUATION">同一 wearer turn 内暂停/续说</option><option value="OUT_OF_SCOPE_POST_COMPLETION_ACTIVITY">Other 在 wearer 结束后才开始</option><option value="OUT_OF_SCOPE_NON_EVENT">其他明确的非目标事件</option></optgroup><optgroup label="INSUFFICIENT EVIDENCE — 数据未坏但语义无法确定"><option value="INSUFFICIENT_EVIDENCE">现有证据不足以可靠判断</option></optgroup><optgroup label="DATA QUALITY — 数据或边界不可用"><option value="DATA_ANNOTATION_ERROR">标注明显错误或矛盾</option><option value="DATA_RECORDING_BOUNDARY">录制/clip 边界截断语境</option><option value="DATA_MISSING_CONTEXT">所需上下文缺失</option><option value="DATA_NON_SPEECH_ARTIFACT">非语音伪影</option><option value="DATA_TIMESTAMP_CORRUPTION">时间戳损坏或错位</option></optgroup></select></label></section>
<details class="boundary-details"><summary>仅在原 annotation 明显错误时校正边界</summary><label><span>可选校正（原时间始终保留）</span><select name="boundary_correction_seconds"><option value="">不校正</option><option value="-0.2">−0.2 s</option><option value="0.2">+0.2 s</option></select></label></details>

90
manuscript/main/main.tex
View File

@@ -233,14 +233,16 @@ participant timeline is retained to derive committed interaction events for
behavioral evaluation.
\textbf{Behavioral event annotation.}
We use floor-level wearer transitions as high-recall anchors for positive-event
annotation. Each wearer onset is examined retrospectively for a preceding
non-wearer floor release as contextual metadata and labeled as a wearer-entry
whenever the subsequent wearer speech constitutes a genuine floor take,
including re-entry while another speaker is active. Similarly, each wearer offset
is examined for a new competing non-wearer onset during the current wearer turn
and labeled as INT-yield only when the competing speaker genuinely takes over
the floor. We use permissive temporal retrieval and let an LLM jointly inspect
We use every observed wearer activity transition as a high-recall semantic
candidate. For each wearer onset, nearby preceding non-wearer releases, the
intervening silence gap, and concurrent activity are retained as contextual
facts; the onset is labeled as wearer entry only when the subsequent wearer
speech constitutes a genuine floor take, including re-entry while another
speaker is active. For each wearer offset, nearby non-wearer onsets and offsets,
pre-existing and newly active speakers, overlap durations, and subsequent
speakers are retained without a timing-based takeover gate; the offset is
labeled as INT-yield only when a non-wearer genuinely takes over and the wearer
relinquishes the floor. We let an LLM jointly inspect
participant-resolved timing and transcripts to provide a semantic pre-label,
followed by mandatory human adjudication. Separately, we mine regions with no
wearer response that nevertheless constitute false-positive opportunities:
@@ -272,7 +274,9 @@ HN: other backchannel & \todo{--} & \todo{--} \\
HN: non-takeover overlap & \todo{--} & \todo{--} \\
HN: wearer holds floor & \todo{--} & \todo{--} \\
\hline
Excluded / fallback & \todo{--} & \todo{--} \\
Excluded: valid but out of scope & \todo{--} & \todo{--} \\
Excluded: insufficient evidence & \todo{--} & \todo{--} \\
Excluded: data quality & \todo{--} & \todo{--} \\
\textbf{Scoring-eligible total} & \todo{--} & \todo{--} \\
\hline
\end{tabular}
@@ -557,42 +561,38 @@ because, by definition, many contain no wearer transition.
\subsection{Positive-event mining}
\textbf{Floor-level transitions.}
Short wearer-silence gaps are conditionally merged before positive mining. In
the current audit configuration, a raw onset after at most 0.8~s of silence is
treated as a within-turn resumption unless the gap contains sustained competing
non-wearer activity. Short feedback-like activity does not by itself split the
wearer episode. This grouping changes only the behavioral-event layer, not the
source intervals or dense targets.
\textbf{Wearer activity transitions.}
We preserve every raw wearer $0\!\rightarrow\!1$ and $1\!\rightarrow\!0$
boundary as a semantic review candidate. Candidate construction does not merge
short silence gaps, infer within-turn resumption, or decide whether overlapping
speech constitutes takeover. It only records participant-resolved temporal and
transcript facts. The source intervals and dense targets remain unchanged.
\textbf{Wearer entry.}
Every floor-level wearer onset at $t_b$ is proposed as a wearer-entry candidate.
We additionally search backward for plausible non-wearer floor releases. A
release at $t_r$ is retained as a contextual option
only when the interval from $t_r$ to $t_b$ contains neither intervening wearer
speech nor a competing non-wearer onset. Each wearer onset contributes at most
one review item: $t_b$ is its immutable \emph{behavior anchor}, and the item
stores every admissible \emph{opportunity-anchor} option and response delay.
Silence duration alone does not reject an option. A preceding release is not
required for a positive entry: the wearer may enter after a release, from
silence, or while another speaker remains active. When a release clearly
prompts the entry, the LLM or human may select one supplied option; neither may
invent or move a trigger timestamp. The release relation is metadata for
response-oriented analyses, not part of the positive entry label.
Every raw wearer onset at $t_b$ is proposed as one wearer-entry review item.
We search backward within the retrieval window for non-wearer offsets and store
each one with its response delay and factual indicators of intervening wearer
speech, intervening other onsets, and concurrent activity. These are possible
context anchors rather than deterministically admissible causes. The LLM and
human reviewer decide whether the boundary is a genuine new entry, a
within-turn resumption, a wearer backchannel, or an excluded non-event. A
preceding release is not required for a positive entry: the wearer may enter
after a release, from silence, or while another speaker remains active. When a
release clearly prompts the entry, the LLM or human may select one supplied
option; neither may invent or move a timestamp.
\textbf{INT-yield.}
Starting from an episode-ending wearer offset $t_y$, we search backward inside
the current wearer episode for a new competing non-wearer onset $t_o$. With
$t_w$ denoting the wearer-episode start, an INT candidate must satisfy
\begin{equation}
t_w < t_o < t_y < t_o^{\mathrm{off}}.
\label{eq:int_candidate}
\end{equation}
The current audit allows a one-frame (50-ms) tolerance only at the offset
boundary. Pre-existing other activity, post-completion onsets, offsets inside a
merged wearer episode, and completion-to-silence remain controls. Timing only
retrieves a possible interruption; the LLM and human annotator decide whether
the competitor genuinely takes over and the wearer yields.
Every raw wearer offset $t_y$ is proposed as an INT-yield review item. The
generator records the current wearer activity-run start, all nearby non-wearer
onsets and offsets, activity immediately before and after $t_y$, whether an
other speaker is pre-existing or begins during wearer speech, strict and
50-ms-tolerant boundary crossing, overlap duration, the next speech onset,
transcripts, and target-of-speech metadata. Relations such as
$t_w<t_o<t_y<t_o^{\mathrm{off}}$ are descriptive evidence only: they neither
admit nor reject a candidate and do not establish takeover. The LLM and human
annotator answer whether the wearer stopped because a non-wearer genuinely took
over the floor. Natural completion, post-completion activity, and within-turn
pauses are therefore resolved semantically rather than pre-filtered.
\subsection{False-positive opportunity mining}
@@ -624,9 +624,11 @@ relabel it, or exclude an
unscorable region. Only human-confirmed \textsc{Wearer-Entry},
\textsc{INT-Yield}, and hard-negative opportunities enter evaluation;
\textsc{Exclude} candidates do not. Thus \textsc{Exclude} is the sole
non-scoring outcome for out-of-scope non-events (e.g., natural wearer completion
that merely coincides with feedback), insufficient evidence, unresolvable data
quality, or pre-label disagreement.
non-scoring verdict, but every exclusion retains a closed-vocabulary reason and
derived aggregation group. We report valid-but-out-of-scope behavior (e.g.,
natural wearer completion coincident with feedback), insufficient semantic
evidence, and data-quality failures separately; model-pass disagreement belongs
to insufficient evidence and is never counted as corrupt data.
Annotation-only transcripts, identities, target-of-speech metadata, LLM output,
and future observations are never supplied to the evaluated forecasting model.

467
prepare_easycom_eot_int_review.py
View File

@@ -2,13 +2,12 @@
"""Generate wearer-centric EasyCom entry/INT candidates for AI and human review.
This stage never changes EasyCom speech boundaries and never claims semantic
ground truth. It groups short wearer resumptions into floor-level episodes,
starts entry candidates from every observed floor-level wearer onset and
retains preceding non-wearer release options as context, requires a new
non-wearer onset inside the wearer floor episode for INT candidates, and keeps
ineligible raw boundaries as deterministic controls. All timing
thresholds are CLI parameters, and raw evidence is retained for DeepSeek
pre-labeling and human correction.
ground truth. Every observed wearer onset and offset is retained as a semantic
candidate. Deterministic code only records participant-resolved timing,
transcripts, and target-of-speech facts; it never decides floor entry,
within-turn resumption, or takeover semantics. All timing windows are CLI
parameters, and raw evidence is retained for DeepSeek pre-labeling and human
correction.
"""
from __future__ import annotations
@@ -28,8 +27,8 @@ from derive_easycom_interaction_events import (
from easycom_manifest import ALL_SESSIONS, TEST_SESSIONS
SCHEMA_VERSION = 5
PROTOCOL_VERSION = "easycom-wearer-entry-int-candidates-v7"
SCHEMA_VERSION = 6
PROTOCOL_VERSION = "easycom-wearer-entry-int-candidates-v8"
@dataclass(frozen=True)
@@ -41,7 +40,6 @@ class CandidateConfig:
short_vocalization_max_seconds: float = 1.0
continuation_min_seconds: float = 0.4
int_offset_tolerance_seconds: float = 0.05
wearer_resume_merge_max_seconds: float = 0.8
@dataclass(frozen=True)
@@ -88,26 +86,13 @@ class ProtocolCandidate:
def requires_semantic_adjudication(row: dict[str, Any]) -> bool:
"""Return whether a stored row belongs in the primary AI/human queue.
Explicit legacy control sources are authoritative. For old v2 candidate
files, reconstruct the same-speaker crossing requirement from the stored
activity IDs. Missing legacy evidence remains reviewable rather than being
silently discarded.
Explicit legacy control sources are authoritative. Current WEARER_OFFSET
rows are always semantic candidates: crossing/order fields are evidence,
never admission gates.
"""
source = row.get("candidate_source")
if source in {"WEARER_ONSET_CONTROL", "WEARER_OFFSET_CONTROL", "WEARER_RESUME"}:
return False
if source != "WEARER_OFFSET":
return True
evidence = row.get("temporal_evidence")
if not isinstance(evidence, dict):
return True
explicit = evidence.get("other_crosses_wearer_offset")
if isinstance(explicit, bool):
return explicit
before = evidence.get("active_other_ids_before_offset")
after = evidence.get("active_other_ids_after_offset")
if isinstance(before, list) and isinstance(after, list):
return bool(set(before) & set(after))
return True
@@ -157,7 +142,7 @@ def _other_ids_crossing_offset(
) -> tuple[int, ...]:
"""Return other speakers whose annotated interval crosses a wearer offset.
With zero tolerance this encodes the strict prerequisite
With zero tolerance this records the strict temporal relation
``other.start < wearer_offset < other.end``. A one-frame tolerance accepts
an annotation boundary at the same 20 Hz frame, but not an other onset one
or more frames after the wearer offset.
@@ -170,26 +155,19 @@ def _other_ids_crossing_offset(
}))
def _wearer_gap_competition(
def _wearer_gap_evidence(
rows: Sequence[Utterance], *, gap_start_frame: int, onset_frame: int,
wearer_id: int, short_frames: int, continuation_frames: int,
wearer_id: int,
) -> dict[str, Any]:
"""Summarize plausible competing floor activity inside a short wearer gap.
Feedback-like short lexical/nonverbal activity is retained as evidence but
does not by itself split the wearer floor episode. Any other contribution
that occupies the gap for ``continuation_frames`` or continues across the
resumed wearer onset conservatively preserves a new ENTRY candidate.
"""
"""Return descriptive other-speaker facts within a wearer-silence gap."""
gap_rows = [
row for row in rows
if row.participant_id != wearer_id
and row.dense_start_frame < onset_frame
and row.dense_end_frame_exclusive > gap_start_frame
]
feedback_ids: set[int] = set()
competing_ids: set[int] = set()
max_gap_coverage = 0
intervals: list[dict[str, Any]] = []
for row in gap_rows:
coverage = max(
0,
@@ -197,20 +175,20 @@ def _wearer_gap_competition(
- max(row.dense_start_frame, gap_start_frame),
)
max_gap_coverage = max(max_gap_coverage, coverage)
feedback_like = (
row.dense_end_frame_exclusive - row.dense_start_frame <= short_frames
and (is_lexical_feedback(row.transcription) or is_pure_nonverbal(row.transcription))
)
if feedback_like:
feedback_ids.add(row.participant_id)
elif coverage >= continuation_frames or row.dense_end_frame_exclusive > onset_frame:
competing_ids.add(row.participant_id)
intervals.append({
"uid": row.uid,
"participant_id": row.participant_id,
"onset_seconds": row.dense_start_seconds,
"offset_seconds": row.dense_end_seconds,
"gap_overlap_seconds": coverage / SOURCE_FPS,
"continues_across_wearer_onset": row.dense_end_frame_exclusive > onset_frame,
"transcription": row.transcription,
"target_of_speech": list(row.target_of_speech),
})
return {
"gap_other_participant_ids": sorted({row.participant_id for row in gap_rows}),
"feedback_like_other_ids_during_gap": sorted(feedback_ids),
"sustained_competing_other_ids_during_gap": sorted(competing_ids),
"gap_other_intervals": intervals,
"max_other_gap_coverage_seconds": max_gap_coverage / SOURCE_FPS,
"sustained_other_takeover_during_gap": bool(competing_ids),
}
@@ -327,7 +305,6 @@ def derive_protocol_candidates(utterances: Sequence[Utterance], clips: Sequence[
short_frames = round(cfg.short_vocalization_max_seconds * SOURCE_FPS)
continuation_frames = round(cfg.continuation_min_seconds * SOURCE_FPS)
int_offset_tolerance_frames = round(cfg.int_offset_tolerance_seconds * SOURCE_FPS)
wearer_resume_merge_frames = round(cfg.wearer_resume_merge_max_seconds * SOURCE_FPS)
rows_by_segment: dict[int, list[Utterance]] = defaultdict(list)
clips_by_segment: dict[int, list[Clip]] = defaultdict(list)
for row in utterances:
@@ -352,10 +329,8 @@ def derive_protocol_candidates(utterances: Sequence[Utterance], clips: Sequence[
segment_end=segment_end, wearer_id=wearer_id)
other_rows = [row for row in rows if row.participant_id != wearer_id]
# A: classify raw wearer onsets before proposing any behavioral event.
# Short uncompeted gaps are internal to a floor episode. All other raw
# onsets begin a new floor episode and are all entry candidates. A
# preceding non-wearer release is optional context, not a gate.
# A: retain every raw wearer onset and offset. Gap and nearby-speaker
# properties are evidence only; they never suppress a semantic item.
onset_records: dict[int, dict[str, Any]] = {}
raw_offsets: list[int] = []
for index in range(1, len(wearer)):
@@ -365,25 +340,16 @@ def derive_protocol_candidates(utterances: Sequence[Utterance], clips: Sequence[
previous_wearer_index = index - gap_frames - 1
has_previous_wearer = previous_wearer_index >= 0 and bool(wearer[previous_wearer_index])
previous_wearer_offset = segment_start + index - gap_frames
competition = _wearer_gap_competition(
gap_evidence = _wearer_gap_evidence(
rows,
gap_start_frame=previous_wearer_offset,
onset_frame=frame,
wearer_id=wearer_id,
short_frames=short_frames,
continuation_frames=continuation_frames,
) if has_previous_wearer else {
"gap_other_participant_ids": [],
"feedback_like_other_ids_during_gap": [],
"sustained_competing_other_ids_during_gap": [],
"gap_other_intervals": [],
"max_other_gap_coverage_seconds": 0.0,
"sustained_other_takeover_during_gap": False,
}
within_turn_resume = (
has_previous_wearer
and gap_frames <= wearer_resume_merge_frames
and not competition["sustained_other_takeover_during_gap"]
)
evidence = _relation_evidence(rows, frame=frame, wearer_id=wearer_id,
window_frames=relation_frames)
evidence.update(
@@ -392,166 +358,112 @@ def derive_protocol_candidates(utterances: Sequence[Utterance], clips: Sequence[
previous_wearer_offset_seconds=(
previous_wearer_offset / SOURCE_FPS if has_previous_wearer else None
),
wearer_resume_merge_max_seconds=cfg.wearer_resume_merge_max_seconds,
floor_episode_merged=within_turn_resume,
original_timestamp_seconds=frame / SOURCE_FPS,
**competition,
**gap_evidence,
)
onset_records[frame] = {
"within_turn_resume": within_turn_resume,
"evidence": evidence,
}
onset_records[frame] = {"evidence": evidence}
elif wearer[index - 1] and not wearer[index]:
raw_offsets.append(frame)
resume_onsets = {
frame for frame, record in onset_records.items() if record["within_turn_resume"]
}
floor_onsets = sorted(set(onset_records) - resume_onsets)
for frame in sorted(resume_onsets):
record = onset_records[frame]
add(_candidate(
session=session, task="wearer_entry_control", source="WEARER_RESUME",
rule_type="WITHIN_TURN_RESUME_CONTROL", anchor_type="wearer_onset", frame=frame,
region_start=frame, region_end=min(segment_end, frame + relation_frames),
label="control", subtype=None,
previous_ids=_active_other_ids(rows, frame - 1, wearer_id=wearer_id),
incoming_ids=_active_other_ids(rows, frame, wearer_id=wearer_id),
evidence=record["evidence"], clips=segment_clips, rows=rows,
wearer_id=wearer_id, context_frames=context_frames,
reason="short raw wearer pause without sustained competing other takeover; merged into the existing floor episode",
), ("WEARER_RESUME", frame, ()))
# A1: every observed floor-level wearer onset is an entry candidate.
# Search backward for release anchors only to preserve response context;
# no preceding release is required for a positive WEARER_ENTRY.
for onset_frame in floor_onsets:
# A1: every observed raw wearer onset is an unresolved entry candidate.
# Every nearby preceding other offset is supplied with factual relation
# flags; no release is declared admissible or causal by this stage.
for onset_frame in sorted(onset_records):
release_options: list[tuple[Utterance, int]] = []
for release in other_rows:
opportunity_frame = release.dense_end_frame_exclusive
if opportunity_frame > onset_frame:
if opportunity_frame > onset_frame or onset_frame - opportunity_frame > relation_frames:
continue
wearer_silent = (
opportunity_frame == onset_frame
or _all_state(
wearer,
opportunity_frame - segment_start,
onset_frame - segment_start,
False,
)
)
competing_onset = any(
opportunity_frame <= row.dense_start_frame < onset_frame
for row in other_rows
)
other_already_holds = bool(
_active_other_ids(rows, opportunity_frame, wearer_id=wearer_id)
)
if wearer_silent and not competing_onset and not other_already_holds:
release_options.append((release, opportunity_frame))
release_options.append((release, opportunity_frame))
release_options.sort(
key=lambda item: (item[1], item[0].participant_id, item[0].uid)
)
if release_options:
_, latest_opportunity_frame = release_options[-1]
earliest_opportunity_frame = release_options[0][1]
option_rows = [{
earliest_opportunity_frame = (
release_options[0][1] if release_options else max(segment_start, onset_frame - relation_frames)
)
option_rows = []
for release, release_frame in release_options:
interval_left = release_frame - segment_start
interval_right = onset_frame - segment_start
option_rows.append({
"uid": release.uid,
"anchor_type": "other_offset",
"anchor_frame": frame,
"anchor_seconds": frame / SOURCE_FPS,
"anchor_frame": release_frame,
"anchor_seconds": release_frame / SOURCE_FPS,
"participant_id": release.participant_id,
"transcription": release.transcription,
"target_of_speech": list(release.target_of_speech),
"response_delay_seconds": (onset_frame - frame) / SOURCE_FPS,
} for release, frame in release_options]
evidence = dict(onset_records[onset_frame]["evidence"])
evidence.update(
opportunity_options=option_rows,
opportunity_anchor_type=None,
opportunity_anchor_frame=None,
opportunity_anchor_seconds=None,
behavior_anchor_type="wearer_floor_onset",
behavior_anchor_frame=onset_frame,
behavior_anchor_seconds=onset_frame / SOURCE_FPS,
minimum_response_delay_seconds=(onset_frame - latest_opportunity_frame) / SOURCE_FPS,
maximum_response_delay_seconds=(onset_frame - earliest_opportunity_frame) / SOURCE_FPS,
competing_other_onsets_before_behavior=[],
preceding_release_paired=True,
entry_context_timing=(
"OTHER_ACTIVE"
if _active_other_ids(rows, onset_frame, wearer_id=wearer_id)
else "AFTER_OTHER_RELEASE"
"response_delay_seconds": (onset_frame - release_frame) / SOURCE_FPS,
"wearer_active_between_release_and_onset": any(
bool(wearer[i]) for i in range(max(0, interval_left), min(len(wearer), interval_right))
),
)
previous_ids = tuple(sorted({row.participant_id for row, _ in release_options}))
add(_candidate(
session=session, task="wearer_entry", source="WEARER_ONSET",
rule_type="WEARER_ENTRY_CANDIDATE", anchor_type="wearer_onset",
frame=onset_frame, region_start=earliest_opportunity_frame,
region_end=min(segment_end, onset_frame + relation_frames),
label="candidate", subtype=None, previous_ids=previous_ids,
incoming_ids=_active_other_ids(rows, onset_frame, wearer_id=wearer_id),
evidence=evidence, clips=segment_clips, rows=rows, wearer_id=wearer_id,
context_frames=context_frames,
reason="floor-level wearer onset; preceding release options retained as context",
), ("WEARER_ONSET", onset_frame, previous_ids))
continue
frame = onset_frame
evidence = dict(onset_records[frame]["evidence"])
"other_onset_between_release_and_onset": any(
release_frame <= row.dense_start_frame < onset_frame for row in other_rows
),
"other_active_at_wearer_onset": bool(
_active_other_ids(rows, onset_frame, wearer_id=wearer_id)
),
})
evidence = dict(onset_records[onset_frame]["evidence"])
evidence.update(
opportunity_options=[],
opportunity_options=option_rows,
opportunity_anchor_type=None,
opportunity_anchor_frame=None,
opportunity_anchor_seconds=None,
behavior_anchor_type="wearer_floor_onset",
behavior_anchor_frame=frame,
behavior_anchor_seconds=frame / SOURCE_FPS,
preceding_release_paired=False,
behavior_anchor_type="wearer_raw_onset",
behavior_anchor_frame=onset_frame,
behavior_anchor_seconds=onset_frame / SOURCE_FPS,
preceding_release_options_present=bool(option_rows),
minimum_response_delay_seconds=(
None if not option_rows
else min(option["response_delay_seconds"] for option in option_rows)
),
maximum_response_delay_seconds=(
None if not option_rows
else max(option["response_delay_seconds"] for option in option_rows)
),
entry_context_timing=(
"OTHER_ACTIVE"
if _active_other_ids(rows, frame, wearer_id=wearer_id)
else "FROM_SILENCE_OR_UNRESOLVED"
if _active_other_ids(rows, onset_frame, wearer_id=wearer_id)
else "NO_OTHER_ACTIVE_AT_ONSET"
),
)
previous_ids = _active_other_ids(rows, frame - 1, wearer_id=wearer_id)
previous_ids = tuple(sorted({row.participant_id for row, _ in release_options}))
add(_candidate(
session=session, task="wearer_entry", source="WEARER_ONSET",
rule_type="WEARER_ENTRY_CANDIDATE", anchor_type="wearer_onset", frame=frame,
region_start=max(segment_start, frame - relation_frames),
region_end=min(segment_end, frame + relation_frames),
rule_type="WEARER_ENTRY_CANDIDATE", anchor_type="wearer_onset", frame=onset_frame,
region_start=earliest_opportunity_frame,
region_end=min(segment_end, onset_frame + relation_frames),
label="candidate", subtype=None,
previous_ids=previous_ids,
incoming_ids=_active_other_ids(rows, frame, wearer_id=wearer_id),
incoming_ids=_active_other_ids(rows, onset_frame, wearer_id=wearer_id),
evidence=evidence, clips=segment_clips, rows=rows, wearer_id=wearer_id,
context_frames=context_frames,
reason="floor-level wearer onset without a surviving preceding release option",
), ("WEARER_ONSET", frame, previous_ids))
reason=(
"raw wearer onset retained without floor-entry or within-turn-resumption gating; "
"nearby release relations are descriptive evidence"
),
), ("WEARER_ONSET", onset_frame, previous_ids))
# A2: a raw wearer offset is an INT candidate only at the end of a
# floor-level wearer episode and only if a new non-wearer onset obeys
# t_w < t_o < t_y and that same interval crosses the yield boundary.
# All other offsets remain auditable controls.
ordered_raw_onsets = sorted(onset_records)
current_floor_start = segment_start if wearer and wearer[0] else None
floor_start_by_offset: dict[int, tuple[int | None, bool]] = {}
# A2: every raw wearer offset is an unresolved INT candidate. The code
# records temporal relationships but does not infer takeover semantics.
# In particular, new/pre-existing/post-offset other activity and
# boundary crossing are evidence for DeepSeek and the human reviewer,
# never deterministic admission gates.
current_activity_run_start = segment_start if wearer and wearer[0] else None
activity_run_start_by_offset: dict[int, int | None] = {}
for index in range(1, len(wearer)):
frame = segment_start + index
if not wearer[index - 1] and wearer[index] and frame not in resume_onsets:
current_floor_start = frame
if not wearer[index - 1] and wearer[index]:
current_activity_run_start = frame
elif wearer[index - 1] and not wearer[index]:
next_onset = next((value for value in ordered_raw_onsets if value > frame), None)
internal_pause = next_onset in resume_onsets
floor_start_by_offset[frame] = (current_floor_start, internal_pause)
if not internal_pause:
current_floor_start = None
activity_run_start_by_offset[frame] = current_activity_run_start
current_activity_run_start = None
for frame in raw_offsets:
index = frame - segment_start
floor_start, internal_pause = floor_start_by_offset[frame]
activity_run_start = activity_run_start_by_offset[frame]
strict_crossing_ids = _other_ids_crossing_offset(
other_rows, frame=frame, wearer_id=wearer_id, tolerance_frames=0
)
@@ -559,71 +471,178 @@ def derive_protocol_candidates(utterances: Sequence[Utterance], clips: Sequence[
other_rows, frame=frame, wearer_id=wearer_id,
tolerance_frames=int_offset_tolerance_frames,
)
eligible_rows = [
new_other_rows = [
row for row in other_rows
if floor_start is not None
and floor_start < row.dense_start_frame < frame
and row.dense_end_frame_exclusive > frame - int_offset_tolerance_frames
if activity_run_start is not None
and activity_run_start < row.dense_start_frame < frame
]
eligible_ids = tuple(sorted({row.participant_id for row in eligible_rows}))
preexisting_ids = tuple(sorted({
row.participant_id for row in other_rows
if floor_start is not None
and row.dense_start_frame <= floor_start
and row.dense_end_frame_exclusive > frame - int_offset_tolerance_frames
}))
is_int_candidate = bool(eligible_ids) and not internal_pause
new_crossing_rows = [
row for row in new_other_rows
if row.dense_end_frame_exclusive > frame - int_offset_tolerance_frames
]
new_other_ids = tuple(sorted({row.participant_id for row in new_other_rows}))
new_crossing_ids = tuple(sorted({row.participant_id for row in new_crossing_rows}))
preexisting_rows = [
row for row in other_rows
if activity_run_start is not None
and row.dense_start_frame <= activity_run_start
and row.dense_end_frame_exclusive > activity_run_start
]
preexisting_ids = tuple(sorted({row.participant_id for row in preexisting_rows}))
nearby_other_rows = [
row for row in other_rows
if row.dense_end_frame_exclusive > frame - relation_frames
and row.dense_start_frame < frame + relation_frames
]
nearby_other_intervals = [{
"uid": row.uid,
"participant_id": row.participant_id,
"onset_seconds": row.dense_start_seconds,
"offset_seconds": row.dense_end_seconds,
"onset_relative_to_wearer_offset_seconds": (
row.dense_start_frame - frame
) / SOURCE_FPS,
"offset_relative_to_wearer_offset_seconds": (
row.dense_end_frame_exclusive - frame
) / SOURCE_FPS,
"starts_during_wearer_activity_run": bool(
activity_run_start is not None
and activity_run_start < row.dense_start_frame < frame
),
"preexisting_at_wearer_activity_run_start": bool(
activity_run_start is not None
and row.dense_start_frame <= activity_run_start
and row.dense_end_frame_exclusive > activity_run_start
),
"active_immediately_before_offset": bool(
row.dense_start_frame <= frame - 1 < row.dense_end_frame_exclusive
),
"active_immediately_after_offset": bool(
row.dense_start_frame <= frame < row.dense_end_frame_exclusive
),
"strictly_crosses_offset": bool(
row.dense_start_frame < frame < row.dense_end_frame_exclusive
),
"crosses_offset_with_tolerance": bool(
row.dense_start_frame < frame + int_offset_tolerance_frames
and row.dense_end_frame_exclusive > frame - int_offset_tolerance_frames
),
"overlap_with_wearer_before_offset_seconds": (
0.0 if activity_run_start is None else max(
0, min(row.dense_end_frame_exclusive, frame)
- max(row.dense_start_frame, activity_run_start)
) / SOURCE_FPS
),
"transcription": row.transcription,
"target_of_speech": list(row.target_of_speech),
} for row in nearby_other_rows]
next_rows = sorted(
[
row for row in rows
if frame <= row.dense_start_frame <= frame + relation_frames
],
key=lambda row: (row.dense_start_frame, row.participant_id, row.uid),
)
first_next_frame = next_rows[0].dense_start_frame if next_rows else None
first_next_rows = (
[] if first_next_frame is None
else [row for row in next_rows if row.dense_start_frame == first_next_frame]
)
active_other_ids_after_offset = _active_other_ids(
rows, frame, wearer_id=wearer_id
)
next_speaker_ids = (
list(active_other_ids_after_offset)
if active_other_ids_after_offset
else [row.participant_id for row in first_next_rows]
)
next_speaker_roles = (
["other"] * len(active_other_ids_after_offset)
if active_other_ids_after_offset
else [
"wearer" if row.participant_id == wearer_id else "other"
for row in first_next_rows
]
)
evidence = _relation_evidence(
rows, frame=frame, wearer_id=wearer_id, window_frames=relation_frames
)
evidence.update(
wearer_floor_episode_start_frame=floor_start,
wearer_floor_episode_start_seconds=(
None if floor_start is None else floor_start / SOURCE_FPS
wearer_activity_run_start_frame=activity_run_start,
wearer_activity_run_start_seconds=(
None if activity_run_start is None else activity_run_start / SOURCE_FPS
),
wearer_activity_run_duration_to_offset_seconds=(
None if activity_run_start is None
else (frame - activity_run_start) / SOURCE_FPS
),
wearer_offset_frame=frame,
wearer_offset_seconds=frame / SOURCE_FPS,
wearer_speech_duration_before_offset_seconds=_run_before(wearer, index, True) / SOURCE_FPS,
wearer_silence_duration_after_offset_seconds=_run_after(wearer, index, False) / SOURCE_FPS,
raw_offset_is_within_merged_floor_episode=internal_pause,
active_other_ids_before_offset=list(_active_other_ids(rows, frame - 1, wearer_id=wearer_id)),
active_other_ids_after_offset=list(_active_other_ids(rows, frame, wearer_id=wearer_id)),
active_other_ids_after_offset=list(active_other_ids_after_offset),
strict_crossing_other_ids=list(strict_crossing_ids),
tolerance_crossing_other_ids=list(tolerance_crossing_ids),
eligible_new_other_onset_ids=list(eligible_ids),
eligible_new_other_onsets=[{
new_other_onset_ids_during_wearer=list(new_other_ids),
new_other_onsets_during_wearer=[{
"participant_id": row.participant_id,
"onset_seconds": row.dense_start_seconds,
"offset_seconds": row.dense_end_seconds,
"uid": row.uid,
} for row in eligible_rows],
preexisting_crossing_other_ids=list(preexisting_ids),
} for row in new_other_rows],
new_other_onset_ids_crossing_offset=list(new_crossing_ids),
preexisting_other_ids_at_wearer_activity_run_start=list(preexisting_ids),
preexisting_other_intervals=[{
"participant_id": row.participant_id,
"onset_seconds": row.dense_start_seconds,
"offset_seconds": row.dense_end_seconds,
"uid": row.uid,
} for row in preexisting_rows],
nearby_other_intervals=nearby_other_intervals,
next_speech_onsets=[{
"participant_id": row.participant_id,
"participant_role": "wearer" if row.participant_id == wearer_id else "other",
"onset_seconds": row.dense_start_seconds,
"offset_seconds": row.dense_end_seconds,
"uid": row.uid,
"transcription": row.transcription,
"target_of_speech": list(row.target_of_speech),
} for row in next_rows],
next_speaker_ids=next_speaker_ids,
next_speaker_roles=next_speaker_roles,
next_speaker_basis=(
"active_across_offset"
if active_other_ids_after_offset
else ("first_new_onset_after_offset" if first_next_rows else None)
),
next_speech_onset_seconds=(
frame / SOURCE_FPS
if active_other_ids_after_offset
else (None if first_next_frame is None else first_next_frame / SOURCE_FPS)
),
other_crosses_wearer_offset=bool(tolerance_crossing_ids),
int_ordering_satisfied=bool(eligible_ids),
new_other_onset_during_wearer=bool(new_other_rows),
new_other_onset_crosses_offset=bool(new_crossing_rows),
int_offset_tolerance_seconds=cfg.int_offset_tolerance_seconds,
original_timestamp_seconds=frame / SOURCE_FPS,
)
control_reason = (
"raw wearer offset is internal to a merged floor episode"
if internal_pause
else "no new non-wearer onset satisfies t_w < t_o < t_y and crosses the wearer offset"
)
add(_candidate(
session=session,
task="int_yield" if is_int_candidate else "int_yield_control",
source="WEARER_OFFSET" if is_int_candidate else "WEARER_OFFSET_CONTROL",
rule_type="YIELD_CANDIDATE" if is_int_candidate else "WEARER_OFFSET_CONTROL",
session=session, task="int_yield", source="WEARER_OFFSET",
rule_type="YIELD_CANDIDATE",
anchor_type="wearer_offset", frame=frame,
region_start=max(segment_start, frame - relation_frames),
region_end=min(segment_end, frame + relation_frames),
label="candidate" if is_int_candidate else "control", subtype=None,
label="candidate", subtype=None,
previous_ids=_active_other_ids(rows, frame - 1, wearer_id=wearer_id),
incoming_ids=eligible_ids,
incoming_ids=new_other_ids,
evidence=evidence, clips=segment_clips, rows=rows, wearer_id=wearer_id,
context_frames=context_frames,
reason=(
"new non-wearer onset occurs inside the wearer floor episode and crosses the yield boundary; semantic takeover status is unresolved"
if is_int_candidate else control_reason
"raw wearer offset retained without timing-based takeover gating; "
"all nearby activity relations are descriptive evidence"
),
), ("WEARER_OFFSET" if is_int_candidate else "WEARER_OFFSET_CONTROL", frame, eligible_ids))
), ("WEARER_OFFSET", frame, new_other_ids))
# B1: same speaker pauses and resumes while the wearer remains silent.
by_speaker: dict[int, list[Utterance]] = defaultdict(list)
@@ -762,7 +781,10 @@ def parse_args() -> argparse.Namespace:
parser.add_argument("--short-vocalization-max-seconds", type=float, default=1.0)
parser.add_argument("--continuation-min-seconds", type=float, default=0.4)
parser.add_argument("--int-offset-tolerance-seconds", type=float, default=0.05)
parser.add_argument("--wearer-resume-merge-max-seconds", type=float, default=0.8)
parser.add_argument(
"--wearer-resume-merge-max-seconds", type=float,
help=argparse.SUPPRESS,
) # Deprecated compatibility option; raw wearer boundaries are never merged.
parser.add_argument("--candidate-sources", nargs="+")
parser.add_argument("--max-candidates", type=int)
parser.add_argument("--selection-seed", type=int, default=2027)
@@ -782,7 +804,6 @@ def main() -> None:
short_vocalization_max_seconds=args.short_vocalization_max_seconds,
continuation_min_seconds=args.continuation_min_seconds,
int_offset_tolerance_seconds=args.int_offset_tolerance_seconds,
wearer_resume_merge_max_seconds=args.wearer_resume_merge_max_seconds,
)
dataset_root, output_dir = args.dataset_root.expanduser().resolve(), args.output_dir.expanduser().resolve()
all_rows: list[ProtocolCandidate] = []

35
review_floor_events.py
View File

@@ -27,6 +27,9 @@ from threading import Lock
from typing import Any, Iterable
from urllib.parse import parse_qs, urlparse
from easycom_event_taxonomy import (
EXCLUDE_REASONS, LEGACY_EXCLUDE_REASONS, exclude_reason_group,
)
from prepare_easycom_eot_int_review import requires_semantic_adjudication
@@ -34,7 +37,7 @@ SOURCE_FPS = 20.0
STATIC_ROOT = Path(__file__).resolve().parent / "floor_event_review_web"
VIDEO_ROOT_PARTS = ("Main", "Video_Compressed")
TRANSCRIPT_ROOT_PARTS = ("Main", "Speech_Transcriptions")
REVIEW_SCHEMA_VERSION = 5
REVIEW_SCHEMA_VERSION = 6
MAX_REQUEST_BYTES = 128 * 1024
PROTOCOL_DISPOSITION_VALUES = {"positive", "hard_negative", "exclude"}
@@ -71,11 +74,7 @@ HUMAN_SUBTYPE_VALUES = {
"INT_OTHER_BACKCHANNEL", "INT_NON_TAKEOVER_OVERLAP", "INT_WEARER_HOLDS_FLOOR",
"OTHER_HARD_NEGATIVE",
}
EXCLUDE_REASON_VALUES = {
"ANNOTATION_ERROR", "RECORDING_BOUNDARY", "MISSING_CONTEXT", "NON_SPEECH_ARTIFACT",
"TIMESTAMP_CORRUPTION", "OUT_OF_SCOPE_NORMAL_COMPLETION", "OUT_OF_SCOPE_NON_EVENT",
"UNRESOLVABLE",
}
EXCLUDE_REASON_VALUES = EXCLUDE_REASONS | LEGACY_EXCLUDE_REASONS
@dataclass(frozen=True)
@@ -341,6 +340,8 @@ class CandidateRepository:
"ai_verdict": row.get("ai_verdict"),
"ai_event_type": row.get("ai_event_type"),
"ai_subtype": row.get("ai_subtype"),
"ai_exclude_reason": row.get("ai_exclude_reason"),
"ai_exclude_reason_group": row.get("ai_exclude_reason_group"),
"participant_role": row.get("participant_role") or row.get("holder_role"),
"event_time_seconds": event_time,
"region_start_seconds": row.get("region_start_seconds", event_time),
@@ -458,14 +459,19 @@ class ReviewStore:
selected_opportunity_uid = candidate.get("ai_selected_opportunity_uid") or (
candidate.get("llm_suggestion") or {}
).get("selected_opportunity_uid")
exclude_reason = candidate.get("ai_exclude_reason") or (
candidate.get("llm_suggestion") or {}
).get("exclude_reason")
if verdict not in HUMAN_VERDICT_VALUES:
raise ValueError("Accept requires an AI pre-label; use Relabel when no AI label is available")
elif action == "exclude":
verdict, event_type, subtype = "EXCLUDE", None, None
selected_opportunity_uid = None
exclude_reason = payload.get("exclude_reason")
else:
verdict, event_type, subtype = payload.get("human_verdict"), payload.get("human_event_type"), payload.get("human_subtype")
selected_opportunity_uid = payload.get("selected_opportunity_uid")
exclude_reason = payload.get("exclude_reason")
if verdict not in HUMAN_VERDICT_VALUES:
raise ValueError(f"Invalid human_verdict: {verdict!r}")
if verdict == "POSITIVE" and event_type not in HUMAN_EVENT_TYPE_VALUES:
@@ -514,11 +520,12 @@ class ReviewStore:
candidate.get("event_time", candidate.get("dense_start_seconds")),
),
)
exclude_reason = payload.get("exclude_reason")
if action == "exclude" and exclude_reason not in EXCLUDE_REASON_VALUES:
raise ValueError("Exclude requires an exclude_reason")
if action != "exclude" and exclude_reason is not None:
raise ValueError("exclude_reason is only valid for Exclude")
if verdict == "EXCLUDE" and exclude_reason not in EXCLUDE_REASON_VALUES:
raise ValueError(
"EXCLUDE requires an exclude_reason; legacy AI exclusions must be manually categorized"
)
if verdict != "EXCLUDE" and exclude_reason is not None:
raise ValueError("exclude_reason is only valid for EXCLUDE")
correction = payload.get("boundary_correction_seconds")
if correction not in {-0.2, 0, 0.2, None}:
raise ValueError("boundary_correction_seconds must be -0.2, 0, 0.2, or null")
@@ -535,6 +542,7 @@ class ReviewStore:
else [selected_opportunity_seconds, behavior_anchor_seconds]
),
exclude_reason=exclude_reason,
exclude_reason_group=exclude_reason_group(exclude_reason),
relabeled=action == "relabel",
boundary_correction_seconds=correction,
original_event_time=candidate.get("event_time", candidate.get("anchor_dense_seconds")),
@@ -597,6 +605,8 @@ class ReviewStore:
"ai_verdict": candidate.payload.get("ai_verdict"),
"ai_event_type": candidate.payload.get("ai_event_type"),
"ai_subtype": candidate.payload.get("ai_subtype"),
"ai_exclude_reason": candidate.payload.get("ai_exclude_reason"),
"ai_exclude_reason_group": candidate.payload.get("ai_exclude_reason_group"),
"review": normalized,
"updated_at": datetime.now(timezone.utc).isoformat(),
"candidate": candidate.payload,
@@ -650,9 +660,10 @@ class ReviewStore:
"automatic_protocol_label",
"automatic_hard_negative_type",
"ai_verdict", "ai_event_type", "ai_subtype",
"ai_exclude_reason", "ai_exclude_reason_group",
"human_action", "human_status", "human_verdict", "human_event_type", "human_subtype",
"selected_opportunity_uid", "selected_opportunity_seconds",
"valid_prediction_interval_seconds", "exclude_reason", "relabeled",
"valid_prediction_interval_seconds", "exclude_reason", "exclude_reason_group", "relabeled",
"boundary_correction_seconds", "original_event_time",
"confidence",
"reviewer",

50
test_adjudicate_easycom_eot_int_candidates.py
View File

@@ -8,7 +8,8 @@ import json
import tempfile
from adjudicate_easycom_eot_int_candidates import (
PROMPT_VERSION, SYSTEM_PROMPT, _progress_line, _transport_rows, build_user_prompt, read_candidates,
PROMPT_VERSION, SYSTEM_PROMPT, _progress_line, _transport_rows, build_user_prompt,
exclude_reason_group, read_candidates,
requires_semantic_adjudication, validate_decisions,
)
@@ -32,9 +33,10 @@ class EntryINTAdjudicationTests(unittest.TestCase):
self.assertIn("Yes.", prompt)
def test_system_prompt_treats_laughter_as_nonlexical_floor_feedback(self) -> None:
self.assertEqual(PROMPT_VERSION, "easycom-wearer-entry-int-prelabel-v12")
self.assertEqual(PROMPT_VERSION, "easycom-wearer-boundary-semantic-prelabel-v14")
self.assertIn("opportunity_options", SYSTEM_PROMPT)
self.assertIn("t_w < t_o < t_y", SYSTEM_PROMPT)
self.assertIn("t_w < t_o < t_y never proves takeover", SYSTEM_PROMPT)
self.assertIn("Every supplied WEARER_ONSET and WEARER_OFFSET is a raw observed", SYSTEM_PROMPT)
self.assertIn("a transcription consisting of [L] denotes laughter-only vocalization", SYSTEM_PROMPT)
self.assertIn("ENTRY_WEARER_BACKCHANNEL", SYSTEM_PROMPT)
self.assertIn("INT_OTHER_BACKCHANNEL", SYSTEM_PROMPT)
@@ -45,11 +47,13 @@ class EntryINTAdjudicationTests(unittest.TestCase):
self.assertIn('"Nope." beginning 0.15 s after an addressed question ends', SYSTEM_PROMPT)
self.assertIn("does not require a preceding non-wearer release", SYSTEM_PROMPT)
self.assertIn("duration alone never determines floor function", SYSTEM_PROMPT)
self.assertIn("out-of-scope normal completion", SYSTEM_PROMPT)
self.assertIn("same incoming non-wearer actually takes", SYSTEM_PROMPT)
self.assertIn("normal wearer completion", SYSTEM_PROMPT)
self.assertIn("relevant non-wearer actually takes", SYSTEM_PROMPT)
self.assertIn("must not retroactively invalidate the preceding INT_YIELD", SYSTEM_PROMPT)
self.assertIn("do not infer that a visible smile occurred", SYSTEM_PROMPT)
self.assertIn("POSITIVE / INT_YIELD: subtype=null; selected_opportunity_uid=null", SYSTEM_PROMPT)
self.assertIn("POSITIVE / INT_YIELD: subtype=null; exclude_reason=null", SYSTEM_PROMPT)
self.assertIn("OUT_OF_SCOPE_WITHIN_TURN_CONTINUATION", SYSTEM_PROMPT)
self.assertIn("DATA_TIMESTAMP_CORRUPTION", SYSTEM_PROMPT)
self.assertNotIn("NOT_APPLICABLE", SYSTEM_PROMPT)
self.assertNotIn("UNSURE", SYSTEM_PROMPT)
self.assertNotIn('"needs_human_review"', SYSTEM_PROMPT)
@@ -78,7 +82,8 @@ class EntryINTAdjudicationTests(unittest.TestCase):
payload = {
"items": [{
"uid": "u1", "verdict": "EXCLUDE", "event_type": None,
"subtype": None, "confidence": 0.4, "reason": "Ambiguous context.",
"subtype": None, "exclude_reason": "INSUFFICIENT_EVIDENCE",
"confidence": 0.4, "reason": "Ambiguous context.",
"needs_human_review": "true",
}]
}
@@ -128,6 +133,9 @@ class EntryINTAdjudicationTests(unittest.TestCase):
"verdict": verdict,
"event_type": event_type,
"subtype": subtype,
"exclude_reason": (
"INSUFFICIENT_EVIDENCE" if verdict == "EXCLUDE" else None
),
"selected_opportunity_uid": "copied-release",
"confidence": 0.8,
"reason": "Evidence-based decision.",
@@ -139,6 +147,28 @@ class EntryINTAdjudicationTests(unittest.TestCase):
)[0]
self.assertIsNone(decision["selected_opportunity_uid"])
def test_exclude_reason_taxonomy_separates_scope_evidence_and_data(self) -> None:
self.assertEqual(
exclude_reason_group("OUT_OF_SCOPE_NORMAL_COMPLETION"), "OUT_OF_SCOPE"
)
self.assertEqual(
exclude_reason_group("INSUFFICIENT_EVIDENCE"), "INSUFFICIENT_EVIDENCE"
)
self.assertEqual(
exclude_reason_group("DATA_ANNOTATION_ERROR"), "DATA_QUALITY"
)
base = {
"uid": "u1", "verdict": "EXCLUDE", "event_type": None,
"subtype": None, "selected_opportunity_uid": None,
"confidence": 0.4, "reason": "Not a scored event.",
}
with self.assertRaisesRegex(ValueError, "valid exclude_reason"):
validate_decisions({"items": [base]}, ["u1"])
with self.assertRaisesRegex(ValueError, "valid exclude_reason"):
validate_decisions(
{"items": [{**base, "exclude_reason": "ANNOTATION_ERROR"}]}, ["u1"]
)
def test_event_and_hard_negative_subtypes_match_candidate_family(self) -> None:
positive = {
"uid": "u1", "verdict": "POSITIVE", "event_type": "INT_YIELD",
@@ -195,7 +225,7 @@ class EntryINTAdjudicationTests(unittest.TestCase):
self.assertEqual([row["uid"] for row in transported], ["item-0000", "item-0001"])
self.assertEqual(mapping["item-0001"], rows[1]["uid"])
def test_old_nonoverlap_wearer_offset_is_not_sent_for_adjudication(self) -> None:
def test_every_current_wearer_offset_is_sent_for_adjudication(self) -> None:
control = {
"candidate_source": "WEARER_OFFSET",
"temporal_evidence": {
@@ -217,9 +247,9 @@ class EntryINTAdjudicationTests(unittest.TestCase):
"active_other_ids_after_offset": [4],
},
}
self.assertFalse(requires_semantic_adjudication(control))
self.assertTrue(requires_semantic_adjudication(control))
self.assertTrue(requires_semantic_adjudication(crossing))
self.assertFalse(requires_semantic_adjudication(speaker_change))
self.assertTrue(requires_semantic_adjudication(speaker_change))
self.assertFalse(requires_semantic_adjudication({"candidate_source": "WEARER_OFFSET_CONTROL"}))
self.assertFalse(requires_semantic_adjudication({"candidate_source": "WEARER_ONSET_CONTROL"}))
self.assertFalse(requires_semantic_adjudication({"candidate_source": "WEARER_RESUME"}))

97
test_prepare_easycom_eot_int_review.py
View File

@@ -106,36 +106,47 @@ class EntryINTReviewCandidateTests(unittest.TestCase):
self.assertEqual(onset.task, "wearer_entry")
self.assertEqual(onset.automatic_protocol_label, "candidate")
self.assertEqual(onset.temporal_evidence["entry_context_timing"], "OTHER_ACTIVE")
self.assertFalse(onset.temporal_evidence["preceding_release_paired"])
self.assertTrue(onset.temporal_evidence["preceding_release_options_present"])
self.assertTrue(
any(
option["other_onset_between_release_and_onset"]
for option in onset.temporal_evidence["opportunity_options"]
)
)
def test_short_wearer_pause_without_other_takeover_becomes_resume_control(self) -> None:
def test_short_wearer_pause_still_becomes_raw_onset_candidate(self) -> None:
rows = [
utterance("wearer-1", 2, 0.2, 1.0, "I was saying"),
utterance("wearer-2", 2, 1.5, 2.5, "and then this."),
]
derived = self.derive(rows)
resume = next(row for row in derived if row.candidate_source == "WEARER_RESUME")
self.assertEqual(resume.task, "wearer_entry_control")
self.assertEqual(resume.rule_candidate_type, "WITHIN_TURN_RESUME_CONTROL")
self.assertEqual(resume.automatic_protocol_label, "control")
self.assertTrue(resume.temporal_evidence["floor_episode_merged"])
self.assertAlmostEqual(resume.temporal_evidence["previous_wearer_silence_duration_seconds"], 0.5)
self.assertFalse(any(
row.candidate_source == "WEARER_ONSET" and row.anchor_dense_seconds == 1.5
for row in derived
))
onset = next(
row for row in derived
if row.candidate_source == "WEARER_ONSET" and row.anchor_dense_seconds == 1.5
)
self.assertEqual(onset.task, "wearer_entry")
self.assertEqual(onset.rule_candidate_type, "WEARER_ENTRY_CANDIDATE")
self.assertEqual(onset.automatic_protocol_label, "candidate")
self.assertAlmostEqual(onset.temporal_evidence["previous_wearer_silence_duration_seconds"], 0.5)
self.assertFalse(any(row.candidate_source == "WEARER_RESUME" for row in derived))
def test_short_feedback_does_not_split_wearer_floor_episode(self) -> None:
def test_gap_other_activity_is_recorded_without_feedback_semantics(self) -> None:
rows = [
utterance("wearer-1", 2, 0.2, 1.0, "First clause"),
utterance("feedback", 4, 1.1, 1.4, "Yeah."),
utterance("wearer-2", 2, 1.6, 2.5, "same turn continues."),
]
resume = next(row for row in self.derive(rows) if row.candidate_source == "WEARER_RESUME")
self.assertEqual(resume.temporal_evidence["feedback_like_other_ids_during_gap"], [4])
self.assertFalse(resume.temporal_evidence["sustained_other_takeover_during_gap"])
onset = next(
row for row in self.derive(rows)
if row.candidate_source == "WEARER_ONSET" and row.anchor_dense_seconds == 1.6
)
gap = onset.temporal_evidence["gap_other_intervals"]
self.assertEqual([item["participant_id"] for item in gap], [4])
self.assertEqual(gap[0]["transcription"], "Yeah.")
self.assertAlmostEqual(gap[0]["gap_overlap_seconds"], 0.3)
self.assertNotIn("feedback_like_other_ids_during_gap", onset.temporal_evidence)
def test_short_gap_with_sustained_other_takeover_preserves_positive_entry_candidate(self) -> None:
def test_short_gap_with_other_activity_preserves_unresolved_entry_candidate(self) -> None:
rows = [
utterance("wearer-1", 2, 0.2, 1.0, "First wearer turn."),
utterance("other", 4, 0.8, 2.2, "I have the floor now."),
@@ -146,16 +157,17 @@ class EntryINTReviewCandidateTests(unittest.TestCase):
row for row in derived
if row.candidate_source == "WEARER_ONSET" and row.anchor_dense_seconds == 1.6
)
self.assertFalse(reentry.temporal_evidence["floor_episode_merged"])
self.assertTrue(reentry.temporal_evidence["sustained_other_takeover_during_gap"])
self.assertEqual(reentry.temporal_evidence["sustained_competing_other_ids_during_gap"], [4])
self.assertEqual(reentry.temporal_evidence["gap_other_participant_ids"], [4])
self.assertTrue(
reentry.temporal_evidence["gap_other_intervals"][0]["continues_across_wearer_onset"]
)
self.assertEqual(reentry.temporal_evidence["entry_context_timing"], "OTHER_ACTIVE")
self.assertEqual(reentry.automatic_protocol_label, "candidate")
preceding_yield = next(
row for row in derived
if row.candidate_source == "WEARER_OFFSET" and row.anchor_dense_seconds == 1.0
)
self.assertTrue(preceding_yield.temporal_evidence["int_ordering_satisfied"])
self.assertTrue(preceding_yield.temporal_evidence["new_other_onset_during_wearer"])
def test_same_other_resume_is_entry_midturn_hard_negative(self) -> None:
rows = [
@@ -199,32 +211,43 @@ class EntryINTReviewCandidateTests(unittest.TestCase):
self.assertEqual(event.anchor_type, "wearer_offset")
self.assertAlmostEqual(event.event_time, 2.0)
self.assertEqual(event.temporal_evidence["tolerance_crossing_other_ids"], [4])
self.assertEqual(event.temporal_evidence["eligible_new_other_onset_ids"], [4])
self.assertEqual(event.temporal_evidence["new_other_onset_ids_during_wearer"], [4])
self.assertAlmostEqual(
event.temporal_evidence["eligible_new_other_onsets"][0]["onset_seconds"], 1.0
event.temporal_evidence["new_other_onsets_during_wearer"][0]["onset_seconds"], 1.0
)
self.assertTrue(event.temporal_evidence["int_ordering_satisfied"])
self.assertTrue(event.temporal_evidence["new_other_onset_during_wearer"])
self.assertTrue(event.temporal_evidence["new_other_onset_crosses_offset"])
self.assertEqual(event.temporal_evidence["next_speaker_ids"], [4])
self.assertEqual(event.temporal_evidence["next_speaker_basis"], "active_across_offset")
nearby = event.temporal_evidence["nearby_other_intervals"][0]
self.assertAlmostEqual(nearby["overlap_with_wearer_before_offset_seconds"], 1.0)
def test_preexisting_other_crossing_offset_is_not_an_int_candidate(self) -> None:
def test_preexisting_other_crossing_offset_remains_an_int_candidate(self) -> None:
rows = [
utterance("other", 4, 0.0, 3.0, "Other already holds the floor."),
utterance("wearer", 2, 1.0, 2.0, "Wearer enters and stops."),
]
event = next(row for row in self.derive(rows) if row.anchor_type == "wearer_offset")
self.assertEqual(event.candidate_source, "WEARER_OFFSET_CONTROL")
self.assertEqual(event.temporal_evidence["preexisting_crossing_other_ids"], [4])
self.assertFalse(event.temporal_evidence["int_ordering_satisfied"])
self.assertEqual(event.candidate_source, "WEARER_OFFSET")
self.assertEqual(event.task, "int_yield")
self.assertEqual(
event.temporal_evidence["preexisting_other_ids_at_wearer_activity_run_start"], [4]
)
self.assertFalse(event.temporal_evidence["new_other_onset_during_wearer"])
self.assertTrue(event.temporal_evidence["other_crosses_wearer_offset"])
def test_other_starting_after_wearer_offset_is_a_separate_control(self) -> None:
def test_other_starting_after_wearer_offset_remains_an_int_candidate(self) -> None:
rows = [
utterance("wearer", 2, 0.0, 2.0, "Wearer finishes."),
utterance("other", 4, 2.2, 3.0, "Later turn."),
]
event = next(row for row in self.derive(rows) if row.anchor_type == "wearer_offset")
self.assertEqual(event.candidate_source, "WEARER_OFFSET_CONTROL")
self.assertEqual(event.rule_candidate_type, "WEARER_OFFSET_CONTROL")
self.assertEqual(event.automatic_protocol_label, "control")
self.assertEqual(event.candidate_source, "WEARER_OFFSET")
self.assertEqual(event.rule_candidate_type, "YIELD_CANDIDATE")
self.assertEqual(event.automatic_protocol_label, "candidate")
self.assertFalse(event.temporal_evidence["other_crosses_wearer_offset"])
self.assertEqual(event.temporal_evidence["next_speaker_ids"], [4])
self.assertAlmostEqual(event.temporal_evidence["next_speech_onset_seconds"], 2.2)
def test_one_frame_boundary_tolerance_is_configurable(self) -> None:
rows = [
@@ -241,9 +264,11 @@ class EntryINTReviewCandidateTests(unittest.TestCase):
tolerant_offset = next(row for row in tolerant if row.anchor_type == "wearer_offset")
strict_offset = next(row for row in strict if row.anchor_type == "wearer_offset")
self.assertEqual(tolerant_offset.candidate_source, "WEARER_OFFSET")
self.assertEqual(strict_offset.candidate_source, "WEARER_OFFSET_CONTROL")
self.assertEqual(strict_offset.candidate_source, "WEARER_OFFSET")
self.assertTrue(tolerant_offset.temporal_evidence["other_crosses_wearer_offset"])
self.assertFalse(strict_offset.temporal_evidence["other_crosses_wearer_offset"])
def test_internal_raw_offset_is_not_an_int_candidate(self) -> None:
def test_internal_raw_offset_remains_an_int_candidate(self) -> None:
rows = [
utterance("wearer-1", 2, 0.0, 1.0, "First clause."),
utterance("other", 4, 0.8, 1.3, "Yeah."),
@@ -253,8 +278,8 @@ class EntryINTReviewCandidateTests(unittest.TestCase):
row for row in self.derive(rows)
if row.anchor_type == "wearer_offset" and row.anchor_dense_seconds == 1.0
)
self.assertEqual(event.candidate_source, "WEARER_OFFSET_CONTROL")
self.assertTrue(event.temporal_evidence["raw_offset_is_within_merged_floor_episode"])
self.assertEqual(event.candidate_source, "WEARER_OFFSET")
self.assertEqual(event.automatic_protocol_label, "candidate")
def test_brief_feedback_is_int_backchannel_hard_negative(self) -> None:
rows = [

35
test_review_floor_events.py
View File

@@ -147,7 +147,7 @@ class FloorEventReviewTests(unittest.TestCase):
[candidate.resolve()],
)
def test_primary_queue_skips_legacy_nonoverlap_wearer_offset(self) -> None:
def test_primary_queue_keeps_nonoverlap_wearer_offset_for_semantics(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
path = Path(temporary) / "legacy.jsonl"
rows = [
@@ -171,7 +171,7 @@ class FloorEventReviewTests(unittest.TestCase):
)
primary = load_candidates([path])
all_rows = load_candidates([path], include_offset_controls=True)
self.assertEqual([row.uid for row in primary], [rows[1]["uid"]])
self.assertEqual([row.uid for row in primary], [row["uid"] for row in rows])
self.assertEqual(len(all_rows), 2)
def test_relabel_preserves_ai_and_records_human_decision(self) -> None:
@@ -259,6 +259,37 @@ class FloorEventReviewTests(unittest.TestCase):
self.assertEqual(
record["review"]["exclude_reason"], "OUT_OF_SCOPE_NORMAL_COMPLETION"
)
self.assertEqual(record["review"]["exclude_reason_group"], "OUT_OF_SCOPE")
def test_accept_ai_exclude_preserves_reason_taxonomy(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
_, events = self._fixture(root)
candidates = load_candidates([events])
candidate = candidates[0]
candidate.payload.update(
ai_verdict="EXCLUDE",
ai_event_type=None,
ai_subtype=None,
ai_exclude_reason="DATA_MISSING_CONTEXT",
ai_exclude_reason_group="DATA_QUALITY",
llm_suggestion={
"verdict": "EXCLUDE",
"event_type": None,
"subtype": None,
"exclude_reason": "DATA_MISSING_CONTEXT",
},
)
store = ReviewStore(root / "reviews.jsonl", candidates)
record = store.save(candidate.candidate_id, {
"human_action": "accept", "human_verdict": None,
"human_event_type": None, "human_subtype": None,
"selected_opportunity_uid": None, "exclude_reason": None,
"boundary_correction_seconds": None, "confidence": "medium",
"notes": "context is truncated",
})
self.assertEqual(record["review"]["exclude_reason"], "DATA_MISSING_CONTEXT")
self.assertEqual(record["review"]["exclude_reason_group"], "DATA_QUALITY")
def test_entry_review_may_select_only_a_supplied_opportunity_anchor(self) -> None:
with tempfile.TemporaryDirectory() as temporary: