wx44wx/icassp2027-experiment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Revise wearer entry and yield annotation protocol

Browse Source
This commit is contained in:
Xin Wang
2026-09-15 01:25:39 +08:00
parent 026d89978f
commit 71dfd8808a
12 changed files with 478 additions and 248 deletions
Download Patch File Download Diff File Expand all files Collapse all files

84
docs/easycom_event_review_tool.md
View File

@@ -1,4 +1,4 @@
# EasyCom wearer-centric EOT/INT pre-label and review pipeline
# EasyCom wearer-entry/INT pre-label and review pipeline
The workflow has three explicit stages. Speech activity timestamps are
read-only throughout; reviewed events never modify the dense 5 Hz activity
@@ -24,9 +24,9 @@ python3 prepare_easycom_eot_int_review.py \
The generator emits two independent files:
- `easycom_eot_int_candidates.jsonl` contains floor-level wearer `0->1`
behavior anchors with one or more admissible preceding non-wearer releases
(`WEARER_ONSET`, `EOT_OPPORTUNITY_CANDIDATE`), only episode-ending wearer
- `easycom_eot_int_candidates.jsonl` (legacy-compatible filename) contains every
floor-level wearer `0->1` behavior anchor (`WEARER_ONSET`,
`WEARER_ENTRY_CANDIDATE`), only episode-ending wearer
`1->0` boundaries where a new non-wearer onset satisfies
`wearer_floor_start < other_onset < wearer_offset` and the same activity
crosses the offset (`WEARER_OFFSET`, `YIELD_CANDIDATE`), and hard-negative
@@ -34,8 +34,7 @@ The generator emits two independent files:
`SAME_SPEAKER_PAUSE`, `OTHER_TO_OTHER`, `OTHER_BACKCHANNEL`, and
`NON_TAKEOVER_OVERLAP`;
- `easycom_eot_int_controls.jsonl` contains short within-turn raw onsets
(`WEARER_RESUME`), floor-level wearer entries with no surviving preceding
release opportunity (`WEARER_ONSET_CONTROL`), and wearer offsets that fail
(`WEARER_RESUME`) and wearer offsets that fail
the strict INT ordering/crossing prerequisite (`WEARER_OFFSET_CONTROL`). They
are retained for audits, but are not sent to DeepSeek or placed in the
primary human-review queue.
@@ -49,18 +48,20 @@ continues across the resumed onset conservatively preserves a new wearer-entry
candidate. Every decision is stored in `temporal_evidence`, including the raw
gap, feedback IDs, competing IDs, threshold, and `floor_episode_merged` flag.
EOT construction iterates floor-level wearer onsets, then searches backward for
preceding non-wearer releases. A release is an admissible option only when the
Entry construction retains every floor-level wearer onset, then searches backward
for preceding non-wearer releases as optional context. A release is an admissible option only when the
wearer stays silent and no new non-wearer onset or already-active other floor
invalidates the interval before the wearer onset. Silence length alone does not
invalidate an option. Each wearer onset produces at most one primary row. Its
`opportunity_options` list contains the only release anchors that DeepSeek or a
human may select; neither may invent a timestamp. The selected release is the
opportunity anchor, the wearer onset is the behavior anchor, and their interval
deterministically defines the valid EOT prediction interval.
invalidate an option. Each wearer onset produces exactly one primary row. A
missing release does not demote an entry to a control: an entry may occur from
silence or while another speaker is active. `entry_context_timing` records
`AFTER_OTHER_RELEASE`, `OTHER_ACTIVE`, or `FROM_SILENCE_OR_UNRESOLVED` as
metadata. When present, `opportunity_options` contains the only release anchors
that DeepSeek or a human may optionally select; neither may invent a timestamp.
Paired wearer onsets and strictly qualified offsets are initially `unsure`;
timing alone is not a semantic positive label. The INT prerequisite is
Floor-level wearer onsets and strictly qualified offsets carry the rule-stage value
`candidate`; timing alone is not a semantic positive label. Deterministic rows
outside the primary queue carry `control`, never a semantic verdict. The INT prerequisite is
`wearer_floor_start < other.start < wearer_offset < other.end`, with a
configurable one-frame (50 ms) tolerance only at the offset boundary.
Pre-existing other activity and raw offsets inside a merged wearer episode stay
@@ -89,15 +90,17 @@ timeline; it is not sent to DeepSeek.
## 2. Add DeepSeek pre-labels
Prompt v7 treats the EasyCom transcript marker `[L]` as laughter-only evidence,
Prompt v11 first reconstructs the exact participant-resolved chronology and
then uses transcript content and `Target_of_Speech` to determine conversational
function. It treats the EasyCom transcript marker `[L]` as laughter-only evidence,
not as a lexical floor take. Wearer laughter during another participant's
retained turn is directed toward `EOT_WEARER_BACKCHANNEL`; non-wearer laughter
retained turn is directed toward `ENTRY_WEARER_BACKCHANNEL`; non-wearer laughter
overlapping a retained wearer turn is directed toward `INT_OTHER_BACKCHANNEL`.
Ambiguous laughter remains `UNSURE`, and `[L]` alone is neither an exclusion
reason nor evidence of a visible smile. The prompt also treats the two EOT
anchor options and strict INT ordering as deterministic prerequisites. For a
positive EOT response it must return `selected_opportunity_uid` from the supplied
options; all other decisions use `null`.
Ambiguous laughter falls back to `EXCLUDE`, and `[L]` alone is neither evidence
of a floor take nor evidence of a visible smile. Preceding release options are
optional entry metadata, whereas strict INT ordering remains a deterministic
retrieval prerequisite. A positive `WEARER_ENTRY` may optionally select one
supplied release; entries from silence or during other activity use `null`.
`needs_human_review` is not requested from DeepSeek: every pre-label enters the
human queue by protocol, so the adjudicator assigns that boolean locally. This
@@ -127,12 +130,26 @@ python3 adjudicate_easycom_eot_int_candidates.py \
```
DeepSeek sees candidate source, structured timing, per-speaker activity, and
transcript together. It returns one of `POSITIVE`, `HARD_NEGATIVE`,
`NOT_APPLICABLE`, `EXCLUDE`, or `UNSURE`; positives additionally receive
`EOT_RESPONSE` or `INT_YIELD`, and hard negatives receive a named subtype. It
transcript together. It returns one of `POSITIVE`, `HARD_NEGATIVE`, or
`EXCLUDE`; positives additionally receive
`WEARER_ENTRY` or `INT_YIELD`, and hard negatives receive a named subtype. It
cannot modify timestamps. The output preserves the original rule candidate,
the selected EOT opportunity UID when applicable, and the complete AI
decision/provenance.
the selected preceding-release UID when applicable, and the complete AI
decision/provenance. `EXCLUDE` is the sole non-scoring outcome. It covers
out-of-scope non-events (including a normal wearer completion that merely
coincides with feedback), insufficient evidence, unresolvable data quality, or
disagreement between the two model passes.
Hard negatives use behavior-specific subtypes. ENTRY distinguishes a same-other
mid-turn pause, a different-other transition, and a wearer backchannel that
does not take the floor. INT first distinguishes an other backchannel, then a
non-backchannel overlap after which the wearer demonstrably holds or resumes
the floor, and finally another interpretable overlap that fails to become a
sustained takeover. `OTHER_HARD_NEGATIVE` is reserved for valid false-positive
opportunities outside these named behaviors; it is never an uncertainty
fallback. A short but complete answer to an addressed question remains a
positive wearer entry; duration alone and later resumption by another speaker
do not turn it into a wearer backchannel.
During live adjudication, the CLI displays a dependency-free progress bar over
all candidate/pass work, including the current pass, batch, percentage, and
@@ -160,10 +177,10 @@ question, then presents the exact AI verdict/subtype with a plain-language
meaning; the Accept control repeats that exact label so its effect is explicit.
The common path is one operation:
- **Accept** keeps the AI verdict/type and its selected EOT opportunity anchor;
- **Relabel** expands verdict and subtype controls and, for a positive EOT,
requires one of the supplied release anchors;
- **Exclude** requires a concrete data-quality reason.
- **Accept** keeps the AI verdict/type and optional preceding-release metadata;
- **Relabel** expands verdict and subtype controls; a positive entry may
optionally select one of the supplied release anchors;
- **Exclude** requires a concrete out-of-scope or data-quality reason.
The current primary-review UI deliberately has no second-review action. Older
saved `second_review` records remain readable for backward compatibility.
@@ -185,8 +202,9 @@ Protocol wording for the paper appendix:
> Raw wearer speech onsets are first grouped into floor-level speech episodes
> by merging short within-turn resumptions that are not separated by a
> competing floor takeover. Only the resulting floor-level 0→1 transitions are
> proposed as EOT-response candidates.
> competing floor takeover. Every resulting floor-level 0→1 transition is
> proposed as a wearer-entry candidate; a preceding non-wearer release is
> retained as context rather than required for positive status.
> The original speech intervals are preserved unchanged for dense forecasting
> and auditability; this grouping is used only for behavioral event annotation.