wx44wx/icassp2027-experiment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add floor-event timing objective and addressee diagnostics

Browse Source
This commit is contained in:
Xin Wang
2026-09-11 00:15:34 +08:00
parent 5937971222
commit a57e827606
14 changed files with 2144 additions and 234 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
docs/easycom_oracle_information.md
View File

@@ -15,10 +15,12 @@ This is an information upper-bound experiment, not a deployable input setting.
`0.2, 0.4, 0.6, ...` second cutoffs, with
`Start_Frame <= marker < End_Frame`. This is right-edge point sampling, not
an any-overlap reduction over a 200 ms bin.
- Each person vector stores the last 2 seconds of binary activity followed by
a visibility/availability mask. Activity is exposed only in bins where that
participant was visible according to the provided boxes. No future bin,
future wearer state, transcript text or `Target_of_Speech` value is input.
- Each original person vector stores the last 2 seconds of binary activity
followed by a visibility/availability mask. Activity is exposed only in bins
where that participant was visible according to the provided boxes. No future
bin, future wearer state or transcript text is input. The separately named
addressee diagnostic below additionally uses privileged `Target_of_Speech`;
it must not be described as an online/deployable feature.
- Only non-wearer participant IDs present in the latest provided-box frame
strictly before the cutoff receive slots. IDs route annotations during
preparation but are not exposed as embeddings to the head.
@@ -35,6 +37,8 @@ This is an information upper-bound experiment, not a deployable input setting.
| `audio` | Omni `G` plus the existing recent local audio `A` | deployable baseline |
| `visibility-only` | current visible-person set, but all activity bits are zero | parameter/person-count control |
| `aggregate-visible` | one OR-pooled history over currently visible people | aggregate floor-information upper bound |
| `aggregate-visible-address-control` | aggregate-visible plus an all-zero history channel | capacity control for addressee input |
| `aggregate-visible-addressee` | aggregate-visible plus past direct-to-wearer annotation bins | analysis-only addressee information upper bound |
| `per-person` | one correct activity history per currently visible participant | main oracle information test |
| `shuffled-id` | preserves the number active at every lag but independently reassigns activity between people | destroys identity continuity |
| `delayed` | correct per-person histories delayed by 1 second | destroys current temporal alignment without future leakage |
@@ -43,6 +47,12 @@ The decisive comparison is `per-person` against `aggregate-visible`,
`shuffled-id` and `delayed`, not only against `audio`. A gain over `audio` alone
could come from perfect aggregate VAD rather than person continuity.
For the separate addressee question, the decisive comparison is
`aggregate-visible-addressee` against both `aggregate-visible` and
`aggregate-visible-address-control`. Group/None/unknown targets are audited and
not treated as trustworthy semantic negatives. The full diagnostic commands
are in [`easycom_floor_event_diagnostics.md`](easycom_floor_event_diagnostics.md).
## Remote preparation and one-seed Session-12 screen
The source local sidecars provide only mixed audio and the visibility/box