Fix opportunity competing-risk evaluation
This commit is contained in:
@@ -54,19 +54,23 @@ result.
|
||||
- An opportunity-conditioned dense-output evaluator aligns ENTRY positives and
|
||||
core negatives at non-wearer releases, aligns INT positives and hold-floor
|
||||
negatives at competing non-wearer onsets, and applies strict competing-risk
|
||||
censoring. It uses only the original wearer probabilities, excludes wearer
|
||||
backchannels from headline ENTRY FPR, and records unanchored/ambiguous cases.
|
||||
censoring. ENTRY positives whose selected release is followed by a competing
|
||||
other onset or intervening wearer activity are excluded, and ambiguous
|
||||
multi-competitor INT cases are not assigned an arbitrary onset. It uses only
|
||||
the original wearer probabilities, excludes wearer backchannels from
|
||||
headline ENTRY FPR, and records unanchored/ambiguous cases.
|
||||
|
||||
### Measured screening result
|
||||
|
||||
- The released EgoSpeak MiniROADA A+V checkpoint on Sessions 1--3 obtains
|
||||
`74.124` 0.2-second mAP and `61.155` ten-horizon mean mAP. Raw wearer
|
||||
onset/offset anticipation is harder (`24.491/38.024` mean AP). On provisional
|
||||
LLM-assisted matched opportunities, ENTRY AP is `52.487` versus `32.731%`
|
||||
prevalence, while INT/YIELD AP is `20.135`, indistinguishable from its
|
||||
`20.438%` prevalence. Figure-7-style continuous wearer curves therefore
|
||||
support dense forecasting, not semantic floor-control claims. Full curves,
|
||||
counts, exclusions, and caveats are frozen in
|
||||
LLM-assisted matched opportunities, the original v1 evaluator reported
|
||||
ENTRY/INT AP of `52.487/20.135`; those opportunity values are superseded
|
||||
because v1 could cross a competing ENTRY onset and arbitrarily selected among
|
||||
multiple INT competitors. Figure-7-style continuous wearer curves support
|
||||
dense forecasting, not semantic floor-control claims. The unaffected dense
|
||||
curves and historical v1 opportunity record are in
|
||||
[`results/easycom_egospeak_three_level_eval_2026-09-16/`](results/easycom_egospeak_three_level_eval_2026-09-16/README.md).
|
||||
- F1 Frame-set Attention uses fused G+A as one query over the latest half-open
|
||||
causal person set. On the existing three-seed, four-epoch test screen it
|
||||
@@ -74,12 +78,11 @@ result.
|
||||
`64.95099 +/- 0.09451` ten-horizon mean mAP. This is stable, but is not a
|
||||
matched improvement over the historical no-dev pooled result. Its three-seed
|
||||
boundary response/yield AP is `28.423 +/- 0.480` / `48.641 +/- 0.391`; on
|
||||
provisional reviewed opportunities its ENTRY/INT AP is
|
||||
`41.561 +/- 0.581` / `22.991 +/- 0.202`. These values are respectively
|
||||
`-4.461/-1.275` points below the historical pooled-V seed-0 result, so the
|
||||
current F1 screen does not support the hoped-for semantic-selectivity gain.
|
||||
A same-split three-seed pooled control and correct/wrong visual controls
|
||||
remain missing. See the immutable
|
||||
the superseded v1 opportunity evaluator reported ENTRY/INT AP of
|
||||
`41.561 +/- 0.581` / `22.991 +/- 0.202`; these values require a v2 rerun and
|
||||
cannot currently support a semantic-selectivity conclusion. A same-split
|
||||
three-seed pooled control and correct/wrong visual controls remain missing.
|
||||
See the historical
|
||||
[`screening record`](results/easycom_omni_floor_screen_2026-09-16/README.md).
|
||||
|
||||
### Measured local-fusion screens
|
||||
|
||||
Reference in New Issue
Block a user