Record EasyCom encoder-only results and horizon analysis

2026-08-28 11:15:09 +08:00
parent 321238a92f
commit 4011ba9b2a
10 changed files with 1781 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -16,6 +16,12 @@ For **EasyCom encoder-only A / V / AV** with no Thinker states, start with
 They reuse the verified local sidecars, train on Session 4–12, test on 3/2/1,
 and keep four epochs with no dev set.

+The user-reported EasyCom encoder-only seed-0 results and complete supplied
+horizon curves are archived in
+[the 2026-08-28 result record](results/easycom_encoder_only_1s_4ep_2026-08-28/README.md).
+It keeps causal-valid summaries separate from full-timeline curves and records
+the head-capacity/context caveats for comparisons with G+A and G+A+V.
+
 The main experiment is deliberately simple:

 ```text
--- a/docs/easycom_encoder_only.md
+++ b/docs/easycom_encoder_only.md
@@ -7,8 +7,16 @@
 固定训练 Session **4–12（升序，与原 B1 一致）**，测试 **3/2/1**；4 epochs、无 dev，
 评测最后一轮。沿用原来的 5Hz 行索引、十个 horizon、LSTR-MCE 和 AP 协议，不改变标签。

+已收到的 seed 0 结果与完整日志见
+[2026-08-28 实验记录](../results/easycom_encoder_only_1s_4ep_2026-08-28/README.md)。
+其中完整曲线为 `egospeak_full_timeline`，此前主表为 `causal_valid_only`，两者分开记录。
+
 ## 1. 确定已经验证过的局部特征目录

+用户已确认本次使用的实际目录是
+`/data1/yfzx/wangx/Datasets/easycom/omni_features/local_qwen_1s_encoded_tf4576`，
+不是早期默认的 `local_qwen_1s`；以下 checkpoint 读取方法可用于再次核对。
+
 在 L20 机器更新本次代码后执行。下面从你之前跑完的 `av_seed0.pth` 读取实际路径，
 避免猜错目录或混入 bbox 修正前的旧视觉特征；checkpoint 仅用于找路径，不作为新模型输入。

--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/README.md
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/README.md
@@ -0,0 +1,242 @@
+# EasyCom encoder-only 1s：seed 0 完整曲线记录与分析
+
+记录日期：2026-08-28。来源是用户在 L20 远程机器运行后提供的控制台输出；
+本地没有重新训练或执行真实数据推理。本记录仍是**单种子初步结果**，标准差为 n/a，
+不是 0；不能据此报告统计显著性。
+
+## 1. 收到了什么，尚未收到什么
+
+- 本次三份附件：encoder A、V、AV 的 `egospeak_full_timeline` 十个 horizon 的三类 AP、mAP 和均值。
+- 此前两份附件：G+A、G+A+V 同协议的完整曲线；在本目录一并归档用于同协议比较。
+- 此前用户汇总表：七组 `causal_valid_only` 的标称 0.2s 三类 AP/mAP 和十-horizon 平均。
+- 本地没有收到原始 `metrics.json`、预测数组、checkpoint 内容或训练摘要 JSON。
+  附件中的 `training summary: ...pth.json` 只是路径，不是文件内容；没有训练曲线。
+- 本次附件在 full-timeline 均值之后结束，**没有包含 causal-valid 的完整曲线**。
+  不将前者改名为后者，也不根据均值补造未提供的 horizon。
+- 以下四轮训练、训练 split、窗口和 head 设置来自已约定命令与代码；
+  尚未从远程 checkpoint 元数据独立核验 epoch、seed、参数量或运行 commit。
+
+G = 全局 Thinker 特征；A = 近期混合音频编码器向量；V = 逐人局部视觉编码器向量的池化。
+旧 B1 中名为 `audio / visual / av` 的运行实际输入为 G+A / G+V / G+A+V，
+不是本次 encoder-only 的 A / V / AV。
+
+## 2. 实验配置与来源路径
+
+约定配置：
+
+- EasyCom：训练 Session 4–12 升序、无 dev、测试 Session 3/2/1。
+- 冻结 Qwen2.5-Omni-3B 编码器；本次实验名为 seed 0、4 epochs、最后一轮评测。
+- A/V 原生各 2048D；A 是最近 1s 混合音频的有效编码器 token pooling。
+- V 使用提供的人物框；2FPS 空间 token mean → 同帧逐人等权 mean → 最近 1s 有效帧 mean。
+  这不是全帧视觉，不是逐人 attention，也没有评测在线人脸检测器。
+- 每 200ms 一行，输入窗口按 `[max(0,t-1),t)` 约定，标签与原实验相同。
+- encoder-only 不输入 Thinker 状态；将选定原生向量和 validity 位拼接后输入 MLP。
+- 对照 G：8s open-stream-replay、2FPS、Thinker last4 stack + concat-MLP。
+- LSTR-MCE（三路 softmax），batch 512、lr 7e-5、weight decay 5e-5、
+  warmup 1.6 epochs、warmup factor 0.3、dropout 0.2、无 transition objective。
+- 局部提取版本按先前实验报告为 Transformers 4.57.6；新附件不是编码器提取日志，
+  没有再次核验所有 sidecar 的实际 config。
+
+用户已通过目录列表确认局部源特征在：
+
+```text
+/data1/yfzx/wangx/Datasets/easycom/omni_features/local_qwen_1s_encoded_tf4576
+```
+
+新导出目录为同一 `omni_features` 下的
+`encoder_only_audio_1s`、`encoder_only_visual_1s`、`encoder_only_av_1s`。
+这些是已有向量的重新打包，不是重新运行 Qwen。
+
+其他路径：
+
+- 项目：`/data1/yfzx/wangx/Code/icassp2027-experiment`
+- target：`/data1/yfzx/wangx/Datasets/easycom/easycom_target_frame`
+- 新 checkpoint：`checkpoints/easycom_encoder_only_1s/encoder_<modality>_seed0.pth`
+- 新评测：`results/easycom_encoder_only_1s/encoder_<modality>_seed0/metrics.json`
+- 旧 B1 checkpoint：`checkpoints/easycom_local_b1_encoded_tf4576/{audio,av}_seed0.pth`
+- 实验入口与命令：[EasyCom encoder-only 指南](../../docs/easycom_encoder_only.md)；
+  本地入口版本为 `321238a`，不是对远程运行版本的证明。
+
+## 3. 两种评测协议必须分开
+
+三个新运行及两个旧对照都打印了相同预测形状：
+Session_3 = (8400,3,10)、Session_2 = (7706,3,10)、Session_1 = (6942,3,10)，
+总计 23048 行。
+
+`egospeak_full_timeline` 每个 horizon 都评测 23048 行，
+包含每个 session 开头没有历史预测的 h 行，数组中这些分数初始化为零。
+`causal_valid_only` 排除这部分行，因此标称 0.2s 为 23045 行，
+依次减少，到 2.0s 为 23018 行。这里 h 指 1–10 的 horizon 索引。
+
+新日志和此前汇总值的小差异与这两种协议一致，不应当成新的训练结果或抄录误差：
+
+| 输入 | causal − full：0.2s mAP | causal − full：十-horizon 均值 |
+|---|---:|---:|
+| V | +0.00108 | +0.00646 |
+| A+V | +0.00236 | +0.01176 |
+| A | +0.00240 | +0.01154 |
+
+数值是 AP/mAP 百分数；差值单位是百分点。代码依据：
+[`infer_turn_head.py`](../../infer_turn_head.py) 的 `_align_to_egospeak_timeline` 与 `_evaluate`。
+`causal_valid_only` 是有效预测行的评测掩码，**不是原始输入没有泄漏的独立证据**。
+本次没有重新审计标签时间戳、原视频时钟或在线延迟；表中 0.2–2.0s 一律指协议标称 horizon，
+不直接当作已证实的物理提前量。
+
+## 4. 主协议汇总：causal_valid_only，seed 0
+
+来自此前用户直接提供的汇总，不是由 full-timeline 反推。仅有首个 horizon 和均值。
+
+| 输入 | 标称 0.2s mAP | BG AP | wearer AP | others AP | 十-horizon 平均 mAP |
+|---|---:|---:|---:|---:|---:|
+| G | 73.44637 | 56.76438 | 77.86650 | 85.70824 | 60.84861 |
+| A | 79.69221 | 63.69477 | 84.09982 | 91.28205 | 60.50771 |
+| V | 48.38434 | 30.16284 | 41.68834 | 73.30186 | 46.25250 |
+| A+V | 79.00727 | 63.67397 | 82.69832 | 90.64952 | 60.76303 |
+| G+A | 81.72257 | 68.15248 | 85.06554 | 91.94969 | 65.10895 |
+| G+V | 73.47412 | 56.67089 | 78.60954 | 85.14194 | 60.99238 |
+| G+A+V | 82.02661 | 68.93423 | 85.08583 | 92.05978 | 65.39978 |
+
+来源转录：[reported_causal_summary.txt](reported_causal_summary.txt)。
+不要将这里的 G 行与下一节 full-timeline 的逐 horizon 行混算差值。
+
+## 5. 完整 mAP 曲线：egospeak_full_timeline，seed 0
+
+A/V/AV 为本次三份附件；G+A/G+A+V 为此前两份附件。所有列是同一协议。
+
+| 标称 horizon / s | A | V | A+V | G+A | G+A+V |
+|---|---:|---:|---:|---:|---:|
+| 0.2 | 79.68981 | 48.38326 | 79.00491 | 81.72011 | 82.02415 |
+| 0.4 | 72.41576 | 47.68478 | 72.06793 | 75.91398 | 75.99169 |
+| 0.6 | 66.33666 | 47.26191 | 66.16179 | 70.37189 | 70.92665 |
+| 0.8 | 61.60584 | 46.62600 | 61.62137 | 66.45047 | 66.79615 |
+| 1.0 | 58.28586 | 46.00582 | 58.49237 | 63.79981 | 63.86856 |
+| 1.2 | 55.93708 | 45.75002 | 56.07422 | 61.35120 | 61.56280 |
+| 1.4 | 54.22585 | 45.41812 | 54.61933 | 59.98279 | 60.34070 |
+| 1.6 | 52.83014 | 45.56259 | 53.78180 | 58.29734 | 58.83407 |
+| 1.8 | 51.97218 | 44.79454 | 52.91556 | 56.93221 | 57.39446 |
+| 2.0 | 51.66256 | 44.97339 | 52.77341 | 56.13656 | 56.12449 |
+| 十个 horizon 平均 | 60.49617 | 46.24604 | 60.75127 | 65.09564 | 65.38637 |
+
+所有 50 个 horizon 的三类 AP、样本数和 mAP 都保存在
+[summary.json](summary.json) 的 `egospeak_full_timeline.runs`；
+不只保存表格中的 mAP。
+
+## 6. 视觉不是“全程无效”：需要分 horizon、分类别看
+
+下表是 **A+V − A**，全部使用 full-timeline，单位为百分点：
+
+| 标称 horizon / s | ΔmAP | ΔBG AP | Δwearer AP | Δothers AP |
+|---|---:|---:|---:|---:|
+| 0.2 | -0.68490 | -0.02080 | -1.40150 | -0.63239 |
+| 0.4 | -0.34783 | +0.70624 | -1.59157 | -0.15812 |
+| 0.6 | -0.17487 | +0.82855 | -1.58313 | +0.22998 |
+| 0.8 | +0.01553 | +1.44947 | -1.66891 | +0.26602 |
+| 1.0 | +0.20651 | +1.56046 | -1.51747 | +0.57655 |
+| 1.2 | +0.13714 | +1.00313 | -1.31375 | +0.72202 |
+| 1.4 | +0.39348 | +1.42271 | -1.10816 | +0.86587 |
+| 1.6 | +0.95166 | +1.90632 | -0.41886 | +1.36752 |
+| 1.8 | +0.94338 | +1.81173 | -0.05834 | +1.07678 |
+| 2.0 | +1.11085 | +1.66905 | +0.66242 | +1.00108 |
+
+观察：
+
+- mAP 在 0.2/0.4/0.6s 下降；0.8s 的 +0.01553 只是很小的数值差。
+  1.6/1.8/2.0s 分别 +0.95166/+0.94338/+1.11085，十-horizon 均值为 +0.25510。
+- wearer AP 在前 9 个 horizon 均下降，直到 2.0s 才提高。
+  十-horizon 的 BG、wearer、others AP 平均变化分别为
+  **+1.23369 / −0.99993 / +0.53153**。
+- 因此“AV 的平均 mAP 略好”掩盖了 wearer 分支的退步；也不能再笼统说视觉在所有 horizon 都有害。
+- 这些 horizon 来自同一 seed、同一批序列，彼此相关；7/10 个正差不能当成七次独立复现实验。
+  正负号是描述，不是显著性检验。
+
+V-only 的 mAP 从 48.38326 到 44.97339，整体较低；但 AP 不是分类准确率，
+未给出测试标签 prevalence 或随机排序基准，不能仅凭 48% 宣称“随机水平”。
+可见人物框不直接提供 wearer 的脸，且当前逐人／时间平均不保留显式交互关系；
+这可作为解释假设，尚不能证明退步确由视觉信息不足或池化导致。
+
+## 7. 加入 G 的组合增益：在完整曲线上不只是均值现象
+
+以下仍是 full-timeline。各类别列为十个 horizon 的 AP 差值平均：
+
+| 对照 | Δ十-horizon mAP | ΔBG AP 均值 | Δwearer AP 均值 | Δothers AP 均值 | mAP 正差 horizon |
+|---|---:|---:|---:|---:|---:|
+| G+A − A | +4.59947 | +8.71604 | +2.42825 | +2.65410 | 10/10 |
+| G+A+V − A+V | +4.63510 | +8.16923 | +3.48897 | +2.24711 | 10/10 |
+| G+A+V − G+A | +0.29073 | +0.68687 | +0.06078 | +0.12454 | 9/10 |
+
+- **G+A 相比 A 在全部 10 个 horizon、每个类别上都更高。**
+  mAP 的增益从 0.2s 的 +2.03030 增至 1.4s 的 +5.75694，
+  在 2.0s 仍为 +4.47400。完整曲线支持组合方案具有互补价值，
+  不是仅由某个 horizon 抬高均值。
+- **G+A+V 相比 A+V 同样在所有 horizon、所有类别上更高。**
+  十-horizon mAP 为 +4.63510。
+- **G+A+V 相比 G+A 的收益仍小且偏向 BG。**
+  十-horizon mAP +0.29073；wearer AP 均值仅 +0.06078；
+  0.2s wearer AP 仅 +0.02029，2.0s 的 mAP 反而 −0.01207。
+  因此不能把整体增益描述为已解决 wearer 发言预测或多说话人视觉理解。
+- 主协议对应的均值差是 G+A − A = +4.60124、
+  G+A+V − A+V = +4.63675、G+A+V − G+A = +0.29083。
+  这些数值与 full-timeline 接近，但仍分别标注，不互相替代。
+
+## 8. 归因限制：这还不是等容量、等上下文的 decoder 消融
+
+按当前 `TurnPredictionHead` 和约定维度计算的可训练参数量：
+
+| 输入 | 参数量 |
+|---|---:|
+| A 或 V | 2,134,048 |
+| A+V | 8,468,547 |
+| G | 18,928,670 |
+| G+A 或 G+V | 19,720,992 |
+| G+A+V | 20,513,314 |
+
+来源是本地代码实例化计算，不是远程 checkpoint 读取。A/V 单模态原生向量＋有效性位
+为 2049D；AV 为 4098D，MLP 隐藏宽度分别 1024 和 2049。
+G 使用 last4 concat-MLP；G+A/G+V/G+AV 还使用各路 256D 投影。
+因此 A→AV 同时改变参数量和优化问题；G+A→A 还改变了 history、视野和融合结构。
+
+这组结果不能单独证明：
+
+- Thinker 的语言推理或长时记忆是增益的唯一原因；
+- 模型已学到多人互动、speaker identity 或 turn transition；
+- AV 退步一定是过拟合，或一定由跨人／跨时间 pooling 引起；
+- 同为四轮就代表不同模型同样充分收敛；
+- encoder-only 或某一模型端到端更快。附件的 batch/s 不包含特征提取，
+  也没有同步计时、预热和统一基准，不能当作系统速度对照。
+
+## 9. 下一步实验优先级与论文表述
+
+1. **先补证据**：收集三组原始 `metrics.json`、checkpoint epoch/head 配置和训练摘要，
+   补全 causal-valid 曲线；在固定四轮／无 dev 的既定协议下补 seed 1/2。
+   不根据这份测试曲线重新选 epoch、阈值或只挑有利 horizon。
+2. **先做受控融合对照**：固定分支投影与后端 MLP 架构，以明确的模态屏蔽比较 A/AV，
+   并报告总参数和实际参与学习的参数；用预先定义的相同原始可见时间范围控制 history。
+   再比较 G 的增量，避免把更大 head 或更长历史完全归给 decoder。
+3. **再测试多人视觉假设**：优先复用已经保存的逐人向量，比较平均池化与音频条件的逐人加权，
+   保留音频分支作为对照。事先定义多可见人物、重叠说话等分析子集及样本量，
+   同时报告 BG/wearer/others，而不只看总体 mAP。提供框的结果与在线检测器场景分开说明。
+
+当前可以写成：
+
+> 在本次单种子设置中，近期音频编码器提供强近时预测信号，加入全局 Thinker 表示的组合
+> 在所有评测 horizon 上优于对应的 encoder-only 方案。局部视觉的影响依赖类别和 horizon：
+> 部分较后 horizon 的宏平均指标改善，但 wearer AP 多数下降。等容量、等上下文控制和
+> 多种子复现仍是隔离各组件贡献的必要步骤。
+
+这支持继续研究“近期感知与上下文表示如何互补、多人视觉如何有效融入”，
+但尚不足以把“多人视觉调优成功”写成已经证实的贡献。
+
+## 10. 归档文件与校验范围
+
+- [summary.json](summary.json)：两种协议分区、完整 AP 曲线、计算差值、配置来源与证据边界。
+- [reported_causal_summary.txt](reported_causal_summary.txt)：此前主协议汇总的转录。
+- [raw_encoder_audio_seed0.txt](raw_encoder_audio_seed0.txt)、
+  [raw_encoder_visual_seed0.txt](raw_encoder_visual_seed0.txt)、
+  [raw_encoder_av_seed0.txt](raw_encoder_av_seed0.txt)：本次三份控制台附件。
+- [raw_previous_thinker_audio_seed0.txt](raw_previous_thinker_audio_seed0.txt)、
+  [raw_previous_thinker_av_seed0.txt](raw_previous_thinker_av_seed0.txt)：此前两个对照附件。
+
+控制台归档仅移除了行尾水平空白、统一最后的换行；保留换行折断的路径和全部数值。
+本地核对了 50 个 horizon 的 AP→mAP、十-horizon 均值、样本数、主协议转录与差值；
+四舍五入允许 1e-5 的百分数误差。没有从原始预测重新计算 AP，
+这些文件不是对远程 `metrics.json` 或真实训练产物的替代。
--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_encoder_audio_seed0.txt
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_encoder_audio_seed0.txt
@@ -0,0 +1,40 @@
+checkpoint: /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_audio_seed0.pth
+training summary: /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_audio_seed0.pth.json
+/usr/local/bin/python /data1/yfzx/wangx/Code/icassp2027-experiment/infer_turn_head.py --dataset easycom --feature-root /data1/yfzx/wangx/Datasets/easycom/omni_features/encoder_only_audio_1s --target-root /data1/yfzx/wangx/Datasets/easyco
+m/easycom_target_frame --expected-mode encoder-only-audio --device cuda:0 --checkpoint /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_audio_seed0.pth --output-dir /data1/yfzx/wangx/Code/icassp202
+7-experiment/results/easycom_encoder_only_1s/encoder_audio_seed0 --sessions Session_3 Session_2 Session_1
+/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the
+ package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+device: cuda:0
+loss: lstr-mce; output_space: three-class-softmax
+sessions: Session_3, Session_2, Session_1
+infer Session_3: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 21.47batch/s]
+Session_3: saved pred (8400, 3, 10), causal-valid counts=[8399, 8398, 8397, 8396, 8395, 8394, 8393, 8392, 8391, 8390]
+infer Session_2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 1441.65batch/s]
+Session_2: saved pred (7706, 3, 10), causal-valid counts=[7705, 7704, 7703, 7702, 7701, 7700, 7699, 7698, 7697, 7696]
+infer Session_1: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 1407.96batch/s]
+Session_1: saved pred (6942, 3, 10), causal-valid counts=[6941, 6940, 6939, 6938, 6937, 6936, 6935, 6934, 6933, 6932]
+
+egospeak_full_timeline:
+Action anticipation (0.20s) perframe mAP: 79.68981
+  AP: background=63.69477, camera_wearer_speaking=84.09982, other_people_speaking=91.27483; n=23048
+Action anticipation (0.40s) perframe mAP: 72.41576
+  AP: background=52.47780, camera_wearer_speaking=77.43606, other_people_speaking=87.33340; n=23048
+Action anticipation (0.60s) perframe mAP: 66.33666
+  AP: background=44.20066, camera_wearer_speaking=71.41221, other_people_speaking=83.39710; n=23048
+Action anticipation (0.80s) perframe mAP: 61.60584
+  AP: background=38.60179, camera_wearer_speaking=66.08811, other_people_speaking=80.12762; n=23048
+Action anticipation (1.00s) perframe mAP: 58.28586
+  AP: background=34.75843, camera_wearer_speaking=62.53734, other_people_speaking=77.56181; n=23048
+Action anticipation (1.20s) perframe mAP: 55.93708
+  AP: background=32.93325, camera_wearer_speaking=59.29096, other_people_speaking=75.58705; n=23048
+Action anticipation (1.40s) perframe mAP: 54.22585
+  AP: background=31.56977, camera_wearer_speaking=56.79707, other_people_speaking=74.31072; n=23048
+Action anticipation (1.60s) perframe mAP: 52.83014
+  AP: background=30.67884, camera_wearer_speaking=54.61175, other_people_speaking=73.19983; n=23048
+Action anticipation (1.80s) perframe mAP: 51.97218
+  AP: background=30.37030, camera_wearer_speaking=52.80894, other_people_speaking=72.73728; n=23048
+Action anticipation (2.00s) perframe mAP: 51.66256
+  AP: background=30.86998, camera_wearer_speaking=51.49671, other_people_speaking=72.62098; n=23048
+Action anticipation (mean) perframe mAP: 60.49617
--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_encoder_av_seed0.txt
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_encoder_av_seed0.txt
@@ -0,0 +1,39 @@
+training summary: /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_av_seed0.pth.json
+/usr/local/bin/python /data1/yfzx/wangx/Code/icassp2027-experiment/infer_turn_head.py --dataset easycom --feature-root /data1/yfzx/wangx/Datasets/easycom/omni_features/encoder_only_av_1s --target-root /data1/yfzx/wangx/Datasets/easycom/e
+asycom_target_frame --expected-mode encoder-only-av --device cuda:0 --checkpoint /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_av_seed0.pth --output-dir /data1/yfzx/wangx/Code/icassp2027-experim
+ent/results/easycom_encoder_only_1s/encoder_av_seed0 --sessions Session_3 Session_2 Session_1
+/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the
+ package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+device: cuda:0
+loss: lstr-mce; output_space: three-class-softmax
+sessions: Session_3, Session_2, Session_1
+infer Session_3: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 69.00batch/s]
+Session_3: saved pred (8400, 3, 10), causal-valid counts=[8399, 8398, 8397, 8396, 8395, 8394, 8393, 8392, 8391, 8390]
+infer Session_2: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 683.75batch/s]
+Session_2: saved pred (7706, 3, 10), causal-valid counts=[7705, 7704, 7703, 7702, 7701, 7700, 7699, 7698, 7697, 7696]
+infer Session_1: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 634.13batch/s]
+Session_1: saved pred (6942, 3, 10), causal-valid counts=[6941, 6940, 6939, 6938, 6937, 6936, 6935, 6934, 6933, 6932]
+
+egospeak_full_timeline:
+Action anticipation (0.20s) perframe mAP: 79.00491
+  AP: background=63.67397, camera_wearer_speaking=82.69832, other_people_speaking=90.64244; n=23048
+Action anticipation (0.40s) perframe mAP: 72.06793
+  AP: background=53.18404, camera_wearer_speaking=75.84449, other_people_speaking=87.17528; n=23048
+Action anticipation (0.60s) perframe mAP: 66.16179
+  AP: background=45.02921, camera_wearer_speaking=69.82908, other_people_speaking=83.62708; n=23048
+Action anticipation (0.80s) perframe mAP: 61.62137
+  AP: background=40.05126, camera_wearer_speaking=64.41920, other_people_speaking=80.39364; n=23048
+Action anticipation (1.00s) perframe mAP: 58.49237
+  AP: background=36.31889, camera_wearer_speaking=61.01987, other_people_speaking=78.13836; n=23048
+Action anticipation (1.20s) perframe mAP: 56.07422
+  AP: background=33.93638, camera_wearer_speaking=57.97721, other_people_speaking=76.30907; n=23048
+Action anticipation (1.40s) perframe mAP: 54.61933
+  AP: background=32.99248, camera_wearer_speaking=55.68891, other_people_speaking=75.17659; n=23048
+Action anticipation (1.60s) perframe mAP: 53.78180
+  AP: background=32.58516, camera_wearer_speaking=54.19289, other_people_speaking=74.56735; n=23048
+Action anticipation (1.80s) perframe mAP: 52.91556
+  AP: background=32.18203, camera_wearer_speaking=52.75060, other_people_speaking=73.81406; n=23048
+Action anticipation (2.00s) perframe mAP: 52.77341
+  AP: background=32.53903, camera_wearer_speaking=52.15913, other_people_speaking=73.62206; n=23048
+Action anticipation (mean) perframe mAP: 60.75127
--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_encoder_visual_seed0.txt
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_encoder_visual_seed0.txt
@@ -0,0 +1,40 @@
+checkpoint: /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_visual_seed0.pth
+training summary: /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_visual_seed0.pth.json
+/usr/local/bin/python /data1/yfzx/wangx/Code/icassp2027-experiment/infer_turn_head.py --dataset easycom --feature-root /data1/yfzx/wangx/Datasets/easycom/omni_features/encoder_only_visual_1s --target-root /data1/yfzx/wangx/Datasets/easyc
+om/easycom_target_frame --expected-mode encoder-only-visual --device cuda:0 --checkpoint /data1/yfzx/wangx/Code/icassp2027-experiment/checkpoints/easycom_encoder_only_1s/encoder_visual_seed0.pth --output-dir /data1/yfzx/wangx/Code/icassp
+2027-experiment/results/easycom_encoder_only_1s/encoder_visual_seed0 --sessions Session_3 Session_2 Session_1
+/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the
+ package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+device: cuda:0
+loss: lstr-mce; output_space: three-class-softmax
+sessions: Session_3, Session_2, Session_1
+infer Session_3: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 71.22batch/s]
+Session_3: saved pred (8400, 3, 10), causal-valid counts=[8399, 8398, 8397, 8396, 8395, 8394, 8393, 8392, 8391, 8390]
+infer Session_2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 1487.47batch/s]
+Session_2: saved pred (7706, 3, 10), causal-valid counts=[7705, 7704, 7703, 7702, 7701, 7700, 7699, 7698, 7697, 7696]
+infer Session_1: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 1409.24batch/s]
+Session_1: saved pred (6942, 3, 10), causal-valid counts=[6941, 6940, 6939, 6938, 6937, 6936, 6935, 6934, 6933, 6932]
+
+egospeak_full_timeline:
+Action anticipation (0.20s) perframe mAP: 48.38326
+  AP: background=30.16284, camera_wearer_speaking=41.68834, other_people_speaking=73.29862; n=23048
+Action anticipation (0.40s) perframe mAP: 47.68478
+  AP: background=29.82215, camera_wearer_speaking=40.98323, other_people_speaking=72.24896; n=23048
+Action anticipation (0.60s) perframe mAP: 47.26191
+  AP: background=29.28129, camera_wearer_speaking=40.74974, other_people_speaking=71.75469; n=23048
+Action anticipation (0.80s) perframe mAP: 46.62600
+  AP: background=28.24103, camera_wearer_speaking=40.47370, other_people_speaking=71.16328; n=23048
+Action anticipation (1.00s) perframe mAP: 46.00582
+  AP: background=27.73638, camera_wearer_speaking=39.50369, other_people_speaking=70.77740; n=23048
+Action anticipation (1.20s) perframe mAP: 45.75002
+  AP: background=27.39577, camera_wearer_speaking=39.53354, other_people_speaking=70.32075; n=23048
+Action anticipation (1.40s) perframe mAP: 45.41812
+  AP: background=27.04920, camera_wearer_speaking=39.35560, other_people_speaking=69.84957; n=23048
+Action anticipation (1.60s) perframe mAP: 45.56259
+  AP: background=27.36515, camera_wearer_speaking=39.16056, other_people_speaking=70.16205; n=23048
+Action anticipation (1.80s) perframe mAP: 44.79454
+  AP: background=26.64061, camera_wearer_speaking=38.50155, other_people_speaking=69.24145; n=23048
+Action anticipation (2.00s) perframe mAP: 44.97339
+  AP: background=27.34810, camera_wearer_speaking=37.96646, other_people_speaking=69.60560; n=23048
+Action anticipation (mean) perframe mAP: 46.24604
--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_previous_thinker_audio_seed0.txt
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_previous_thinker_audio_seed0.txt
@@ -0,0 +1,36 @@
+training summary: checkpoints/easycom_local_b1_encoded_tf4576/audio_seed0.pth.json
+/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml d
+irectly, please report this to the maintainers of the package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+device: cuda:0
+loss: lstr-mce; output_space: three-class-softmax
+sessions: Session_3, Session_2, Session_1
+infer Session_3: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 39.03batch/s]
+Session_3: saved pred (8400, 3, 10), causal-valid counts=[8399, 8398, 8397, 8396, 8395, 8394, 8393, 8392, 8391, 8390]
+infer Session_2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 142.52batch/s]
+Session_2: saved pred (7706, 3, 10), causal-valid counts=[7705, 7704, 7703, 7702, 7701, 7700, 7699, 7698, 7697, 7696]
+infer Session_1: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 144.02batch/s]
+Session_1: saved pred (6942, 3, 10), causal-valid counts=[6941, 6940, 6939, 6938, 6937, 6936, 6935, 6934, 6933, 6932]
+
+egospeak_full_timeline:
+Action anticipation (0.20s) perframe mAP: 81.72011
+  AP: background=68.15248, camera_wearer_speaking=85.06554, other_people_speaking=91.94232; n=23048
+Action anticipation (0.40s) perframe mAP: 75.91398
+  AP: background=59.90715, camera_wearer_speaking=79.22465, other_people_speaking=88.61015; n=23048
+Action anticipation (0.60s) perframe mAP: 70.37189
+  AP: background=52.48480, camera_wearer_speaking=73.21461, other_people_speaking=85.41626; n=23048
+Action anticipation (0.80s) perframe mAP: 66.45047
+  AP: background=47.92066, camera_wearer_speaking=68.77900, other_people_speaking=82.65175; n=23048
+Action anticipation (1.00s) perframe mAP: 63.79981
+  AP: background=45.59262, camera_wearer_speaking=65.09890, other_people_speaking=80.70791; n=23048
+Action anticipation (1.20s) perframe mAP: 61.35120
+  AP: background=43.15384, camera_wearer_speaking=61.86698, other_people_speaking=79.03278; n=23048
+Action anticipation (1.40s) perframe mAP: 59.98279
+  AP: background=41.83907, camera_wearer_speaking=60.22346, other_people_speaking=77.88584; n=23048
+Action anticipation (1.60s) perframe mAP: 58.29734
+  AP: background=40.44538, camera_wearer_speaking=57.66189, other_people_speaking=76.78474; n=23048
+Action anticipation (1.80s) perframe mAP: 56.93221
+  AP: background=39.04677, camera_wearer_speaking=55.70067, other_people_speaking=76.04919; n=23048
+Action anticipation (2.00s) perframe mAP: 56.13656
+  AP: background=38.77318, camera_wearer_speaking=54.02582, other_people_speaking=75.61069; n=23048
+Action anticipation (mean) perframe mAP: 65.09564
--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_previous_thinker_av_seed0.txt
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/raw_previous_thinker_av_seed0.txt
@@ -0,0 +1,37 @@
+checkpoint: checkpoints/easycom_local_b1_encoded_tf4576/av_seed0.pth
+training summary: checkpoints/easycom_local_b1_encoded_tf4576/av_seed0.pth.json
+/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml d
+irectly, please report this to the maintainers of the package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+device: cuda:0
+loss: lstr-mce; output_space: three-class-softmax
+sessions: Session_3, Session_2, Session_1
+infer Session_3: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 40.61batch/s]
+Session_3: saved pred (8400, 3, 10), causal-valid counts=[8399, 8398, 8397, 8396, 8395, 8394, 8393, 8392, 8391, 8390]
+infer Session_2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 143.92batch/s]
+Session_2: saved pred (7706, 3, 10), causal-valid counts=[7705, 7704, 7703, 7702, 7701, 7700, 7699, 7698, 7697, 7696]
+infer Session_1: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 160.27batch/s]
+Session_1: saved pred (6942, 3, 10), causal-valid counts=[6941, 6940, 6939, 6938, 6937, 6936, 6935, 6934, 6933, 6932]
+
+egospeak_full_timeline:
+Action anticipation (0.20s) perframe mAP: 82.02415
+  AP: background=68.93423, camera_wearer_speaking=85.08583, other_people_speaking=92.05239; n=23048
+Action anticipation (0.40s) perframe mAP: 75.99169
+  AP: background=60.58903, camera_wearer_speaking=78.83427, other_people_speaking=88.55177; n=23048
+Action anticipation (0.60s) perframe mAP: 70.92665
+  AP: background=53.84208, camera_wearer_speaking=73.45503, other_people_speaking=85.48283; n=23048
+Action anticipation (0.80s) perframe mAP: 66.79615
+  AP: background=49.21382, camera_wearer_speaking=68.56072, other_people_speaking=82.61390; n=23048
+Action anticipation (1.00s) perframe mAP: 63.86856
+  AP: background=45.62073, camera_wearer_speaking=65.27034, other_people_speaking=80.71461; n=23048
+Action anticipation (1.20s) perframe mAP: 61.56280
+  AP: background=43.46670, camera_wearer_speaking=62.05277, other_people_speaking=79.16893; n=23048
+Action anticipation (1.40s) perframe mAP: 60.34070
+  AP: background=42.54682, camera_wearer_speaking=60.20540, other_people_speaking=78.26988; n=23048
+Action anticipation (1.60s) perframe mAP: 58.83407
+  AP: background=41.15952, camera_wearer_speaking=58.18522, other_people_speaking=77.15747; n=23048
+Action anticipation (1.80s) perframe mAP: 57.39446
+  AP: background=39.94176, camera_wearer_speaking=55.99271, other_people_speaking=76.24890; n=23048
+Action anticipation (2.00s) perframe mAP: 56.12449
+  AP: background=38.87001, camera_wearer_speaking=53.82707, other_people_speaking=75.67639; n=23048
+Action anticipation (mean) perframe mAP: 65.38637
--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/reported_causal_summary.txt
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/reported_causal_summary.txt
@@ -0,0 +1,15 @@
+User-reported causal_valid_only summaries from earlier messages in this thread.
+Not a dump of remote metrics.json; only the first horizon and ten-horizon mean were supplied.
+G=Thinker; A/V=local encoders. All runs below are seed 0. AP/mAP values are percent.
+
+variant,input,seed,nominal_0.2s_map,background_ap,wearer_ap,others_ap,ten_horizon_mean_map
+thinker,G,0,73.44637,56.76438,77.86650,85.70824,60.84861
+encoder_audio,A,0,79.69221,63.69477,84.09982,91.28205,60.50771
+encoder_visual,V,0,48.38434,30.16284,41.68834,73.30186,46.25250
+encoder_av,A+V,0,79.00727,63.67397,82.69832,90.64952,60.76303
+thinker_audio,G+A,0,81.72257,68.15248,85.06554,91.94969,65.10895
+thinker_visual,G+V,0,73.47412,56.67089,78.60954,85.14194,60.99238
+thinker_av,G+A+V,0,82.02661,68.93423,85.08583,92.05978,65.39978
+
+The newly supplied full logs use egospeak_full_timeline, not this protocol.
+The reported sample standard deviation for each encoder-only run was n/a, not zero.
--- a/results/easycom_encoder_only_1s_4ep_2026-08-28/summary.json
+++ b/results/easycom_encoder_only_1s_4ep_2026-08-28/summary.json