wx44wx/icassp2027-experiment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
11 Commits 1 Branch 0 Tags
43f47e8f3a2e6bafdce1f4bf2299e0d7cee45800
Commit Graph

11 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xin Wang
43f47e8f3a Add caching mechanism for completed AV blocks in streaming model. Introduce CompletedBlockPieceCache class to manage reusable blocks during strict-window replay, enhancing efficiency. Update profiling hooks for cache operations and refine README documentation to reflect changes in block handling and replay strategies. 2026-08-18 11:02:42 +08:00
Xin Wang
d610305355 Add profiling script for GPU bottleneck analysis in EasyCom extraction. Introduce command-line options for GPU ID, log directory, and sampling period. Implement stage profiling to measure extraction performance and synchronize CUDA operations. Enhance CausalSourceAV class with new parameters for decoder threads and open readers, optimizing video decoding process. 2026-08-18 10:30:19 +08:00
Xin Wang
24cb92f1f4 Add support for NVDEC backend in video processing and enhance README documentation. Introduce a new script for extracting EasyCom sessions on specific GPUs, detailing session management and error handling. Update requirements.txt to include PyNvVideoCodec as an optional dependency. Refactor streaming_turn_model_directly.py to accommodate multiple decoder backends and improve frame indexing with ffprobe. 2026-08-18 08:41:54 +08:00
Xin Wang
e4fb62ba60 Add script for extracting EasyCom sessions with three workers per GPU. This script manages session assignments, ensures serialized worker launches, and includes error handling for session readiness and logging. Enhances the EasyCom pipeline for improved session management and feature extraction. 2026-08-17 23:43:34 +08:00
Xin Wang
d815b59def Refactor README and streaming_turn_model.py to clarify fixed-grid frame ownership and update documentation. Remove merge conflict markers and enhance explanations regarding session launching conditions and audio slicing behavior. 2026-08-17 22:59:02 +08:00
Xin Wang
a6dea1529a Merge branch 'main' of https://gitea.xiaowang.eu.org/wx44wx/icassp2027-experiment 2026-08-17 22:46:44 +08:00
Xin Wang
5ee371f0ae Update README and streaming_turn_model.py to implement half-open interval ownership for fixed-grid AV frames. Adjust verification parameters to 1 for testing, and enhance documentation for clarity on audio slicing and frame selection. Add runtime checks to prevent overlapping temporal coordinates in open-stream intervals. 2026-08-17 22:45:17 +08:00
WangXin93
8145be4108 Add scripts for extracting EasyCom session features: run_extract_easycom_direct.sh for full session extraction and run_smoke_easycom_direct.sh for smoke testing on a single GPU. Both scripts utilize streaming_turn_model_directly.py for processing and include necessary configurations and logging. This enhances the EasyCom pipeline for better session management and feature extraction. 2026-08-17 22:16:27 +08:00
Xin Wang
de29e75734 Add scripts for preparing and extracting 5 FPS audio-video clips, including prepare_ego4d_2fps.py, run_extract_5fps_2pergpu.sh, and run_prepare_all_5fps.sh. Introduce streaming_turn_model_directly.py for direct extraction from original media. Update README.md to reflect changes in Qwen's streaming design and causal extraction modes. Add new AGENTS.md file for documentation. 2026-08-17 15:17:11 +08:00
Xin Wang
080c45f049 Add reproducible EasyCom causal head pipeline 2026-08-16 22:19:43 +08:00
Xin Wang
bde72487bb Initial commit: Omni-only turn-taking experiment scaffold. 
Add EasyCom prep, Qwen2.5-Omni feature extraction, streaming turn model prototype, and docs for the ICASSP 2027 ego-speak study.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-16 00:02:58 +08:00