Add caching mechanism for completed AV blocks in streaming model. Introduce CompletedBlockPieceCache class to manage reusable blocks during strict-window replay, enhancing efficiency. Update profiling hooks for cache operations and refine README documentation to reflect changes in block handling and replay strategies.

This commit is contained in:
Xin Wang
2026-08-18 11:02:42 +08:00
parent d610305355
commit 43f47e8f3a
4 changed files with 220 additions and 22 deletions

View File

@@ -477,7 +477,9 @@ two practical changes for this dataset:
1. the current 0--2 second partial block is evaluated speculatively every
0.2 seconds and discarded unless it reaches a complete block boundary;
2. the KV history is bounded and rebuilt from recent complete blocks rather
than accumulating for an entire multi-minute session.
than accumulating for an entire multi-minute session. Complete-block AV
embeddings are retained in a bounded ring, so rolling rebuilds rerun only
the Thinker and do not rerun the processor, audio tower, or video tower.
It should therefore be described as **bounded Qwen-style open streaming**, not
as bit-for-bit reproduction of an unbounded official serving session.
@@ -522,7 +524,10 @@ prediction is emitted every 0.2 seconds.
budget it rebuilds the open cache from the latest nine completed blocks
(18 seconds) when a new block starts, then adds the current 0--2 second tail.
Its effective history therefore varies from 18 to 20 seconds and never drops
to an empty 0.2-second context at a 20-second boundary.
to an empty 0.2-second context at a 20-second boundary. The nine blocks'
prepared AV embeddings and local TMRoPE coordinates are cached once; strict
window replay still recomputes the Thinker KV so retained blocks no longer
carry indirect influence from observations that fell out of the window.
For a short native-stream fidelity check, set `--memory-seconds` to at least
the tested clip duration. For example, the options