RESEARCH PROJECT · PROJECT PAGE
Autoregressive video diffusion enables scalable long-video generation by producing chunks from a bounded recent context, but recency-based caching eventually evicts the historical cues needed when subjects, objects, scenes, or attributes reappear. LayerRecall addresses this limitation with a current-conditioned, layer-selective memory router that retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers while preserving local attention elsewhere. To reduce reliance on scarce high-quality long-horizon videos and explicit memory-allocation labels, CHPM uses a privileged long-context reference to supervise the bounded-memory router in prediction space. Across 100 multi-shot evaluation prompts, LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, strengthening long-range recovery without sacrificing local continuity; additional analyses show cross-backbone portability and negligible inference overhead.
PART 01
10 SHOWCASES · THREE-SHOT RECALL
LayerRecall improves long-horizon consistency by recovering subjects, attributes, and scene-specific cues after they leave the local context and later reappear.
Each 63.9-second showcase follows a controlled three-shot recall sequence: Shot 1 (00:00.0–00:31.9) establishes the subject and memory-critical visual cues; Shot 2 (00:31.9–00:47.9) moves the subject out of frame, creating a need for nonlocal memory; and Shot 3 (00:47.9–01:03.9) tests whether those earlier identity, attribute, and scene cues are faithfully recalled after the temporal gap. Because Shot 1 spans twice as much source time as either later shot, it plays at 2× while Shots 2 and 3 remain at 1×.
An older puppet maker animates a large cloth puppet at his worktable, disappears behind the small stage curtain, then returns from behind it and settles at the table.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
An older reader turns the pages of an illustrated book beside a stained-glass window, walks out through the reading-room doorway, then returns to the same armchair.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
An older lighthouse keeper polishes the Fresnel lens, walks out through the narrow lantern-room doorway, then returns through the same doorway and settles beside the lens.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
A young cake decorator pipes frosting at a marble counter, exits through the cooler-room doorway, then returns to the same counter beside the unfinished cake.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
An older watercolorist paints at a seaside drawing board, leaves through the terrace doorway, then returns through the same doorway and settles beside the board.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
An older pastry chef kneads dough at a flour-dusted counter, leaves through the pantry doorway, then returns and settles at the same work position.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
An older tailor guides fabric through a sewing machine, walks out through the tailor-shop doorway, then returns to the sewing table beside the arched window.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
A young percussionist taps a large drum in a candlelit rehearsal room, exits through the doorway, then returns and settles at the same drum.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
An older bonsai gardener trims a large bonsai at a courtyard stone table, exits through the courtyard gate, then returns and settles at the same table.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
A young paper artist folds a large square sheet at a studio table, leaves through the paper-studio doorway, then returns to the same workstation.
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
SYNCHRONIZED SHOT POSITION
PART 02
DYNAMIC RETRIEVAL · LOCAL RECOVERY
In the annotated sequence, the woman first returns with a light, patterned inner layer that conflicts with the blue garment established before she leaves. Moments later, the blue color and texture recover while her gray cardigan, pose, book, table, and camera composition remain continuous.
The orange MISMATCH inset marks the drift; the green MATCH inset marks the recovery.
LayerRecall can revisit more detailed historical evidence as the returning subject becomes easier to identify. At first, the returning figure is still visually ambiguous. As her identity, silhouette, and relationship to the scene become clearer, the current-conditioned query can favor the earlier memory that best matches her appearance.
That memory is introduced only into memory-sensitive layers—the layers most responsive to long-range appearance cues—while the rest continue the local motion and scene. This division lets the model repair the inner garment without reconstructing the person or resetting the shot.
INTERPRETATION · This sequence is behavioral evidence consistent with the mechanism; before/after selection logs would provide stronger mechanism-level confirmation.ANNOTATED THREE-SHOT SEQUENCE · 15.96S
PART 03
5 CASES · LAYERRECALL + 2 BASELINES
On 60-second videos, we compare LayerRecall with mainstream open-source long-video generation models under a controlled present–absent–return setting: a subject is introduced, moves out of view, and must be recalled after the intervening scene.
LayerRecall more consistently restores the earlier subject and visual attributes while preserving overall temporal continuity and following the requested leave-and-return structure. Some baselines do not fully execute the departure instruction, so the intended memory gap is never established. For models without native multi-shot prompt input, we adapt the prompts to their supported format; full details are provided in the supplementary material.
Pianist on a small jazz-club stage · exits through the side-stage opening · returns to the piano
SYNCHRONIZED SHOT POSITION
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
PART 04
3 STUDIES · MEMORY QUALITY + TEMPORAL STABILITY
This section presents three ablation studies that isolate CHPM supervision, layer-selective memory routing, and profile-guided where-to-use, clarifying how each design choice contributes to memory quality and temporal stability.
02 CASES · TRAINED VS RANDOM INIT
Across the 100-prompt evaluation set, with the same ten memory-sensitive layers, CHPM-trained routing raises MemoBench overall from 0.519 to 0.548 over randomly initialized routing and improves every reported subdimension. The gains support prediction matching as effective routing supervision beyond the fixed layer policy.
Stone-relief restorer beside a large museum relief · exits through the restoration-room doorway · returns beside the relief
SYNCHRONIZED SHOT POSITION
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
01 CASE · SELECTIVE VS ALL-LAYER ROUTING
Across the 100-prompt evaluation set, with the checkpoint, retrieved memory, memory budget, and evaluation protocol fixed, layer-selective routing reduces high-frequency frame-change power from 0.60 to 0.38 relative to all-layer routing. It also improves motion smoothness and adjacent-frame CLIP/DINO consistency, showing that targeted memory access limits temporal disturbance.
SYNCHRONIZED SHOT POSITION
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
Lower abrupt-change energy indicates fewer sudden frame transitions; the shared axis exposes when broad all-layer memory access introduces temporal disturbance.
02 CASES · PROFILED-10 VS RANDOM-10
Across the 100-prompt evaluation set, with the checkpoint, retrieval mechanism, and memory budget fixed, the profiled ten-layer policy raises MemoBench overall from 0.538 to 0.570 and improves four of five reported dimensions over an equally sized random policy. This matched comparison shows that where retrieved memory enters the DiT materially affects how well it is used.
Leather artisan at a long workbench · exits through the workshop doorway · returns to the bench
SYNCHRONIZED SHOT POSITION
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
Target identity and attributes
Intended exit and off-screen interval
Long-range return after the gap
PART 05
2 SYSTEM DIAGRAMS · INFERENCE + TRAINING
INFERENCE FRAMEWORK
LayerRecall separates long-horizon memory into two decisions. A current-conditioned router selects relevant historical K/V states from finite, layer-indexed memory banks; only profiled memory-sensitive layers receive the retrieved states, while all other layers preserve the backbone’s original local attention context.
TRAINING FRAMEWORK
CHPM trains LayerRecall by matching predictions from two frozen views of the same backbone. The long-horizon teacher sees expanded historical context; the short-horizon student sees bounded local context augmented by LayerRecall memory routing. Only the routing module is optimized.