Déjà View: Looping Transformers for Multi-View 3D Reconstruction

Abstract

Déjà View applies a single looped transformer block recurrently to per-view features for K refinement steps, making explicit the iterative refinement that multi-view reconstruction transformers otherwise buy inefficiently through unique parameters across depth. Trained once, it exposes K as an inference-time compute knob. At only 117M parameters, Déjà View matches or outperforms substantially larger feed-forward baselines across five reconstruction benchmarks spanning indoor, outdoor, object-centric, and driving scenes, using 8-10x fewer parameters.