ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models

Abstract

Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed regions. ArtiFixer bridges 3D neural reconstruction and auto-regressive video generation to enhance sparse 3D Gaussian Splatting reconstructions by distilling a bidirectional video diffusion model into a causal auto-regressive generator, achieving state-of-the-art novel view synthesis and outperforming prior methods by 1-3 dB PSNR.

Tobias Fischer
Tobias Fischer

Dr. Tobias Fischer is a Researcher at Apple.