GeoComplete: Geometry-Aware Diffusion for Reference-Driven Image Completion

NeurIPS 2025
Beibei Lin, Tingting Chen, Robby T. Tan
National University of Singapore
arXiv Code
Qualitative results

Given a few reference images of the same scene and a target image with missing regions, our method completes the target's missing regions while preserving geometric consistency more effectively than the state-of-the-art Paint-by-Example. Semi-transparent white masks indicate the known, unaltered regions of the target image.

Abstract

Reference-driven image completion, which restores missing regions in a target view using additional images, is particularly challenging when the target view differs significantly from the references. Existing generative methods rely solely on diffusion priors and, without geometric cues such as camera pose or depth, often produce misaligned or implausible content.

We propose GeoComplete, a novel framework that incorporates explicit 3D structural guidance to enforce geometric consistency in the completed regions, setting it apart from prior image-only approaches. GeoComplete introduces two key ideas: conditioning the diffusion process on projected point clouds to infuse geometric information, and applying target-aware masking to guide the model toward relevant reference cues. The framework features a dual-branch diffusion architecture. One branch synthesizes the missing regions from the masked target, while the other extracts geometric features from the projected point cloud. Joint self-attention across branches ensures coherent and accurate completion. To address regions visible in references but absent in the target, we project the target view into each reference to detect occluded areas, which are then masked during training. This target-aware masking directs the model to focus on useful cues, enhancing performance in difficult scenarios. To our knowledge, GeoComplete is the first to tightly couple explicit 3D geometry with diffusion-based image completion in a unified framework.

Experiments show that GeoComplete achieves a 17.1% PSNR improvement over state-of-the-art methods, significantly boosting geometric accuracy while maintaining high visual quality.

Pipeline

Framework overview

Overview of our GeoComplete framework: We first construct a point cloud from the reference and target images. During training, target-aware masking selectively masks both the reference images and their projected point clouds to highlight informative regions. These inputs are fed into a dual-branch diffusion model: the target branch encodes the masked image, while the cloud branch processes the projected point cloud. Joint self-attention fuses the two branches, enabling geometric cues to guide content synthesis. At inference, the masked target image and its projected point cloud are passed into the finetuned model to complete the missing regions.

Qualitative and Quantitative Results

Qualitative Results

Qualitative Results
Qualitative comparisons from RealFill, Paint-by-Example and our method. The red bounding box marks the known, unaltered region of the target image (i.e., the area inside the box), except for the first-row images, where the known region lies outside the box. Our method synthesizes missing regions while ensuring better geometric consistency.
Qualitative Results
Qualitative comparisons from RealFill, Paint-by-Exampleand our method. The red bounding box marks the known, unaltered region of the target image (i.e., the area inside the box). Our method synthesizes missing regions while ensuring better geometric consistency.

Quantitative Results

Quantitative Results

BibTeX

@article{lin2025geocomplete,
  title={GeoComplete: Geometry-Aware Diffusion for Reference-Driven Image Completion},
  author={Lin, Beibei and Chen, Tingting and Robby T., Tan},
  journal={The Thirty-Ninth Annual Conference on Neural Information Processing Systems},
  year={2025}
}