April 17, 2026
Semantic- and Geometry-Aware 3D Reconstruction of Outdoor Scenes Using Low-Cost Vision Sensors
Thesis Title: Semantic- and Geometry-Aware 3D Reconstruction of Outdoor Scenes Using Low-Cost Vision Sensors
Degree program: MSc in Geomatics Engineering
Student鈥檚 name: Rhea Joyce Zambra
Supervisor鈥檚 name: Hongzhou Yang
Accurate 3D reconstruction in large-scale outdoor environments remains challenging due to geometric degeneracies, scene heterogeneity, and the reliance of modern methods on appearance-driven optimization. In forward-motion scenarios common to autonomous driving datasets, sparse Structure-from-Motion (SfM) suffers from poor depth observability, leading to unstable geometry that propagates into downstream dense reconstruction methods such as 3D Gaussian Splatting (3DGS). These limitations are further exacerbated in complex scenes containing a mixture of planar structures, vegetation, and dynamic objects, where uniform regularization strategies fail to capture class-specific geometric behavior.
This thesis proposes a semantic-aware reconstruction pipeline that introduces physically meaningful geometric constraints at both the sparse and dense stages. First, a Ground-Optimized SfM (GO-SfM) framework is developed by incorporating a developable surface prior into bundle adjustment, applied selectively to confidently assigned driveable points. This improves local geometric consistency and stabilizes camera pose estimation without requiring external sensors or learned depth priors. Second, the refined sparse reconstruction is used to initialize a semantically decomposed 3D Gaussian Splatting process, where class-specific geometric constraints (e.g., planar enforcement, silhouette consistency, etc.) are applied to guide dense optimization.
Experimental results on the Waymo Open and nuScenes datasets demonstrate that the proposed approach improves both geometric fidelity and rendering quality. The GO-SfM framework produces more accurate camera pose estimates (rotational and translational RMSE) and provides stable spatial anchors for downstream densification, while the semantic-aware Gaussian Splatting reduces common artifacts such as floaters and depth ambiguity. Quantitatively, the method achieves consistent gains in rendering metrics (PSNR and SSIM) over standard SfM-initialized 3DGS. Together, these results underscore the value of integrating semantic information with geometry-aware priors to address fundamental limitations in vision-only 3D reconstruction pipelines.