LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives

Large garages are ubiquitous yet intricate scenes in our daily lives, posing challenges characterized by monotonous colors, repetitive patterns, reflective surfaces, and transparent vehicle glass. Conventional Structure from Motion (SfM) methods for camera pose estimation and 3D reconstruction fail in these environments due to poor correspondence construction. To address these challenges, this paper introduces LetsGo, a LiDAR-assisted Gaussian splatting approach for large-scale garage modeling and rendering. We develop a handheld scanner, Polar, equipped with IMU, LiDAR, and a fisheye camera, to facilitate accurate LiDAR and image data scanning. With this Polar device, we present a GarageWorld dataset consisting of five expansive garage scenes with diverse geometric structures and will release the dataset to the community for further research. We demonstrate that the collected LiDAR point cloud by the Polar device enhances a suite of 3D Gaussian splatting algorithms for garage scene modeling and rendering. We also propose a novel depth regularizer for 3D Gaussian splatting algorithm training, effectively eliminating floating artifacts in rendered images, and a lightweight Level of Detail (LOD) Gaussian renderer for real-time viewing on web-based devices. Additionally, we explore a hybrid representation that combines the advantages of traditional mesh in depicting simple geometry and colors (e.g., walls and the ground) with modern 3D Gaussian representations capturing complex details and high-frequency textures. This strategy achieves an optimal balance between memory performance and rendering quality. Experimental results on our dataset, along with ScanNet++ and KITTI-360, demonstrate the superiority of our method in rendering quality and resource efficiency.

We design a handheld Polar scanner for expansive garage data collection, which combines IMU, LiDAR and a fisheye camera for robust relative pose estimation, and demonstrate that the LiDAR points collected by this Polar device successfully assist a suite of Gaussian splatting algorithms for garage scene representation. To improve the quality of the 3D Gaussian rendering, we also introduce a depth regularizer that uses depth priors as supervisory signals, significantly reducing floating artifacts and enabling high rendering quality.

In order to deliver large scene online rendering on lightweight devices, we construct a webpage LOD viewer. We store the unstructured residual 3D Gaussians in the partitioned sub-block of the large-scale garage in a layered manner in an octree. Following previous works, we apply Chunking, merging, and downsampling operations to create the octree at lower levels. Then a coarse-to-fine loading and rendering scheme is implented for the 3D Gaussians to enable real-time rendering on mobile devices.