SVGS: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors

Rui Xu¹ Wenyue Chen² Jiepeng Wang¹ Yuan Liu^3,4 Peng Wang¹ Cheng Lin⁵ Shiqing Xin⁶ Xin Li⁷ Wenping Wang⁷ Taku Komura¹

¹The University of Hong Kong ²Peking University ³Nanyang Technological University
⁴Hong Kong University of Science and Technology ⁵Macau University of Science and Technology ⁶Shandong University ⁷Texas A&M University

Arxiv Preprint.

Abstract

Gaussian Splattings demonstrate impressive results in multi-view reconstruction based on Gaussian explicit representations. However, the current Gaussian primitives only have a single view-dependent color and an opacity to represent the appearance and geometry of the scene, resulting in a non-compact representation. In this paper, we introduce a new method called SVGS that utilizes spatially varying colors and opacity in a single Gaussian primitive to improve its representation ability. We have implemented bilinear interpolation, movable kernels, and even tiny neural networks as spatially varying functions. Quantitative and qualitative experimental results demonstrate that all three functions outperform the baseline, with the best movable kernels achieving superior novel view synthesis performance on multiple datasets, highlighting the strong potential of spatially varying functions.

SVGS

SVGS gives each Gaussian the ability to vary spatially. Compared with 2DGS and 3DGS, SVGS is more expressive and can better reconstruct details (as seen in the white area). Additionally, SVGS can express more color variations using only one Gaussian, instead of being limited to just one color (bottom right).

Methods

3DGS uses Gaussian ellipsoids to express scenes, and a learnable color is defined on each ellipsoid. 2DGS uses Gaussian surfels to express scenes, and a learnable color is defined on each Gaussian surfel. Our SVGS uses spatially varying Gaussian surfels to express scenes, and the color and opacity changes with the spatial position on each surfel.

Results

Visual comparison with 2DGS and 3DGS on both the Mip-NeRF360 dataset and the Tanks&Temples dataset shows that SVGS can reconstruct details better due to stronger expressiveness.

2DGS has demonstrated impressive capabilities in geometric reconstruction. Since SVGS is implemented based on 2DGS, it inherits these excellent geometric reconstruction capabili- ties, although this is not our primary objective. The results demonstrate that our geometric reconstruction capa- bilities are comparable to 2DGS when there is no limit on the number of Gaussians, while also ensuring the high- est PSNR, indicating the best image reconstruction quality. Furthermore, when a limited number of Gaussians is used, our powerful expressiveness becomes apparent, greatly out- performing 2DGS in both geometric reconstruction quality and image rendering quality.