Gaussian Splattings demonstrate impressive results in multi-view reconstruction based on Gaussian explicit representations. However, the current Gaussian primitives only have a single view-dependent color and an opacity to represent the appearance and geometry of the scene, resulting in a non-compact representation. In this paper, we introduce a new method called SVGS that utilizes spatially varying colors and opacity in a single Gaussian primitive to improve its representation ability. We have implemented bilinear interpolation, movable kernels, and even tiny neural networks as spatially varying functions. Quantitative and qualitative experimental results demonstrate that all three functions outperform the baseline, with the best movable kernels achieving superior novel view synthesis performance on multiple datasets, highlighting the strong potential of spatially varying functions.
SVGS gives each Gaussian the ability to vary spatially. Compared with 2DGS and 3DGS, SVGS is more expressive and can better reconstruct details (as seen in the white area). Additionally, SVGS can express more color variations using only one Gaussian, instead of being limited to just one color (bottom right).
3DGS uses Gaussian ellipsoids to express scenes, and a learnable color is defined on each ellipsoid. 2DGS uses Gaussian surfels to express scenes, and a learnable color is defined on each Gaussian surfel. Our SVGS uses spatially varying Gaussian surfels to express scenes, and the color and opacity changes with the spatial position on each surfel.
@article{xu2024SVGS,
title={SVGS: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors},
author={Xu, Rui and Chen, Wenyue and Wang, Jiepeng and Liu, Yuan and Wang, Peng and Lin, Cheng and Xin, Shiqing and Li, Xin and Wang, Wenping and Komura, Taku},
year={2024},
eprint={2411.18966},
archivePrefix={arXiv},
primaryClass={cs.LG},
}