Dynamic Novel View Synthesis (Dynamic NVS) enhances NVS technologies to model moving 3-D scenes. However, current methods are resource intensive and challenging to compress. To address this, we present WavePlanes, a fast and more compact hex plane representation, applicable to both dynamic Neural Radiance Fields and Gaussian Splatting methods. Rather than modeling many feature scales separately (as done previously), we use the inverse discrete wavelet transform to reconstruct features at varying scales. This leads to a more compact representation and allows us to explore wavelet-based compression schemes for further gains. The proposed compression scheme exploits the sparsity of wavelet coefficients, by applying hard thresholding to the wavelet planes and storing nonzero coefficients and their locations on each plane in a Hash Map. Compared to the state-of-the-art (SotA), WavePlanes is significantly smaller, less resource demanding and competitive in reconstruction quality. Compared to small SotA models, WavePlanes outperforms methods in both model size and quality of novel views.
The ablation results, below, were carried out on "K-Planes-W" (our WavePlanes applied to K-Plane pipeline) on the D-NeRF dataset. The configurations were carried over to 4D-GS-W (WavePlanes applied to 4D-GS pipeline) without significant modification, aside from that we only model a single feature plane for 4D-GS-W while K-Planes-W still requires modelling two sets of feature planes.
@article{azzarelli2023waveplanes,
title={WavePlanes: Compact Feature Planes for Dynamic Novel View Synthesis},
author={Azzarelli, Adrian and Anantrasirichai, Nantheera and Bull, David R},
journal={arXiv preprint arXiv:2312.02218},
year={2023}
}