Gaussian Shell Maps for Efficient 3D Human Generation

CVPR 2024
1Stanford University, 2HKUST
* Equal Contribution
Gaussian Shell Maps: Gaussian Shell Maps is an efficient framework for 3D human generation connecting 3D Gaussians with CNN-based generators. 3D Gaussians are anchored to “shells” derived from the SMPL template (only two shells are visualized for clarity), and the appearance is modeled in texture space. Trained only on 2D images, we show that our method can generate diverse articulable humans in real-time with state-of-the-art quality directly in high resolution without the need for upsampling and hence avoiding aliasing artifacts.
Method Overview : We propose an expressive yet highly efficient representation, Gaussian Shell Map (GSM), for 3D human generation. Combining the idea of 3D Gaussians and Shell Maps, we sample 3D Gaussians on “shells”, which are mesh layers offsetted from the SMPL template, forming a shell volume to model complex and diverse geometry and appearance; the Gaussian parameters are learned in the texture space, allowing us to leverage existing CNN-based generative architecture. Articulation is straightforward by interpolating the deformation of the shell. The generation is supervised by single-view 2D images using several discriminator critics, including part-specific face, hands, and feet discriminators.



Semantic Editing



Walking sequence

Jumping sequence

Swing sequence

Novel Views




      title={Gaussian Shell Maps for Efficient 3D Human Generation}, 
      author={Rameen Abdal and Wang Yifan and Zifan Shi and Yinghao Xu and Ryan Po and Zhengfei Kuang and Qifeng Chen and Dit-Yan Yeung and Gordon Wetzstein},