DECOR-GAN: 3D Shape Detailization by Conditional Refinement

PubDate: June 2021

Teams: Simon Fraser University 2 Adobe Research 3 IIT Bombay

Writers: Zhiqin Chen 1 Vladimir G. Kim 2 Matthew Fisher 2 Noam Aigerman 2 Hao Zhang 1 Siddhartha Chaudhuri

PDF: DECOR-GAN: 3D Shape Detailization by Conditional Refinement

Abstract

We introduce a deep generative network for 3D shape detailization, akin to stylization with the style being geometric details. We address the challenge of creating large varieties of high-resolution and detailed 3D geometry from a small set of exemplars by treating the problem as that of geometric
detail transfer. Given a low-resolution coarse voxel shape, our network refines it, via voxel upsampling, into a higherresolution shape enriched with geometric details. The output shape preserves the overall structure (or content) of the input, while its detail generation is conditioned on an input “style code” corresponding to a detailed exemplar. Our 3D detailization via conditional refinement is realized by a generative adversarial network, coined DECOR-GAN. The network utilizes a 3D CNN generator for upsampling coarse voxels and a 3D PatchGAN discriminator to enforce local patches of the generated model to be similar to those in the training detailed shapes. During testing, a style code is fed into the generator to condition the refinement. We demonstrate that our method can refine a coarse shape into a variety of detailed shapes with different styles. The generated results are evaluated in terms of content preservation, plausibility, and diversity. Comprehensive ablation studies are conducted to validate our network designs. Code is available at https://github.com/czq142857/DECOR-GAN.