3D Meta Point Signature: Learning to Learn 3D Point Signature for 3D Dense Shape Correspondence

Hao Huang1
Lingjing Wang1,2,3
Xiang Li1,3
Yi Fang1,2,3

1NYU Multimedia and Visual Computing Lab
2New York University Abu Dhabi
3New York University Tandon School of Engineering


Point signature, a representation describing the structural neighborhood of a point in 3D shapes, can be applied to establish correspondences between points in 3D shapes. Conventional methods apply a weight-sharing network, e.g., graph neural network, across all neighborhoods to generate point signatures directly and gain the generalization ability by extensive training over a large number of training samples from scratch. However, these methods lack the flexibility in rapidly adapting to unseen neighborhood structures and thus generalizes poorly on new point sets. In this paper, we propose a novel meta-learning based 3D point signature model, named 3D meta point signature (MEPS) network, that is capable of learning robust point signatures in 3D shapes. By regarding each point signature learning process as a task, our method obtains an optimized model over the best performance on the distribution of all tasks, generating reliable signatures for new tasks, i.e., signatures of unseen point neighborhoods. Specifically, the MEPS consists of two modules: a base signature learner and a meta signature learner. During training, the base-learner is trained to perform specific signature learning tasks. In the meantime, the meta-learner is trained to update the base-learner with optimal parameters. During testing, the meta-learner that is learned with the distribution of all tasks can adaptively change parameters of the base-learner, accommodating to unseen local neighborhoods. We evaluate the MEPS model on a dataset for 3D shape correspondence. Experimental results demonstrate that our method gains significant improvements over the baseline model and achieves state-of-the-art results.



News



Paper

Hao Huang, Lingjing Wang, Xiang Li, Yi Fang

3D Meta Point Signature: Learning to Learn 3D Point Signature
for 3D Dense Shape Correspondence

Preprint, 2020

[Paper]
[Bibtex]


MEPS Model



Results



Quantitative results on 3D shape registration

Qualitative Results on 3D shape registration

Clustering of feature embeddings


This webpage template was borrowed from NTG.