DANCE-NET: DENSITY-AWARE CONVOLUTION NETWORKS WITH CONTEXT ENCODING FOR AIRBORNE LIDAR POINT CLOUD CLASSIFICATION

Xiang Li1,3
Lingjing Wang1,2,3
Congcong Wen1,3
Mingyang Wang1,3
Yi Fang1,2,3

1NYU Multimedia and Visual Computing Lab
2New York University Abu Dhabi
3New York University Tandon School of Engineering


Airborne LiDAR point cloud classification has been a long-standing problem in photogrammetry and remote sensing. Early efforts either combine hand-crafted feature engineering with machine learning-based classification models or leverage the power of conventional convolutional neural networks (CNNs) on projected feature images. Recent proposed deep learning-based methods tend to develop new convolution operators which can be directly applied on raw point clouds for representative point feature learning. Although these methods have achieved satisfying performance for the classification of airborne LiDAR point clouds, they cannot adequately recognize fine-grained local structures due to the uneven density distribution of 3D point clouds. In this paper, to address this challenging issue, we introduce a density-aware convolution module which uses the point-wise density to reweight the learnable weights of convolution kernels. The proposed convolution module can approximate continuous convolution on unevenly distributed 3D point sets. Based on this convolution module, we further develop a multi-scale CNN model with downsampling and upsampling blocks to perform perpoint semantic labeling. In addition, to regularize the global semantic context, we implement a context encoding module to predict a global context encoding and formulated a context encoding regularizer to enforce the predicted context encoding to be aligned with the ground truth one. The overall network can be trained in an end-to-end fashion and directly produces the desired classification results in one network forward pass. Experiments on the ISPRS 3D Labeling Dataset and 2019 Data Fusion Contest Dataset demonstrate the effectiveness and superiority of the proposed method for airborne LiDAR point cloud classification.



News



Paper

Xiang Li, Lingjing Wang, Congcong Wen, Mingyang Wang, Yi Fang

DANCE-NET: Density-aware convolution networks with context encoding for airborne LiDAR point cloud classification



[Paper]
[Bibtex]


Model



Results



Classification performance of our model and the top three models on the ISPRS benchmark. We report the per-category F1 score in the first 9 columns as well as the overall accuracy (OA) and average F1 score (Average F1) in the last two columns. The boldface text indicates the best performance.

Qualitative Results on 3D shape registration

Performance of our model and all comparing methods on the 2019 Data Fusion Contest Dataset. We report the F1 score for each category in the first 5 columns as well as the overall accuracy (OA) and average F1 score (Average F1) in the last two columns. The boldface text indicates the best performance.


This webpage template was borrowed from MEPS.