Page 10 - Read Online

Page 108 Wu et al. Intell Robot 2022;2(2):105­29 I http://dx.doi.org/10.20517/ir.2021.20

Figure 1. Three approaches for LiDAR point cloud representation: (a) multi-view-based methods; (b) volumetric-based methods; and (c)
point-based methods. The image in (a) is original originally from MV3D [12] . The images in (b,c) are original originally from RPVNet [13]

view-pooling layers. Volumetric-based methods discretize the whole 3D space into plenty of 3D voxels, where
each point in the original 3D space is assigned to the corresponding voxel following some specific regulations.
This representation can preserve rich 3D shape information. Nevertheless, the limitation of performance is
inevitable as a result of the spatial resolution and fine-grained 3D geometry loss during the voxelization. On
the contrary, point-based methods conduct deep learning methods directly on the point cloud in continuous
vector space without transforming the point cloud into other intermediate data representations. This approach
avoids the loss caused by transformation and data quantification and preserves the detailed information of the
point cloud. The visualization of the three representations is illustrated in Figure 1.

The point cloud carries point-level information (e.g., the x, y, and z coordinates in 3D space, color, and in-
tensities) and keeps invariant under rigid transformation, scaling, and permutation. An azimuth-like physical
quantity can be easily acquired from the point cloud, and thus diverse features can be generated for deep learn-
ing. Although the point cloud is less affected by the variation of illumination and scale when compared to the
image, the point cloud suffers more from the intensity and often ignores sparse information reflected by the
surface of objects. The laser emitted by LiDAR cannot bypass obstacles and will be greatly disturbed or even
unable to work in the rain, fog, sand, and other severe weather. Thus, challenges exist when extracting fea-
tures from the spatial-sparse and unordered point sets. Algorithms have evolved from hand-crafted features
extraction to deep-learning ones. Among them, point-wise and region-wise methods treat different paths that
lead to the same destination. Meanwhile, the cooperation with other sensors shows huge potential to improve
the performance through supplementing insufficient information, which may unexpectedly lead to extra com-
putational cost or information redundancy if not well designed. Therefore, studies focus on how to reach a
compromise on the cost and the performance when conducting LiDAR-fusion tasks.

With the development of LiDAR, increasing LiDAR point cloud datasets are available, facilitating the training
andevaluationamongdifferentalgorithms. Table1 [14–28] listsdatasetsrecordedbyLiDAR-basedvisualsystem.
Among them, KITTI [14] provides a comprehensive real-world dataset for autonomous driving, providing a
benchmark for 3D object detection, tracking, and scene flow estimation. The evaluation metrics vary for
different tasks. For 3D classification, the overall accuracy (OA) and the mean class accuracy (mAcc) are widely
used. For 3D object detection, the average precision (AP) and mean average precision (mAP) are mostly-used.
For 3D object tracking, precision and success are commonly used as evaluation metrics of single object tracker.
Average multi-object tracking Accuracy (AMOTA) and average multi-object tracking precision (AMOTP) are
used as evaluation metrics for a 3D multi-object tracker. For 3D segmentation, mean intersection over union
(mIoU), OA, and mAcc are widely used for the algorithm evaluation.