Analysis of the PointNet neural network architecture

Objective. Most researchers convert point cloud data into ordinary three-dimensional voxel grids or image collections, which makes the data unnecessarily voluminous and causes problems when processing them. The purpose of the study is to analyze the architecture of the PointNet neural network. Method. A unified approach has been applied to solving various 3D recognition problems, ranging from object classification, detail segmentation to semantic scene analysis. Result. A comparative analysis of the classification of 2d and 3d objects was carried out, the layers and functions through which classification occurs were studied in detail. A type of neural network is considered that directly uses point clouds, which takes into account the invariance of permutations of points in the input data. The network is determined to provide a unified architecture for applications ranging from object classification, part segmentation, and scene semantics. For semantic segmentation, the input data can be either a single object from the part area segmentation or a small part of the 3D scene. A neural network that is widely used for raster image editing, graphic design, and digital art is a deep point cloud architecture called PointNet. Conclusion. A new deep point cloud architecture, PointNet, is introduced. For object classification task, the input point cloud is directly selected from the shape or pre-segmented from the scene point cloud. To obtain a virtual model of the real world, neural network solutions are used, based on the assumption that there is an RGB point cloud obtained by an RGB-D camera from one or several angles.

1. Mozhaev A. N. Segmentation of point clouds by means of the point cloud library. Extreme robotics. 2018; 1(1):301-308. – EDN YNCUTJ (In Russ)

2. Zhu X. X., Tuia D., Mou L., Xia G. S., Zhang L., Xu F., Fraundorfer F. Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources. IEEE Geoscience and Remote Sensing Magazine. 2017; 8–36. DOI: 10.1109/MGRS.2017.2762307

3. Ferrit S. E. Patterns and practice of deep learning / translated from English by A.V. Logunov. – M.: DMK Press, 2022; 538. ISBN 978-5-93700-113-9(In Russ)

4. Aliyev R.M., Morozova O.N. “Comparative analysis of the application of point cloud processing methods Paint Net and Paint Net++ for the task of segmentation of 3D objects”. Proceedings of the Institute of System Programming of the Russian Academy of Sciences. 2020; 29(1):37-54. (In Russ)

5. Melnik S.P., Ivanov I.V. Analysis of the PointNet method for the task of segmentation of three-dimensional objects”. Information technologies and computer engineering. 2018; 16( 5): 951-960. (In Russ)

6. Fedorov V.I., Popovkin V.V. “Convolutional neural networks in the problem of point cloud classification” Bulletin of the Russian Academy of Sciences. 2019; 89(5):428-436. (In Russ)

7. Kurmanbaev B.B. “Application of the PointNet method for classification of three-dimensional objects” Bulletin of KazNTU. 2019; 4(134): 111-117. (In Russ)

8. Khaustov A.M., Prygulsky M.V. “Analysis of the proposed PointNet architecture”. Information systems and technologies. 2019; 4(98): 49-57. (In Russ)

9. Gura D.A., Dyachenko R.A., Khusht N.I. On the issue of preprocessing three-dimensional laser scanning data. Electronic network polythematic journal “Scientific works of KubSTU”. 2021; 3:39-46. EDN: GKDDTK (In Russ)

10. Gura D.A., Dyachenko R.A., Khusht N.I., Markovsky I.G. On the methods of generation, classification and recognition of three-dimensional images. Electronic network polythematic journal “Scientific works of KubSTU”. 2021; 2: 86-97. EDN: KNEICL (In Russ)

11. Gura D.A., Dyachenko R.A., Andryushchenko A.V. Technology of flight route simulation of an unmanned aerial vehicle and aerial laser scanning. Electronic network polythematic journal “Scientific works of KubSTU”. 2022; 6: 126-133. EDN: QPPBDL (In Russ)

12. Cadres I. V. Types of activation functions for training convolutional neural networks / I. V. Cadres, L. B. Kizhner. Issues of sustainable development of society. 2022; 6: 1109-1115. – EDN TELYIO. (In Russ)

13. Kosolapov P.A., Dyachenko R.A., Gura D.A., Khusht N.I. On the normalization of three-dimensional laser scanning data. Electronic network polythematic journal “Scientific works of KubSTU”. 2021; 4: 56-66. EDN: ILVHIQ (In Russ)

14. Svitov D. V. Distillation of models for face recognition trained using the Softmax function with indents / D. V. Svitov, S. A. Alyamkin. Automation and telemechanics. 2022; 10:35-46. – DOI 10.31857/S000523102210004X. – EDN AJXJDG. (In Russ)

15. Krizhevsky I., Sutskever G. E. Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, May 2017. ISSN 0001-0782, 1557-7317. doi: 10.1145/3065386.

16. Malashin R. O. Chapter 12. Modern neural networks of deep learning for automatic image analysis / R. O. Malashin, S. V. Ponomarev. Neurotechnologies : a collective monograph. St. Petersburg : VVM, 2018; 271-335. – EDN DHAIFS. (In Russ)

17. Maenpaa T. The local binary pattern approach to texture analysis – Extensions and Applications. Oulu University Press, 2003.

18. Zaganidis A., Sun L., Duckett T., Cielniak G. Integrating Deep Semantic Segmentation into 3-D Point Cloud Registration. IEEE Robotics andAutomation Letters. 2018;3(4): 942–2949. DOI:10.1109/LRA.2018.2848308

19. Orlova S. R. Three-dimensional recognition: current state and trends / S. R. Orlova, A.V. Lopota. Automation and telemechanics. 2022; 4: 5-26. DOI 10.31857/S000523102204002X. – EDN AAALWB. (In Russ)

20. Berezovsky B. A., Baryshnikov Yu. M., Borzenko V. I., Kempner L. M. Multicriteria optimization: Mathematical aspects / B. A. Berezovsky, Yu. M. Baryshnikov, V. I. Borzenko, L. M. Kempner. M.: Nauka, 1989; 128.- ISBN 5-02-006543-9 (In Russ)