vdgtuВестник Дагестанского государственного технического университета. Технические наукиHerald of Dagestan State Technical University. Technical Sciences2073-61852542-095XDaghestan State Technical University10.21822/2073-6185-2023-50-3-142-149vdgtu-1351Research ArticleИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИ И ТЕЛЕКОММУНИКАЦИИINFORMATION TECHNOLOGY AND TELECOMMUNICATIONSМоделирование процесса обучения нейросети DeepLabv3 для сегментации сельскохозяйственных полейSimulation of the of the DeepLabv3 neural network learning process for the agricultural fields segmentationhttps://orcid.org/0000-0002-3077-6622РогачевА. Ф.RogachevA. F.

Алексей Фруминович Рогачев, профессор кафедры математического моделирования и информатики, 400002, г. Волгоград, пр. Университетский, 26;

400005, г. Волгоград, пр. им. Ленина, 28

Alexey F. Rogachev, Professor of the Department of Mathematical Modeling and Computer Science, 

26 Universitetsky Ave., Volgograd 400002

rafr@mail.ruБелоусовИ. С.BelousovI. S.

Илья Станиславович Белоусов, аспирант кафедры математического моделирования и информатики,

400002, г. Волгоград, пр. Университетский, 26

Ilya S. Belousov, Graduate Student of the Department of Mathematical Modeling and Computer Science,

28 Lenin Ave., Volgograd 400005

Волгоградский государственный аграрный университет; Волгоградский государственный технический университетРоссияVolgograd State Agricultural UniversityRussian FederationВолгоградский государственный аграрный университетРоссияVolgograd State Technical UniversityRussian Federation202329102023503142149Copyright © Рогачев А.Ф., Белоусов И.С., 20232023Рогачев А.Ф., Белоусов И.С.Rogachev A.F., Belousov I.S.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.dgtu.ru/jour/article/view/1351Цель

Цель. Проведение мониторинга и определение состояния посевов в сельскохозяйственном производстве требует использование и совершенствование нейросетевых методов искусственного интеллекта. Целью исследования является создание математической модели процесса обучения нейросети DeepLabv3 для интеллектуального анализа и сегментации участков сельскохозяйственных полей.

Метод

Метод. На основе сформированной базы RGB изображений сельскохозяйственных полей, размеченной на четыре класса, была разработана нейросеть архитектуры DeepLabv3 и проведено ее обучение. Получены аппроксимации кривой обучения модифицированной функцией Джонсона методами наименьших квадратов и наименьших модулей.

Результат

Результат. Проведена статистическая оценка качества обучения и аппроксимации нейросетей архитектуре DeepLabV3 в сочетании с ResNet50. Построенное семейство DNN на основе DeepLabV3 с ResNet50 показали эффективность распознавания и достаточное быстродействие при определении состояния посевов.

Вывод

Вывод. Аппроксимация диаграммы обучения нейросетей архитектуре DeepLabV3, с использованием модифицированной функции Джонсона, позволяет оценивать значение «насыщения» моделируемой зависимости и прогнозировать максимальное значение метрики нейросети без учета возможного ее переобучения.

Objective

Objective. Monitoring and determining the state of crops in agricultural production requires the use and improvement of neural network methods of artificial intelligence.

The aim of the study is to create a mathematical model of the learning process of the DeepLabV3 neural network for intelligent analysis and segmentation of agricultural fields.

Method

Method. Based on the newly formed RGB database of images of agricultural fields, marked up into four classes, a neural network of the DeepLabV3 architecture was developed and trained. Approximations of the learning curve by the modified Johnson function are obtained by the methods of least squares and least modules.

Result

Result. A statistical assessment of the quality of training and approximation of neural networks to the DeepLabV3 architecture in combination with ResNet 50 was carried out. The constructed DNN family based on DeepLabV3 with ResNet50 showed the efficiency of recognition and sufficient speed in determining the state of crops.

Conclusions

Conclusions. Approximation of the neural network learning diagram to the DeepLabV3 architecture, using a modified Johnson function, allows us to estimate the value of the “saturation” of the simulated dependence and predict the maximum value of the neural network metric without taking into account its possible retraining.

посевы сельскохозяйственных культурзадача сегментацииискусственные нейросетиматематическое моделированиеcropssegmentation problemartificial neural networksmathematical modelingReferencesSaiz-Rubio V. From Smart Farming towards Agriculture 5.0: A Review on Crop Data Management. https://www.mdpi.com/2073-4395/10/2/207/htm.Saiz-Rubio V. From Smart Farming towards Agriculture 5.0: A Review on Crop Data Management. https://www.mdpi.com/2073-4395/10/2/207/htm.A Review on Deep Learning Techniques Applied to Semantic Segmentation / Garcia-Garcia Alberto, Orts-Escolano Sergio, Oprea Sergiu, Villena-Martinez Victor, Garcia-Rodriguez Jose. https://doi.org/10.48550/arXiv.1704.06857.A Review on Deep Learning Techniques Applied to Semantic Segmentation / Garcia-Garcia Alberto, OrtsEscolano Sergio, Oprea Sergiu, Villena-Martinez Victor, Garcia-Rodriguez Jose. https://doi.org/10.48550/arXiv.1704.06857.Badrinarayanan V., Kendall A., Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. http://docs.cntd.ru/document/902361843.Badrinarayanan V., Kendall A., Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. http://docs.cntd.ru/document/902361843.Bin Xu et al., “Remote sensing monitoring on dynamic status of grassland productivity and animal loading balance in Northern China,” IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium, Anchorage, AK, USA, 2004, pp. 2306-2309 vol.4, doi: 10.1109/IGARSS.2004.1369747.Bin Xu et al., “Remote sensing monitoring on dynamic status of grassland productivity and animal loading balance in Northern China,” IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium, Anchorage, AK, USA, 2004, pp. 2306-2309 vol.4, doi: 10.1109/IGARSS.2004.1369747.Pelletier, C.; Valero, S.; Inglada, J.; Champion, N.; Dedieu, G. Assessing the robustness of Random Forests to map land cover with high resolution satellite image time series over large areas. Remote Sens. Environ. 2016, 187, 156–168.Pelletier, C.; Valero, S.; Inglada, J.; Champion, N.; Dedieu, G. Assessing the robustness of Random Forests to map land cover with high resolution satellite image time series over large areas. Remote Sens. Environ. 2016;187:156–168.L. Zhang, W. Yu, G. Li and H. Zhang, “An approach for flood inundated duration extraction based on Level Set Method using remote sensing data,” 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 2016, pp. 1820-1822, doi: 10.1109/IGARSS.2016.7729467.L. Zhang, W. Yu, G. Li and H. Zhang, “An approach for flood inundated duration extraction based on Level Set Method using remote sensing data,” 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 2016;1820-1822, doi: 10.1109/IGARSS.2016.7729467.Zhao-Cheng Zeng, Yuan Wang, Thomas J. Pongetti, Fang-Ying Gong, Sally Newman, Yun Li, Vijay Natraj, RunLie Shia, Yuk L. Yung, Stanley P. Sander, Tracking the atmospheric pulse of a North American megacity from a mountaintop remote sensing observatory,Remote Sensing of Environment,Volume 248,2020,112000.Zhao-Cheng Zeng, Yuan Wang, Thomas J. Pongetti, Fang-Ying Gong, Sally Newman, Yun Li, Vijay Natraj, Run-Lie Shia, Yuk L. Yung, Stanley P. Sander, Tracking the atmospheric pulse of a North American megacity from a mountaintop remote sensing observatory, Remote Sensing of Environment, 2020; 248:112000.Рогачев А. Ф., Мелихова Е. В., Плещенко Т. В. Нейросетевая система управления программируемым аграрным производством с использованием ретроспективных данных и результатов дистанционного зондирования: монография. Волгоград: ФГБОУ ВО Волгоградский ГАУ, 2021. 172 с.Rogachev A.F., Melikhova E.V., Pleschenko T.V. Neural network control system for programmable agricultural production using retrospective data and remote sensing results: monograph. Volgograd: Federal State Budgetary Educational Institution of Higher Education Volgograd State Agrarian University, 2021; 172 .(In Russ)Рогачёв А. Ф. Математическое моделирование экономической динамики в аграрном производстве: монография. Волгоград, 2014. 172 с.Rogachev A.F. Mathematical modeling of economic dynamics in agricultural production: monograph. Volgograd, 2014; 172. (In Russ)Соловьев Р. А., Тельпухов Д. В., Кустов А. Г. Автоматическая сегментация спутниковых снимков на базе модифицированной свёрточной нейронной сети UNET // Инженерный вестник Дона. 2017. № 4.Solovyov R. A., Telpukhov D. V., Kustov A. G. Automatic segmentation of satellite images based on a modified convolutional neural network UNET. Engineering Bulletin of the Don. 2017; 4.(In Russ)Jeong S, Ko J, Yeom J M. Predicting rice yield at pixel scale through synthetic use of crop and deep learning models with satellite data in South and North Korea[J]. Science of The Total Environment, 2021, 802(2):149726.Jeong S, Ko J, Yeom J M. Predicting rice yield at pixel scale through synthetic use of crop and deep learning models with satellite data in South and North Korea[J]. Science of The Total Environment, 2021, 802(2):149726.Xu J, Yang J, Xiong X, et al. Towards interpreting multi-temporal deep learning models in crop mapping[J]. Remote Sensing of Environment, 2021, 264:112599. https://doi.org/10.3390/rs13224668.Xu J, Yang J, Xiong X, et al. Towards interpreting multi-temporal deep learning models in crop mapping[J]. Remote Sensing of Environment, 2021, 264:112599. https://doi.org/10.3390/rs13224668.Ofori-Ampofo, S.; Pelletier,C.; Lang, S. Crop Type Mapping from Optical and Radar Time Series Using Attention Based Deep Learning.Remote Sens. 2021, 13, 4668.Ofori-Ampofo, S.; Pelletier,C.; Lang, S. Crop Type Mapping from Optical and Radar Time Series Using Attention Based Deep Learning.Remote Sens. 2021;13:4668.Kussul N, Lavreniuk M, Skakun S, et al. Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[J]. IEEE Geoscience and Remote Sensing Letters, 2017, PP(99):1-5.Kussul N, Lavreniuk M, Skakun S, et al. Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data[J]. IEEE Geoscience and Remote Sensing Letters, 2017;(99):1-5.Context-sensitive image analysis for coloring nature images / A. V. Alekseev, V. L. Rozaliev, Y. A. Orlova, A. V. Zaboleeva-Zotova // Advances in Intelligent Systems and Computing. 2016. V. 451. P. 133-141.Context-sensitive image analysis for coloring nature images. A. V. Alekseev, V. L. Rozaliev, Y. A. Orlova, A. V. Zaboleeva-Zotova // Advances in Intelligent Systems and Computing. 2016; 451:133-141.Deep residual learning for image recognition.K.He, X. Zhang, X. Ren, J Sun. Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition .CVPR. Las egas, Nevada, USA, 2016; 770-778.Deep residual learning for image recognition. K. He, X. Zhang, X. Ren, J Sun. Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition.CVPR. Las Vegas, Nevada, USA, 2016; 770-778.Fezan Review DeepLabv3 (Semantic Segmentation). 2020. https://medium.com/swlh/review-deeplabv3-semanticsegmentation-52c00ddbf28d.Fezan Review DeepLabv3 (Semantic Segmentation). 2020. https://medium.com/swlh/review-deeplabv3- semantic-segmentation-52c00ddbf28d. 18. Rosebrock Adrian. Intersection over Union (IoU) for object detection. https://www.pyimagesearch.com/2016/11/07/intersection-over-unioniou-for-object-detection.Rosebrock Adrian. Intersection over Union (IoU) for object detection. https://www.pyimagesearch.com/2016/11/07/intersection-over-unioniou-for-object-detection.Seeland M, Mäder P (2021) Multi-view classification with convolutional neural networks. PLoS ONE 16(1): e0245230. https://doi.org/10.1371/journal.pone.0245230Seeland M, Mäder P (2021) Multi-view classification with convolutional neural networks. PLoS ONE 16(1): e0245230. https://doi.org/10.1371/journal.pone.0245230Sik-Ho Tsang Review: DeepLabv3. Atrous Convolution (Semantic Segmentation). https://towardsdatascience.com/review-deeplabv3-atrous-convolution-semantic-segmentation-6d818bfd1d74.Sik-Ho Tsang Review: DeepLabv3. Atrous Convolution (Semantic Segmentation). https://towardsdatascience.com/review-deeplabv3-atrous-convolution-semantic-segmentation-6d818bfd1d74.Rogachev A. F., Belousov I.S. Neural network identification of problem areas of the state of crops by methods of artificial intelligence. Proc. of the Lower Volga Agro-University Comp. 2022; 3(67). 459-466. DOI: 10.32786/2071-9485-2022-03-52.Rogachev A. F., Belousov I.S. Neural network identification of problem areas of the state of crops by methods of artificial intelligence. Proc. of the Lower Volga Agro-University Comp. 2022. 3(67). 459-466. DOI: 10.32786/2071-9485-2022-03-52.Sozykin A.V. Review of methods for training deep neural networks. Bulletin of SUSU. Series: Computational mathematics and computer science. 2017; 6(3): 28–59. DOI: 10.14529/cmse170303.Созыкин А.В. Обзор методов обучения глубоких нейронных сетей // Вестник ЮУрГУ. Серия: Вычислительная математика и информатика. (2017. Т. 6, № 3. С. 28–59. DOI: 10.14529/cmse170303.Ioffe S., Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. JMLR Workshop and Conference Proceedings. Proceedings of the 32nd International Conference on Machine Learning (Lille, France, July 06–11, 2015), 2015; 37: 448–456.Ioffe S., Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. JMLR Workshop and Conference Proceedings. Proceedings of the 32nd International Conference on Machine Learning (Lille, France, July 06–11, 2015), 2015. vol. 37. pp. 448–456.He K., Zhang X., Ren S., et al. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (Las Vegas, NV, USA, 27–30 June 2016), 2016; 770–778. DOI: 10.1109/CVPR.2016.90.He K., Zhang X., Ren S., et al. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (Las Vegas, NV, USA, 27–30 June 2016), 2016. pp. 770–778. DOI: 10.1109/CVPR.2016.90.Guseinov R.V., Guseinova M.R., Alieva K.A. Some issues of multicriteria optimization of parameters of complex systems. Herald of the Dagestan State Technical University. Technical Science. 2023; 50(2): 67-75. DOI:10.21822/2073-6185-2023-50-2-67-75. (In Russ)Гусейнов Р.В., Гусейнова М.Р., Алиева К.А. Некоторые вопросы многокритериальной оптимизации параметров сложных систем. Вестник Дагестанского государственного технического университета. Технические науки. 2023; 50(2): 67-75. DOI:10.21822/2073-6185-2023-50-2-67-75.Grishin A.F., Yagunov V.N., Kotov-Darty S.F. Statistical models in economics. Rostov n/a. : Phoenix, 2005;344.(In Russ)Гришин А. Ф., Ягунов В. Н., Котов-Дарти С. Ф. Статистические модели в экономике. Ростов н/Д. : Феникс, 2005. 344 с.Kurbanov S. A., Melikhova E. V., Borodychev V. V., Rogachev A. F. Mathematical and statistical modeling of root crop yields under combined irrigation. Problems of development of the regional agro-industrial complex. 2018; 4(36):70-76. – DOI 10.15217/issn2079-0996.2018.4.70 .(In Russ)Курбанов С. А., Мелихова Е. В., Бородычев В. В., Рогачев А. Ф. Математико-статистическое моделирование урожайности корнеплодов при комбинированном орошении // Проблемы развития АПК региона. – 2018. – № 4(36). – С. 70-76. – DOI 10.15217/issn2079-0996.2018.4.70.Rogachev A.F. Parameterization of econometric dependencies by the method of least modules. Management of Economic Systems: electronic Scientific Journal. 2011; 3: 0421100034.Рогачев А.Ф. Параметризация эконометрических зависимостей методом наименьших модулей // Управление экономическими системами: электронный научный журнал. 2011. № 3. С. 0421100034.Gabriel Danilo Shimizu, Leandro Simões Azeredo Gonçalves. AgroReg: main regression models in agricultural sciences implemented as an R. Sci. Agric. v.80, e20220041, 2023. URL: http://doi.org/10.1590/1678-992X-2022-0041.Gabriel Danilo Shimizu, Leandro Simões Azeredo Gonçalves. AgroReg: main regression models in agricultural sciences implemented as an R. Sci. Agric. v.80, e20220041, 2023. URL: http://doi.org/10.1590/1678-992X-2022-0041.Melikhova, E., Rogachev, A. Computer Optimization of ANN Hyperparameters for Retrospective Information Processing. In: Guda, A. (eds) Networked Control Systems for Connected and Automated Vehicles. Lecture Notes in Networks and Systems, Springer, Cham. 2023;509:723-730. https://doi.org/10.1007/978-3-031-11058-0_72.Melikhova, E., Rogachev, A. Computer Optimization of ANN Hyperparameters for Retrospective Information Processing. In: Guda, A. (eds) Networked Control Systems for Connected and Automated Vehicles. Lecture Notes in Networks and Systems, (2023) vol 509. Springer, Cham. Pp. 723-730. https://doi.org/10.1007/978-3-031-11058-0_72.Melikhova, E., Rogachev, A. Computer Optimization of ANN Hyperparameters for Retrospective Information Processing. In: Guda, A. (eds) Networked Control Systems for Connected and Automated Vehicles. Lecture Notes in Networks and Systems, (2023) vol 509. Springer, Cham. Pp. 723-730. https://doi.org/10.1007/978-3-031-11058-0_72.

The authors declare that there are no conflicts of interest present.