News of the Kabardino-Balkarian Scientific Center of the Russian Academy of SciencesNews of the Kabardino-Balkarian Scientific Center of the Russian Academy of SciencesИзвестия Кабардино-Балкарского научного центра РАН1991-66392949-194026072710.35330/1991-6639-2024-26-3-11-20EOHSCHSystem analysis, management and information processingСистемный анализ, управление и обработка информацииResearch ArticlePrediction the yield of green crops based on monitoring morphometric parameters using machine vision and neural networksПрогнозирование урожайности зеленых культур на основе мониторинга морфометрических параметров посредством машинного зрения и нейронных сетейhttps://orcid.org/0000-0002-9121-894X3195-0770AstapovaMarina A.АстаповаМарина Алексеевна
Russian Federation

мл. науч. сотр. лаборатории технологий больших данных социокиберфизических систем

Junior Researcher of the Laboratory of Big Data Technologies in Socio-Cyberphysical Systems

astapova.m@iias.spb.suhttps://orcid.org/0000-0002-7032-02917398-2273UzdiaevMikhail Yu.УздяевМихаил Юрьевич
Russian Federation

Junior Researcher of the Laboratory of Big Data Technologies in Socio-Cyberphysical Systems

мл. науч. сотр. лаборатории технологий больших данных социокиберфизических систем

uzdyaev.m@iias.spb.suhttps://orcid.org/0000-0001-5822-41442148-2591КондратьевВиталий МихайловичKondratyevVitaly M.
Russian Federation

Candidate of Agricultural Sciences, Assistant Professor of the Department of Technology of Storage and Processing of Agricultural Products

канд. с.-х. наук, доцент кафедры технологий хранения и переработки сельскохозяйственной продукции

vitsevsk@mail.ruСанкт-Петербургский Федеральный исследовательский центр Российской академии наукSt. Petersburg Federal Research Center of the Russian Academy of SciencesSt. Petersburg State Agrarian UniversityСанкт-Петербургский государственный аграрный университетru15062024202426311–15111202807202428072024Copyright ©; 2024, Астапова М.А., Уздяев М.Ю., Кондратьев В.М.Copyright ©; 2024, Astapova M.A., Uzdiaev M.Y., Kondratyev V.M.2024Астапова М.А., Уздяев М.Ю., Кондратьев В.М.Astapova M.A., Uzdiaev M.Y., Kondratyev V.M.https://creativecommons.org/licenses/by/4.0https://journals.rcsi.science/1991-6639/article/view/260727

Artificial intelligence (AI) and computer vision tools play an important role in automatically determining plant growth stages. The study aims to analyze modern technologies for automatic analysis and measurement of plant characteristics such as height, leaf area and other morphometric parameters. This article discusses the use of computer vision and neural networks for monitoring morphometric parameters and predicting the yield of green crops. An algorithm has been developed for determining the growth stage, which collects data about plants using a multispectral camera and then analyzes the obtained information using neural networks. Training for growth stage classification was performed on a subsample of the original dataset, consisting of 273 randomly selected images maintaining class balance (91 images in each class). The training sample size for each class is 45 images, and the test sample size is 46 images for each class. Classification of growth stage showed high results: more than 95% of correctly recognized specimens; more than 93% correct recognition of individual growth stages. In terms of individual metrics (Precision, Recall, F1-score), the ResNet34 architecture performed best.

Средства искусственного интеллекта и технического зрения играют важную роль в автоматическом определении стадий роста растений. Исследование направлено на изучение современных технологий для автоматического анализа и измерения характеристик растений, таких как высота, площадь листьев и другие морфометрические параметры. В данной статье рассматривается применение машинного зрения и нейронных сетей для мониторинга морфометрических параметров и прогнозирования урожайности зеленых культур. Разработан алгоритм определения стадий роста салата, который осуществляет сбор данных о растениях с помощью мультиспектральной камеры, а затем анализирует полученную информацию с использованием нейронных сетей. Обучение классификации стадий роста выполнялось на подвыборке исходного датасета, состоящей из 273 случайно отобранных изображений с соблюдением баланса классов (91 изображение в каждом классе). Размер обучающей выборки для каждого класса – 45 изображений и размер тестовой выборки – 46 изображений для каждого класса. Классификация стадий роста показала высокие результаты: более 95 % верно распознанных экземпляров; более 93 % верных распознаваний отдельных стадий роста. По отдельным метрикам (Precision, Recall, F1-score) лучше всего себя показала архитектура ResNet34.

technical visionneural networksyield predictionproduction automationтехническое зрениенейронные сетипрогнозирование урожайностиавтоматизация производстваЩербина Т. А. Цифровая трансформация сельского хозяйства РФ: опыт и перспективы // Россия: тенденции и перспективы развития. 2019. № 14-1. С. 450–453. EDN: UGBYZTShcherbina T.A. Digital transformation of agriculture in the Russian Federation: experience and prospects. Rossiya: tendentsii i perspektivy razvitiya [Russia: trends and development prospects]. 2019. No. 14-1. Pp. 450–453. EDN: UGBYZT. (In Russian)Zhao C., Zhang Y., Du J. et al. Crop phenomics: current status and perspectives. Frontiers in Plant Science. 2019. Vol. 10. P. 714. DOI: 10.3389/fpls.2019.00714Shukla R., Dubey G., Malik P. et al. Detecting crop health using machine learning techniques in smart agriculture system. Journal of Scientific & Industrial Research. 2021. Vol. 80. No. 08. Pp. 699–706.Varshney D., Babukhanwala B., Khan J. et al. Plant disease detection using machine learning techniques. 2022 3rd International Conference for Emerging Technology (INCET). IEEE, 2022. Pp. 1–5.Jiang Y., Li C. Convolutional neural networks for image-based high-throughput plant phenotyping: a review. Plant Phenomics. 2020. DOI: 10.34133/2020/4152816Adhikari N.D., Simko I., Mou B. Phenomic and physiological analysis of salinity effects on lettuce. Sensors. 2019. Vol. 19. No. 21. P. 4814. DOI: 10.3390/s19214814Simko I., Hayes R.J., Furbank R.T. Non-destructive phenotyping of lettuce plants in early stages of development with optical sensors. Frontiers in plant science. 2016. Vol. 7. P. 1985. DOI: 10.3389/fpls.2016.01985Grahn C.M., Benedict C., Thornton T., Miles C. Production of baby-leaf salad greens in the spring and fall seasons of northwest Washington. HortScience. 2015. Vol. 50. No. 10. Pp. 1467–1471. DOI: 10.21273/HORTSCI.50.10.1467Wen Y., Zha L., Liu W. Dynamic responses of ascorbate pool and metabolism in lettuce to light intensity at night time under continuous light provided by red and blue LEDs. Plants. 2021. Vol. 10. No. 2. P. 214. DOI: 10.3390/plants10020214Barbedo J.G.A. Detection of nutrition deficiencies in plants using proximal images and machine learning: A review. Computers and Electronics in Agriculture. 2019. Vol. 162. Pp. 482–492. DOI: 10.1016/j.compag.2019.04.035Jung D.H., Park S.H., Han X.Z., Kim H.J. Image processing methods for measurement of lettuce fresh weight. Journal of Biosystems Engineering. 2015. Vol. 40. No. 1. Pp. 89–93. DOI: 10.5307/JBE.2015.40.1.089Lin Z., Fu R., Ren G. et al. Automatic monitoring of lettuce fresh weight by multi-modal fusion based deep learning. Frontiers in Plant Science. 2022. Vol. 13. P. 980581. DOI: 10.3389/ fpls.2022.980581Wang M., Guo X. Extracting the height of lettuce by using neural networks of image recognition in deep learning. ESS Open Archive. 2022. DOI: 10.1002/essoar.10510405.1Gang M.S., Kim H.J., Kim D.W. Estimation of greenhouse lettuce growth indices based on a two-stage CNN using RGB-D images. Sensors. 2022. Vol. 22. No. 15. P. 5499. DOI: 10.3390/s22155499Lu J.Y., Chang C.L., Kuo Y.F. Monitoring growth rate of lettuce using deep convolutional neural networks. 2019 ASABE Annual International Meeting. American Society of Agricultural and Biological Engineers. 2019. P. 1. DOI:10.13031/aim.201900341Mokhtar A., El-Ssawy W., He H. et al. Using machine learning models to predict hydroponically grown lettuce yield. Frontiers in Plant Science. 2022. Vol. 13. P. 706042. DOI: 10.3389/fpls.2022.706042Martinez-Nolasco C., Padilla-Medina J.A., Nolasco J.J.M. et al. Non-Invasive monitoring of the thermal and morphometric characteristics of lettuce grown in an aeroponic system through multispectral image system. Applied Sciences. 2022. Vol. 12. No. 13. P. 6540. DOI: 10.3390/ app12136540Zhang Y., Wu M., Li J. et al. Automatic non-destructive multiple lettuce traits prediction based on DeepLabV3+. Journal of Food Measurement and Characterization. 2023. Vol. 17. No. 1. Pp. 636–652. DOI: 10.1007/s11694-022-01660-3Wada K. Labelme: Image polygonal annotation with Python. 2016. URL: https:// github.com/labelmeai/labelmeSimonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014. DOI: 10.48550/arXiv.1409.1556He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. Pp. 770–778.Tan M., Le Q. Efficient net: Rethinking model scaling for convolutional neural networks. International conference on machine learning, PMLR, 2019. Pp. 6105–6114.Kumar V., Arora H., Sisodia J. Resnet-based approach for detection and classification of plant leaf diseases. 2020 International conference on electronics and sustainable communication systems (ICESC), IEEE, 2020. Pp. 495–502. DOI:10.1109/ICESC48915.2020.9155585Elwirehardja G.N., Prayoga J.S. Oil palm fresh fruit bunch ripeness classification on mobile devices using deep learning approaches. Computers and Electronics in Agriculture. 2021. Vol. 188. P. 106359. DOI:10.1016/j.compag.2021.106359Wang J., Zhang H., Zhou W. Apple automatic classification method based on improved VGG11. Third International Conference on Computer Vision and Pattern Analysis (ICCPA 2023). SPIE, 2023. Vol. 12754. Pp. 473–478.