Detection of peach trees in unmanned aerial vehicle (UAV) images based on improved Faster-RCNN network

doi:10.3969/j.issn.1004-1524.20230912

Acta Agriculturae Zhejiangensis ›› 2024, Vol. 36 ›› Issue (8): 1909-1919.DOI: 10.3969/j.issn.1004-1524.20230912

• Biosystems Engineering • Previous Articles Next Articles

Detection of peach trees in unmanned aerial vehicle (UAV) images based on improved Faster-RCNN network

CHENG Jiayu¹^,²(), CHEN Miaojin³, LI Tong¹^,², SUN Qinan³, ZHANG Xiaobin², ZHAO Yiying², ZHU Yihang², GU Qing²^,^*()

1. College of Mathematics and Computer Science, Zhejiang A&F University, Hangzhou 311300, China
2. Institute of Digital Agriculture, Zhejiang Academy of Agricultural Sciences, Hangzhou 310021, China
3. Ningbo Fenghua Peach Research Institute, Ningbo 315502, Zhejiang, China

Received:2023-07-26 Online:2024-08-25 Published:2024-09-06
Contact: GU Qing

Abstract

Abstract:

Studying precise detection and positioning methods for peach trees can provide support for precision management of peach orchards. This study utilizes unmanned aerial vehicle (UAV) remote sensing combined with deep learning algorithms to detect leafless peach trees and differentiate between budding and flowering peach trees. Three improvements are proposed based on the Faster R-CNN original network: replacing the backbone network with a ResNeXt-50 integrated with a convolutional block attention module (CBAM), using ROI Align instead of ROI Pooling for feature extraction from regions of interest, and introducing the Focal Loss function for imbalanced cross-entropy loss. Ablation experiments are conducted to analyze the effectiveness of these improvements. The experiments demonstrate that compared with the unimproved Faster R-CNN, the improved model achieves a mean average precision (mAP) increase of 8.95 percentage points, reaching 86.46%, enabling better differentiation between flowering and budding peach trees. The most significant improvement contributing to the model’s enhancement is the replacement of the ResNeXt-50 backbone network with CBAM, resulting in a 5.98 percentage points increase in mAP; the use of ROI Align reduces errors in the feature quantization process, leading to a 2.33 percentage points increase in mAP. Compared with other mainstream detection models such as YOLOv3, YOLOv5x, and SSD, the proposed model in this study demonstrates superior detection performance. The UAV remote sensing combined with the improved Faster R-CNN algorithm proposed in this study can effectively detect peach trees, meeting the requirements of precision management in peach orchards.

Key words: peach tree, remote sensing, deep learning, unmanned aerial vehicle

CLC Number:

S126

CHENG Jiayu, CHEN Miaojin, LI Tong, SUN Qinan, ZHANG Xiaobin, ZHAO Yiying, ZHU Yihang, GU Qing. Detection of peach trees in unmanned aerial vehicle (UAV) images based on improved Faster-RCNN network[J]. Acta Agriculturae Zhejiangensis, 2024, 36(8): 1909-1919.

Figures/Tables 13

References 36

[1]	王宇霖, 宗学普, 魏闻东. 全国桃区划研究[J]. 果树科学, 1984, 1(2): 13-24.
	WANG Y L, ZONG X P, WEI W D. Study on the regionalization of peach growing in China[J]. Journal of Fruit Science, 1984, 1(2): 13-24. (in Chinese with English abstract)
[2]	俞明亮, 王力荣, 王志强, 等. 新中国果树科学研究70年: 桃[J]. 果树学报, 2019, 36(10): 1283-1291.
	YU M L, WANG L R, WANG Z Q, et al. Fruit scientific research in New China in the past 70 years: peach[J]. Journal of Fruit Science, 2019, 36(10): 1283-1291. (in Chinese with English abstract)
[3]	孙忠富, 杜克明, 郑飞翔, 等. 大数据在智慧农业中研究与应用展望[J]. 中国农业科技导报, 2013, 15(6): 63-71.
	SUN Z F, DU K M, ZHENG F X, et al. Perspectives of research and application of big data on smart agriculture[J]. Journal of Agricultural Science and Technology, 2013, 15(6): 63-71. (in Chinese with English abstract)
[4]	刘海启. 以精准农业驱动农业现代化加速现代农业数字化转型[J]. 中国农业资源与区划, 2019, 40(1): 1-6, 73.
	LIU H Q. Accelerating the digital transformation of modern agriculture by driving the agricultural modernization with precision agriculture[J]. Chinese Journal of Agricultural Resources and Regional Planning, 2019, 40(1): 1-6, 73. (in Chinese with English abstract)
[5]	施连敏, 陈志峰, 盖之华. 物联网在智慧农业中的应用[J]. 农机化研究, 2013, 35(6): 250-252.
	SHI L M, CHEN Z F, GAI Z H. Application research of the Internet of Things in wisdom agriculture[J]. Journal of Agricultural Mechanization Research, 2013, 35(6): 250-252. (in Chinese with English abstract)
[6]	杨普, 赵远洋, 李一鸣, 等. 基于多源信息融合的农业空地一体化研究综述[J]. 农业机械学报, 2021, 52(S1): 185-196.
	YANG P, ZHAO Y Y, LI Y M, et al. Review of research on integration of agricultural air-ground integration based on multi-source information fusion[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(S1): 185-196. (in Chinese)
[7]	史舟, 梁宗正, 杨媛媛, 等. 农业遥感研究现状与展望[J]. 农业机械学报, 2015, 46(2): 247-260.
	SHI Z, LIANG Z Z, YANG Y Y, et al. Status and prospect of agricultural remote sensing[J]. Transactions of the Chinese Society for Agricultural Machinery, 2015, 46(2): 247-260. (in Chinese with English abstract)
[8]	王利民, 刘佳, 杨玲波, 等. 短波红外波段对玉米大豆种植面积识别精度的影响[J]. 农业工程学报, 2016, 32(19): 169-178.
	WANG L M, LIU J, YANG L B, et al. Impact of short infrared wave band on identification accuracy of corn and soybean area[J]. Transactions of the Chinese Society of Agricultural Engineering, 2016, 32(19): 169-178. (in Chinese with English abstract)
[9]	苏伟, 侯宁, 李琪, 等. 基于Sentinel-2遥感影像的玉米冠层叶面积指数反演[J]. 农业机械学报, 2018, 49(1): 151-156.
	SU W, HOU N, LI Q, et al. Retrieving leaf area index of corn canopy based on sentinel-2 remote sensing image[J]. Transactions of the Chinese Society for Agricultural Machinery, 2018, 49(1): 151-156. (in Chinese with English abstract)
[10]	冯海霞, 秦其明, 李滨勇, 等. 基于SWIR-Red光谱特征空间的农田干旱监测新方法[J]. 光谱学与光谱分析, 2011, 31(11): 3069-3073.
	FENG H X, QIN Q M, LI B Y, et al. The new method monitoring agricultural drought based on SWIR-red spectrum feature space[J]. Spectroscopy and Spectral Analysis, 2011, 31(11): 3069-3073. (in Chinese with English abstract)
[11]	RADOGLOU-GRAMMATIKIS P, SARIGIANNIDIS P, LAGKAS T, et al. A compilation of UAV applications for precision agriculture[J]. Computer Networks, 2020, 172: 107148.
[12]	WATANABE K, GUO W, ARAI K, et al. High-throughput phenotyping of sorghum plant height using an unmanned aerial vehicle and its application to genomic prediction modeling[J]. Frontiers in Plant Science, 2017, 8: 421.
[13]	ARAUS J L, CAIRNS J E. Field high-throughput phenotyping: the new crop breeding frontier[J]. Trends in Plant Science, 2014, 19(1): 52-61.
[14]	汪小钦, 王苗苗, 王绍强, 等. 基于可见光波段无人机遥感的植被信息提取[J]. 农业工程学报, 2015, 31(5): 152-157, 159, 158.
	WANG X Q, WANG M M, WANG S Q, et al. Extraction of vegetation information from visible unmanned aerial vehicle images[J]. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31(5): 152-157, 159, 158. (in Chinese with English abstract)
[15]	王上上, 王军, 孙思. 无人机遥感技术对园林树木的计数统计[J]. 仲恺农业工程学院学报, 2017, 30(3): 37-42.
	WANG S S, WANG J, SUN S. Landscape trees counting by remote sensing technology based on UAV[J]. Journal of Zhongkai University of Agriculture and Engineering, 2017, 30(3): 37-42. (in Chinese with English abstract)
[16]	JIANG H, CHEN S S, LI D, et al. Papaya tree detection with UAV images using a GPU-accelerated scale-space filtering method[J]. Remote Sensing, 2017, 9: 721.
[17]	PU R, LANDRY S. A comparative analysis of high spatial resolution IKONOS and WorldView-2 imagery for mapping urban tree species[J]. Remote Sensing of Environment, 2012, 124: 516-533.
[18]	HUNG C, BRYSON M, SUKKARIEH S. Multi-class predictive template for tree crown detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2012, 68: 170-183.
[19]	DALPONTE M, ORKA H O, ENE L T, et al. Tree crown delineation and tree species classification in boreal forests using hyperspectral and ALS data[J]. Remote Sensing of Environment, 2014, 140: 306-317.
[20]	XIANG J J, LIU J, CHEN D, et al. CTFuseNet: A multi-scale cnn-transformer feature fused network for crop type segmentation on UAV remote sensing imagery[J]. Remote Sensing, 2023, 15(4): 1151.
[21]	叶青, 龙满生, 曾劲涛, 等. 基于深度学习的蜜柚果树航拍图像检测[J]. 井冈山大学学报(自然科学版), 2021, 42(5): 63-69.
	YE Q, LONG M S, ZENG J T, et al. Aerial image detection of pomelo fruit trees based on deep learning[J]. Journal of Jinggangshan University(Natural Science), 2021, 42(5): 63-69. (in Chinese with English abstract)
[22]	李志军, 杨圣慧, 史德帅, 等. 基于轻量化改进YOLOv5的苹果树产量测定方法[J]. 智慧农业(中英文), 2021, 3(2): 100-114.
	LI Z J, YANG S H, SHI D S, et al. Yield estimation method of apple tree based on improved lightweight YOLOv5[J]. Smart Agriculture, 2021, 3(2): 100-114. (in Chinese with English abstract)
[23]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[EB/OL]. 2018: arXiv: 1807.06521. http://arxiv.org/abs/1807.06521.
[24]	XIE S N, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 5987-5995.
[25]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[EB/OL]. 2017: arXiv: 1703.06870. http://arxiv.org/abs/1703.06870.
[26]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy. IEEE, 2017: 2999-3007.
[27]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. 2018: arXiv: 1804.02767. http://arxiv.org/abs/1804.02767.
[28]	ZHU X K, LYU S C, WANG X, et al. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Montreal, BC, Canada. IEEE, 2021: 2778-2788.
[29]	LIU W, ANGUELOV D, ERHAN D, et al. Ssd: Single shot multibox detector[C]// Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14. Springer, 2016: 21-37.
[30]	WEI L H, ZHENG C, HU Y J. Oriented object detection in aertal images based on the scaled smooth Li loss function[J]. Remote sensing, 2023, 15(5): 1350.
[31]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA. IEEE, 2018: 7132-7141.
[32]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA. IEEE, 2021: 13708-13717.
[33]	WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020: 11531-11539.
[34]	HADAS E, KÖLLE M, KARPINA M, et al. Identification of peach tree trunks from laser scanning data obtained with small unmanned aerial system[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020: 735-740.
[35]	MU Y, FUJII Y, TAKATA D, et al. Characterization of peach tree crown by using high-resolution images from an unmanned aerial vehicle[J]. Horticulture Research, 2018, 5: 74.
[36]	史浩, 冯全, 陈佰鸿. 基于深度学习的果树资源调查[J]. 林业机械与木工设备, 2021, 49(6): 53-58.
	SHI H, FENG Q, CHEN B H. Investigation of fruit tree resources based on deep learning[J]. Forestry Machinery & Woodworking Equipment, 2021, 49(6): 53-58. (in Chinese with English abstract)

方法 Method	物候期 Period	精确率 Precision/%	召回率 Recall/%	平均准确率 mAP/%	权重大小 Weight size/MB
Faster-RCNN	花期Flowering	62.99	81.86	77.51	521
	萌芽期Budding	58.31	77.39
Faster-RCNN①	花期Flowering	68.74	89.09	83.49	106
	萌芽期Budding	65.56	81.61
Faster-RCNN①+②	花期Flowering	72.37	90.59	85.82	106
	萌芽期Budding	69.50	82.55
Faster-RCNN①②+③	花期Flowering	74.99	90.18	86.46	106
	萌芽期Budding	72.25	80.28

方法 Method	物候期 Period	精确率 Precision/%	召回率 Recall/%	平均准确率 mAP/%	权重大小 Weight size/MB
Faster-RCNN	花期Flowering	62.99	81.86	77.51	521
	萌芽期Budding	58.31	77.39
Faster-RCNN①	花期Flowering	68.74	89.09	83.49	106
	萌芽期Budding	65.56	81.61
Faster-RCNN①+②	花期Flowering	72.37	90.59	85.82	106
	萌芽期Budding	69.50	82.55
Faster-RCNN①②+③	花期Flowering	74.99	90.18	86.46	106
	萌芽期Budding	72.25	80.28

方法 Method	主干网络 backbone	物候期 Period	精确率 Precision/%	召回率 Recall/%	平均准确率 mAP/%	权重大小 Weight size/MB
Faster-RCNN(Original)	VGG16	花期Flowering	62.99	81.86	77.51	521
		萌芽期Budding	58.31	77.39
YOLOv3	Cspdarknet53	花期Flowering	78.03	56.68	65.04	235
		萌芽期Budding	70.66	42.41
YOLOv5x	Cspdarknet53	花期Flowering	83.22	76.47	83.07	333
		萌芽期Budding	86.32	59.23
SSD	VGG16	花期Flowering	74.96	66.29	73.32	91
		萌芽期Budding	73.68	61.54
Ours	ResNeXt50	花期Flowering	75.61	88.81	86.46	106
		萌芽期Budding	72.49	82.88

方法 Method	主干网络 backbone	物候期 Period	精确率 Precision/%	召回率 Recall/%	平均准确率 mAP/%	权重大小 Weight size/MB
Faster-RCNN(Original)	VGG16	花期Flowering	62.99	81.86	77.51	521
		萌芽期Budding	58.31	77.39
YOLOv3	Cspdarknet53	花期Flowering	78.03	56.68	65.04	235
		萌芽期Budding	70.66	42.41
YOLOv5x	Cspdarknet53	花期Flowering	83.22	76.47	83.07	333
		萌芽期Budding	86.32	59.23
SSD	VGG16	花期Flowering	74.96	66.29	73.32	91
		萌芽期Budding	73.68	61.54
Ours	ResNeXt50	花期Flowering	75.61	88.81	86.46	106
		萌芽期Budding	72.49	82.88

方法 Method	平均准确率 Mean average precision/%	权重大小 Weight size/MB
Vgg16	77.51	521
ResNet-50	81.97	108
ResNet-101	82.60	180
ResNext-50	82.86	106
ResNext-101	82.36	350

Detection of peach trees in unmanned aerial vehicle (UAV) images based on improved Faster-RCNN network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 36

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	GUO Faxu, FENG Quan, YANG Sen, YANG Wanxia. Inversion of leaf nitrogen content in potato canopy based on unmanned aerial vehicle hyperspectral images [J]. Acta Agriculturae Zhejiangensis, 2023, 35(8): 1904-1914.
[2]	PAN Pan, ZHANG Jianhua, ZHENG Xiaoming, ZHOU Guomin, HU Lin, FENG Quan, CHAI Xiujuan. Research progress of deep learning in intelligent identification of disease resistance of crops and their related species [J]. Acta Agriculturae Zhejiangensis, 2023, 35(8): 1993-2012.
[3]	ZHANG Meng, SHE Bao, YANG Yuying, HUANG Linsheng, ZHU Mengqi. Study on extraction method of soybean planting areas based on unmanned aerial vehicle RGB image [J]. Acta Agriculturae Zhejiangensis, 2023, 35(4): 952-961.
[4]	BAI Weiwei, ZHAO Xueni, LUO Bin, ZHAO Wei, HUANG Shuo, ZHANG Han. Study of YOLOv5-based germination detection method for wheat seeds [J]. Acta Agriculturae Zhejiangensis, 2023, 35(2): 445-454.
[5]	ZHU Yongji, TAO Xinyu, CHEN Xiaofang, SU Xiangxiang, LIU Jikai, LI Xinwei. Estimation of above-ground biomass of winter wheat based on vegetation indexes and texture features of multispectral images captured by unmanned aerial vehicle [J]. Acta Agriculturae Zhejiangensis, 2023, 35(12): 2966-2976.
[6]	ZHU Shisong, MA Wanli, ZHAO Lishan, ZHENG Yanmei, ZHENG Xianbo, LU Bibo. Apple leaf image segmentation algorithm based on improved LinkNet [J]. Acta Agriculturae Zhejiangensis, 2023, 35(1): 202-214.
[7]	JIANG Youyi, ZHANG Chengjian, HAN Shaoyu, YANG Xiaodong, YANG Guijun, YANG Hao. Automatic counting of maize plants based on unmanned aerial vehicle (UAV) 3D point cloud [J]. Acta Agriculturae Zhejiangensis, 2022, 34(9): 2032-2042.
[8]	WANG Jun, LU Zhou, LUO Ming, XU Feifei, ZHANG Xu. Inversion of soil moisture content of winter wheat at turning green period based on multispectral remote sensing by unmanned aerial vehicle [J]. Acta Agriculturae Zhejiangensis, 2022, 34(6): 1297-1305.
[9]	CHEN Daohuai, WANG Hangjun. Detection of forest pests based on improved YOLOv4 [J]. Acta Agriculturae Zhejiangensis, 2022, 34(6): 1306-1315.
[10]	ZHOU Xinxing, ZHAO Lin, ZHANG Wenjie, TAN Changwei, LI Gangbo, SHI Mengyun, ZHANG Ting, YANG Feng. Remote sensing extraction of fruit tree planting area based on Sentinel-2 multi-temporal images [J]. Acta Agriculturae Zhejiangensis, 2022, 34(12): 2767-2777.
[11]	LI Chao, LI Feng, HUANG Weijia. Fruit variety recognition based on parallel convolutional neural network [J]. Acta Agriculturae Zhejiangensis, 2022, 34(11): 2533-2541.
[12]	ZHANG Qingqing, LIU Lianzhong, NING Jingming, WU Guodong, JIANG Zhaohui, LI Mengjie, LI Dongliang. Tea buds recognition under complex scenes based on optimized YOLOV3 model [J]. Acta Agriculturae Zhejiangensis, 2021, 33(9): 1740-1747.
[13]	WU Ningshan, WANG Jiaxi, ZHANG Yan, YUAN Mutian, ZHANG Qi, GAO Chiyu. Determining tree species and crown width from unmanned aerial vehicle imagery in hilly loess region of west Shanxi, China: a case study from Caijiachuan watershed [J]. Acta Agriculturae Zhejiangensis, 2021, 33(8): 1505-1518.
[14]	QIAN Jiawei, LIU Xiaoqing, ZHANG Jingjing, ZHOU Weihong, LI Jianlong. Constructions of hyperspectral remote sensing monitoring models for heavy metal contents in farmland soil in Zhangjiagang City [J]. , 2020, 32(8): 1437-1445.
[15]	YIN Linjiang, ZHOU Zhongfa, HUANG Denghong, SHANG Mengjia. Extraction of individual plant of pitaya in Karst Canyon Area based on point cloud data of UAV image matching [J]. , 2020, 32(6): 1092-1102.