基于改进Faster-RCNN网络的无人机遥感影像桃树检测

doi:10.3969/j.issn.1004-1524.20230912

浙江农业学报 ›› 2024, Vol. 36 ›› Issue (8): 1909-1919.DOI: 10.3969/j.issn.1004-1524.20230912

基于改进Faster-RCNN网络的无人机遥感影像桃树检测

程嘉瑜¹^,²(), 陈妙金³, 李彤¹^,², 孙奇男³, 张小斌², 赵懿滢², 朱怡航², 顾清²^,^*()

1.浙江农林大学数学与计算机科学学院,浙江杭州 311300
2.浙江省农业科学院数字农业研究所,浙江杭州 310021
3.宁波市奉化区水蜜桃研究所,浙江宁波 315502

收稿日期:2023-07-26 出版日期:2024-08-25 发布日期:2024-09-06
作者简介:*顾清,E-mail: guq@zaas.ac.cn
程嘉瑜(1999—),男,上海青浦人,硕士研究生,主要从事农业智能技术研究。E-mail: cjy@stu.zafu.edu.cn
通讯作者: 顾清
基金资助:
浙江省基础公益研究计划项目(LQ23F050001);浙江省农业科学院生物育种融通计划项目(2022SWYZ);浙江省农业科学院青年人才培养项目(20CF0401)

Detection of peach trees in unmanned aerial vehicle (UAV) images based on improved Faster-RCNN network

CHENG Jiayu¹^,²(), CHEN Miaojin³, LI Tong¹^,², SUN Qinan³, ZHANG Xiaobin², ZHAO Yiying², ZHU Yihang², GU Qing²^,^*()

1. College of Mathematics and Computer Science, Zhejiang A&F University, Hangzhou 311300, China
2. Institute of Digital Agriculture, Zhejiang Academy of Agricultural Sciences, Hangzhou 310021, China
3. Ningbo Fenghua Peach Research Institute, Ningbo 315502, Zhejiang, China

Received:2023-07-26 Online:2024-08-25 Published:2024-09-06
Contact: GU Qing

摘要/Abstract

摘要：

研究桃树精准检测定位方法可为桃园精准化管理提供支撑,该研究利用无人机遥感结合深度学习算法对无叶期桃树进行检测,并对萌芽期桃树和花期桃树进行区分。在Faster-RCNN原始网络的基础上提出了3种改进:替换主干网络为融合卷积注意力模块(convolutional block attention module, CBAM )后的ResNeXt-50、感兴趣区域特征提取方法使用ROI Align代替ROI Pooling、引入不平衡交叉熵损失函数Focal Loss,并采用消融试验对这些改进方法的效果进行分析。实验表明,与未改进的Faster-RCNN相比,改进后的模型mAP(mean average precision)提升了8.95百分点,达到了86.46%,能够较好地区分花期桃树与萌芽期桃树。对模型提升贡献最大的改进是ResNeXt-50-CBAM主干网络替换,mAP提升5.98百分点;ROI Align的使用减少了特征量化过程中的误差,mAP提升2.33百分点。与其他主流检测模型YOLOv3、 YOLOv5x和SSD相比,该研究提出的模型检测效果更优。该研究提出的无人机遥感结合改进Faster-RCNN算法能够较好地对桃树进行检测,可以满足桃园精准化管理需求。

关键词: 桃树, 遥感, 深度学习, 无人机

Abstract:

Studying precise detection and positioning methods for peach trees can provide support for precision management of peach orchards. This study utilizes unmanned aerial vehicle (UAV) remote sensing combined with deep learning algorithms to detect leafless peach trees and differentiate between budding and flowering peach trees. Three improvements are proposed based on the Faster R-CNN original network: replacing the backbone network with a ResNeXt-50 integrated with a convolutional block attention module (CBAM), using ROI Align instead of ROI Pooling for feature extraction from regions of interest, and introducing the Focal Loss function for imbalanced cross-entropy loss. Ablation experiments are conducted to analyze the effectiveness of these improvements. The experiments demonstrate that compared with the unimproved Faster R-CNN, the improved model achieves a mean average precision (mAP) increase of 8.95 percentage points, reaching 86.46%, enabling better differentiation between flowering and budding peach trees. The most significant improvement contributing to the model’s enhancement is the replacement of the ResNeXt-50 backbone network with CBAM, resulting in a 5.98 percentage points increase in mAP; the use of ROI Align reduces errors in the feature quantization process, leading to a 2.33 percentage points increase in mAP. Compared with other mainstream detection models such as YOLOv3, YOLOv5x, and SSD, the proposed model in this study demonstrates superior detection performance. The UAV remote sensing combined with the improved Faster R-CNN algorithm proposed in this study can effectively detect peach trees, meeting the requirements of precision management in peach orchards.

Key words: peach tree, remote sensing, deep learning, unmanned aerial vehicle

中图分类号:

S126

程嘉瑜, 陈妙金, 李彤, 孙奇男, 张小斌, 赵懿滢, 朱怡航, 顾清. 基于改进Faster-RCNN网络的无人机遥感影像桃树检测[J]. 浙江农业学报, 2024, 36(8): 1909-1919.

CHENG Jiayu, CHEN Miaojin, LI Tong, SUN Qinan, ZHANG Xiaobin, ZHAO Yiying, ZHU Yihang, GU Qing. Detection of peach trees in unmanned aerial vehicle (UAV) images based on improved Faster-RCNN network[J]. Acta Agriculturae Zhejiangensis, 2024, 36(8): 1909-1919.

图/表 13

图1 研究区位置

Fig.1 Location of study area

图2 不同物候期的桃树 a. 萌芽期桃树 b. 花期桃树

Fig.2 Peach trees in different phenological stages a. Budding peach tree b. Flowering peach trees

图3 改进后的Faster-RCNN网络结构图

Fig.3 Improved structure diagram of Faster-RCNN network

图4 ResNeXt网络结构示意图

Fig.4 ResNeXt network structure diagram

图5 CBAM 注意力机制结构图

Fig.5 CBAM attention mechanism structure diagram

图6 感兴趣区域池化(ROI Align)

Fig.6 Region of interest (ROI) align pooling

图7 训练过程中的损失和精度曲线

Fig.7 Loss and accuracy curves during training

表1 不同网络结构对Faster-RCNN的影响

Table 1 Effects of different network structures on Faster-RCNN

方法 Method	物候期 Period	精确率 Precision/%	召回率 Recall/%	平均准确率 mAP/%	权重大小 Weight size/MB
Faster-RCNN	花期Flowering	62.99	81.86	77.51	521
	萌芽期Budding	58.31	77.39
Faster-RCNN①	花期Flowering	68.74	89.09	83.49	106
	萌芽期Budding	65.56	81.61
Faster-RCNN①+②	花期Flowering	72.37	90.59	85.82	106
	萌芽期Budding	69.50	82.55
Faster-RCNN①②+③	花期Flowering	74.99	90.18	86.46	106
	萌芽期Budding	72.25	80.28

表2 不同网络模型的检测结果对比

Table 2 Comparison of detection results of different networks

方法 Method	主干网络 backbone	物候期 Period	精确率 Precision/%	召回率 Recall/%	平均准确率 mAP/%	权重大小 Weight size/MB
Faster-RCNN(Original)	VGG16	花期Flowering	62.99	81.86	77.51	521
		萌芽期Budding	58.31	77.39
YOLOv3	Cspdarknet53	花期Flowering	78.03	56.68	65.04	235
		萌芽期Budding	70.66	42.41
YOLOv5x	Cspdarknet53	花期Flowering	83.22	76.47	83.07	333
		萌芽期Budding	86.32	59.23
SSD	VGG16	花期Flowering	74.96	66.29	73.32	91
		萌芽期Budding	73.68	61.54
Ours	ResNeXt50	花期Flowering	75.61	88.81	86.46	106
		萌芽期Budding	72.49	82.88

图8 不同网络模型的检测效果对比

Fig.8 Comparison of detection results of different networks

表3 不同主干网络Faster-RCNN模型的检测结果

Table 3 Detection results of Faster-RCNN models using different backbone networks

方法 Method	平均准确率 Mean average precision/%	权重大小 Weight size/MB
Vgg16	77.51	521
ResNet-50	81.97	108
ResNet-101	82.60	180
ResNext-50	82.86	106
ResNext-101	82.36	350

表4 花期桃树与萌芽期桃树分类使用不同注意力机制模块的检测结果

Table 4 Comparison of different attention mechanism modules for classification of flowering and budding peach trees

方法 Method	平均准确率 Mean average precision/%
/	82.86
SE	80.03
CA	80.63
ECA	82.13
CBAM	83.49

图9 长势异常桃树检测结果与原始检测结果对比 a. 原始检测结果 b. 长势异常桃树检测结果

Fig.9 Detection of peach trees with abnormal growth and original detection result a. Original detection result b. Peach trees with abnormal growth

参考文献 36

[1]	王宇霖, 宗学普, 魏闻东. 全国桃区划研究[J]. 果树科学, 1984, 1(2): 13-24.
	WANG Y L, ZONG X P, WEI W D. Study on the regionalization of peach growing in China[J]. Journal of Fruit Science, 1984, 1(2): 13-24. (in Chinese with English abstract)
[2]	俞明亮, 王力荣, 王志强, 等. 新中国果树科学研究70年: 桃[J]. 果树学报, 2019, 36(10): 1283-1291.
	YU M L, WANG L R, WANG Z Q, et al. Fruit scientific research in New China in the past 70 years: peach[J]. Journal of Fruit Science, 2019, 36(10): 1283-1291. (in Chinese with English abstract)
[3]	孙忠富, 杜克明, 郑飞翔, 等. 大数据在智慧农业中研究与应用展望[J]. 中国农业科技导报, 2013, 15(6): 63-71.
	SUN Z F, DU K M, ZHENG F X, et al. Perspectives of research and application of big data on smart agriculture[J]. Journal of Agricultural Science and Technology, 2013, 15(6): 63-71. (in Chinese with English abstract)
[4]	刘海启. 以精准农业驱动农业现代化加速现代农业数字化转型[J]. 中国农业资源与区划, 2019, 40(1): 1-6, 73.
	LIU H Q. Accelerating the digital transformation of modern agriculture by driving the agricultural modernization with precision agriculture[J]. Chinese Journal of Agricultural Resources and Regional Planning, 2019, 40(1): 1-6, 73. (in Chinese with English abstract)
[5]	施连敏, 陈志峰, 盖之华. 物联网在智慧农业中的应用[J]. 农机化研究, 2013, 35(6): 250-252.
	SHI L M, CHEN Z F, GAI Z H. Application research of the Internet of Things in wisdom agriculture[J]. Journal of Agricultural Mechanization Research, 2013, 35(6): 250-252. (in Chinese with English abstract)
[6]	杨普, 赵远洋, 李一鸣, 等. 基于多源信息融合的农业空地一体化研究综述[J]. 农业机械学报, 2021, 52(S1): 185-196.
	YANG P, ZHAO Y Y, LI Y M, et al. Review of research on integration of agricultural air-ground integration based on multi-source information fusion[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(S1): 185-196. (in Chinese)
[7]	史舟, 梁宗正, 杨媛媛, 等. 农业遥感研究现状与展望[J]. 农业机械学报, 2015, 46(2): 247-260.
	SHI Z, LIANG Z Z, YANG Y Y, et al. Status and prospect of agricultural remote sensing[J]. Transactions of the Chinese Society for Agricultural Machinery, 2015, 46(2): 247-260. (in Chinese with English abstract)
[8]	王利民, 刘佳, 杨玲波, 等. 短波红外波段对玉米大豆种植面积识别精度的影响[J]. 农业工程学报, 2016, 32(19): 169-178.
	WANG L M, LIU J, YANG L B, et al. Impact of short infrared wave band on identification accuracy of corn and soybean area[J]. Transactions of the Chinese Society of Agricultural Engineering, 2016, 32(19): 169-178. (in Chinese with English abstract)
[9]	苏伟, 侯宁, 李琪, 等. 基于Sentinel-2遥感影像的玉米冠层叶面积指数反演[J]. 农业机械学报, 2018, 49(1): 151-156.
	SU W, HOU N, LI Q, et al. Retrieving leaf area index of corn canopy based on sentinel-2 remote sensing image[J]. Transactions of the Chinese Society for Agricultural Machinery, 2018, 49(1): 151-156. (in Chinese with English abstract)
[10]	冯海霞, 秦其明, 李滨勇, 等. 基于SWIR-Red光谱特征空间的农田干旱监测新方法[J]. 光谱学与光谱分析, 2011, 31(11): 3069-3073.
	FENG H X, QIN Q M, LI B Y, et al. The new method monitoring agricultural drought based on SWIR-red spectrum feature space[J]. Spectroscopy and Spectral Analysis, 2011, 31(11): 3069-3073. (in Chinese with English abstract)
[11]	RADOGLOU-GRAMMATIKIS P, SARIGIANNIDIS P, LAGKAS T, et al. A compilation of UAV applications for precision agriculture[J]. Computer Networks, 2020, 172: 107148.
[12]	WATANABE K, GUO W, ARAI K, et al. High-throughput phenotyping of sorghum plant height using an unmanned aerial vehicle and its application to genomic prediction modeling[J]. Frontiers in Plant Science, 2017, 8: 421.
[13]	ARAUS J L, CAIRNS J E. Field high-throughput phenotyping: the new crop breeding frontier[J]. Trends in Plant Science, 2014, 19(1): 52-61.
[14]	汪小钦, 王苗苗, 王绍强, 等. 基于可见光波段无人机遥感的植被信息提取[J]. 农业工程学报, 2015, 31(5): 152-157, 159, 158.
	WANG X Q, WANG M M, WANG S Q, et al. Extraction of vegetation information from visible unmanned aerial vehicle images[J]. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31(5): 152-157, 159, 158. (in Chinese with English abstract)
[15]	王上上, 王军, 孙思. 无人机遥感技术对园林树木的计数统计[J]. 仲恺农业工程学院学报, 2017, 30(3): 37-42.
	WANG S S, WANG J, SUN S. Landscape trees counting by remote sensing technology based on UAV[J]. Journal of Zhongkai University of Agriculture and Engineering, 2017, 30(3): 37-42. (in Chinese with English abstract)
[16]	JIANG H, CHEN S S, LI D, et al. Papaya tree detection with UAV images using a GPU-accelerated scale-space filtering method[J]. Remote Sensing, 2017, 9: 721.
[17]	PU R, LANDRY S. A comparative analysis of high spatial resolution IKONOS and WorldView-2 imagery for mapping urban tree species[J]. Remote Sensing of Environment, 2012, 124: 516-533.
[18]	HUNG C, BRYSON M, SUKKARIEH S. Multi-class predictive template for tree crown detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2012, 68: 170-183.
[19]	DALPONTE M, ORKA H O, ENE L T, et al. Tree crown delineation and tree species classification in boreal forests using hyperspectral and ALS data[J]. Remote Sensing of Environment, 2014, 140: 306-317.
[20]	XIANG J J, LIU J, CHEN D, et al. CTFuseNet: A multi-scale cnn-transformer feature fused network for crop type segmentation on UAV remote sensing imagery[J]. Remote Sensing, 2023, 15(4): 1151.
[21]	叶青, 龙满生, 曾劲涛, 等. 基于深度学习的蜜柚果树航拍图像检测[J]. 井冈山大学学报(自然科学版), 2021, 42(5): 63-69.
	YE Q, LONG M S, ZENG J T, et al. Aerial image detection of pomelo fruit trees based on deep learning[J]. Journal of Jinggangshan University(Natural Science), 2021, 42(5): 63-69. (in Chinese with English abstract)
[22]	李志军, 杨圣慧, 史德帅, 等. 基于轻量化改进YOLOv5的苹果树产量测定方法[J]. 智慧农业(中英文), 2021, 3(2): 100-114.
	LI Z J, YANG S H, SHI D S, et al. Yield estimation method of apple tree based on improved lightweight YOLOv5[J]. Smart Agriculture, 2021, 3(2): 100-114. (in Chinese with English abstract)
[23]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[EB/OL]. 2018: arXiv: 1807.06521. http://arxiv.org/abs/1807.06521.
[24]	XIE S N, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 5987-5995.
[25]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[EB/OL]. 2017: arXiv: 1703.06870. http://arxiv.org/abs/1703.06870.
[26]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy. IEEE, 2017: 2999-3007.
[27]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. 2018: arXiv: 1804.02767. http://arxiv.org/abs/1804.02767.
[28]	ZHU X K, LYU S C, WANG X, et al. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Montreal, BC, Canada. IEEE, 2021: 2778-2788.
[29]	LIU W, ANGUELOV D, ERHAN D, et al. Ssd: Single shot multibox detector[C]// Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part I 14. Springer, 2016: 21-37.
[30]	WEI L H, ZHENG C, HU Y J. Oriented object detection in aertal images based on the scaled smooth Li loss function[J]. Remote sensing, 2023, 15(5): 1350.
[31]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA. IEEE, 2018: 7132-7141.
[32]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA. IEEE, 2021: 13708-13717.
[33]	WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020: 11531-11539.
[34]	HADAS E, KÖLLE M, KARPINA M, et al. Identification of peach tree trunks from laser scanning data obtained with small unmanned aerial system[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020: 735-740.
[35]	MU Y, FUJII Y, TAKATA D, et al. Characterization of peach tree crown by using high-resolution images from an unmanned aerial vehicle[J]. Horticulture Research, 2018, 5: 74.
[36]	史浩, 冯全, 陈佰鸿. 基于深度学习的果树资源调查[J]. 林业机械与木工设备, 2021, 49(6): 53-58.
	SHI H, FENG Q, CHEN B H. Investigation of fruit tree resources based on deep learning[J]. Forestry Machinery & Woodworking Equipment, 2021, 49(6): 53-58. (in Chinese with English abstract)

编辑推荐

Metrics

阅读次数

全文

379

HTML			PDF

最新录用	在线预览	正式出版	最新录用	在线预览	正式出版
0	0	7	0	0	372

来源	本网站	其他网站

次数	303	76
比例	80%	20%

摘要

161

最新录用	在线预览	正式出版

0	0	161

	来源	本网站

	次数	161
	比例	100%

基于改进Faster-RCNN网络的无人机遥感影像桃树检测

Detection of peach trees in unmanned aerial vehicle (UAV) images based on improved Faster-RCNN network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 36

相关文章 15

编辑推荐

Metrics

本文评价

[1]	郭发旭, 冯全, 杨森, 杨婉霞. 基于无人机高光谱的马铃薯冠层叶片全氮含量反演[J]. 浙江农业学报, 2023, 35(8): 1904-1914.
[2]	潘攀, 张建华, 郑晓明, 周国民, 胡林, 冯全, 柴秀娟. 深度学习在作物及其近缘种抗病性智能鉴定上的研究进展[J]. 浙江农业学报, 2023, 35(8): 1993-2012.
[3]	张梦, 佘宝, 杨玉莹, 黄林生, 朱梦琦. 基于无人机RGB影像的大豆种植区提取方法研究[J]. 浙江农业学报, 2023, 35(4): 952-961.
[4]	白卫卫, 赵雪妮, 罗斌, 赵薇, 黄硕, 张晗. 基于YOLOv5的小麦种子发芽检测方法研究[J]. 浙江农业学报, 2023, 35(2): 445-454.
[5]	朱永基, 陶新宇, 陈小芳, 苏祥祥, 刘吉凯, 李新伟. 基于无人机多光谱影像植被指数与纹理特征的冬小麦地上部生物量估算[J]. 浙江农业学报, 2023, 35(12): 2966-2976.
[6]	朱世松, 马婉丽, 赵理山, 郑艳梅, 郑先波, 芦碧波. 基于改进的LinkNet的苹果叶片图像分割算法[J]. 浙江农业学报, 2023, 35(1): 202-214.
[7]	姜友谊, 张成健, 韩少宇, 杨小冬, 杨贵军, 杨浩. 基于无人机三维点云的玉米植株自动计数研究[J]. 浙江农业学报, 2022, 34(9): 2032-2042.
[8]	王钧, 陆洲, 罗明, 徐飞飞, 张序. 基于机载多光谱的冬小麦返青期土壤墒情反演[J]. 浙江农业学报, 2022, 34(6): 1297-1305.
[9]	陈道怀, 汪杭军. 基于改进YOLOv4的林业害虫检测[J]. 浙江农业学报, 2022, 34(6): 1306-1315.
[10]	周欣兴, 赵林, 张文杰, 谭昌伟, 李刚波, 石梦云, 张婷, 杨峰. 基于Sentinel-2多时相影像的果树种植区遥感提取[J]. 浙江农业学报, 2022, 34(12): 2767-2777.
[11]	李超, 李锋, 黄炜嘉. 基于并联卷积神经网络的水果品种识别[J]. 浙江农业学报, 2022, 34(11): 2533-2541.
[12]	张晴晴, 刘连忠, 宁井铭, 吴国栋, 江朝晖, 李孟杰, 李栋梁. 基于YOLOV3优化模型的复杂场景下茶树嫩芽识别[J]. 浙江农业学报, 2021, 33(9): 1740-1747.
[13]	邬宁珊, 王佳希, 张岩, 元慕田, 张琪, 高驰宇. 基于无人机可见光影像的树种和树冠信息提取——以晋西黄土区蔡家川流域为例[J]. 浙江农业学报, 2021, 33(8): 1505-1518.
[14]	钱家炜, 刘晓青, 张静静, 周卫红, 李建龙. 张家港市农田土壤重金属含量高光谱遥感监测模型构建[J]. 浙江农业学报, 2020, 32(8): 1437-1445.
[15]	尹林江, 周忠发, 黄登红, 尚梦佳. 基于无人机影像匹配点云数据的喀斯特峡谷区火龙果单株提取研究[J]. 浙江农业学报, 2020, 32(6): 1092-1102.