基于YOLOV4模型的果园樱桃实时检测研究

doi:10.3969/j.issn.1004-1524.2022.11.21

浙江农业学报 ›› 2022, Vol. 34 ›› Issue (11): 2522-2532.DOI: 10.3969/j.issn.1004-1524.2022.11.21

基于YOLOV4模型的果园樱桃实时检测研究

周品志¹^,²(), 裴悦琨¹^,²^,^*(), 魏冉¹^,², 张永飞¹^,², 谷宇¹^,²

1.大连大学辽宁省北斗高精度位置服务技术工程实验室,辽宁大连 116622
2.大连大学大连市环境感知与智能控制重点实验室,辽宁大连 116622

收稿日期:2021-05-07 出版日期:2022-11-25 发布日期:2022-11-29
通讯作者: 裴悦琨
作者简介:*裴悦琨,E-mail: peiyuekun@dlu.edu.cn
周品志(1996—), 男,贵州毕节人,硕士研究生,研究方向为计算机图像处理。E-mail: 1978868229@qq.com
基金资助:
国家自然科学基金(61601076);大连市高层次人才创新计划(2019RQ070)

Real-time detection of orchard cherry based on YOLOV4 model

ZHOU Pinzhi¹^,²(), PEI Yuekun¹^,²^,^*(), WEI Ran¹^,², ZHANG Yongfei¹^,², GU Yu¹^,²

1. Beidou High Precision Positioning Service Technology Engineering Laboratory of Liaoning Province, Dalian Unviersity, Dalian 116622, Liaoning, China
2. Environment Sensing and Intelligent Control Key Laboratory of Dalian,Dalian University, Dalian 116622, Liaoning, China

Received:2021-05-07 Online:2022-11-25 Published:2022-11-29
Contact: PEI Yuekun

摘要/Abstract

摘要：

为解决在自然环境下对樱桃不同生长时期的状态监测受环境影响存在目标识别困难、检测准确率低的问题,提出了一种基于CSPDarknet53改进的卷积神经网络樱桃分类检测模型。经典YOLOV4所使用的特征提取网络层数较深,能够提取更高级的抽象特征,但是对目标局部感知能力较弱,通过在CSPDarknet53网络结构上融合CBAM注意力机制,增强了目标局部特征感知能力,进一步提升目标检测精度,其特征提取和目标检测能力优于原算法,调整特征提取网络的特征层输出,将第三层输出变为第二层输出以增加小目标语义信息的获取,利用k-means算法优化先验框尺寸以适应樱桃目标大小,并进行了消融实验分析。结果表明,改进的YOLOV4樱桃检测模型模型的平均精度达到了92.31%,F1分数达到了87.3%,优于Faster RCNN、YOLOV3和原来的YOLOV4算法,检测速度为40.23幅·s^-1,适用于自然环境下的樱桃监测,为实现果园水果生长状态自动监测提供了理论和技术基础。

关键词: 樱桃, 成熟度, 注意力机制

Abstract:

In order to solve the problem of difficulty in target recognition and low detection accuracy in the state monitoring of cherries in different growth periods under natural environment, this paper proposed an improved convolutional neural network cherry classification and detection model based on CSPDarknet53. The feature extraction network used in the classic YOLOV4 had a deeper number of layers and could extract more advanced abstract features, but had a weak local perception of the target. By integrating the CBAM attention mechanism on the CSPDarknet53 network structure, the perception of local features of the target was enhanced. Furthermore, the accuracy of target detection was improved, its feature extraction and target detection capabilities were better than the original algorithm, the feature layer output of the feature extraction network was adjusted, and the third layer output to the second layer output was changed to increase the acquisition of small target semantic information, k-means algorithm was used to optimize the size of the prior frame to adapt to the size of the cherry target, and ablation experiment analysis was conducted. The results showed that the improved YOLOV4 cherry detection model had an average accuracy of 92.31% and an F1 score of 87.3%, which was better than Faster RCNN, YOLOV3 and the original YOLOV4 algorithm. The detection speed was 40.23 frames·s^-1, which was suitable for natural environments. It provided a theoretical and technical basis for realizing automatic monitoring of fruit growth status in orchards.

Key words: cherry, maturity, attention mechanism

中图分类号:

S126

周品志, 裴悦琨, 魏冉, 张永飞, 谷宇. 基于YOLOV4模型的果园樱桃实时检测研究[J]. 浙江农业学报, 2022, 34(11): 2522-2532.

ZHOU Pinzhi, PEI Yuekun, WEI Ran, ZHANG Yongfei, GU Yu. Real-time detection of orchard cherry based on YOLOV4 model[J]. Acta Agriculturae Zhejiangensis, 2022, 34(11): 2522-2532.

图/表 15

图1 YOLOV4网络结构

Fig.1 YOLOV4 network structure

图2 残差块结构

Fig.2 Residual block structure

图3 改进的YOLOV4网络结构

Fig.3 Improved YOLOV4 network structure

图4 CBAM网络结构

Fig.4 CBAM network structure

图5 樱桃数据集标注框分布图

Fig.5 The distribution map of the label box of the cherry data set

图6 樱桃数据集宽高比统计图

Fig.6 The aspect ratio statistical graph of the cherry data set

图7 IoU交并比示例

Fig.7 IoU cross-to-combination example

表1 改进的YOLOV4模型训练参数

Table 1 Improved YOLOV4 model training parameters

处理 Treatment	参数值Parameter
处理 Treatment	第一阶段 First stage	第二阶段 Second stage
Epoch	0~100	100~200
Batch size	6	6
Lr	1×10^-3	1×10^-4
Optimizer	Adam	Adam

图8 不同成熟度的樱桃样本

Fig.8 Cherry samples of different maturity

图9 mosaic数据增强

Fig.9 mosaic data enhancement

表2 不同模块对模型性能的影响

Table 2 The impact of different modules on model performance

网络模型 Network model	AP/%			mAP/%	F1分数 F1-score/%
网络模型 Network model	未成熟果Immature	半成熟果Semi-mature	成熟果mature	mAP/%	F1分数 F1-score/%
YOLOV4	85.88	83.13	86.74	85.25	78.0
YOLOV4+mosaic	87.72	91.90	88.31	89.31	82.0
YOLOV4+CBAM	85.07	88.03	87.22	86.77	77.0
YOLOV4+k-means	85.90	84.36	86.98	85.75	78.6
本文方法Method of this article	90.92	91.98	94.04	92.31	87.3

图10 损失函数变化

Fig.10 Loss function changes

图11 未成熟果、半成熟果、成熟果对应模型的P-R曲线

Fig.11 The P-R curve of the corresponding model of immature, semimature and mature fruits

表3 不同算法对比测试结果

Table 3 Comparison test results of different algorithms

网络模型 Network model	FPS/ (幅·s^-1)	AP/%			精度 Precision/%	F1分数 F1-score/%
网络模型 Network model	FPS/ (幅·s^-1)	未成熟果 Immature	半成熟果 Semi-mature	成熟果 mature	精度 Precision/%	F1分数 F1-score/%
Faster RCNN	19.43	75.81	67.29	69.00	70.70	71.3
YOLOV3	52.13	85.79	75.15	87.92	82.95	78.6
YOLOV4	47.77	85.88	83.13	86.74	85.25	78.0
本文方法Method of this article	40.23	90.92	91.98	94.04	92.31	87.3

图12 Faster RCNN、YOLOV3、YOLOV4、改进YOLOV4检测及复杂环境下部分结果对比

Fig.12 Faster RCNN, YOLOV3, YOLOV4, improved YOLOV4 detection and comparison of some results in complex environments

参考文献 16

[1]	孙坦, 黄永文, 鲜国建, 等. 新一代信息技术驱动下的农业信息化发展思考[J]. 农业图书情报学报, 2021, 33(33): 4-15.
	SUN T, HUANG Y W, XIAN G J, et al. Considerations for the development of agricultural informatization driven by a new generation of information technologies[J]. Journal of Library and Information Science in Agriculture, 2021, 33(3): 4-15. (in Chinese with English abstract)
[2]	邹谜, 伍世虔, 王欣. 一种用于机器人水果采摘的快速识别方法[J]. 农机化研究, 2019, 41(1): 206-210,252.
	ZOU M, WU S Q, WANG X. A fast matching recognition method for fruit picking[J]. Journal of Agricultural Mechanization Research, 2019, 41(1): 206-210,252. (in Chinese with English abstract)
[3]	颜申, 宋文浩, 陈光, 等. 基于视觉的橘类水果识别系统设计[J]. 佳木斯大学学报(自然科学版), 2020, 38(38): 41-44.
	YAN S, SONG W H, CHEN G, et al. Design of orange-based fruit recognition system based on vision[J]. Journal of Jiamusi University (Natural Science Edition), 2020, 38(1): 41-44. (in Chinese with English abstract)
[4]	初广丽, 张伟, 王延杰, 等. 基于机器视觉的水果采摘机器人目标识别方法[J]. 中国农机化学报, 2018, 39(39): 83-88.
	CHU G L, ZHANG W, WANG Y J, et al. A method of fruit picking robot target identification based on machine vision[J]. Journal of Chinese Agricultural Mechanization, 2018, 39(2): 83-88. (in Chinese with English abstract)
[5]	YU L Y, XIONG J T, FANG X Q, et al. A litchi fruit recognition method in a natural environment using RGB-D images[J]. Biosystems Engineering, 2021, 204: 50-63. DOI URL
[6]	许德刚, 王露, 李凡. 深度学习的典型目标检测算法研究综述[J]. 计算机工程与应用, 2021, 57(8):10-25. DOI
	XU D G, WANG L, LI F. Review of typical object detection algorithms for deep learning[J]. Computer Engineering and Applications, 2021, 57(8):10-25. (in Chinese with English abstract) DOI
[7]	李国进, 胡洁, 艾矫燕. 基于改进SSD算法的车辆检测[J]. 计算机工程, 2022, 48(1):266-274.
	LI G J, HU J, AI J Y. Vehicle detection based on improved SSD[J]. Computer Engineering, 2022, 48(1):266-274. (in Chinese with English abstract)
[8]	杨耘, 李龙威, 高思岩, 等. 基于YOLO v3网络训练优化的高分遥感影像目标检测[J]. 激光与光电子学进展, 2021, 58(16):147-153.
	YANG Y, LI L W, GAO S Y, et al. High-resolution remote sensing image target detection based on YOLO v3network training optimization[J]. Laser & Optoelectronics Progress, 2021, 58(16):147-153. (in Chinese with English abstract)
[9]	洪敏杰, 吴刚, 刘星辰, 等. 基于注意力机制的肺结节检测算法[J]. 计算机工程与设计, 2021, 42(42): 83-88.
	HONG M J, WU G, LIU X C, et al. Detection algorithm of lung nodule based on attention mechanism[J]. Computer Engineering and Design, 2021, 42(1): 83-88. (in Chinese with English abstract)
[10]	唐熔钗, 伍锡如. 基于改进YOLO-V3网络的百香果实时检测[J]. 广西师范大学学报(自然科学版), 2020, 38(38): 32-39.
	TANG R C, WU X R. Real-time detection of passion fruit based on improved YOLO-V3 network[J]. Journal of Guangxi Normal University (Natural Science Edition), 2020, 38(6): 32-39. (in Chinese with English abstract)
[11]	SCHUMANN A W, MOOD N S, MUNGOFA P D, et al. Detection of three fruit maturity stages in wild blueberry fields using deep learning artificial neural networks[C]// 2019 Boston, Massachusetts: American Society of Agricultural and Biological Engineers, 2019: 1.
[12]	朱旭, 马淏, 姬江涛, 等. 基于Faster R-CNN的蓝莓冠层果实检测识别分析[J]. 南方农业学报, 2020, 51(51): 1493-1501.
	ZHU X, MA H, JI J T, et al. Detecting and identifying blueberry canopy fruits based on Faster R-CNN[J]. Journal of Southern Agriculture, 2020, 51(6): 1493-1501. (in Chinese with English abstract)
[13]	SONG Z Z, FU L S, WU J Z, et al. Kiwifruit detection in field images using Faster R-CNN with VGG16[J]. IFAC-PapersOnLine, 2019, 52(30): 76-81.
[14]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[J/OL]. arXiv preprint arXiv:1804.02767, 2018.
[15]	WANG C Y, MARK LIAO H Y, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). June 14-19, 2020, Seattle, WA, USA. IEEE, 2020: 1571-1580.
[16]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.

基于YOLOV4模型的果园樱桃实时检测研究

Real-time detection of orchard cherry based on YOLOV4 model

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 16

相关文章 15

编辑推荐

Metrics

本文评价

[1]	金宝霞, 王伟杰, 朱晓林, 王贤, 魏小红. 不同激素组合对番茄离体再生和相关基因表达的影响[J]. 浙江农业学报, 2022, 34(9): 1889-1900.
[2]	王慧茹, 李建设, 闫思华, 高艳明. 整枝方式对樱桃番茄冠层截获和荧光特性的影响[J]. 浙江农业学报, 2022, 34(3): 525-533.
[3]	裴芸, 徐秀红, 陆锦彪, 陈阿敏, 张万萍. 151份贵州地方樱桃番茄资源的遗传多样性分析[J]. 浙江农业学报, 2022, 34(2): 310-316.
[4]	张志刚, 刘玉芳, 李长城, 李宏, 程平, 杨璐. 不同成熟度对杏果实品质的影响[J]. 浙江农业学报, 2021, 33(8): 1402-1408.
[5]	张宁, 吴华瑞, 韩笑, 缪祎晟. 基于多尺度和注意力机制的番茄病害识别方法[J]. 浙江农业学报, 2021, 33(7): 1329-1338.
[6]	胡晓丹, 王来娣, 杨婷, 白皓, 江勇, 王志秀, 王莎莎, 陈国宏, 常国斌. 鸭脂肪细胞中circ-FBLN2鉴定与功能预测[J]. 浙江农业学报, 2021, 33(4): 602-609.
[7]	周艳超, 薛坤, 葛海燕, 陈火英, 刘杨. 基于主成分与聚类分析的樱桃番茄品质综合评价[J]. 浙江农业学报, 2021, 33(12): 2320-2329.
[8]	梁乐, 刘娟, 李晓梅, 廖继超, 李焕秀, 唐懿. 三种基因型樱桃番茄混种对果实品质和硒含量的影响[J]. 浙江农业学报, 2021, 33(10): 1870-1878.
[9]	谭光辉, 覃媛钰, 李杰章, 吴磊, 岳雍, 张依裕. 樱桃谷鸭ATF4基因SNPs筛查及生物信息学分析[J]. 浙江农业学报, 2020, 32(2): 218-225.
[10]	张芳, 王佩欣, 何勇, 骆慧枫, 寿国忠. 基于物联网的阳台微型温室作物生长环境因子探究[J]. 浙江农业学报, 2020, 32(2): 234-242.
[11]	杨志, 李文义, 高云涛, 熊华斌, 陈毅坚, 杨慧娟. 响应面法优化针叶樱桃总黄酮的提取工艺及其抗氧化活性研究[J]. 浙江农业学报, 2020, 32(10): 1866-1872.
[12]	陈兵, 徐婧, 罗启慧, 黄超, 刘文涛, 吴洪, 陈镜妃, 陈苹, 高琪, 马文静, 陈正礼. 感染鸭源小鹅瘟病毒QH-L01株樱桃谷雏鸭病理学研究[J]. 浙江农业学报, 2018, 30(6): 932-938.
[13]	薛莞莞, 龚荣高, 丁建林, 李克强, 邹金, 李如龙, 彭宏贵. 赤霉素喷施对红灯甜樱桃果实品质及解剖结构的影响[J]. 浙江农业学报, 2018, 30(6): 978-984.
[14]	李京霞, 夏惠, 吕秀兰, 王进, 梁东. 甜樱桃PacAMR1基因的克隆及表达分析[J]. 浙江农业学报, 2018, 30(4): 607-613.
[15]	程远, 万红建, 刘超超, 姚祝平, 叶青静, 李志邈, 王荣青, 周国治, 杨悦俭, 陈德梁, 阮美颖. 十六个樱桃番茄品种果实风味品质相关指标比较分析[J]. 浙江农业学报, 2018, 30(11): 1859-1869.