基于迁移学习和改进残差网络的棉花叶螨为害等级识别

doi:10.3969/j.issn.1004-1524.20221148

浙江农业学报 ›› 2023, Vol. 35 ›› Issue (7): 1729-1739.DOI: 10.3969/j.issn.1004-1524.20221148

基于迁移学习和改进残差网络的棉花叶螨为害等级识别

张䶮^a^,^b(), 周保平^a^,^*(), 王昱^a^,^b, 冯洁^a^,^b, 叶凡恺^a^,^b, 何云龙^a^,^b

a.塔里木大学信息工程学院, 新疆阿拉尔 843300
b.塔里木大学现代农业工程重点实验室,新疆阿拉尔 843300

收稿日期:2022-08-03 出版日期:2023-07-25 发布日期:2023-08-17
作者简介:张䶮(1998—),女,河南周口人,硕士研究生,研究方向为农业电气化与自动化。E-mail:2648385897@qq.com
通讯作者: *周保平,E-mail:502805150@qq.com
基金资助:
国家自然科学基金(61563046)

Identification of harm grades of cotton spider mites based on transfer learning and improved residual network

ZHANG Yan^a^,^b(), ZHOU Baoping^a^,^*(), WANG Yu^a^,^b, FENG Jie^a^,^b, YE Fankai^a^,^b, HE Yunlong^a^,^b

a. College of Information Engineering, Tarim University, Alar 843300, Xinjiang, China
b. Key Laboratory of Modern Agricultural Engineering, Tarim University, Alar 843300, Xinjiang, China

Received:2022-08-03 Online:2023-07-25 Published:2023-08-17
Contact: ZHOU Baoping

摘要/Abstract

摘要：

针对人工诊断棉叶螨害分级准确率低、耗时长、成本高的问题,提出一种基于迁移学习和改进残差网络的棉花叶螨为害等级识别方法。以3种受害等级的棉花叶片与健康叶片图像作为对象,分别于单一背景和自然环境下采集图像,构建图像数据集。首先,利用PlantVillage数据集预训练模型,使用数据增强技术对数据集进行数据增强,扩充训练样本;然后,在ResNet50网络模型的基础上,引入焦点损失函数,在不同网络层嵌入注意力机制模块,并加入Dropout正则化构建改进的ResNet50模型;最后,对比不同模型的识别效果。结果表明:同时在深层和浅层引入注意力机制模块,设定动量为0.9、学习率为0.001时,改进的ResNet50模型具有最好的分类效果,优于ResNet50、VGG16、MobileNet、AlexNet和SENet模型,对棉叶螨危害等级的平均识别准确率达到97.8%。

关键词: 棉花叶螨, 受害等级, ResNet50网络, 迁移学习, 焦点损失函数

Abstract:

In view of the low accuracy, long time consuming and high cost of traditional artificial diagnosis of cotton spider mites, a method for detection and classification of harm grade was proposed based on transfer learning and improved residual network. The cotton leaf images either healthy or with 3 harm grades of cotton spider mites were collected both in single background and natural environment to construct image dataset. First, the PlantVillage dataset was used to pretrain the model. Data augmentation was carried out to expand training samples. Then, based on the original ResNet50 network, an improved ResNet50 network was constructed by introducing focal loss functions, embedding attention mechanism modules in different network layers, and optimizing with the Dropout regularization. Finally, the performance of the improved ResNet50 network was compared with other models. It was shown that with the attention mechanism module introduced both in the deep and shallow layers, the momentum being 0.9, and the learning rate being 0.001, the improved ResNet50 network had the best classification effect, which was superior than the original ResNet50, VGG16, MobileNet, AlexNet and SENet models, with the average recognition accuracy of 97.8%.

Key words: cotton spider mite, hazard level, ResNet50 network, transfer learning, focal loss function

中图分类号:

TP391.4

张䶮, 周保平, 王昱, 冯洁, 叶凡恺, 何云龙. 基于迁移学习和改进残差网络的棉花叶螨为害等级识别[J]. 浙江农业学报, 2023, 35(7): 1729-1739.

ZHANG Yan, ZHOU Baoping, WANG Yu, FENG Jie, YE Fankai, HE Yunlong. Identification of harm grades of cotton spider mites based on transfer learning and improved residual network[J]. Acta Agriculturae Zhejiangensis, 2023, 35(7): 1729-1739.

图/表 12

图1 各级棉叶图像示例

Fig.1 Sample images of cotton leaves with spider mites

表1 数据增强后各级棉叶图像的数量

Table 1 Quantity of cotten leaves images after augmentation

级别 Grade	图像数量Image quantity
级别 Grade	训练集 Training set	测试集 Test set	总计 Total
0	9 117	3 908	13 025
1	11 970	5 130	17 100
2	11 900	5 100	17 000
3	14 175	6 075	20 250
总计Total	47 162	2 0213	67 375

图2 数据增强效果示例 a,原图;b,高斯模糊;c,水平翻转;d,旋转90°;e,弹性变换;f,随机裁剪;g,矩形丢弃;h,网格失真;i,网格擦除;j,光学畸变。

Fig.2 Examples of images after data augmentation a, Origin image; b, Gaussian blur; c, Horizontal flip; d, Rotate by 90°; e, Elastic transformation; f, Selective erasing; g, Generate rectangular region; h, Grid distortion; i, Delete rectangular areas by grid mode; j, Optical distortion.

图3 ResNet50模型结构图

Fig.3 Structure diagram of ResNet50 model

图4 改进ResNet50模型结构图

Fig.4 Structure diagram of improved ResNet50 model

图5 棉叶螨害原始图像及其特征提取的可视化

Fig.5 Original image and feature extraction and visualization of images with cotton spider mites

图6 不同动量因子下模型的准确率和损失值对比

Fig.6 Comparison of accuracy and loss of models under different momentum factors a,m=0.3;b,m=0.6,c,m=0.9。

表2 不同学习率下模型的性能指标对比

Table 2 Performance comparison of models under different learning rates %

学习率 Learning rate	准确率 Precision	召回率 Recall	F1得分 F1 score
0.01	94.7	94.6	94.6
0.001	97.8	97.7	97.8
0.000 1	92.0	91.8	91.5

图7 注意力机制在不同嵌入方式下的模型准确率对比

Fig.7 Comparison of accuracy of models under different embedding methods of attention mechanism

图8 不同模型在测试集上的准确率和损失值对比

Fig.8 Accuracy and loss of different models on test set

表3 不同模型性能对比

Table 3 Performance comparison of different models %

模型 Model	准确率Precision				召回率Recall				F1得分F1 score
模型 Model	0级 Grade 0	1级 Grade 1	2级 Grade 2	3级 Grade 3	0级 Grade 0	1级 Grade 1	2级 Grade 2	3级 Grade 3	0级 Grade 0	1级 Grade 1	2级 Grade 2	3级 Grade 3
AlexNet	92.9	74.3	80.3	85.8	92.9	89.6	85.9	87.5	92.9	80.7	86.9	86.6
MobileNet	97.1	92.4	84.3	97.2	94.9	95.4	88.3	93.4	96.9	93.9	86.2	95.3
VGG16	96.6	89.5	84.0	95.3	94.9	94.8	81.5	94.0	96.8	92.1	82.7	94.7
ResNet50	97.0	89.0	73.0	95.7	96.0	89.6	81.6	88.8	97.0	89.3	77.1	92.2
SENet	96.3	91.3	72.0	97.9	85.5	96.1	87.3	94.1	91.8	93.6	84.2	91.8
改进的ResNet50	97.1	96.2	98.0	98.7	100.0	98.1	94.2	98.7	98.5	97.1	96.0	99.3
Improved ResNet50

图9 各模型的混淆矩阵比较

Fig.9 Comparison of confusion matrix of different models

参考文献 31

[1]	高江. 新疆棉种产业发展问题剖析[J]. 棉花科学, 2022, 44(2): 9-15.
	GAO J. Analysis of Xinjiang cotton seed industry development[J]. Cotton Sciences, 2022, 44(2): 9-15. (in Chinese with English abstract)
[2]	杨德松, 姬华, 王星, 等. 影响新疆棉叶螨发生因素的研究[J]. 中国棉花, 2004, 31(6): 10-11.
	YANG D S, JI H, WANG X, et al. Study on the factors affecting the occurrence of cotton spider mite in Xinjiang[J]. China Cotton, 2004, 31(6): 10-11. (in Chinese with English abstract)
[3]	GREWAL P S, OLOUMI F, RUBIN U, et al. Deep learning in ophthalmology: a review[J]. Canadian Journal of Ophthalmology, 2018, 53(4): 309-313.
[4]	KAMILARIS A, PRENAFETA-BOLDÚ F X. Deep learning in agriculture: a survey[J]. Computers and Electronics in Agriculture, 2018, 147: 70-90.
[5]	REVATHI P, HEMALATHA M. Identification of cotton diseases based on cross information gain_deep forward neural network classifier with PSO feature selection[J]. International Journal of Engineering and Technology, 2013, 5(6): 4637-4642.
[6]	BOULENT J, FOUCHER S, THÉAU J, et al. Convolutional neural networks for the automatic identification of plant diseases[J]. Frontiers in Plant Science, 2019, 10: 941.
[7]	许景辉, 邵明烨, 王一琛, 等. 基于迁移学习的卷积神经网络玉米病害图像识别[J]. 农业机械学报, 2020, 51(2): 230-236.
	XU J H, SHAO M Y, WANG Y C, et al. Recognition of corn leaf spot and rust based on transfer learning with convolutional neural network[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(2): 230-236. (in Chinese with English abstract)
[8]	吴沧海, 熊焕亮, 裘武. 基于Matlab的柑橘始叶螨边缘检测的研究[J]. 微计算机信息, 2010, 26(26): 198-199.
	WU C H, XIONG H L, QIU W. Research of yellow-spider’s edge detection based on MATLAB[J]. Microcomputer Information, 2010, 26(26): 198-199. (in Chinese with English abstract)
[9]	杨丽丽, 张大卫, 罗君, 等. 基于SVM和AdaBoost的棉叶螨危害等级识别[J]. 农业机械学报, 2019, 50(2): 14-20.
	YANG L L, ZHANG D W, LUO J, et al. Automatic recognition for cotton spider mites damage level based on SVM and AdaBoost[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(2): 14-20. (in Chinese with English abstract)
[10]	CHENG X, ZHANG Y H, CHEN Y Q, et al. Pest identification via deep residual learning in complex background[J]. Computers and Electronics in Agriculture, 2017, 141: 351-356.
[11]	张建华, 孔繁涛, 吴建寨, 等. 基于改进VGG卷积神经网络的棉花病害识别模型[J]. 中国农业大学学报, 2018, 23(11): 161-171.
	ZHANG J H, KONG F T, WU J Z, et al. Cotton disease identification model based on improved VGG convolution neural network[J]. Journal of China Agricultural University, 2018, 23(11): 161-171. (in Chinese with English abstract)
[12]	何东健, 王鹏, 牛童, 等. 基于改进残差网络的田间葡萄霜霉病病害程度分级模型[J]. 农业机械学报, 2022, 53(1): 235-243.
	HE D J, WANG P, NIU T, et al. Classification model of grape downy mildew disease degree in field based on improved residual network[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(1): 235-243. (in Chinese with English abstract)
[13]	JENIFA A, RAMALAKSHMI R, RAMACHANDRAN V. Cotton leaf disease classification using deep convolution neural network for sustainable cotton production[C]// 2019 IEEE International Conference on Clean Energy and Energy Efficient Electronics Circuit for Sustainable Development (INCCES). December 18-20, 2019. Krishnankoil, India: IEEE, 2020: 1-3.
[14]	CALDEIRA R F, SANTIAGO W E, TERUEL B. Identification of cotton leaf lesions using deep learning techniques[J]. Sensors (Basel, Switzerland), 2021, 21(9): 3169.
[15]	YANG L L, LUO J, WANG Z P, et al. Research on recognition for cotton spider mites’ damage level based on deep learning[J]. International Journal of Agricultural and Biological Engineering, 2019, 12(6): 129-134.
[16]	HUGHES D P, SALATHE M. An open access repository of images on plant health to enable the development of mobile disease diagnostics[EB/OL]. (2015-11-25) [2022-08-03]. https://arxiv.org/abs/1511.08060v2
[17]	国家质量监督检验检疫总局, 中国国家标准化管理委员会.棉花叶螨测报技术规范: GB/T 15802—2011[S]. 北京: 中国标准出版社, 2011.
[18]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016. Las Vegas, NV, USA: IEEE, 2016: 770-778.
[19]	PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[20]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327.
[21]	王卓, 王健, 王枭雄, 等. 基于改进YOLO v4的自然环境苹果轻量级检测方法[J]. 农业机械学报, 2022, 53(8): 294-302.
	WANG Z, WANG J, WANG X X, et al. Lightweight real-time apple detection method based on improved YOLO v4[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(8): 294-302. (in Chinese with English abstract)
[22]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. ( 2017-06-12) [2022-08-03]. https://arxiv.org/abs/1706.03762v4
[23]	LAROCHELLE H, HINTON G. Learning to combine foveal glimpses with a third-order Boltzmann machine[C]//Proceedings of the 23rd International Conference on Neural Information Processing Systems: Volume 1. December 6-9, 2010. Vancouver: ACM, 2010: 1243-1251.
[24]	孙萍, 胡旭东, 张永军. 结合注意力机制的深度学习图像目标检测[J]. 计算机工程与应用, 2019, 55(17): 180-184.
	SUN P, HU X D, ZHANG Y J. Object detection based on deep learning and attention mechanism[J]. Computer Engineering and Applications, 2019, 55(17): 180-184. (in Chinese with English abstract)
[25]	MNIH V, HEESS N, GRAVES A, et al. Recurrent models of visual attention[EB/OL]. ( 2014-06-25) [2022-08-03]. https://arxiv.org/abs/1406.6247
[26]	BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL]. ( 2014-09-01) [2022-08-03]. https://arxiv.org/abs/1409.0473v7
[27]	XU K, BA J, KIROS R, et al. Show, attend and tell: neural image caption generation with visual attention[EB/OL]. ( 2015-02-10) [2022-08-03]. https://arxiv.org/abs/1502.03044
[28]	RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6088): 533-536.
[29]	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[30]	FU J L, ZHENG H L, MEI T. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA: IEEE, 2017: 4476-4484.
[31]	万昊, 任勇, 山秀明. 基于混淆矩阵的全方位角雷达目标识别[J]. 微电子学与计算机, 2005, 22(3): 136-139.
	WAN H, REN Y, SHAN X M. Confusion-matrix based whole-aspect-range HRRP recognition[J]. Microelectronics & Computer, 2005, 22(3): 136-139. (in Chinese with English abstract)

基于迁移学习和改进残差网络的棉花叶螨为害等级识别

Identification of harm grades of cotton spider mites based on transfer learning and improved residual network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 31

相关文章 2

编辑推荐

Metrics

本文评价

[1]	张宁, 吴华瑞, 韩笑, 缪祎晟. 基于多尺度和注意力机制的番茄病害识别方法[J]. 浙江农业学报, 2021, 33(7): 1329-1338.
[2]	王见, 田光宝, 周勤. 基于迁移学习的棉花识别[J]. 浙江农业学报, 2020, 32(8): 1457-1465.