Identification of harm grades of cotton spider mites based on transfer learning and improved residual network

doi:10.3969/j.issn.1004-1524.20221148

Acta Agriculturae Zhejiangensis ›› 2023, Vol. 35 ›› Issue (7): 1729-1739.DOI: 10.3969/j.issn.1004-1524.20221148

• Biosystems Engineering • Previous Articles Next Articles

Identification of harm grades of cotton spider mites based on transfer learning and improved residual network

ZHANG Yan^a^,^b(), ZHOU Baoping^a^,^*(), WANG Yu^a^,^b, FENG Jie^a^,^b, YE Fankai^a^,^b, HE Yunlong^a^,^b

a. College of Information Engineering, Tarim University, Alar 843300, Xinjiang, China
b. Key Laboratory of Modern Agricultural Engineering, Tarim University, Alar 843300, Xinjiang, China

Received:2022-08-03 Online:2023-07-25 Published:2023-08-17
Contact: ZHOU Baoping

Abstract

Abstract:

In view of the low accuracy, long time consuming and high cost of traditional artificial diagnosis of cotton spider mites, a method for detection and classification of harm grade was proposed based on transfer learning and improved residual network. The cotton leaf images either healthy or with 3 harm grades of cotton spider mites were collected both in single background and natural environment to construct image dataset. First, the PlantVillage dataset was used to pretrain the model. Data augmentation was carried out to expand training samples. Then, based on the original ResNet50 network, an improved ResNet50 network was constructed by introducing focal loss functions, embedding attention mechanism modules in different network layers, and optimizing with the Dropout regularization. Finally, the performance of the improved ResNet50 network was compared with other models. It was shown that with the attention mechanism module introduced both in the deep and shallow layers, the momentum being 0.9, and the learning rate being 0.001, the improved ResNet50 network had the best classification effect, which was superior than the original ResNet50, VGG16, MobileNet, AlexNet and SENet models, with the average recognition accuracy of 97.8%.

Key words: cotton spider mite, hazard level, ResNet50 network, transfer learning, focal loss function

CLC Number:

TP391.4

ZHANG Yan, ZHOU Baoping, WANG Yu, FENG Jie, YE Fankai, HE Yunlong. Identification of harm grades of cotton spider mites based on transfer learning and improved residual network[J]. Acta Agriculturae Zhejiangensis, 2023, 35(7): 1729-1739.

Figures/Tables 12

Fig.1 Sample images of cotton leaves with spider mites

Table 1 Quantity of cotten leaves images after augmentation

级别 Grade	图像数量Image quantity
级别 Grade	训练集 Training set	测试集 Test set	总计 Total
0	9 117	3 908	13 025
1	11 970	5 130	17 100
2	11 900	5 100	17 000
3	14 175	6 075	20 250
总计Total	47 162	2 0213	67 375

Fig.2 Examples of images after data augmentation a, Origin image; b, Gaussian blur; c, Horizontal flip; d, Rotate by 90°; e, Elastic transformation; f, Selective erasing; g, Generate rectangular region; h, Grid distortion; i, Delete rectangular areas by grid mode; j, Optical distortion.

Fig.3 Structure diagram of ResNet50 model

Fig.4 Structure diagram of improved ResNet50 model

Fig.5 Original image and feature extraction and visualization of images with cotton spider mites

Fig.6 Comparison of accuracy and loss of models under different momentum factors a,m=0.3;b,m=0.6,c,m=0.9。

Table 2 Performance comparison of models under different learning rates %

学习率 Learning rate	准确率 Precision	召回率 Recall	F1得分 F1 score
0.01	94.7	94.6	94.6
0.001	97.8	97.7	97.8
0.000 1	92.0	91.8	91.5

Fig.7 Comparison of accuracy of models under different embedding methods of attention mechanism

Fig.8 Accuracy and loss of different models on test set

Table 3 Performance comparison of different models %

模型 Model	准确率Precision				召回率Recall				F1得分F1 score
模型 Model	0级 Grade 0	1级 Grade 1	2级 Grade 2	3级 Grade 3	0级 Grade 0	1级 Grade 1	2级 Grade 2	3级 Grade 3	0级 Grade 0	1级 Grade 1	2级 Grade 2	3级 Grade 3
AlexNet	92.9	74.3	80.3	85.8	92.9	89.6	85.9	87.5	92.9	80.7	86.9	86.6
MobileNet	97.1	92.4	84.3	97.2	94.9	95.4	88.3	93.4	96.9	93.9	86.2	95.3
VGG16	96.6	89.5	84.0	95.3	94.9	94.8	81.5	94.0	96.8	92.1	82.7	94.7
ResNet50	97.0	89.0	73.0	95.7	96.0	89.6	81.6	88.8	97.0	89.3	77.1	92.2
SENet	96.3	91.3	72.0	97.9	85.5	96.1	87.3	94.1	91.8	93.6	84.2	91.8
改进的ResNet50	97.1	96.2	98.0	98.7	100.0	98.1	94.2	98.7	98.5	97.1	96.0	99.3
Improved ResNet50

Fig.9 Comparison of confusion matrix of different models

References 31

[1]	高江. 新疆棉种产业发展问题剖析[J]. 棉花科学, 2022, 44(2): 9-15.
	GAO J. Analysis of Xinjiang cotton seed industry development[J]. Cotton Sciences, 2022, 44(2): 9-15. (in Chinese with English abstract)
[2]	杨德松, 姬华, 王星, 等. 影响新疆棉叶螨发生因素的研究[J]. 中国棉花, 2004, 31(6): 10-11.
	YANG D S, JI H, WANG X, et al. Study on the factors affecting the occurrence of cotton spider mite in Xinjiang[J]. China Cotton, 2004, 31(6): 10-11. (in Chinese with English abstract)
[3]	GREWAL P S, OLOUMI F, RUBIN U, et al. Deep learning in ophthalmology: a review[J]. Canadian Journal of Ophthalmology, 2018, 53(4): 309-313.
[4]	KAMILARIS A, PRENAFETA-BOLDÚ F X. Deep learning in agriculture: a survey[J]. Computers and Electronics in Agriculture, 2018, 147: 70-90.
[5]	REVATHI P, HEMALATHA M. Identification of cotton diseases based on cross information gain_deep forward neural network classifier with PSO feature selection[J]. International Journal of Engineering and Technology, 2013, 5(6): 4637-4642.
[6]	BOULENT J, FOUCHER S, THÉAU J, et al. Convolutional neural networks for the automatic identification of plant diseases[J]. Frontiers in Plant Science, 2019, 10: 941.
[7]	许景辉, 邵明烨, 王一琛, 等. 基于迁移学习的卷积神经网络玉米病害图像识别[J]. 农业机械学报, 2020, 51(2): 230-236.
	XU J H, SHAO M Y, WANG Y C, et al. Recognition of corn leaf spot and rust based on transfer learning with convolutional neural network[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(2): 230-236. (in Chinese with English abstract)
[8]	吴沧海, 熊焕亮, 裘武. 基于Matlab的柑橘始叶螨边缘检测的研究[J]. 微计算机信息, 2010, 26(26): 198-199.
	WU C H, XIONG H L, QIU W. Research of yellow-spider’s edge detection based on MATLAB[J]. Microcomputer Information, 2010, 26(26): 198-199. (in Chinese with English abstract)
[9]	杨丽丽, 张大卫, 罗君, 等. 基于SVM和AdaBoost的棉叶螨危害等级识别[J]. 农业机械学报, 2019, 50(2): 14-20.
	YANG L L, ZHANG D W, LUO J, et al. Automatic recognition for cotton spider mites damage level based on SVM and AdaBoost[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(2): 14-20. (in Chinese with English abstract)
[10]	CHENG X, ZHANG Y H, CHEN Y Q, et al. Pest identification via deep residual learning in complex background[J]. Computers and Electronics in Agriculture, 2017, 141: 351-356.
[11]	张建华, 孔繁涛, 吴建寨, 等. 基于改进VGG卷积神经网络的棉花病害识别模型[J]. 中国农业大学学报, 2018, 23(11): 161-171.
	ZHANG J H, KONG F T, WU J Z, et al. Cotton disease identification model based on improved VGG convolution neural network[J]. Journal of China Agricultural University, 2018, 23(11): 161-171. (in Chinese with English abstract)
[12]	何东健, 王鹏, 牛童, 等. 基于改进残差网络的田间葡萄霜霉病病害程度分级模型[J]. 农业机械学报, 2022, 53(1): 235-243.
	HE D J, WANG P, NIU T, et al. Classification model of grape downy mildew disease degree in field based on improved residual network[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(1): 235-243. (in Chinese with English abstract)
[13]	JENIFA A, RAMALAKSHMI R, RAMACHANDRAN V. Cotton leaf disease classification using deep convolution neural network for sustainable cotton production[C]// 2019 IEEE International Conference on Clean Energy and Energy Efficient Electronics Circuit for Sustainable Development (INCCES). December 18-20, 2019. Krishnankoil, India: IEEE, 2020: 1-3.
[14]	CALDEIRA R F, SANTIAGO W E, TERUEL B. Identification of cotton leaf lesions using deep learning techniques[J]. Sensors (Basel, Switzerland), 2021, 21(9): 3169.
[15]	YANG L L, LUO J, WANG Z P, et al. Research on recognition for cotton spider mites’ damage level based on deep learning[J]. International Journal of Agricultural and Biological Engineering, 2019, 12(6): 129-134.
[16]	HUGHES D P, SALATHE M. An open access repository of images on plant health to enable the development of mobile disease diagnostics[EB/OL]. (2015-11-25) [2022-08-03]. https://arxiv.org/abs/1511.08060v2
[17]	国家质量监督检验检疫总局, 中国国家标准化管理委员会.棉花叶螨测报技术规范: GB/T 15802—2011[S]. 北京: 中国标准出版社, 2011.
[18]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016. Las Vegas, NV, USA: IEEE, 2016: 770-778.
[19]	PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[20]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327.
[21]	王卓, 王健, 王枭雄, 等. 基于改进YOLO v4的自然环境苹果轻量级检测方法[J]. 农业机械学报, 2022, 53(8): 294-302.
	WANG Z, WANG J, WANG X X, et al. Lightweight real-time apple detection method based on improved YOLO v4[J]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(8): 294-302. (in Chinese with English abstract)
[22]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. ( 2017-06-12) [2022-08-03]. https://arxiv.org/abs/1706.03762v4
[23]	LAROCHELLE H, HINTON G. Learning to combine foveal glimpses with a third-order Boltzmann machine[C]//Proceedings of the 23rd International Conference on Neural Information Processing Systems: Volume 1. December 6-9, 2010. Vancouver: ACM, 2010: 1243-1251.
[24]	孙萍, 胡旭东, 张永军. 结合注意力机制的深度学习图像目标检测[J]. 计算机工程与应用, 2019, 55(17): 180-184.
	SUN P, HU X D, ZHANG Y J. Object detection based on deep learning and attention mechanism[J]. Computer Engineering and Applications, 2019, 55(17): 180-184. (in Chinese with English abstract)
[25]	MNIH V, HEESS N, GRAVES A, et al. Recurrent models of visual attention[EB/OL]. ( 2014-06-25) [2022-08-03]. https://arxiv.org/abs/1406.6247
[26]	BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL]. ( 2014-09-01) [2022-08-03]. https://arxiv.org/abs/1409.0473v7
[27]	XU K, BA J, KIROS R, et al. Show, attend and tell: neural image caption generation with visual attention[EB/OL]. ( 2015-02-10) [2022-08-03]. https://arxiv.org/abs/1502.03044
[28]	RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6088): 533-536.
[29]	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[30]	FU J L, ZHENG H L, MEI T. Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA: IEEE, 2017: 4476-4484.
[31]	万昊, 任勇, 山秀明. 基于混淆矩阵的全方位角雷达目标识别[J]. 微电子学与计算机, 2005, 22(3): 136-139.
	WAN H, REN Y, SHAN X M. Confusion-matrix based whole-aspect-range HRRP recognition[J]. Microelectronics & Computer, 2005, 22(3): 136-139. (in Chinese with English abstract)

Identification of harm grades of cotton spider mites based on transfer learning and improved residual network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 31

Related Articles 2

Recommended Articles

Metrics

Comments

[1]	ZHANG Ning, WU Huarui, HAN Xiao, MIAO Yisheng. Tomato disease recognition scheme based on multi-scale and attention mechanism [J]. Acta Agriculturae Zhejiangensis, 2021, 33(7): 1329-1338.
[2]	WANG Jian, TIAN Guangbao, ZHOU Qin. Cotton recognition based on transfer learning [J]. , 2020, 32(8): 1457-1465.