基于改进U-Net网络的苹果叶部病害语义分割方法

doi:10.3969/j.issn.1004-1524.20221445

浙江农业学报 ›› 2023, Vol. 35 ›› Issue (11): 2731-2741.DOI: 10.3969/j.issn.1004-1524.20221445

基于改进U-Net网络的苹果叶部病害语义分割方法

王英允¹^,²^,³(), 龙燕¹^,²^,³^,^*(), 杨智优¹^,², 黄铝文²^,³^,⁴

1.西北农林科技大学机械与电子工程学院,陕西杨凌 712100
2.农业农村部农业物联网重点实验室,陕西杨凌712100
3.陕西省农业信息感知与智能服务重点试验室,陕西杨凌 712100
4.西北农林科技大学信息工程学院,陕西杨凌712100

收稿日期:2022-10-09 出版日期:2023-11-25 发布日期:2023-12-04
作者简介:王英允(1996—),男,陕西西安人,硕士,研究方向为农业电子与自动化技术。E-mail:yingyun@nwafu.edu.cn
通讯作者: * 龙燕,E-mail:longyan@nwsuaf.edu.cn
基金资助:
陕西省重点研发计划(2020NY-144)

Semantic segmentation method of apple leaf disease based on improved U-Net network

WANG Yingyun¹^,²^,³(), LONG Yan¹^,²^,³^,^*(), YANG Zhiyou¹^,², HUANG Lyuwen²^,³^,⁴

1. College of Mechanical and Electronic Engineering, Northwest A & F University, Yangling 712100, Shaanxi, China
2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling 712100, Shaanxi, China
3. Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Service, Yangling 712100, Shaanxi, China
4. College of Information Engineering, Northwest A&F University, Yangling 712100, Shaanxi, China

Received:2022-10-09 Online:2023-11-25 Published:2023-12-04

摘要/Abstract

摘要：

针对自然条件下苹果叶部病斑分割与识别效果欠佳的问题,本文提出一种融合条件随机场和卷积块状注意力模块的苹果叶部病害语义分割模型,实现苹果叶部锈病、褐斑病、灰斑病及斑点落叶病的病斑准确分割和识别。本文在U-Net模型基础上,使用ResNet50为骨干网络防止梯度消失问题,并分别在跳跃连接分支与上采样层加入卷积块状注意力模块,减少训练过程中的分割精度损失,融合Dice Loss和Focal Loss降低损失波动,最后利用条件随机场优化分割结果,获取病斑掩模图像,实现对苹果叶部病害语义分割。本研究在自制苹果叶部病害数据集上进行试验,分析了光照、阴影及水滴等因素对分割结果的影响。试验结果表明:本文构建的语义分割模型相比传统U-Net模型,平均分割精度(mIoU)提升8.24百分点,平均分类精度(mPrecision)提升11百分点,类别平均像素准确率(mPA)提升6.09百分点,受光照不均、雨滴的影响更小,具有更好的鲁棒性和可靠性。

关键词: 病害分割, 注意力机制, 条件随机场, 深度语义分割

Abstract:

Aiming at the problem of poor segmentation and recognition of apple leaf spots under natural conditions, this paper proposed a semantic segmentation model for apple leaf diseases that incorporated conditional random fields and convolutional block attention modules to achieve accurate segmentation and recognition of spots of apple leaf rust, brown spot, gray spot and Alternaria leaf spot disease. In this paper, based on the U-Net model, ResNet50 was used as the backbone network to prevent the gradient vanishing problem, and the convolutional block attention module was added to the jump-connected branch and the up-sampling layer respectively, to reduce the loss of segmentation accuracy during the training process, and the fusion of dice loss and the cross-entropy loss function to reduce the loss fluctuation, and finally, the segmentation results were optimized using the conditional random field to obtain the diseased spot mask image, which was used to realize the accurate segmentation and recognition of apple leaf rust, brown spot, grey spot and Alternaria leaf spot disease so as to realize semantic segmentation of apple leaf diseases. In this study, we conducted experiments on the homemade apple leaf disease dataset, and analyzed the effects of light, shadow and water droplets on the segmentation results. The experimental results showed that the semantic segmentation model constructed in this paper improved the average segmentation accuracy mIoU by 8.24 percentage points, the average classification accuracy mPrecision by 11 percentage points, and the average pixel accuracy of category mPA by 6.09 percentage points compared with the traditional U-Net model, and was less affected by uneven illumination and raindrops, and had better robustness and reliability.

Key words: disease segmentation, attention mechanism, conditional random field, deep semantic segmentation

中图分类号:

S661.1
S126

王英允, 龙燕, 杨智优, 黄铝文. 基于改进U-Net网络的苹果叶部病害语义分割方法[J]. 浙江农业学报, 2023, 35(11): 2731-2741.

WANG Yingyun, LONG Yan, YANG Zhiyou, HUANG Lyuwen. Semantic segmentation method of apple leaf disease based on improved U-Net network[J]. Acta Agriculturae Zhejiangensis, 2023, 35(11): 2731-2741.

图/表 11

图1 U-Net网络结构图

Fig.1 Network structure of U-Net

图2 改进U-Net网络结构图

Fig.2 Improved U-Net network structure diagram

图3 CBAM注意力机制网络结构图

Fig.3 CBAM attention mechanism network structure diagram

表1 数据集参数

Table 1 Dataset parameters

分类 Classifi- cation	灰斑病/张 Gray spot disease	褐斑病/张 Brown spot disease	锈病/张 Rust disease	斑点落叶病/张 Alternaria leaf spot disease
室内采集	165	301	400	127
Indoor collection
室外采集	191	74	60	284
Outdoor collection
合计Total	356	375	460	411

图4 不同病斑大小分割结果对比

Fig.4 Comparison of segmentation results of different lesion sizes

图5 复杂环境分割结果对比

Fig.5 Comparison of segmentation results in complex environments

表2 不同网络模型分割评价指标对比表

Table 2 Comparison table of segmentation evaluation indicators of different network models %

网络模型 Network model	mIoU	mPA
PSPNet	50.76	57.68
DeepLabV3+	58.20	62.06
U-Net	77.68	86.16
U-Net-ResNet	78.53	88.17
U-Net-ResNet-CRF	82.02	89.17
本文方法The method in this study	85.92	92.25

表3 不同网络不同病害的mIoU对比表

Table 3 Comparison table of mIoU of different networks and different diseases %

网络模型 Network model	锈病 Rust disease	褐斑病 Brown spot disease	灰斑病 Gray spot disease	斑点落叶病 Alternaria leaf spot disease
PSPNet	65	62	20	10
DeepLabV3+	73	74	29	15
U-Net	78	80	69	62
U-Net-ResNet	80	81	71	60
U-Net-ResNet-CRF	83	83	73	72
本文方法	86	86	80	78
The method in this study

图6 不同网络模型混淆矩阵图

Fig.6 Confusion matrix diagram of different network models

表4 不同网络模型分类评价指标对比表

Table 4 Comparison table of classification evaluation indicators of different network models %

网络模型 Network model	PSPNet	DeepLabV3+	U-Net	U-Net-ResNet	U-Net-ResNet-CRF	本文方法 The method in this study
锈病Rust disease	75	80	83	84	89	91
褐斑病Brown spot disease	78	85	85	85	90	92
灰斑病Gray spot disease	71	60	80	84	89	91
斑点落叶病Alternaria leaf spot disease	45	50	69	72	84	87
平均分类精度mPrecision	67.25	68.75	79.25	81.25	88	90.25

图7 损失函数对比图 a,传统U-Net损失函数图;b,融合后的损失函数图

Fig.7 Loss function comparison diagram a, Diagram of traditional U-Net loss function; b, The fused loss function graph

参考文献 21

[1]	KHAN A I, QUADRI S M K, BANDAY S, et al. Deep diagnosis: a real-time apple leaf disease detection system based on deep learning[J]. Computers and Electronics in Agriculture, 2022, 198: 107093.
[2]	翟肇裕, 曹益飞, 徐焕良, 等. 农作物病虫害识别关键技术研究综述[J]. 农业机械学报, 2021, 52(7): 1-18.
	ZHAI Z Y, CAO Y F, XU H L, et al. Review of key techniques for crop disease and pest detection[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(7): 1-18. (in Chinese with English abstract)
[3]	刘文波, 叶涛, 李颀. 基于改进SOLO v2的番茄叶部病害检测方法[J]. 农业机械学报, 2021, 52(8): 213-220.
	LIU W B, YE T, LI Q. Tomato leaf disease detection method based on improved SOLO v2[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(8): 213-220. (in Chinese with English abstract)
[4]	MATTIHALLI C, GARED F, GETNET L. Automatic plant leaf disease detection and auto-medicine using IoT technology[M]//KRAUSE P, XHAFA F. IoT-based Intelligent Modelling for Environmental and Ecological Engineering. Cham: Springer, 2021: 257-273.
[5]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[6]	ZHAO W, ZHANG H D, YAN Y J, et al. A semantic segmentation algorithm using FCN with combination of BSLIC[J]. Applied Sciences, 2018, 8(4): 500.
[7]	任守纲, 贾馥玮, 顾兴健, 等. 反卷积引导的番茄叶部病害识别及病斑分割模型[J]. 农业工程学报, 2020, 36(12): 186-195.
	REN S G, JIA F W, GU X J, et al. Recognition and segmentation model of tomato leaf diseases based on deconvolution-guiding[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(12): 186-195. (in Chinese with English abstract)
[8]	QI X K, DONG J S, LAN Y B, et al. Method for identifying Litchi picking position based on YOLOv5 and PSPNet[J]. Remote Sensing, 2022, 14(9): 2004.
[9]	戴雨舒, 仲晓春, 孙成明, 等. 基于图像处理和Deeplabv3+模型的小麦赤霉病识别[J]. 中国农机化学报, 2021, 42(9): 209-215.
	DAI Y S, ZHONG X C, SUN C M, et al. Identification of fusarium head blight in wheat-based on image processing and Deeplabv3+model[J]. Journal of Chinese Agricultural Mechanization, 2021, 42(9): 209-215. (in Chinese with English abstract)
[10]	李昊, 刘海隆, 刘生龙. 基于深度学习的柑橘病虫害动态识别系统研发[J]. 中国农机化学报, 2021, 42(9): 195-201, 208.
	LI H, LIU H L, LIU S L. Research on dynamic identification system of citrus diseases and pests based on deep learning[J]. Journal of Chinese Agricultural Mechanization, 2021, 42(9): 195-201, 208. (in Chinese with English abstract)
[11]	SLADOJEVIC S, ARSENOVIC M, ANDERLA A, et al. Deep neural networks based recognition of plant diseases by leaf image classification[J]. Computational Intelligence and Neuroscience, 2016, 2016: 1-11.
[12]	DIVYANTH L G, AHMAD A, SARASWAT D. A two-stage deep-learning based segmentation model for crop disease quantification based on corn field imagery[J]. Smart Agricultural Technology, 2023, 3: 100108.
[13]	黄林生, 邵松, 卢宪菊, 等. 基于卷积神经网络的生菜多光谱图像分割与配准[J]. 农业机械学报, 2021, 52(9): 186-194.
	HUANG L S, SHAO S, LU X J, et al. Segmentation and registration of lettuce multispectral image based on convolutional neural network[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(9): 186-194. (in Chinese with English abstract)
[14]	CHEN X E, LIAN Y C, JIAO L C, et al. Supervised edge attention network for accurate image instance segmentation[C]// VEDALDI A, BISCHOF H, BROX T, et al. European Conference on Computer Vision, Cham: Springer, 2020: 617-631.
[15]	YU H L, MEN Z B, BI C G, et al. Research on field soybean weed identification based on an improved UNet model combined with a channel attention mechanism[J]. Frontiers in Plant Science, 2022, 13: 890051.
[16]	饶秀勤, 朱逸航, 张延宁, 等. 基于语义分割的作物垄间导航路径识别[J]. 农业工程学报, 2021, 37(20): 179-186.
	RAO X Q, ZHU Y H, ZHANG Y N, et al. Navigation path recognition between crop ridges based on semantic segmentation[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(20): 179-186. (in Chinese with English abstract)
[17]	陈燕, 朱成宇, 胡小春, 等. 基于改进Unet的小麦茎秆截面参数检测[J]. 农业机械学报, 2021, 52(7): 169-176.
	CHEN Y, ZHU C Y, HU X C, et al. Detection of wheat stem section parameters based on improved unet[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(7): 169-176. (in Chinese with English abstract)
[18]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[19]	CHEN S, SEDGHI GAMECHI Z, DUBOST F, et al. An end-to-end approach to segmentation in medical images with CNN and posterior-CRF[J]. Medical Image Analysis, 2022, 76: 102311.
[20]	YEUNG M, SALA E, SCHÖNLIEB C B, et al. Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation[J]. Computerized Medical Imaging and Graphics, 2022, 95: 102026.
[21]	LI M C, CHEN D L, LIU S X. Weakly supervised segmentation loss based on graph cuts and superpixel algorithm[J]. Neural Processing Letters, 2022, 54(3): 2339-2362.

基于改进U-Net网络的苹果叶部病害语义分割方法

Semantic segmentation method of apple leaf disease based on improved U-Net network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 21

相关文章 3

编辑推荐

Metrics

本文评价

[1]	李颀, 李煜哲. 弱光条件下猪舍清洗目标检测[J]. 浙江农业学报, 2023, 35(9): 2240-2249.
[2]	周品志, 裴悦琨, 魏冉, 张永飞, 谷宇. 基于YOLOV4模型的果园樱桃实时检测研究[J]. 浙江农业学报, 2022, 34(11): 2522-2532.
[3]	张宁, 吴华瑞, 韩笑, 缪祎晟. 基于多尺度和注意力机制的番茄病害识别方法[J]. 浙江农业学报, 2021, 33(7): 1329-1338.