Disease spot segmentation and disease degree classification of grape black rot based on improved UNet++ model

doi:10.3969/j.issn.1004-1524.20221753

Acta Agriculturae Zhejiangensis ›› 2023, Vol. 35 ›› Issue (11): 2720-2730.DOI: 10.3969/j.issn.1004-1524.20221753

• Biosystems Engineering • Previous Articles Next Articles

Disease spot segmentation and disease degree classification of grape black rot based on improved UNet++ model

RU Jiaqi¹^,²^,³(), WU Bin¹, WENG Xiang⁴, XU Dayu¹, LI Yan'e¹^,²^,³^,^*()

1. School of Mathematics and Computer Science, Zhejiang A&F University, Hangzhou 311300, China
2. Key Laboratory of Forestry Intelligent Monitoring and Information Technology of Zhejiang Province, Hangzhou 311300, China
3. Key Laboratory of Forestry Sensing Technology and Intelligent Equipment, National Forestry and Grassland Administration, Hangzhou 311300, China
4. College of Optical, Mechanical and Electrical Engineering, Zhejiang A&F University, Hangzhou 311300, China

Received:2022-12-06 Online:2023-11-25 Published:2023-12-04

Abstract

Abstract:

Based on the grape black rot images from PlantVillage dataset, an improved model for disease spot segmentation based on UNet++was proposed to solve the fuzzy edge segmentation and segmentation difficulties encountered at the early disese stage. For image feature extraction, on one hand, the adaptive soft thresholding method was introduced in the proposed model to improve the edge segmentation accuracy of grape disease image by filtering the influence of noise, on the other hand, the skip connection structure of UNet++was constructed by combining long and short connections to reduce the computational complexity of the model. Multi-scale features were fused in the lateral output layer of the model to enhance the semantic information of the disease spot and further improve the segmentation accuracy. In addition, the loss function of the model was weighted by adding Dice loss function to the cross-entropy loss function, to solve the imbalance between the pixel area of the disease spot and the leaf area. Five-fold cross validation was used for the model training and test. The results showed that the pixel accuracy of the proposed model was 98.433%, the mean intersection over union was 92.056%, the intersection over union for the disease spot was 81.230%, and the Dice coefficient was 0.941, which were all superior to the traditional UNet++model. Based on the area ratio of disease spot to leaf, the disease degree was classified, and the mean accuracy of disease degree classification was 97.41%. The proposed model could accurately segment the edge of diseased spots and minor diseased spots, realize the classification of disease degree with good robustness.

Key words: grape black rot, image segmentation, adaptive soft thresholding

CLC Number:

S126
S431.9

RU Jiaqi, WU Bin, WENG Xiang, XU Dayu, LI Yan'e. Disease spot segmentation and disease degree classification of grape black rot based on improved UNet++ model[J]. Acta Agriculturae Zhejiangensis, 2023, 35(11): 2720-2730.

Figures/Tables 12

Fig.1 Original image of disease spot

Fig.2 Original image of disease spot (a) and annotation image (b)

Fig.3 Illustration of data enhancement a, Original image; b, Brightness increased by 20%; c, Brightness decreased by 10%; d, Rotated 90°; e, Rotatated 180°.

Fig.4 Structure of the improved UNet++model

Fig.5 Structure of double convolution (a), convolutional shrinkage block (b) and shrinkage layer (c) Conv(3×3) represents the convolution of 3×3 size. BN represents batch normalization. Relu represents the ReLU activation function. Absolute represents the absolute value of the feature map. GAP represents global average pooling. FC represents fully connected. Sigmoid represents Sigmoid activation function.

Fig.6 Skip connection structure of UNet++ (a) and the improved UNet++ (b)

Table 1 Effect of different combinations of weights on segmentation

α	β	背景交并比 Background IoU/%	叶片交并比 Leaf IoU/%	病斑交并比 Disease spot IoU/%	平均交并比 MIoU/%
1.0	0.5	96.826	95.807	81.101	91.245
1.0	0.8	97.844	96.827	81.230	91.967
1.0	1.0	97.414	96.387	81.663	91.821
0.8	1.0	97.846	96.833	81.488	92.056
0.5	1.0	97.785	96.772	81.301	91.953

Fig.7 Segmentation results of weighted combined loss function, Dice loss function and cross entropy (CE) loss function respectively IoU, Intersection over union.

Fig.8 Disease spot and leaf segmentation results of different models a, Original image; b, UNet model; c, Fully convolutional networks (FCN) model; d, UNet++model; e, The module of convolutional shrinkage block added to the UNet++model; f, The proposed improved UNet++model, with the module of convolutional shrinkage block and the improved skip connection added to the UNet++model.

Table 2 Comparison of segmentation results of different models based on evaluation indexes

模型 Model	像素准确率 Pixel accuracy/%	背景交并比 Background IoU/%	叶片交并比 Leaf IoU/%	病斑交并比 Disease spot IoU/%	平均交并比 MIoU/%	Dice系数 Dice coefficient
FCN	96.318	94.856	94.543	59.881	83.093	0.859
UNet	96.925	95.643	94.504	61.762	83.970	0.868
UNet++	97.144	95.964	94.650	66.716	85.777	0.886
CSB-UNet++	97.907	96.964	95.837	77.282	90.028	0.925
nCSB-UNet++	98.433	97.833	96.772	81.230	92.056	0.941

Fig.9 Example of feature maps of convolution layer a~c, Original images; d~f, Heat maps when the proposed model focuses on leaves; g~i,Heat maps when the proposed model focuses on disease spot. Red represents high contribution, blue represents low contribution, and the darker the color, the higher the contribution.

Table 3 Result of disease degree classification

病害等级 Disease level	图像数 Images number	模型正确分级的图像数 Number of images with correct classification by the proposed model	准确率 Accuracy/ %
1级 Level 1	146	144	98.63
3级 Level 3	101	99	98.02
5级 Level 5	20	17	85.00
7级 Level 7	3	3	100.00
合计Total	270	263	97.41

References 22

[1]	SUNG K T, AE P K, SUN L, et al. Application of bimodal histogram method to oil spill detection from a satellite synthetic aperture radar image[J]. Korean Journal of Remote Sensing, 2013, 29(6): 645-655.
[2]	李亚丽, 张松林, 韩杰. 基于小波系数分割的局部自适应阈值图像去噪[J]. 测绘通报, 2020(5): 43-46.
	LI Y L, ZHANG S L, HAN J. Wavelet coefficients segmentation for locally adaptive threshold image denoising method[J]. Bulletin of Surveying and Mapping, 2020(5): 43-46. (in Chinese with English abstract)
[3]	黄林彩, 王智文, 符晓彪, 等. 手势图像边缘检测: 基于多方向和最佳阈值的Canny算法[J]. 广西科技大学学报, 2022, 33(1): 71-77.
	HUANG L C, WANG Z W, FU X B, et al. An improved canny algorithm based on multi-directional and optimal threshold for gesture image edge detection[J]. Journal of Guangxi University of Science and Technology, 2022, 33(1): 71-77. (in Chinese with English abstract)
[4]	SEMMA A, HANNAD Y, SIDDIQI I, et al. Writer identification using deep learning with FAST keypoints and Harris corner detector[J]. Expert Systems With Applications, 2021, 184: 115473.
[5]	孙东辉, 鞠秀亮, 冯登超, 等. 基于FAST检测器和SURF描述子的聚合图像人脸识别[J]. 国外电子测量技术, 2016, 35(1): 94-98.
	SUN D H, JU X L, FENG D C, et al. Aggregated image face recognition based on FAST detector and SURF descriptor[J]. Foreign Electronic Measurement Technology, 2016, 35(1): 94-98. (in Chinese with English abstract)
[6]	王中元, 胡瑞敏, 章凯. 区域分割或连通分量标记分裂-合并法的一种实现[J]. 小型微型计算机系统, 2004, 25(9):1648-1651.
	WANG Z Y, HU R M, ZHANG K. Realization of split-merge algorithm in image segmentation or components labeling[J]. Mini-Micro Systems, 2004, 25(9):1648-1651. (in Chinese with English abstract)
[7]	BOYKOV Y Y, JOLLY M P. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images[C]// Eighth IEEE International Conference on Computer Vision. ICCV. July 7-14, 2001, Vancouver, BC, Canada. IEEE, 2002: 105-112.
[8]	梁计锋. 基于Snake改进型模型的胸部CT图像分割方法[J]. 自动化与仪器仪表, 2022(6): 23-26.
	LIANG J F. A segmentation method of thoracic CT images based on Snake improved model[J]. Automation & Instrumentation, 2022(6): 23-26. (in Chinese with English abstract)
[9]	BEN SLAMA A, MBARKI Z, SEDDIK H, et al. Improving parotid gland tumor segmentation and classification using geometric active contour model and deep neural network framework[J]. TS, 2021, 38(4): 955-965.
[10]	何希平, 刘波. 深度学习理论与实践[M]. 北京: 科学出版社, 2017.
[11]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[EB/OL]. (2016-05-20) [2022-12-06]. https://arxiv.org/abs/1605.06211.
[12]	赵兵, 冯全. 基于全卷积网络的葡萄病害叶片分割[J]. 南京农业大学学报, 2018, 41(4): 752-759.
	ZHAO B, FENG Q. Segmentation of grape diseases leaf based on full convolution network[J]. Journal of Nanjing Agricultural University, 2018, 41(4): 752-759. (in Chinese with English abstract)
[13]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[14]	王振, 张善文, 赵保平. 基于级联卷积神经网络的作物病害叶片分割[J]. 计算机工程与应用, 2020, 56(15): 242-250.
	WANG Z, ZHANG S W, ZHAO B P. Crop diseases leaf segmentation method based on cascade convolutional neural network[J]. Computer Engineering and Applications, 2020, 56(15): 242-250. (in Chinese with English abstract)
[15]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
[16]	赵小虎, 李晓, 叶圣, 等. 基于改进U-Net网络的多尺度番茄病害分割算法[J]. 计算机工程与应用, 2022, 58(10):216-223.
	ZHAO X H, LI X, YE S, et al. Multi-scale tomato disease segmentation algorithm based on improved U-Net network[J]. Computer Engineering and Applications, 2022, 58(10):216-223. (in Chinese with English abstract)
[17]	ZHOU Z W, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867.
[18]	BHAGAT S, KOKARE M, HASWANI V, et al. Eff-UNet++: a novel architecture for plant leaf segmentation and counting[J]. Ecological Informatics, 2022, 68: 101583.
[19]	TAN M X, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. (2019-05-28) [2022-12-06]. https://arxiv.org/abs/1905.11946.
[20]	HUANG M F, XU G Q, LI J Y, et al. A method for segmenting disease lesions of maize leaves in real time using attention YOLACT++[J]. Agriculture, 2021, 11(12): 1216.
[21]	ZHAO M H, ZHONG S S, FU X Y, et al. Deep residual shrinkage networks for fault diagnosis[J]. IEEE Transactions on Industrial Informatics, 2020, 16(7): 4681-4690.
[22]	王键. 木薯叶片病害与病指机器视觉与云服务的识别与测量[D]. 乌鲁木齐: 新疆农业大学, 2020.
	WANG J. Identification and measurement of cassava leaf disease and disease index based on machine vision and cloud service[D]. Urumqi: Xinjiang Agricultural University, 2020. (in Chinese with English abstract)

Disease spot segmentation and disease degree classification of grape black rot based on improved UNet++ model

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 22

Related Articles 5

Recommended Articles

Metrics

Comments

[1]	YUAN Jiahong1， ZHU Dequan1，2，*， SUN Bingyu3， SUN Lei1， WU Liquan2，4， SONG Yu1， JIANG Rui1. Machine vision based segmentation algorithm for rice seedling [J]. , 2016, 28(6): 1069-.
[2]	QIN Lei1，2， SUN Kai\\|qiong1，3， LI Shi\\|gao1， LIU Chun\\|tai1， RUAN Song1. Image segmentation of ripe strawberry based on RGB color similarity [J]. , 2016, 28(2): 330-.
[3]	TIAN Hai-tao, ZHAO Jun, PU Fu-peng. A method for recognizing potato’s bud eye [J]. , 2016, 28(11): 1947-1953.
[4]	HU Meng\\|han， DONG Qing\\|li， LIU Bao\\|lin. A multiple\\|threshold based image segmentation method for bananas in the crate#br# [J]. , 2015, 27(10): 1828-.
[5]	ZHANG Fang1， WANG Lu2， FU Li\\|si1，*， TIAN You\\|wen1. Segmentation method for cucumber disease leaf images under complex background [J]. , 2014, 26(5): 1346-.