YOLOv5s high-density koi fry detection method based on fusion non-local operation

doi:10.3969/j.issn.1004-1524.20230621

Acta Agriculturae Zhejiangensis ›› 2024, Vol. 36 ›› Issue (4): 952-967.DOI: 10.3969/j.issn.1004-1524.20230621

Previous Articles Next Articles

YOLOv5s high-density koi fry detection method based on fusion non-local operation

TANG Yonghua¹(), SHI Feifan¹^,^*(), LIN Sen², ZHANG Zhipeng¹, MENG Yanjun¹, LIU Xingtong¹

1. School of Information Science and Engineering, Shenyang University of Technology, Shenyang 110870, China
2. School of Automation and Electrical Engineering, Shenyang Ligong University, Shenyang 110159, China

Received:2023-05-12 Online:2024-04-25 Published:2024-04-29
Contact: SHI Feifan

Abstract

Abstract:

Aiming at the poor applicability of existing methods in the target detection task of high-density koi fry, a Ms-Non-local BIFPN coordinate attention YOLOv5s (NBC-YOLOv5s) target detection algorithm based on non-local operation is proposed. Firstly, in the backbone network of YOLOv5s, a multi-scale non-local operator (MS-Non-local) is added to enhance the feature extraction ability of the model for high-density koi fry. Secondly, the bi-directional feature pyramid network (BIFPN) is used in the neck network to improve the model feature fusion efficiency. Finally, at the feature fusion of the network, the coordinate attention (CA) mechanism is introduced to increase the model’s attention to the key information of the image. In order to verify the effectiveness of the proposed algorithm, a koi fry dataset was established based on the real fishery environment. The experimental results show that the precision, recall rate and mean average precision (mAP) of NBC-YOLOv5s are 88.5%, 89.7% and 93.7%, respectively, which are 0.6, 9.0 and 4.4 percentage points higher than the original model in the improved network compared with YOLOv5s. In order to verify the performance improvement effect of MS-Non-local on YOLOv5s, this paper compares the three mechanisms of convolutional block attention module (CBAM), squeeze and excitation (SE), and bi-level routing attention (BRA). The results showed that the mAP of MS-Non-local increased by 2.6, 2.1 and 0.9 percentage points compared with CBAM, SE and BRA, respectively. Through model disassembly, the effectiveness of the proposed method on the detection of images of koi fry of different densities is analyzed, and it is concluded that the algorithm can realize the detection of high-density koi fry in real scenarios, and can provide effective technical support for screening high-quality koi.

Key words: koi, fry detection, high-density target, YOLOv5s, Multi Scale-Non-local

CLC Number:

TP391.4

TANG Yonghua, SHI Feifan, LIN Sen, ZHANG Zhipeng, MENG Yanjun, LIU Xingtong. YOLOv5s high-density koi fry detection method based on fusion non-local operation[J]. Acta Agriculturae Zhejiangensis, 2024, 36(4): 952-967.

Figures/Tables 19

Fig.1 YOLOv5s network structure CONV, Convolution; BN, Batch normalization; CSP, Cross stage partial; SPPF, Spatial pyramid pooling-fast. The same as below.

Fig.2 NBC-YOLOv5s network structure SPP, Spatial Pyramid Pooling; MS-NL, MS-Non-local; UP, Upsample.

Fig.3 Non-local module structure T, Batch size; H, Image height; W, Image width; C, Number of image channels; X, Input feature map; Z, Output feature map.

Fig.4 Non-local Demo module structure θ, ϕ and g were three convolution kernels; A, Attention map; M1, Output feature map; M, Input feature map.

Fig.5 MS-Non-local module structure F, Input feature map; F1, Output feature map; θ, ϕ and g were three convolution kernels.

Fig.6 FPN、PAN、BIFPN structure P, Feature levels used for the final prediction; in, Input; out, Output; td, Intermediate layer.

Fig.7 CA module structure H, Image height; W, Image width; C, Number of image channels.

Fig.8 Samples of good fish

Fig.9 Samples of bad fish

Fig.10 Sample drawings of koi fry at different densities

Fig.11 Dataset analysis A, Distribution of types of koi fry; B, Distribution of target center points; C, Size distribution of koi fry.

Table 1 Performance comparison of different attention mechanisms

方法 Method	精确率 Precision/%	召回率 Recall/%	平均精度均值 mAP/%	占用内存 Memory/Mb	检测速度 Speed/ms
YOLOv5s+CBAM	86.3	85.1	89.5	28.2	35.1
YOLOv5s+SE	88.0	86.2	90.0	23.5	36.0
YOLOv5s+BRA	87.8	86.8	91.2	181.0	41.1
YOLOv5s+MS-Nonlocal	88.2	87.1	92.1	167.0	40.2

Table 2 Comparison of the performance of different algorithms

方法 Method	精确率 Precision/%	召回率 Recall/%	平均精度均值 mAP/%	占用内存 Memory/Mb	检测速度 Speed/ms
SSD	68.3	68.2	76.3	189	98
Faster Rcnn	65.1	60.0	58.1	186	191
YOLOv4	86.5	78.5	81.5	246	41
YOLOv5s	87.9	80.7	89.3	18	25
NBC-YOLOv5s	88.5	89.7	93.7	213	35

Fig.12 Detection results of different algorithms

Table 3 Comparison of leading-edge algorithm performance

方法 Method	精确率 Precision/%	召回率 Recall/%	平均精度均值 mAP/%	占用内存 Memory/Mb	检测速度 Speed/ms
YOLOv6s	95.5	87.0	93.60	40	28
YOLOv7	82.1	90.2	93.06	298	13
NBC-YOLOv5s	88.5	89.7	93.70	213	35

Fig.13 Comparison of algorithm detection results

Table 4 Results of ablation experiment

方法 Method	添加CA注意力 Add CA	修改特征金字塔 Modify BIFPN	添加MS-Non-local Add MS-Non-local	平均精度均值 mAP/%	检测速度 Speed/ms
YOLOv5s	×	×	×	89.3	25
1	√	×	×	90.1	26
2	×	√	×	89.7	26
3	×	×	√	92.1	40
4	√	×	√	91.3	32
5	√	√	×	91.0	27
6	×	√	√	92.3	33
NBC-YOLOv5s	√	√	√	93.7	35

Table 5 Performance comparison of improved models in high-density koi fry %

方法 Method	精确率 Precision	召回率 Recall	平均精度均值 mAP
YOLOv5s	84.8	77.9	84.6
1	83.4	85.7	89.4
2	87.2	86.1	90.7
3	85.7	86.3	90.8
4	88.1	85.1	90.3
5	90.2	85.8	91.7
6	89.4	84.6	91.2
NBC-YOLOv5s	90.9	87.6	92.2

Fig.14 Comparison of ablation experimental results of different densities

References 34

[1]	陈俊杰, 李媛瑗, 阳瑞雪, 等. 四川地区一株锦鲤疱疹病毒的分离鉴定及系统进化分析[J]. 水产学报, 2018, 42(1): 141-148.
	CHEN J J, LI Y Y, YANG R X, et al. Isolation and identification of a koi herpesvirus and phylogenetic analysis from Sichuan Province[J]. Journal of Fisheries of China, 2018, 42(1): 141-148. (in Chinese with English abstract)
[2]	康慧敏, 何嘉乐, 刘晨恺, 等. 四川地区养殖锦鲤浮肿病毒检测与系统进化分析[J]. 浙江农业学报, 2019, 31(4): 539-544.
	KANG H M, HE J L, LIU C K, et al. Detection and phylogenetic analysis of carp edema virus (CEV) in farmed koi carp in Sichuan Province[J]. Acta Agriculturae Zhejiangensis, 2019, 31(4): 539-544. (in Chinese with English abstract)
[3]	杨志强, 李潇轩, 殷悦, 等. 锦鲤苗种筛选技术及成鱼鉴赏标准[J]. 水产养殖, 2022, 43(9): 49-50.
	YANG Z Q, LI X X, YIN Y, et al. Screening technology of fry and appreciation standard of adult fish in koi fish[J]. Journal of Aquaculture, 2022, 43(9): 49-50. (in Chinese)
[4]	GRAY P C, FLEISHMAN A B, KLEIN D J, et al. A convolutional neural network for detecting sea turtles in drone imagery[J]. Methods in Ecology and Evolution, 2019, 10(3): 345-355.
[5]	陈明, 张重阳, 冯国富, 等. 基于特征加权融合的鱼类摄食活动强度评估方法[J]. 农业机械学报, 2020, 51(2): 245-253.
	CHEN M, ZHANG C Y, FENG G F, et al. Intensity assessment method of fish feeding activities based on feature weighted fusion[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(2): 245-253. (in Chinese with English abstract)
[6]	张明华, 龙腾, 宋巍, 等. 一种水下鱼类动态视觉序列运动目标检测方法[J]. 图学学报, 2021, 42(1): 52-58.
	ZHANG M H, LONG T, SONG W, et al. Method for moving object detection of underwater fish using dynamic video sequence[J]. Journal of Graphics, 2021, 42(1): 52-58. (in Chinese with English abstract)
[7]	TANAKA Y, YAMABE S, FUKAE K, et al. Quantification of fish behavior using optical flow[C]//2022 IEEE 11th Global Conference on Consumer Electronics. October 18-21, 2022.: Osaka, Japan. IEEE, 2022: 125-126.
[8]	SALMAN A, MAQBOOL S, KHAN A H, et al. Real-time fish detection in complex backgrounds using probabilistic background modelling[J]. Ecological Informatics, 2019, 51: 44-51.
[9]	THARWAT A, HEMEDAN A A, HASSANIEN A E, et al. A biometric-based model for fish species classification[J]. Fisheries Research, 2018, 204: 324-336.
[10]	樊闯, 赵子皓, 张雪松, 等. 基于BP神经网络的一季稻发育期预测模型[J]. 浙江农业学报, 2023, 35(2): 434-444.
	FAN C, ZHAO Z H, ZHANG X S, et al. Prediction model of one season rice development period based on BP neural network[J]. Acta Agriculturae Zhejiangensis, 2023, 35(2): 434-444. (in Chinese)
[11]	李斌, 刘东阳, 时国龙, 等. 基于改进YOLOv4模型的群养生猪姿态检测[J]. 浙江农业学报, 2023, 35(1): 215-225.
	LI B, LIU D Y, SHI G L, et al. Pig posture detection based on improved YOLOv4 model[J]. Acta Agriculturae Zhejiangensis, 2023, 35(1): 215-225. (in Chinese with English abstract)
[12]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016. Las Vegas, NV, USA.: IEEE, 2016: 770-778.
[13]	刘雨青, 王亚茹, 黄璐瑶. 基于FML-Centernet算法的鱼类识别检测[J]. 激光与光电子学进展, 2022, 59(16): 1615002.
	LIU Y Q, WANG Y R, HUANG L Y. Fish recognition and detection based on FML-centernet algorithm[J]. Laser & Optoelectronics Progress, 2022, 59(16): 1615002. (in Chinese with English abstract)
[14]	吴立栋, 彭宗举, 李欣, 等. 基于样本迭代融合的海洋生物检测[J]. 激光与光电子学进展, 2023, 60(2): 3788/LOP212567.
	WU L D, PENG Z J, LI X, et al. Marine creature detection based on sample iterative fusion[J]. Laser & Optoelectronics Progress, 2023, 60(2): 3788/LOP212567. (in Chinese with English abstract)
[15]	杨文静, 陈明, 冯国富. 基于图像增强的水下视频鱼类识别方法[J]. 激光与光电子学进展, 2021, 58(22): 2211002.
	YANG W J, CHEN M, FENG G F. Fish recognition method for underwater video based on image enhancement[J]. Laser & Optoelectronics Progress, 2021, 58(22): 2211002. (in Chinese with English abstract)
[16]	SALMAN A, AHMAD SIDDIQUI S, SHAFAIT F, et al. Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system[J]. ICES Journal of Marine Science, 2020, 77(4): 1295-1307.
[17]	ZHAO Z X, LIU Y, SUN X D, et al. Composited FishNet: fish detection and species recognition from low-quality underwater videos[J]. IEEE Transactions on Image Processing, 2021, 30: 4719-4734.
[18]	FAROOQ M, HAFEEZ A. Covid-resnet: a deep learning framework for screening of covid19 from radiographs[EB/OL]. (2020-05-31) [2023-05-10].https://arxiv.org/abs/2003.14395.
[19]	WU W X, QI Z A, LI F X. PointConv: deep convolutional networks on 3D point clouds[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 15-20, 2019. Long Beach, CA, USA. IEEE, 2019: 9621-9630.
[20]	ALBAHLI S, NAWAZ M, JAVED A, et al. An improved faster-RCNN model for handwritten character recognition[J]. Arabian Journal for Science and Engineering, 2021, 46(9): 8509-8523.
[21]	KASPER-EULAERS M, HAHN N, BERGER S, et al. Short communication: detecting heavy goods vehicles in rest areas in winter conditions using YOLOv5[J]. Algorithms, 2021, 14(4): 114.
[22]	WANG C Y, MARK LIAO H Y, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. June 14-19, 2020. Seattle, WA, USA. IEEE, 2020: 390-391.
[23]	WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT, USA. IEEE, 2018: 7794-7803.
[24]	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. July 21-26, 2017. Honolulu, HI. IEEE, 2017: 2117-2125.
[25]	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT. IEEE, 2018: 8759-8768.
[26]	TAN M X, PANG R M, LE Q V. EfficientDet: scalable and efficient object detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 13-19, 2020. Seattle, WA, USA. IEEE, 2020: 10781-10790.
[27]	SINHA D, EL-SHARKAWY M. Thin MobileNet: an enhanced MobileNet architecture[C]//2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference. October 10-12, 2019. New York City, NY, USA. IEEE, 2019: 0280-0285.
[28]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 20-25, 2021. Nashville, TN, USA. IEEE, 2021: 13713-13722.
[29]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT. IEEE, 2018: 7132-7141.
[30]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]// Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[31]	ZHU L, WANG X J, KE Z H, et al. BiFormer: vision transformer with Bi-level routing attention[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 17-24, 2023. Vancouver, BC, Canada. IEEE, 2023: 10323-10333.
[32]	LI C Y, LI L, JIANG H L, et al. YOLOv6: a single-stage object detection framework for industrial applications[EB/OL]. ( 2022-09-27[2023-05-10].https://arxiv.org/abs/2209.02976.
[33]	WANG C Y A. BOCHKOVSKIY A, H Y M LIAO. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, IEEE, 2023: 7464-7475.
[34]	DING X H, ZHANG X Y, MA N N, et al. RepVGG: making VGG-style ConvNets great again[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 20-25, 2021. Nashville, TN, USA. IEEE, 2021: 13733-13742.

YOLOv5s high-density koi fry detection method based on fusion non-local operation

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 19

References 34

Related Articles 1

Recommended Articles

Metrics

Comments