融合图像增强和迁移学习的YOLOv8n夜间苹果检测方法
CSTR:
作者:
作者单位:

1.湖北工业大学农机工程研究设计院,武汉 430068;2.湖北省农机装备智能化工程技术研究中心,武汉 430068

作者简介:

仝召茂,E-mail:tzhaomao@163.com

通讯作者:

陈学海,E-mail:cxh@hbut.edu.cn

中图分类号:

TP391.4

基金项目:

湖北省科技创新人才计划项目(2023DJC088);湖北省农机装备补短板核心技术应用攻关项目(HBSNYT202208)


A method for detecting apple at night based on YOLOv8n with fusion of image enhancement and transfer learning
Author:
Affiliation:

1.Institute of Agricultural Machinery, Hubei University of Technology, Wuhan 430068, China;2.Hubei Engineering Research Center for Intellectualization of Agricultural Equipment, Wuhan 430068, China

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献 [22]
  • | | | |
  • 文章评论
    摘要:

    针对夜间场景下苹果识别率低、实时性差的问题,提出一种融合图像增强和迁移学习的YOLOv8n夜间苹果检测方法。首先,在YOLOv8n前端嵌入Zero-DCE模块增强夜间图像,更清晰地呈现苹果的轮廓和细节,降低夜间苹果图像的识别难度;其次,使用SPD-Conv进行下采样,增强模型细粒度特征的提取能力;在此基础上,针对夜间苹果数据集样本量少的问题,采用迁移学习训练策略,选取含有苹果类别的MS COCO数据集作为源域数据集,对于夜间场景下的目标域数据集,利用Zero-DCE增加其与日间苹果图像的相似度并在源域模型上微调目标域模型。基于上述方法,在夜间苹果图像数据集上进行了试验,结果显示,所提方法的模型精确率P为97.0%、召回率R为93.4%、平均精度均值mAP@0.5:0.95为74.6%,较YOLOv8n原始模型分别提升2.3、1.9和4.3百分点,同时该模型的推理速度为22 帧/s,可以满足实时性要求。消融试验显示,图像增强与迁移学习结合使用的效果超过两者单独使用时的效果之和。研究表明,改进后的模型在处理重叠、遮挡、绿果和光线过暗等复杂情形时都比原始模型表现更优,具有良好的鲁棒性。

    Abstract:

    This paper proposed a method for detecting apple at night based on YOLOv8n with fusion of image enhancement and transfer learning to address the issues of low recognition rate and poor real-time performance of apples in nighttime scenarios. Firstly, embedding a Zero-DCE module in the front-end of YOLOv8n enhanced images of apple at night, presented the contours and details of apples more clearly, and reduced the difficulty of recognizing images of apple at night. Secondly, using SPD-Conv for down-sampling enhanced the ability of the model to extract fine-grained features. On this basis, transfer learning training strategy was used to solve the problem of small sample size in the dataset of apple at night. The MS COCO dataset containing categories of apple was selected as the source domain dataset. In term of the target domain dataset in nighttime scenarios, Zero-DCE was used to increase its similarity with images of apple during the day and finely tune the model of target domain on the model of source domain. Experiments were conducted on the image dataset of apple at night based on the method above. The results showed that the model accuracy P, a recall R, and an average accuracy mean mAP@0.5:0.95 of method proposed was 97.0%,93.4% and 74.6%, being 2.3,1.9, and 4.3 percentages higher than that of the YOLOv8n original model. The inference speed of this model was 22 frames/s, meeting requirements of real-time detection. The results of the ablation experiment showed that the combined effect of image enhancement and transfer learning exceeded the sum of the effects when applied separately. The improved model performed better than the original model in dealing with complex situations including overlap, occlusion, green fruits, and dim lighting, and had good robustness.

    表 1 消融试验结果Table 1 Results of ablation test
    图1 数据标注结果示例Fig.1 Example of data annotation results
    图2 数据增强结果示例Fig.2 Example of data augmentation results
    图3 Zero-DCE的框架Fig.3 The framework of Zero-DCE
    图4 步长为2与步长为1的卷积核采样特点(以列为例)Fig.4 Sampling characteristics of convolutional kernels with steps of 2 and 1(using columns as an example)
    图5 SPD-Conv的操作过程Fig.5 Operation process of SPD-Conv
    图6 改进的网络结构Fig.6 Improved network structure
    图7 本研究方法迁移学习过程Fig.7 Transfer learning process of the method in this article
    图8 基线模型和改进模型训练过程中在测试集上的mAP@0.5:0.95变化Fig.8 Changes in mAP@0.5:0.95 on the test set during training of baseline and improved model
    图9 基线模型和本研究模型的复杂场景检测示例Fig.9 Examples of complex scene detection using baseline and improved model
    表 2 鲁棒性试验结果Table 2 Robustness test results
    参考文献
    [1] 李会宾,史云.果园采摘机器人研究综述[J].中国农业信息,2019,31(6):1-9.LI H B,SHI Y.Review on orchard harvesting robots[J].China agricultural informatics,2019,31(6):1-9 (in Chinese with English abstract).
    [2] 中国苹果产业协会,中国农业大学农业规划设计院.2021年度中国苹果产业发展报告(总篇)精简版[J].中国果菜,2023,43(1):1-8.China Apple Industry Association,China Agricultural University Agricultural Planning and Design Institute.China apple industry development report in 2021(general),condensed versio N[J].China fruit & vegetable,2023,43(1):1-8(in Chinese with English abstract).
    [3] 宋怀波,尚钰莹,何东健.果实目标深度学习识别技术研究进展[J].农业机械学报,2023,54(1):1-19.SONG H B,SHANG Y Y,HE D J.Review on deep learning technology for fruit target recognition[J].Transactions of the CSAM,2023,54(1):1-19(in Chinese with English abstract).
    [4] WEI X Q,JIA K,LAN J H,et al.Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot[J].Optik,2014,125(19):5684-5689.
    [5] 周桂红,马帅,梁芳芳.基于改进YOLOv4模型的全景图像苹果识别[J].农业工程学报,2022,38(21):159-168.ZHOU G H,MA S,LIANG F F.Recognition of the apple in panoramic images based on improved YOLOv4 model[J].Transactions of the CSAE,2022,38(21):159-168(in Chinese with English abstract).
    [6] 袁紫薇.基于机器视觉的番茄收割机实时分拣系统研究[D].西安:长安大学,2017.YUAN Z W.Research on real-time sorting system of tomato harvester based on machine vision[D].Xi’an:Changan University,2017(in Chinese with English abstract).
    [7] 岳琳茜,李文宽,杨晓峰,等.基于改进YOLOv4的苹果检测与果径估测方法[J].激光杂志,2022,43(2):58-65.YUE L X,LI W K,YANG X F,et al.Apple detection and fruit diameter estimation method based on improved YOLOv4[J].Laser journal,2022,43(2):58-65(in Chinese with English abstract).
    [8] 陈青,殷程凯,郭自良,等.苹果采摘机器人关键技术研究现状与发展趋势[J].农业工程学报,2023,39(4):1-15.CHEN Q,YIN C K,GUO Z L,et al.Current status and future development of the key technologies for apple picking robots[J].Transactions of the CSAE,2023,39(4):1-15(in Chinese with English abstract).
    [9] FONT D,PALLEJà T,TRESANCHEZ M,et al.Counting red grapes in vineyards by detecting specular spherical reflection peaks in RGB images obtained at night with artificial illumination[J].Computers and electronics in agriculture,2014,108:105-111.
    [10] 赵德安,刘晓洋,陈玉,等.苹果采摘机器人夜间识别方法[J].农业机械学报,2015,46(3):15-22.ZHAO D A,LIU X Y,CHEN Y,et al.Image recognition at night for apple picking robot[J].Transactions of the CSAM,2015,46(3):15-22(in Chinese with English abstract).
    [11] 戴家裕.苹果采摘机器人夜间青苹果识别方法[D].杭州:浙江工业大学,2020.DAI J Y.Method for identify green apples at night by apple pic robot[D].Hangzhou:Zhejiang University of Technology,2020(in Chinese with English abstract).
    [12] REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:unified,real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).June 27-30,2016.Las Vegas,NV,USA.Las Vegas:IEEE,2016:779-788.
    [13] 熊俊涛,郑镇辉,梁嘉恩,等.基于改进YOLO v3网络的夜间环境柑橘识别方法[J].农业机械学报,2020,51(4):199-206.XIONG J T,ZHENG Z H, LIANG J E,et al.Citrus detection method in night environment based on improved YOLO v3 network[J].Transactions of the CSAM,2020,51(4):199-206(in Chinese with English abstract).
    [14] HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely connected convolutional networks[DB/OL].arXiv,2016:1608.06993[2023-12-01].http://arxiv.org/abs/1608.06993v5.
    [15] 孙宝霞,梁翠晓,刘凯,等.夜间环境下树上柑橘表征缺陷深度学习检测方法[J].林业工程学报,2021,6(6):148-155.SUN B X,LIANG C X,LIU K,et al.Fluorescence detection of citrus characterization defects using SVM[J].Journal of forestry engineering,2021,6(6):148-155(in Chinese with English abstract).
    [16] 何斌,张亦博,龚健林,等.基于改进YOLOv5的夜间温室番茄果实快速识别[J].农业机械学报,2022,53(5):201-208.HE B,ZHANG Y B,GONG J L,et al.Fast recognition of tomato fruit in greenhouse at night based on improved YOLOv5[J].Transactions of the CSAM,2022,53(5):201-208(in Chinese with English abstract).
    [17] LIN T Y,MARIRE M,BELONGIE S,et al.Microsoft COCO:Common objects in context[C]//Computer Vision-EC CV 2014 Workshops.Zurich,Switzerland,September 6-7 and 12,2014.Cham:Springer,2014.
    [18] GUO C L,LI C Y,GUO J C,et al.Zero-reference deep curve estimation for low-light image enhancement[DB/OL].arXiv,2020:2001.06826[2023-12-01].https://doi.org/10.48550/ arXiv.2001.06826
    [19] SUNKARA R,LUO T.No more strided convolutions or pooling:a new CNN building block for low-resolution images and small objects[DB/OL].arXiv,2022:2208.03641[2023-12-01].https://doi.org/10.48550/arXiv.2208.03641.
    [20] 庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报,2015,26(1):26-39.ZHUANG F Z,LUO P,HE Q,et al.Survey on transfer learning research[J].Journal of software,2015,26(1):26-39(in Chinese with English abstract).
    [21] PAN S J,YANG Q.A survey on transfer learning[J].IEEE transactions on knowledge and data engineering,2010,22(10):1345-1359.
    [22] EVERINGHAM M,ALI ESLAMI S M,VAN GOOL L,et al.The pascal visual object classes challenge:a retrospective[J].International journal of computer vision,2015,111(1):98-136.
    相似文献
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

仝召茂,陈学海,马志艳,杨光友,张灿.融合图像增强和迁移学习的YOLOv8n夜间苹果检测方法[J].华中农业大学学报,2024,43(5):1-9

复制
分享
文章指标
  • 点击次数:331
  • 下载次数: 797
  • HTML阅读次数: 116
  • 引用次数: 0
历史
  • 收稿日期:2023-12-01
  • 在线发布日期: 2024-10-08
文章二维码