A method for detecting apple at night based on YOLOv8n with fusion of image enhancement and transfer learning
CSTR:
Author:
Affiliation:

1.Institute of Agricultural Machinery, Hubei University of Technology, Wuhan 430068, China;2.Hubei Engineering Research Center for Intellectualization of Agricultural Equipment, Wuhan 430068, China

Clc Number:

TP391.4

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference [22]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    This paper proposed a method for detecting apple at night based on YOLOv8n with fusion of image enhancement and transfer learning to address the issues of low recognition rate and poor real-time performance of apples in nighttime scenarios. Firstly, embedding a Zero-DCE module in the front-end of YOLOv8n enhanced images of apple at night, presented the contours and details of apples more clearly, and reduced the difficulty of recognizing images of apple at night. Secondly, using SPD-Conv for down-sampling enhanced the ability of the model to extract fine-grained features. On this basis, transfer learning training strategy was used to solve the problem of small sample size in the dataset of apple at night. The MS COCO dataset containing categories of apple was selected as the source domain dataset. In term of the target domain dataset in nighttime scenarios, Zero-DCE was used to increase its similarity with images of apple during the day and finely tune the model of target domain on the model of source domain. Experiments were conducted on the image dataset of apple at night based on the method above. The results showed that the model accuracy P, a recall R, and an average accuracy mean mAP@0.5:0.95 of method proposed was 97.0%,93.4% and 74.6%, being 2.3,1.9, and 4.3 percentages higher than that of the YOLOv8n original model. The inference speed of this model was 22 frames/s, meeting requirements of real-time detection. The results of the ablation experiment showed that the combined effect of image enhancement and transfer learning exceeded the sum of the effects when applied separately. The improved model performed better than the original model in dealing with complex situations including overlap, occlusion, green fruits, and dim lighting, and had good robustness.

    Table 1 Results of ablation test
    Fig.1 Example of data annotation results
    Fig.2 Example of data augmentation results
    Fig.3 The framework of Zero-DCE
    Fig.4 Sampling characteristics of convolutional kernels with steps of 2 and 1(using columns as an example)
    Fig.5 Operation process of SPD-Conv
    Fig.6 Improved network structure
    Fig.7 Transfer learning process of the method in this article
    Fig.8 Changes in mAP@0.5:0.95 on the test set during training of baseline and improved model
    Fig.9 Examples of complex scene detection using baseline and improved model
    Table 2 Robustness test results
    Reference
    [1] 李会宾,史云.果园采摘机器人研究综述[J].中国农业信息,2019,31(6):1-9.LI H B,SHI Y.Review on orchard harvesting robots[J].China agricultural informatics,2019,31(6):1-9 (in Chinese with English abstract).
    [2] 中国苹果产业协会,中国农业大学农业规划设计院.2021年度中国苹果产业发展报告(总篇)精简版[J].中国果菜,2023,43(1):1-8.China Apple Industry Association,China Agricultural University Agricultural Planning and Design Institute.China apple industry development report in 2021(general),condensed versio N[J].China fruit & vegetable,2023,43(1):1-8(in Chinese with English abstract).
    [3] 宋怀波,尚钰莹,何东健.果实目标深度学习识别技术研究进展[J].农业机械学报,2023,54(1):1-19.SONG H B,SHANG Y Y,HE D J.Review on deep learning technology for fruit target recognition[J].Transactions of the CSAM,2023,54(1):1-19(in Chinese with English abstract).
    [4] WEI X Q,JIA K,LAN J H,et al.Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot[J].Optik,2014,125(19):5684-5689.
    [5] 周桂红,马帅,梁芳芳.基于改进YOLOv4模型的全景图像苹果识别[J].农业工程学报,2022,38(21):159-168.ZHOU G H,MA S,LIANG F F.Recognition of the apple in panoramic images based on improved YOLOv4 model[J].Transactions of the CSAE,2022,38(21):159-168(in Chinese with English abstract).
    [6] 袁紫薇.基于机器视觉的番茄收割机实时分拣系统研究[D].西安:长安大学,2017.YUAN Z W.Research on real-time sorting system of tomato harvester based on machine vision[D].Xi’an:Changan University,2017(in Chinese with English abstract).
    [7] 岳琳茜,李文宽,杨晓峰,等.基于改进YOLOv4的苹果检测与果径估测方法[J].激光杂志,2022,43(2):58-65.YUE L X,LI W K,YANG X F,et al.Apple detection and fruit diameter estimation method based on improved YOLOv4[J].Laser journal,2022,43(2):58-65(in Chinese with English abstract).
    [8] 陈青,殷程凯,郭自良,等.苹果采摘机器人关键技术研究现状与发展趋势[J].农业工程学报,2023,39(4):1-15.CHEN Q,YIN C K,GUO Z L,et al.Current status and future development of the key technologies for apple picking robots[J].Transactions of the CSAE,2023,39(4):1-15(in Chinese with English abstract).
    [9] FONT D,PALLEJà T,TRESANCHEZ M,et al.Counting red grapes in vineyards by detecting specular spherical reflection peaks in RGB images obtained at night with artificial illumination[J].Computers and electronics in agriculture,2014,108:105-111.
    [10] 赵德安,刘晓洋,陈玉,等.苹果采摘机器人夜间识别方法[J].农业机械学报,2015,46(3):15-22.ZHAO D A,LIU X Y,CHEN Y,et al.Image recognition at night for apple picking robot[J].Transactions of the CSAM,2015,46(3):15-22(in Chinese with English abstract).
    [11] 戴家裕.苹果采摘机器人夜间青苹果识别方法[D].杭州:浙江工业大学,2020.DAI J Y.Method for identify green apples at night by apple pic robot[D].Hangzhou:Zhejiang University of Technology,2020(in Chinese with English abstract).
    [12] REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:unified,real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).June 27-30,2016.Las Vegas,NV,USA.Las Vegas:IEEE,2016:779-788.
    [13] 熊俊涛,郑镇辉,梁嘉恩,等.基于改进YOLO v3网络的夜间环境柑橘识别方法[J].农业机械学报,2020,51(4):199-206.XIONG J T,ZHENG Z H, LIANG J E,et al.Citrus detection method in night environment based on improved YOLO v3 network[J].Transactions of the CSAM,2020,51(4):199-206(in Chinese with English abstract).
    [14] HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely connected convolutional networks[DB/OL].arXiv,2016:1608.06993[2023-12-01].http://arxiv.org/abs/1608.06993v5.
    [15] 孙宝霞,梁翠晓,刘凯,等.夜间环境下树上柑橘表征缺陷深度学习检测方法[J].林业工程学报,2021,6(6):148-155.SUN B X,LIANG C X,LIU K,et al.Fluorescence detection of citrus characterization defects using SVM[J].Journal of forestry engineering,2021,6(6):148-155(in Chinese with English abstract).
    [16] 何斌,张亦博,龚健林,等.基于改进YOLOv5的夜间温室番茄果实快速识别[J].农业机械学报,2022,53(5):201-208.HE B,ZHANG Y B,GONG J L,et al.Fast recognition of tomato fruit in greenhouse at night based on improved YOLOv5[J].Transactions of the CSAM,2022,53(5):201-208(in Chinese with English abstract).
    [17] LIN T Y,MARIRE M,BELONGIE S,et al.Microsoft COCO:Common objects in context[C]//Computer Vision-EC CV 2014 Workshops.Zurich,Switzerland,September 6-7 and 12,2014.Cham:Springer,2014.
    [18] GUO C L,LI C Y,GUO J C,et al.Zero-reference deep curve estimation for low-light image enhancement[DB/OL].arXiv,2020:2001.06826[2023-12-01].https://doi.org/10.48550/ arXiv.2001.06826
    [19] SUNKARA R,LUO T.No more strided convolutions or pooling:a new CNN building block for low-resolution images and small objects[DB/OL].arXiv,2022:2208.03641[2023-12-01].https://doi.org/10.48550/arXiv.2208.03641.
    [20] 庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报,2015,26(1):26-39.ZHUANG F Z,LUO P,HE Q,et al.Survey on transfer learning research[J].Journal of software,2015,26(1):26-39(in Chinese with English abstract).
    [21] PAN S J,YANG Q.A survey on transfer learning[J].IEEE transactions on knowledge and data engineering,2010,22(10):1345-1359.
    [22] EVERINGHAM M,ALI ESLAMI S M,VAN GOOL L,et al.The pascal visual object classes challenge:a retrospective[J].International journal of computer vision,2015,111(1):98-136.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

仝召茂,陈学海,马志艳,杨光友,张灿. A method for detecting apple at night based on YOLOv8n with fusion of image enhancement and transfer learning[J]. Jorunal of Huazhong Agricultural University,2024,43(5):1-9.

Copy
Share
Article Metrics
  • Abstract:358
  • PDF: 839
  • HTML: 119
  • Cited by: 0
History
  • Received:December 01,2023
  • Online: October 08,2024
Article QR Code