首页 > 过刊浏览>2022年第41卷第4期 >158-169. DOI:10.13300/j.cnki.hnlkxb.2022.04.020
PDF HTML阅读 XML下载 导出引用 引用提醒
基于CNN-Transformer的视觉缺陷柑橘分选方法
CSTR:
[cstr]
作者:
作者单位:

1.华中农业大学工学院,武汉430070;2.国家柑橘保鲜技术研发专业中心,武汉430070;3.农业农村部长江中下游农业装备重点实验室,武汉430070

作者简介:

安小松，E-mail： 279560176@qq.com

通讯作者:

李善军，E-mail：shanjunlee@163.com

中图分类号:

TP391.41

基金项目:

财政部和农业农村部：国家现代农业产业技术体系、柑橘全程机械化科研基地建设项目（农计发［2017］19号）；湖北省农业科技创新行动项目；国家重点研发计划（2020YFD1000101）


A CNN-Transformer-based method for sorting citrus with visual defects
Author:
Affiliation:

1.College of Engineering，Huazhong Agricultural University，Wuhan 430070，China;2.National R&D Center for Citrus Preservation，Wuhan 430070，China;3.Ministry of Agriculture and Rural Affairs Key Laboratory of Agricultural Equipment in Mid-Lower Yangtze River，Wuhan 430070，China

  • 摘要
    • |
  • 图/表
    • |
  • 访问统计
    • |
  • 参考文献 [22]
    • |
  • 相似文献 [20]
    • | | |
  • 文章评论
    摘要:

    针对产线分拣缺陷柑橘费时费力等问题，以柑橘加工生产线输送机上随机旋转的柑橘果实为研究对象，开发了一种基于卷积神经网络（CNN）的检测算法Mobile-citrus，用于检测和暂时分类缺陷果实，并采用Tracker-citrus跟踪算法来记录其路径上的分类信息，通过跟踪的历史信息识别柑橘的真实类别。结果显示，跟踪精度达到98.4%，分类精度达到92.8%。同时还应用基于Transformer的轨迹预测算法对果实的未来路径进行了预测，平均轨迹预测误差达到最低2.98个像素，可用于指导机器人手臂分选缺陷柑橘。试验结果表明，所提出的基于CNN-Transformer的缺陷柑橘视觉分选系统，可直接应用在柑橘加工生产线上实现快速在线分选。

    Abstract:

    Manual sorting of citrus fruit with visual defects on the production line is time-consuming and cost-expensive.This article proposes a sorting solution based on machine vision and CNN-Transformer.The system can be directly implemented on various citrus processing lines for online sorting.For the citrus fruits randomly rotating on the conveyor，a detection algorithm Mobile-citrus based on convolutional neural network （CNN） was developed to detect and temporarily classify the defective one.A tracking algorithm Tracker-citrus was used to record the classification information along the path.The real category of the fruit was identified using the historical information，with tracking accuracy of 98.4% and classification accuracy of 92.8%.In addition，a trajectory prediction algorithm based on Transformer was used to predict the future path of fruit with the average prediction error of 2.98 pixels，which can be used to guide the robot arm to sort defective citrus in real time.The results showed that the method proposed can be applied to citrus production lines for online sorting.

    图1 3类柑橘示意图Fig.1 Examples of 3 types of citrus
    图2 平台设置和计算机视觉系统Fig.2 Platform setup and computer vision system
    图3 检测器的网络结构Fig.3 Network architecture of the detector
    图4 Transformer网络架构Fig.4 Network architecture of the Transformer
    图5 多目标跟踪工作流程Fig.5 Workflow of the multi-object tracking
    图6 轨迹预测工作流程Fig.6 Workflow of trajectory prediction
    图7 跟踪分类过程Fig.7 Tracking classification process
    图8 Mobile-citrus模型缺陷检测结果Fig.8 Mobile-citrus model defect detection results
    图9 跟踪指标参数示意图Fig.9 Schematic diagram of tracking indicator parameters
    图10 缺陷检测和跟踪结果Fig.10 Defect detection and tracking results
    图11 轨迹预测误差计算示意图Fig.11 Schematic diagram of trajectory prediction error calculation
    图12 轨迹预测结果Fig.12 Trajectory prediction results
    图13 最终分类结果Fig.13 Final classification results
    表 1 检测器的性能评估Table 1 Performance evaluation of detector
    参考文献
    [1] LIU N，LI X，ZHAO P，et al.A review of chemical constituents and health-promoting effects of Citrus peels[J/OL].Food chemistry，2021，365：130585[2021-12-02].https：//doi.org/10.1016/j.foodchem.2021.130585.
    [2] ELKAOUD N S M，ELGLALY A M M.Development of grading machine for citrus fruits （Valencia orange）[J].Journal of soil sciences and agricultural engineering，2019，10（11）：671-677.
    [3] SAKUDO A，YAGYU Y.Application of a roller conveyor type plasma disinfection device with fungus-contaminated citrus fruits[J/OL].AMB express，2021，11（1）：16[2021-12-02].https：//doi.org/10.1186/s13568-020-01177-2.
    [4] BHATNAGAR A，PATEL R，GUPTA M，et al.Customized sorting and packaging machine[J].Telecommunication computing electronics and control，2021，19（4）：1326-1333.
    [5] REDMON J，FARHADI A.YOLOv3：an incremental improvement[EB/OL].arXiv.2018.1804.02767[cs.CV][2021-12-02].https：//arxiv.org/abs/1804.02767.
    [6] KANG H W，CHEN C.Fruit detection，segmentation and 3D visualisation of environments in apple orchards[J/OL].Computers and electronics in agriculture，2020，171：105302[2021-12-02].https：//doi.org/10.1016/j.compag.2020.105302.
    [7] KANG H W，CHEN C.Fast implementation of real-time fruit detection in apple orchards using deep learning[J/OL].Computers and electronics in agriculture，2020，168：105108[2021-12-02].https：//doi.org/10.1016/j.compag.2019.105108.
    [8] WANG Q J，ZHANG S Y，DONG S F，et al.Pest24：a large-scale very small object data set of agricultural pests for multi-target detection[J/OL].Computers and electronics in agriculture，2020，175：105585[202-12-02].https：//doi.org/10.1016/j.compag.2020.105585.
    [9] 杨万里，段凌凤，杨万能.基于深度学习的水稻表型特征提取和穗质量预测研究[J].华中农业大学学报，2021，40（1）：227-235.YANG W L，DUAN L F，YANG W N.Deep learning-based extraction of rice phenotypic characteristics and prediction of rice panicle weight[J].Journal of Huazhong Agricultural University，2021，40（1）：227-235（in Chinese with English abstract）.
    [10] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[EB/OL].arXiv：1706.03762 [cs.CL] [2021-12-02].https：//doi.org/10.48550/arXiv.1706.03762.
    [11] RAGANATO A，SCHERRER Y，TIEDEMANN J.Fixed encoder self-attention patterns in transformer-based machine translation[EB/OL].arXiv：2002.10260 [2021-12-02].https：//doi.org/10.48550/arXiv.2002.10260.
    [12] SUN P Z，JIANG Y，ZHANG R F，et al.TransTrack：multiple-object tracking with transformer[EB/OL] . arXiv：2012.15460 [cs.CV] [2021-12-02].https：//doi.org/10.48550/arXiv.2012.15460.
    [13] WANG Z W，MA Y，LIU Z T，et al.R-transformer：recurrent neural network enhanced transformer[EB/OL].2019：arXiv：1907.05572[cs.LG][2021-12-02].https：//arxiv.org/abs/1907.05572.
    [14] 章海亮，高俊峰，何勇.基于高光谱成像技术的柑橘缺陷无损检测[J].农业机械学报，2013，44（9）：177-181.ZHANG H L，GAO J F，HE Y.Nondestructive detection of citrus defection using hyper-spectra imaging technology[J].Transactions of the CSAM，2013，44（9）：177-181（in Chinese with English abstract）.
    [15] 龚中良，杨张鹏，梁力，等.基于机器视觉的柑橘表面缺陷检测[J].江苏农业科学，2019，47（7）：236-239.GONG Z L，YANG Z P，LIANG L，et al.Detection of citrus surface defects based on machine vision[J].Jiangsu agricultural sciences，2019，47（7）：236-239（in Chinese）.
    [16] 李善军，胡定一，高淑敏，等.基于改进SSD的柑橘实时分类检测[J].农业工程学报，2019，35（24）：307-313.LI S J，HU D Y，GAO S M，et al.Real-time classification and detection of citrus based on improved single short multibox detecter[J].Transactions of the CSAE，2019，35（24）：307-313（in Chinese with English abstract）.
    [17] BOCHKOVSKIY A，WANG C Y，LIAO H Y M.YOLOv4：optimal speed and accuracy of object detection[EB/OL].arXiv：2004.10934 [cs.CV] [2021-12-02].https：//doi.org/10.48550/arXiv.2004.10934.
    [18] SANDLER M，HOWARD A，ZHU M L，et al.MobileNetV2：inverted residuals and linear bottlenecks[EB/OL].arXiv：1801.04381 [cs.CV] [2021-12-02].https：//doi.org/10.48550/arXiv.1801.04381.
    [19] LIU S，QI L，QIN H F，et al.Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.June 18-23，2018，Salt Lake City，UT，USA.[S.l.]：IEEE，2018：8759-8768.
    [20] XU J J，SUN X，ZHANG Z Y，et al.Understanding and improving layer normalization[EB/OL].arXiv：1911.07013 [cs.LG] [2021-12-02].https：//doi.org/10.48550/arXiv.1911.07013.
    [21] BEWLEY A，GE Z Y，OTT L，et al.Simple online and realtime tracking[C]//2016 IEEE International Conference on Image Processing.September 25-28，2016，Phoenix，AZ，USA.[S.l.]：IEEE，2016：3464-3468.
    [22] MILAN A，LEAL-TAIXE L，REID I，et al.MOT16：a benchmark for multi-object tracking[EB/OL].arXiv：2016.1603.00831[cs.CV] [2021-12-02].https：//arxiv.org/abs/1603.00831.
    引证文献
引用本文

安小松,宋竹平,梁千月,杜璇,李善军.基于CNN-Transformer的视觉缺陷柑橘分选方法[J].华中农业大学学报,2022,41(4):158-169

复制
分享
文章指标
  • 点击次数:306
  • 下载次数: 1043
  • HTML阅读次数: 73
  • 引用次数: 0
历史
  • 收稿日期:2021-12-02
  • 在线发布日期: 2022-10-12
文章二维码

联系我们

电话： 027-87287437，027-87287364
邮政编码： 430070
E-mail：hnlkxb@mail.hzau.edu.cn
地址： 武汉市洪山区狮子山街1号

期刊订阅 Email Alert 国际标准刊号：ISSN 1000-2421 国外邮发代号：BM 3816 国内刊号：CN 42-1181/S 国内发行代号：38-120

华中农业大学学报（自然科学版） ® 2025 版权所有   技术支持：北京勤云科技发展有限公司