基于深度学习的露天矿电铲斗齿状态监测技术研究.pdf

非全日制非全日制硕士学位论文硕士学位论文 基于深度学习的露天矿 电铲斗齿状态监测技术研究 Research on Monitoring Technology of Tooth Detection Based on Deep Learning 作者姓名 田 宽 导师姓名 毛君 教授 工程领域 机械工程 研究方向 机电一体化 完成日期 2020 年 8 月 11 日 辽宁工程技术大学 Liaoning Technical University 万方数据 基 于 深 度 学 习 的 露 天 矿 电 铲 斗 齿 状 态 监 测 技 术 研 究 田 宽 辽 宁 工 程 技 术 大 学 万方数据 万方数据 中图分类号 TH39 学校代码 10147 UDC 621 密 级 公 开 辽宁工程技术大学 非全日制非全日制硕士学位论文硕士学位论文 基于深度学习的露天矿 电铲斗齿状态监测技术研究 Research on Monitoring Technology of Tooth Detection Based on Deep Learning 作者姓名 田宽 学 号 471720086 导师姓名 毛君 （教授） 副导师姓名 杨志勇（高工） 申请学位 工程硕士 培养单位 机械工程学院 工程领域 机械工程 研究方向 机电一体化 二○二〇年八月 万方数据 致致 谢谢 时光匆匆，宛如白驹过隙，转眼间三年的研究生生涯即将结束，加上本科的四年学习 之路，我已经在工大度过了七年时光。回首这三年，这是一段人生中最重要的历程，也是 我最难忘的一段时光，在这三年里，我的导师、同学、同事和朋友在我前进的道路上给予 了我莫大的关怀与帮助。在这里，我要向他们献上最真挚的谢意。 首先我由衷的感谢我的导师毛君教授。本论文的工作是在毛老师的悉心指导下完成的， 在和毛老师的交流中，他总能在学术研究方面给予我极大的帮助，使我的科研水平得到极 大的提升，为我的研究工作指明了方向。毛老师对科研工作的执着和热情给我留下了很深 的印象，这在我今后的工作学习中能让我更加认真和坚持，这将是我一生的财富。 同时我还要感谢谢苗教授。谢老师不仅对我论文的研究工作提出了诸多建议，还解决 了我在专业学习与工作中的疑惑，在此向谢苗老师表示衷心的感谢。 我还要感谢我的所有同学们。在这三年里，是你们给了我许多帮助，在我遇到困难和 疑惑的时候，是你们帮我解决困难，你们是我最美好的回忆。 最后感谢评阅老师和答辩老师，谢谢各位老师在 2020 年这个特殊的时期在百忙之中 对我的论文进行评阅和指导，谢谢 万方数据 I 摘摘 要要 矿用电铲在挖煤过程中，电铲铲斗在复杂物理力学环境中长时间作业，会出现斗齿 断裂及铲斗局部脱落现象，脱落的部分混入煤炭中装入卡车，卡车直接运煤至破碎站。 当脱落部件大于 300mm 时会造成破碎机或电机故障，影响生产的正常运行，导致严重的 经济损失和人力物力浪费。矿用挖掘机体积庞大、工作环境复杂，不便于电铲操作人员 通过人工观察的方式对斗齿的工作状态进行监视；而结合计算机视觉对在工作过程中的 斗齿进行实时监控,实时监测并自动判断斗齿是否脱落，在斗齿发生脱落时警信息，指示 工作人员及时进行处置，防止造成更大的经济损失，因此电铲斗齿的失效监测对露天煤 矿的安全生产有着重要的实际意义和经济价值。 本文主要介绍了一种基于深度学习的电铲斗齿监测方法，旨在对电铲斗齿进行实时 监测，在发生斗齿脱落时发出警报，提示人员进行处理，避免更大的损失发生。 首先，介绍了斗齿监测的相应背景、研究现状和意义，并且结合当下热门的深度学 习技术提出了基于深度学习的斗齿监测方法。 其次，分析了斗齿多种故障类型的成因，在故障分析的基础之上，提出了监测系统 的整体设计方案，在硬件和软件两个方面进行了分析与介绍。 再次， 结合卷积神经网络和深度学习理论， 改进 YOLO-v3 算法， 优化系统稳定性和同 步性，对网络模型进行了构建和训练。设计监测系统人机交互界面，实现数据读取、斗齿 识别、故障报警和故障查看等功能，完成了软件系统的设计。 最后，分别进行了模拟验证实验与现场实际测试。测试结果表明，基于深度学习的电 铲斗齿监测系统有较好的识别效果，不仅监测速度快，且精度高，在一定程度上可以帮助 电铲监测斗齿状态， 在发生斗齿缺失时发出报警， 将理论付诸于实际， 实现了预期的效果。 该论文有图 88 幅，表 4 个，参考文献 64 篇。 关键词关键词电铲；斗齿；深度学习；斗齿监测；YOLO-v3 万方数据 II Abstract During the mining of a mining electric shovel, the electric shovel bucket is operated for a long time in a complicated physical and mechanical environment, and the teeth will break and the bucket will fall off locally. The dropped part will be mixed into the coal and loaded into the truck. The truck will directly transport the coal to the crushing station. When the falling part is larger than 300mm, it will cause the crusher or motor to fail, affecting the normal operation of the production, resulting in serious economic loss and waste of manpower and material resources. The mining excavator is bulky and the working environment is complex, which is not convenient for the electric shovel operator to monitor the working status of the bucket teeth by manual observation; and combined with computer vision, the bucket teeth are monitored in real time during the work process with the real-time detection and automatically judges whether the bucket teeth fall off. When the bucket teeth fall off, the alarm message instructs the staff to take timely measures to prevent greater economic losses. Therefore, the failure monitoring of the power shovel teeth has important practical significance for the safe production of open-pit coal mines and economic value. This article mainly introduces a shovel shovel tooth monitoring based on deep learning, which aims to monitor the shovel shovel teeth in real time, and issues an alarm when the shovel teeth fall off, prompts personnel to take action to avoid greater losses. Firstly, the corresponding background, research status and significance of shovel tooth monitoring are introduced, and a deep learning-based shovel tooth monitoring is proposed in combination with current popular deep learning technologies. Secondly, the causes of various failure types of the shovel teeth are analyzed. Based on the failure analysis, the overall design scheme of the monitoring system is proposed, and the analysis and introduction are made on both hardware and software. Thirdly, combining the convolutional neural network and deep learning theory, the YOLO- v3 algorithm was improved, the system stability and synchronization were optimized, and the network model was constructed and trained. Designed the human-computer interaction interface of the monitoring system to realize data reading, tooth recognition, fault alarm and fault viewing, etc., and completed the design of the software system. Finally, simulation verification experiments and field actual tests were carried out. The test results show that the shovel tooth monitoring system based on deep learning has a good recognition effect. It not only has fast detection speed and high accuracy, it can help the shovel to monitor the state of the shovel teeth to a certain extent. The alarm put theory into practice and achieved the 万方数据 III desired effect. The paper has 88 pictures, 4 tables, and 64 references. Keywords electric shovel; shovel teeth; deep learning; shovel teeth monitoring; YOLO-v3 万方数据 IV 目目 录录 摘摘 要要 .......................................................................................................................................... I I 目目 录录 ........................................................................................................................................ IVIV 图清单图清单 ................................................................................................................................. . VIIIVIII 表清单表清单 ................................................................................................................................. . XIIIXIII 变量注释表变量注释表 ............................................................................................................................ XIVXIV 1 1 绪论绪论.......................................................................................................................................... 1 1 1.1 课题研究背景及意义 .................................................... 1 1.2 斗齿脱落监测的研究现状与发展 .......................................... 3 1.3 基于深度学习的监测技术的研究现状 ...................................... 3 1.4 本文的主要研究内容与创新点 ............................................ 5 1.5 本章小结 .............................................................. 5 2 2 系统方案设计系统方案设计 .......................................................................................................................... 7 7 2.1 电铲斗齿结构与分析 .................................................... 7 2.2 斗齿主要的失效类型与分析 .............................................. 8 2.3 斗齿目标监测系统设计方案 ............................................. 13 2.4 系统硬件设计 ......................................................... 15 2.5 斗齿状态监测系统的软件平台的功能结构与实现流程 ....................... 18 2.6 本章小结 ............................................................. 19 3 3 数据的获取与处理实验数据的获取与处理实验 ........................................................................................................ 2020 3.1 数据采集与样本准备 ................................................... 20 3.2 图像灰度化 ........................................................... 22 3.3 图像灰度变换 ......................................................... 24 3.4 图像去噪 ............................................................. 28 3.5 基于像素坐标系的斗齿区域提取 ......................................... 31 3.6 本章小结 ............................................................. 32 4 4 基于卷积神经网络的斗齿目标监测基于卷积神经网络的斗齿目标监测 .................................................................................... 3333 4.1 卷积神经网络的基本结构 ............................................... 33 4.2 卷积神经网络的训练 ................................................... 39 4.3 使用 YOLO V3 算法进行模型训练 ......................................... 42 万方数据 V 4.4 基于卷积神经网络的结构优化改进 ....................................... 45 4.5 数据导入与算法预测流程 ............................................... 47 4.6 监测模型的构建与训练 ................................................. 49 4.7 系统实现 ............................................................. 52 4.8 本章小结 ............................................................. 55 5 5 监测系统的实验验证与分析监测系统的实验验证与分析 ................................................................................................ 5656 5.1 模拟场景实验及流程 ................................................... 56 5.2 现场验证及流程 ....................................................... 59 5.3 实验数据与结果分析 ................................................... 62 5.4 本章小结 ............................................................. 64 6 6 结论与展望结论与展望 ............................................................................................................................ 6565 6.1 本文结论 ............................................................. 65 6.2 未来展望 ............................................................. 65 参考文献参考文献 ................................................................................................................................. . 6767 作者简历作者简历 ................................................................................................................................. . 7171 学位论文原创性声明学位论文原创性声明 .............................................................................................................. 7272 学位论文数据集学位论文数据集 ...................................................................................................................... 7373 万方数据 VI Contents Abstract.......Ⅰ Contents....Ⅳ List of Figures....Ⅷ List of Tables.....ⅩⅢ List of Variables....ⅩⅣ 1 Introduction.1 1.1 Research Background and Significance.....1 1.2 Research Status and Development of Bucket Tooth Loss Monitoring ...3 1.3 Research Status of Monitoring Technology Based on Deep Learning ...3 1.4 The Main Research Content and Innovation Points of This Article ....5 1.5 Summary of This Chapter.......5 2 System Scheme Design... 7 2.1 Structure and Analysis of Shovel Teeth...... 7 2.2 Main Failure Types and Analysis of Bucket Teeth...................8 2.3 Design Scheme of Bucket Tooth Target Monitoring System ...13 2.4 System Hardware Design ......15 2.5 Functional Structure and Implementation Process of Software Plat for Bucket Tooth Condition Monitoring System........18 2.6 Summary of This Chapter...19 3 Data Acquisition and Processing Experiment ...20 3.1 Data Collection and Sample Preparation .....20 3.2 Image Graying .....22 3.3 Image Gray Transation .....24 3.4 Image Denoising ..28 3.5 Bucket Tooth Region Extraction Based on Pixel Coordinate System ......31 3.6 Summary of This Chapter ...32 4 Bucket Tooth Target Monitoring Based on Convolution Neural Network...33 4.1 Basic Structure of Convolutional Neural Networks .....33 4.2 Training of Convolutional Neural Networks ..39 万方数据 VII 4.3 Model Training Using YOLO V3 Algorithm ......42 4.4 Improvement of Structure Optimization Based on CNN...45 4.5 Data Import and Algorithm Prediction Process ..47 4.6 Construction and Training of Monitoring Model ......49 4.7 System Implementation ....52 4.8 Summary of This Chapter ......55 5 Experimental Verification and Analysis of Monitoring System ...56 5.1 Simulation Scene Experiment and Process ......56 5.2 Field Verification and Process.. 59 5.3 Analysis of Experimental Data and Results ...62 5.4 Summary of This Chapter ...64 6 Conclusion and Prospect...... 65 6.1 Conclusion ...65 6.2 Future Prospects ......65 References...67 Author’s Resume ...71 Declaration of Thesis Originality... 72 Thesis Data Collection .....73 万方数据 VIII 图清单图清单 图序号 图名称 页码 图 1.1 电铲 1 Figure1.1 Shovel 1 图 2.1 铲斗与铲齿组件 7 Figure2.1 Bucket and Tooth Assembly 7 图 2.2 电铲采掘工况 8 Figure2.2 Excavation conditions of electric shovel 8 图 2.3 铲齿断裂部位 9 Figure2.3 Fracture of shovel teeth 9 图 2.4 裂纹的三种类型 9 Figure2.4 Three types of cracks 9 图 2.5 Ⅰ型裂纹裂端区坐标描述 10 Figure2.5 Coordinate description of crack end zone of mode Ⅰ crack 10 图 2.6 II 型和 III 型裂纹裂端区坐标描述 10 Figure2.6 Figure 2.6 Coordinate description of the crack end region of mode II and III cracks 10 图 2.7 疲劳裂纹扩展曲线 12 Figure2.7 Fatigue crack growth curve 12 图 2.8 监测系统组成结构 12 Figure2.8 Composition of the detection system 12 图 2.9 相机安装位置示意 14 Figure2.9 Camera installation position 14 图 2.10 铲齿监测系统监测流程示意图 14 Figure2.10 Schematic diagram of the monitoring process of the tooth detection system 14 图 2.11 监测系统硬件结构图 15 Figure2.11 Hardware structure of the detection system 15 图 2.12 DS-2XE3046FWD-I 型红外补光摄像机 15 Figure2.12 DS-2XE3046FWD-I infrared fill light camera 15 图 2.13 Jetson AGX Xavier 开发平台 16 Figure2.13 Jetson AGX Xavier development plat 16 图 2.14 附件 17 Figure2.14 Attachment 17 图 2.15 系统软件功能结构 18 Figure2.15 System software functional structure 18 图 2.16 监测系统流程图 19 Figure2.16 Flow chart of detection system 19 图 3.1 获取的全部视频 20 Figure3.1 All acquired videos 20 图 3.2 弱光源红外补偿状态下获取的图像 21 Figure3.2 Image acquired under weak infrared compensation 21 图 3.3 强光源正常状态下获取的图像 21 Figure3.3 Image obtained under strong light source 21 万方数据 IX 图 3.4 部分关键帧数据 21 Figure3.4 Some key frame data 21 图 3.5 图像坐标系 22 Figure3.5 Image coordinale system 22 图 3.6 铲齿彩色数据 23 Figure3.6 Bucket tooth color data 23 图 3.7 平均值法灰度化处理结果 24 Figure3.7 Results of graying processing by the average 24 图 3.8 最大值法灰度化处理结果 24 Figure3.8 Grayscale processing results of the maximum 24 图 3.9 加权平均法灰度化处理结果 24 Figure3.9 Graying result of weighted average 24 图 3.10 铲齿灰度图像的线性变换结果 25 Figure3.10 Linear transation result of bucket tooth gray image 25 图 3.11 分段线性函数 26 Figure3.11 Piecewise linear function 26 图 3.12 铲齿灰度图像的分段线性变换结果 26 Figure3.12 The piecewise linear transation result of the bucket tooth gray image 26 图 3.13 指数函数 27 Figure3.13 Exponential function 27 图 3.14 铲齿灰度图像的指数变换结果 27 Figure3.14 Exponential transation result of bucket t