机器视觉在煤矿泵房智能监控系统的应用研究.pdf

硕士学位论文 机器视觉在煤矿泵房智能监控系统的应用 研究 Study on the Intelligent Monitoring System of Pump Room in Coal Mine Based on Machine Vision 作 者饶中钰 导 师李明教授 中国矿业大学 2020 年 6 月 万方数据 学位论文使用授权声明学位论文使用授权声明 本人完全了解中国矿业大学有关保留、使用学位论文的规定，同意本人所 撰写的学位论文的使用授权按照学校的管理规定处理 作为申请学位的条件之一，学位论文著作权拥有者须授权所在学校拥有学 位论文的部分使用权，即①学校档案馆和图书馆有权保留学位论文的纸质版 和电子版，可以使用影印、缩印或扫描等复制手段保存和汇编学位论文；②为 教学和科研目的，学校档案馆和图书馆可以将公开的学位论文作为资料在档案 馆、图书馆等场所或在校园网上供校内师生阅读、浏览。另外，根据有关法 规，同意中国国家图书馆保存研究生学位论文。 （保密的学位论文在解密后适用本授权书） 。 作者签名 导师签名 年 月 日 年 月 日 万方数据 中图分类号 TD76 学校代码 10290 UDC 004.8 密 级 公开 中国矿业大学 硕士学位论文 机器视觉在煤矿泵房智能监控系统的应用研究 Study on the Intelligent Monitoring System of Pump Room in Coal Mine Based on Machine Vision 作 者 饶中钰 导 师 李明 申请学位 工学硕士学位 培养单位 信息与控制工程学院 学科专业 控制科学与工程 研究方向 机器视觉 答辩委员会主席 孙晓燕 评 阅 人 二○○二年六月 万方数据 致谢致谢 时光荏苒，岁月如梭，转眼间三年研究生生活已经接近尾声。回想起自己 三年生活中的点点滴滴，充满了的是开心的日常生活。起初刚来学校时，也迷 糊了很长一段时间，在此要感谢我的导师李明教授、雷萌老师在学习生活多方 面上的照顾，让我慢慢开始适应自己的研究生生活。同时在此也要感谢已经毕 业的王洪栋师兄 、王大方以及卜令正等师兄在起初入学时的照顾。 随后自己慢慢适应了研究生生活，也开始了自己的科研道路。在自己的科 研学习过程中，要感谢李明教授的科研指导和敦敦教诲，感谢李老师不断指明 我的科研方向。同时也要特别感谢雷萌老师以及邹亮老师的耐心指导，不断启 发我的科研思路，在课题选择，方法探讨以及论文写作等多个方面给予我细致 的指导。在此祝各位老师身体健康，事业蒸蒸日上，也祝雷老师、邹老师刚出 生的女儿健康快乐成长。 感谢实验室的同门、师弟师妹们给予我的关心与帮助，感谢大家在一起的 快乐时光，祝愿大家事业有成，在学业生活上一帆风顺，祝愿大家能找到自己 心仪的工作以及生活。 最后要感谢我的女朋友两年以来的陪伴，感谢这两年你带给我的快乐，是 你让我研究生的生活更加丰富多彩，让我在生活中有了更多的期待和依赖，也 祝女朋友你能开开心心，未来的日子才刚刚开始，希望能和你携手走进更美好 的时光。 人事未尽，人时很长，我在中间，自当加油。感谢所有关心，帮助过我的 人，祝大家开心健康，事业有成。 万方数据 I 摘摘 要要 井下排水系统的运行状况直接影响煤炭开采的稳定与安全。 作为排水系统的 核心部门， 矿井泵房属于煤矿井下高级别限制区域。 为保障矿井泵房的安全运行， 需及时获取水泵的异常工作状态， 并具备警示技术员的危险区域操作及无关人员 的进入等能力。而目前，矿井泵房内监控系统只提供现场工况环境的视频监测、 存储等简单功能，无法实现智能在线预警功能。鉴于此，本文在已有矿井泵房监 控系统的基础上，利用机器视觉、深度学习等智能算法，实现对现场工况的智能 在线监控，主要包括以下内容 针对井下泵房的漏水检测问题， 提出了一种结合运动目标检测以及卷积神经 网络分类算法的检测策略首先，利用漏水区域的运动特性，采用三帧差分法， 确定包含运动目标的区域；其次，将运动目标定义为疑似漏水区域；最后，搭建 基于卷积神经网络的判别模型， 对疑似漏水区域进行判定。 实验表明基于 ResNet- 50 的分类模型在漏水区域判别的准确率可达到 96.8。 为获取井下泵房人员进入的情况， 基于 Deepsort 模型， 研究对泵房内人员实 现多目标跟踪的方法首先，分别采用 YOLOv3、YOLOv3-tiny 以及 YOLOv3- MobileNet 3 种网络， 作为 Deepsort 模型检测器； 然后利用 ReID 网络进行特征提 取，结合余弦相似度损失函数进行训练；最后，采用匈牙利匹配算法对跟踪以及 检测结果进行数据关联，从而实现矿井泵房内人员多目标跟踪，实验表明基于 YOLOv3 的 Deepsort 模型的多目标跟踪准确度可达到 83.7。 关于井下泵房人员动作行为的识别， 研究了基于三维卷积神经网络的 6 种基 础动作鉴定首先采用固定窗口方式制作数据集；然后采用 C3D、ResNet-3D 以 及 Inception-3D 网络对矿井泵房内人员进行行为识别，并结合 P3D 与 Inception- 3D 网络提出一种更为高效的 PI3D 模型。相较于原始 Inception-3D 模型，PI3D 网络的模型参数减少了 58，其行为识别准确度可达到 91.5。 基于上述模型算法，通过引入模块化的设计思想，设计友好的软件平台，设 计一套完备的煤矿井下泵房环境智能在线监控系统， 以实现漏水事故预警及工作 人员轨迹跟踪与行为识别的智能化、实时化、无人化的监测分析。 该论文有图 48 幅，表 6 个，参考文献 87 篇。 关键词关键词煤矿泵房；智能监控系统；机器视觉；深度学习 万方数据 II Abstract Operational status of underground drainage system affects the stability and safety of coal mine production. As a key part of drainage system, pump house is the high-level restricted area in the underground mining system. To ensure safe operation in pump house, it is necessary to obtain the abnormal working state of the pump in time, and to restrict the workers’ access to danger area. Nowadays, the available monitoring systems provide simple support services including on-site working conditions monitoring and video storage, but these systems cannot meet the need of early-warning and intelligent monitoring. To address these problems, several techniques such as machine vision and deep learning are employed to achieve intelligent online monitoring of pump house. This research mainly includes four aspects as follows To achieve automatic water leakage detection, a strategy, the combination of moving target detection and the convolution networks, is proposed. First, coterminous frames differencing is applied to determine the target area which is considered as the potential leaking area. Afterwards, convolution network is trained to further investigate the characteristic of the potential leaking area. Experimental results show that the Resnet-50 model yields the best perance with an accuracy of 96.8. The Deepsort model is employed to achieve multi-target tracking. First, three algorithms including YOLOv3, YOLOv3-tiny, and YOLOv3-MobileNet are trained as the detector of Deepsort model. To extract feature ination, ReID is employed and trained according to the loss in cosine similarity. Finally, the detector and ReID network are combined to build a Deepsort model to track of the workers in the pump house. The experimental results show that the Deepsort model based on YOLOv3 detector achieves the highest multiple objects tracking accuracy of 83.7. To recognize the actions and behaviors of people in pump house, 3D-CNN is employed to identify six basic actions. First, the dataset is comprised of the cropped images with a fixed window. Second, three models including C3D, ResNet-3D and Inception-3D model are introduced to recognize the actions. In addition, an efficient PI3D model, which combines P3D and Inception-3D networks, is proposed. The number parameters of PI3D network are reduced by 58, compared with the number parameters of standard Inception-3D model. Furthermore, PI3D achieves an accuracy of 91.5. 万方数据 III Based on the above algorithms, an intelligent online monitoring software is designed to provide well-rounded support services including water leakage warning, staff trajectory tracking and behavior recognition. Keywords underground drainage system; machine vision; deep learning 万方数据 IV 目目 录录 摘要摘要 ............................................................................................................................... I 目录目录 ............................................................................................................................ IV 图清单图清单 ..................................................................................................................... VIII 表清单表清单 ........................................................................................................................ XI 变量注释表变量注释表 ............................................................................................................... XII 1 绪论绪论 ........................................................................................................................... 1 1.1 研究背景与意义 ..................................................................................................... 1 1.2 国内外研究现状 ..................................................................................................... 3 1.3 主要工作与章节安排 ............................................................................................. 7 2 相关基础理论与知识相关基础理论与知识 ............................................................................................... 9 2.1 运动目标检测相关理论基础研究 ......................................................................... 9 2.2 目标跟踪相关理论基础研究 ............................................................................... 11 2.3 卷积神经网络相关理论基础研究 ....................................................................... 13 2.4 本章小结 ............................................................................................................... 18 3 基于帧差法与深度卷积神经网络结合的矿井泵房漏水检测基于帧差法与深度卷积神经网络结合的矿井泵房漏水检测 ............................. 19 3.1 漏水检测基本流程概述 ....................................................................................... 19 3.2 基于运动目标检测方法的疑似漏水区域检测 ................................................... 19 3.3 基于卷积神经网络的漏水区域分类 ................................................................... 24 3.4 本章小结 ............................................................................................................... 30 4 基于基于 Deepsort 算法的矿井泵房人员目标跟踪研究算法的矿井泵房人员目标跟踪研究 ............................................ 32 4.1 Deepsort 理论基础 ............................................................................................... 33 4.2 Deepsort 算法在泵房人员目标跟踪上的应用 ........................................................... 38 4.3 本章小结 ............................................................................................................... 42 5 基于基于 3D 卷积神经网络的矿井泵房人员动作识别卷积神经网络的矿井泵房人员动作识别 .............................................. 43 5.1 3D 卷积神经网络理论基础 .................................................................................. 43 5.2 3D 卷积神经网络在矿井泵房人员动作识别应用 .............................................. 46 5.3 本章小结 ............................................................................................................... 50 6 矿井泵房智能监控系统软件平台设计矿井泵房智能监控系统软件平台设计 ................................................................. 51 万方数据 V 6.1 矿井泵房智能监控系统软件需求分析 ............................................................... 51 6.2 矿井泵房智能监控系统软件设计 ....................................................................... 51 6.3 本章小结 ............................................................................................................... 54 7 总结与展望总结与展望 ............................................................................................................. 55 7.1 总结 ....................................................................................................................... 55 7.2 展望 ....................................................................................................................... 56 参考文献参考文献 ..................................................................................................................... 58 作者简介作者简介 ..................................................................................................................... 62 学位论文原创性声明学位论文原创性声明 ................................................................................................. 63 学位论文数据集学位论文数据集 ......................................................................................................... 64 万方数据 VI Contents Abstract ......................................................................................................................... I Contents ..................................................................................................................... IV List of Figures ........................................................................................................... VII List of Tables ........................................................................................................... VIII List of Variables........................................................................................................ XII 1 Introduction ............................................................................................................... 1 1.1 Research Background .............................................................................................. 1 1.2 Current Research Status ........................................................................................... 3 1.3 Structure of the Disseration ..................................................................................... 7 2 Related Basic Theories and Knowledge .................................................................. 9 2.1 Related Basic Theory for the Detection of Moving Target ...................................... 9 2.2 Related Basic Theory for the Target Tracking ....................................................... 11 2.3 Related Basic Theory for the Convolution Network ............................................. 13 2.4 Conclusions of this Chapter ................................................................................... 18 3 The Leaking Detection Based on The Frame Difference s and Convolution Networks ............................................................................................... 19 3.1 The Basic Introduction for The Process of Leaking Detection .............................. 19 3.2 The Suspected Leak Area Detection ...................................................................... 19 3.3 The Classification of Leaking Area Based on Convolution Networks .................. 24 3.4 Conclusions of this Chapter ................................................................................... 30 4 The Human Tracking in Pump House Based on Deep-sort ................................ 32 4.1 Basic Theory of Deepsort ...................................................................................... 33 4.2 Application of Deepsort in Target Tracking of Pumping House ............................ 38 4.3 Conclusions of this Chapter ................................................................................... 42 5 The Action recognition of Peoplein Pump Housw Based on 3D Convolutional Neural network........................................................................................................... 43 5.1 Basic Theory of 3D Convolution Networks .......................................................... 43 5.2 Application of 3D Convolutional Neural Network in Action Recognition in Pump House ........................................................................................................................... 46 5.3 Conclusions of this Chapter ................................................................................... 50 万方数据 VII 6 Software Development of Intelligent Monitoring System in Pump House ........ 51 6.1 Demand analysis of pump room intelligent monitoring system ............................ 51 6.2 Software design of intelligent monitoring system for water pump room .............. 51 6.3 Conclusions of this Chapter ................................................................................... 54 7 Conclusions and Prospects ..................................................................................... 55 7.1 Conclusions ............................................................................................................ 55 7.2 Prospects ................................................................................................................ 56 References ................................................................................................................... 58 Author’s Resume ...................................................................................................... 62 Declaration of Thesis Originality ............................................................................. 63 Thesis Data Collection ............................................................................................... 64 万方数据 VIII 图清单图清单 图序号 图名称 页码 图 1-1 煤矿安全事故类型分布图 1 Figure 1-1 The type distribution of coal mining safety accident 1 图 1-2 矿井泵房工况现场示意图 2 Figure 1-2 Schematic diagram of water pump room 2 图 1-3 跟踪基本原理示意图 5 Figure 1-3 schematic diagram of tracking fundamentals 5 图 2-1 帧间差分法流程图 9 Figure 2-1 Flow chart of frame difference 9 图 2-2 背景消除法流程图 11 Figure 2-2 Flow chart of background removal 11 图 2-3 匈牙利匹配算法多目标跟踪匹配示意图 13 Figure 2-3 Hungarian matching algorithm for multiple target tracking matching 13 图 2-4 卷积运算过程示意图 14 Figure 2-