opencv+tensorFlow探索实现目标检测

红太狼 2022-12-02 00:47 171阅读 0赞

# 前述： #

本文为了记载自己在开发过程中的心得以及问题解决而写，阅读者在参考时尽量加入自己的思考，毕竟依赖库以及主程序随着时间的变化正在不断升级；  
另外，牢骚一下，对于初学者，很多作者在写文章的时候，会漏掉诸多环节，对于他们而言，那是不言而喻的东西，对于初学者，那些是什么，怎么做？

参考地址：

> https://tensorflow.google.cn/install/pip\#windows  
> https://blog.csdn.net/kh971024/article/details/103391094

--------------------

# 重新安装python： #

python-3.7.9.exe是32位版本的python，在寻找对应的tensorflow时，无法找到，试了很多的方法均无法匹配，因此删除3.7.9版本，改为使用【python-3.7.9-amd64.exe】；同样的，openCV也需要安装64位版本；  
安装文件如何寻找，可以参见博主之前的文章[python+openCV安装（WINDOWS环境）][python_openCV_WINDOWS]

--------------------

# 安装tensorflow： #

在**命令行**中执行：`pip3 install tensorflow`  
在下图中可以看到可以自动匹配到tensorflow的版本并且进行自动下载安装，但是由于网速有限，依然需要在https://pypi.org/上找到【tensorflow-2.3.0-cp37-cp37m-win\_amd64.whl】文件下载后本地安装。  
![在这里插入图片描述][20200829101819897.png_pic_center]

根据博主之前的文章[python+openCV安装（WINDOWS环境）][python_openCV_WINDOWS]目前初始环境已经具备，即：Python 3.7.9；tensorflow 2.3.0；opencv 4.4.0

--------------------

# COCO数据集： #

博主把下边这个表格拷贝过来，这个表格代表了每一种模型的速度、识别率等情况，表格中的超链接可以直接链接到下载地址，如果无法下载，可以拷贝超链接后，使用百度网盘等工具中的离线下载，亲测好用。  
另外，其他模型暂时不考虑，主要是初学，贪多嚼不烂。

<table> 
 <thead> 
 <tr> 
 <th>Model name</th> 
 <th>Speed (ms)</th> 
 <th>COCO mAP<a rel="nofollow">1</a></th> 
 <th>Outputs</th> 
 </tr> 
 </thead> 
 <tbody> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2018_01_28.tar.gz" rel="nofollow">ssd_mobilenet_v1_coco </a></td> 
 <td>30</td> 
 <td>21</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_0.75_depth_300x300_coco14_sync_2018_07_03.tar.gz" rel="nofollow">ssd_mobilenet_v1_0.75_depth_coco ☆ </a></td> 
 <td>26</td> 
 <td>18</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tar.gz" rel="nofollow">ssd_mobilenet_v1_quantized_coco ☆ </a></td> 
 <td>29</td> 
 <td>18</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.tar.gz" rel="nofollow">ssd_mobilenet_v1_0.75_depth_quantized_coco ☆ </a></td> 
 <td>29</td> 
 <td>16</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_ppn_shared_box_predictor_300x300_coco14_sync_2018_07_03.tar.gz" rel="nofollow">ssd_mobilenet_v1_ppn_coco ☆ </a></td> 
 <td>26</td> 
 <td>20</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz" rel="nofollow">ssd_mobilenet_v1_fpn_coco ☆ </a></td> 
 <td>56</td> 
 <td>32</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz" rel="nofollow">ssd_resnet_50_fpn_coco ☆ </a></td> 
 <td>76</td> 
 <td>35</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz" rel="nofollow">ssd_mobilenet_v2_coco </a></td> 
 <td>31</td> 
 <td>22</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.tar.gz" rel="nofollow">ssd_mobilenet_v2_quantized_coco </a></td> 
 <td>29</td> 
 <td>22</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz" rel="nofollow">ssdlite_mobilenet_v2_coco </a></td> 
 <td>27</td> 
 <td>22</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2018_01_28.tar.gz" rel="nofollow">ssd_inception_v2_coco </a></td> 
 <td>42</td> 
 <td>24</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_inception_v2_coco </a></td> 
 <td>58</td> 
 <td>28</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet50_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_resnet50_coco </a></td> 
 <td>89</td> 
 <td>30</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet50_lowproposals_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_resnet50_lowproposals_coco </a></td> 
 <td>64</td> 
 <td> </td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/rfcn_resnet101_coco_2018_01_28.tar.gz" rel="nofollow">rfcn_resnet101_coco </a></td> 
 <td>92</td> 
 <td>30</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet101_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_resnet101_coco </a></td> 
 <td>106</td> 
 <td>32</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet101_lowproposals_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_resnet101_lowproposals_coco </a></td> 
 <td>82</td> 
 <td> </td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_inception_resnet_v2_atrous_coco </a></td> 
 <td>620</td> 
 <td>37</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_lowproposals_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_inception_resnet_v2_atrous_lowproposals_coco </a></td> 
 <td>241</td> 
 <td> </td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_nas_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_nas </a></td> 
 <td>1833</td> 
 <td>43</td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/faster_rcnn_nas_lowproposals_coco_2018_01_28.tar.gz" rel="nofollow">faster_rcnn_nas_lowproposals_coco </a></td> 
 <td>540</td> 
 <td> </td> 
 <td>Boxes</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/mask_rcnn_inception_resnet_v2_atrous_coco_2018_01_28.tar.gz" rel="nofollow">mask_rcnn_inception_resnet_v2_atrous_coco </a></td> 
 <td>771</td> 
 <td>36</td> 
 <td>Masks</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/mask_rcnn_inception_v2_coco_2018_01_28.tar.gz" rel="nofollow">mask_rcnn_inception_v2_coco </a></td> 
 <td>79</td> 
 <td>25</td> 
 <td>Masks</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/mask_rcnn_resnet101_atrous_coco_2018_01_28.tar.gz" rel="nofollow">mask_rcnn_resnet101_atrous_coco </a></td> 
 <td>470</td> 
 <td>33</td> 
 <td>Masks</td> 
 </tr> 
 <tr> 
 <td><a href="http://download.tensorflow.org/models/object_detection/mask_rcnn_resnet50_atrous_coco_2018_01_28.tar.gz" rel="nofollow">mask_rcnn_resnet50_atrous_coco </a></td> 
 <td>343</td> 
 <td>29</td> 
 <td>Masks</td> 
 </tr> 
 </tbody> 
</table>

这里以ssd\_mobilenet\_v2\_coco\_2018\_03\_29模型为例，  
1、将其解压后，文件夹样式如图：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQwNDE4MzAy_size_16_color_FFFFFF_t_70_pic_center]  
2、增加一个文件[classes.txt][]，这个里面是将需要识别的类别名称列举到其中，博主把已知的所有名称都列举进去了（自己理解越少，识别起来越速度）。  
3、书写python代码，并且调整某些参数：

########################################################
    ## DIC：tensorFlow auto recognition SSD Demo
    ## Base on ssd_mobilenet_v2_coco
    ## Depend On：Python 3.7.9；tensorflow 2.3.0；opencv 4.4.0
    ## Author:J.Y.Zhang 2020-09-11
    ## Ver:1.0
    ## See:## See: https://editor.csdn.net/md/?articleId=108280933
    ########################################################
    import os
    import numpy as np
    import cv2
    import matplotlib.pyplot as plt
    import tensorflow as tf
    
    # 加载coco数据集模型 TODO 需要改为自己文件路径
    model_path = "F:/traindata/ssd_mobilenet_v2_coco_2018_03_29"
    frozen_pb_file = os.path.join(model_path, 'frozen_inference_graph.pb')
    
    # 加载coco数据集分类 TODO 需要改为自己文件路径
    f = open("F:/traindata/ssd_mobilenet_v2_coco_2018_03_29/classes.txt", "r")
    class_names = f.readlines()
    
    # 此值决定了何种情况下判定为元素，取值越低认定的可能性越高，相应的图中的框就越多，识别为假的可能性越高
    # TODO 如果需要则调整
    score_threshold = 0.3
    # 需要识别的图片的位置以及名称 TODO 需要改为自己文件路径
    img_file = 'F:/traindata/labelImgdata/img/4.jpg'
    # resize img for a format size TODO 可以修改成为自己想要的尺寸
    RESIZE_IMG_WIDTH = 500
    RESIZE_IMG_HEIGHT = 330
    
    # Read the graph.
    with tf.gfile.FastGFile(frozen_pb_file, 'rb') as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
    
    with tf.Session() as sess:
        # Restore session
        sess.graph.as_default()
        tf.import_graph_def(graph_def, name='')
    
        # Read and preprocess an image.
        img_cv2 = cv2.imread(img_file)
        img_height, img_width, _ = img_cv2.shape
        # 对图片的长宽进行格式化，防止图片过大
        img_in = cv2.resize(img_cv2, (RESIZE_IMG_WIDTH, RESIZE_IMG_HEIGHT))
        img_in = img_in[:, :, [2, 1, 0]]  # BGR2RGB
    
        # Run the model
        outputs = sess.run([sess.graph.get_tensor_by_name('num_detections:0'),
                            sess.graph.get_tensor_by_name('detection_scores:0'),
                            sess.graph.get_tensor_by_name('detection_boxes:0'),
                            sess.graph.get_tensor_by_name('detection_classes:0')],
                            feed_dict={ 'image_tensor:0': img_in.reshape(1, img_in.shape[0],
                                img_in.shape[1], 3)})
    
        # Visualize detected bounding boxes.
        num_detections = int(outputs[0][0])
        for i in range(num_detections):
            classId = int(outputs[3][0][i])
            score = float(outputs[1][0][i])
            bbox = [float(v) for v in outputs[2][0][i]]
            if score > score_threshold:
                x = bbox[1] * img_width
                y = bbox[0] * img_height
                right = bbox[3] * img_width
                bottom = bbox[2] * img_height
                # 标框
                cv2.rectangle(img_cv2, (int(x), int(y)), (int(right), int(bottom)), (125, 255, 51), thickness=3)
                # 文字"class_name, score"
                cv2.putText(img_cv2, class_names[classId - 1][:-1] + "," + str("%.2f" % score), (int(x), int(y)),
                            cv2.FONT_HERSHEY_DUPLEX, .5, (0, 0, 255), 1)
                print(str(classId) + ",class:" + class_names[classId - 1][:-1] + ",score:%.2f" % score)
    
    # figsize : 指定figure的宽和高，单位为英寸
    plt.figure(figsize=(10, 8))
    plt.imshow(img_cv2[:, :, ::-1])
    plt.title("TensorFlow MobileNetV2-SSD")
    plt.axis("off")
    plt.show()

4、执行代码：![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQwNDE4MzAy_size_16_color_FFFFFF_t_70_pic_center 1]  
我们可以尝试使用不同的图片、修改代码中的score\_threshold值等方式查看执行情况。

[python_openCV_WINDOWS]: https://editor.csdn.net/md/?articleId=108196768
[20200829101819897.png_pic_center]: /images/20221123/17de6cb4520647a1aadab5cf30823f6d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQwNDE4MzAy_size_16_color_FFFFFF_t_70_pic_center]: /images/20221123/8397ca38770a4439b351192873391c57.png
[classes.txt]: https://download.csdn.net/download/qq_40418302/12837906
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQwNDE4MzAy_size_16_color_FFFFFF_t_70_pic_center 1]: /images/20221123/fd24cf3b7d5245d1a6fad5fb74f31d5c.png