人脸检测（三）

谁践踏了优雅 2022-06-16 05:57 442阅读 0赞

**3.OpenCV人脸检测方法**  
采用的是harr-like特征提取 + adaboost级联分类器构成的人脸检测算法，可以设置要检测的人脸大小范围，召回率高，但有时会出现误检；当设置要检测的人脸比较大时，检测速度很快，可以达到实时检测；检测到的人脸框位置不够精细。

![这里写图片描述][SouthEast]

参考opencv安装目录下source\\samples\\cpp文件夹tutorial\_code\\objectDetection中的示例程序，[官网][Link 1] 中有人脸检测相关的函数说明。  
下面给出利用opencv3进行人脸检测的python和C++代码，可以进行单张图像、摄像头中或视频中的人脸检测。  
python版本：

# -*- coding: utf-8 -*-
    """
    Created on Sun May 07 18:49:11 2017
    
    @author: Administrator
    """
    import cv2
    import time
    
    def face_detection(ccf,im):
        if im.shape[2] == 3:
            gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY) #级联分类器只能检测灰度图像
        else:
            gray = im
        faces = ccf.detectMultiScale(gray,1.1,3,0,(64,64),(256,256)) #多尺度人脸检测，设置检测的人脸大小范围
        for face in faces: #face:(x,y,width,height)
            cv2.rectangle(im,(face[0],face[1]),(face[0]+face[2],face[1]+face[3]),(0,0,255),2,8)
        return im
    
    def read_capture(ccf):
    #    video = cv2.VideoCapture('xxx.mp4') #从视频中读取
        video = cv2.VideoCapture(0) #从摄像头读取
        if video.isOpened():
            success,frame = video.read()
            while success:
                im = face_detection(ccf,frame)
                cv2.imshow('capture face detection',im)
                if cv2.waitKey(1) >= 0 :
                    break
                success,frame = video.read()
            cv2.destroyAllWindows()
            video.release()
    
    if __name__ == '__main__' :
        ccf = cv2.CascadeClassifier('D:/OPENCV3.0.0/opencv/sources/data/haarcascades/haarcascade_frontalface_alt2.xml') #加载级联人脸检测器
        start_time = time.time()                           
        #从单张图像中检测人脸
        im = cv2.imread('C:/Users/Administrator/Desktop/caffe/matlab/demo/1.jpg')
        output = face_detection(ccf,im)
    
        end_time = time.time()
        print (end_time-start_time)*1000,'ms'
        cv2.imshow('face detection',output)
        cv2.waitKey(0)
        cv2.destroyAllWindows()
    
    #    read_capture(ccf) #视频或摄像头中人脸检测

C++版本：

#include <iostream>
    #include <opencv2/opencv.hpp>
    #include <time.h>
    using namespace cv;
    using namespace std;
    
    Mat faceDetection(CascadeClassifier ccf, Mat img_src);
    void readCapture(CascadeClassifier ccf);
    void readVideo(CascadeClassifier ccf);
    
    void main()
    {
        //string xmlPath = "D:\\OPENCV3.0.0\\opencv\\sources\\data\\haarcascades\\haarcascade_frontalface_default.xml"; //训练好的分类xml文件
        string xmlPath = "D:\\OPENCV3.0.0\\opencv\\sources\\data\\haarcascades\\haarcascade_frontalface_alt2.xml"; //训练好的分类xml文件(改进版)
        CascadeClassifier ccf; //级联分类器，很重要的类
        if (!ccf.load(xmlPath))
        {
            cout << "can not open this xml file" << endl;
            system("pause");
            return;
        }
    
        //readCapture(ccf); //摄像头中人脸检测
        //readVideo(ccf); //视频中人脸检测
        //单张图片人脸检测
        Mat img_src = imread("1.jpg");
        Mat img_dst = faceDetection(ccf, img_src); 
        imshow("face detection", img_dst);
        waitKey(0);
    }
    
    //opencv人脸检测，原理是提取harr或lbp特征，送入adaboost级联分类器
    //输入待检测图像，返回画框/圆图像
    Mat faceDetection(CascadeClassifier ccf, Mat img_src)
    {
        clock_t start, end; //用于计时，毫秒级
        start = clock();
    
        Mat img_dst = img_src.clone(); //将输入图像拷贝为输出结果图像
    
        Mat gray;
        cvtColor(img_src, gray, COLOR_BGR2GRAY); //在opencv3命名规范中,CV_BGR2GRAY变成了COLOR_BGR2GRAY
        equalizeHist(gray, gray);
    
        /*
        void CascadeClassifier::detectMultiScale(InputArray image, vector<Rect>& objects, double scaleFactor=1.1,
        int minNeighbors=3, int flags=0, Size minSize=Size(), Size maxSize=Size())
        第一个参数image:  要检测的图片，一般为灰度图
        第二个参数objects:  Rect型的容器，存放所有检测出的人脸，每个人脸是一个矩形
        第三个参数scaleFactor:  缩放因子，对图片进行缩放，默认为1.1
        第四个参数minNeighbors： 最小邻居数，默认为3
        第五个参数flags:  兼容老版本的一个参数，在3.0版本中没用处。默认为0
        第六个参数minSize: 最小尺寸，检测出的人脸最小尺寸
        第七个参数maxSize： 最大尺寸，检测出的人脸最大尺寸
        */
        vector<Rect> faces; //定义人脸框
        ccf.detectMultiScale(gray, faces, 1.1, 3, 0, Size(64 + 32, 64 + 32), Size(256 + 128, 256 + 128)); //级联分类器最核心函数，多尺度检测，设置要检测的人脸大小范围
    
        end = clock();
        cout << (double)(end - start) / CLOCKS_PER_SEC << endl;
    
        for (auto face : faces)
        {
            rectangle(img_dst, face, Scalar(0, 0, 255), 2, 8); //画矩形框
    
            //画圆形
            //Point center;
            //center.x = face.x + face.width * 0.5;
            //center.y = face.y + face.height * 0.5;
            //int radius = (face.width + face.height) * 0.25;
            //circle(img_dst,center,radius,Scalar(0,0,255),2.8);
        }
        return img_dst;
    }
    
    //读取摄像头
    void readCapture(CascadeClassifier ccf)
    {
        VideoCapture capture(0);
        if (!capture.isOpened())
        {
            cout << "can not open capture..." << endl;
            return;
        }
        int width = (int)capture.get(CAP_PROP_FRAME_WIDTH); //opencv3中将CV_CAP_PROP_FRAME_WIDTH去掉了CV_前缀，变成了CAP_PROP_FRAME_WIDTH
        int height = (int)capture.get(CAP_PROP_FRAME_HEIGHT);
        cout << "width: " << width << "," << "height: " << height << endl;
        Mat img;
        while (1)
        {
            Mat frame;
            capture >> frame;
    
            img = faceDetection(ccf, frame);
    
            imshow("capture face detection", img);
            if (waitKey(1) >= 0) break;
        }
    }
    //读取视频
    void readVideo(CascadeClassifier ccf)
    {
        VideoCapture capture("test.avi");
        int width = (int)capture.get(CAP_PROP_FRAME_WIDTH); //opencv3中将CV_CAP_PROP_FRAME_WIDTH去掉了CV_前缀，变成了CAP_PROP_FRAME_WIDTH
        int height = (int)capture.get(CAP_PROP_FRAME_HEIGHT);
        cout << "width: " << width << "," << "height: " << height << endl;
        Mat img;
        while (1)
        {
            Mat frame;
            capture >> frame;
            if (frame.empty())
                break;
    
            img = faceDetection(ccf, frame); //人脸检测
    
            imshow("video face detection", img);
            if (waitKey(1) >= 0) break;
        }
    }

--------------------

**4.Dlib人脸检测方法**

dlib人脸检测采用的是经典的梯度方向直方图（HOG特征）提取 + 线性分类器 + 图像金字塔 + 滑动窗口的人脸检测方法，速度很快，只能检测80x80或更大的人脸，但可以通过图像上采样来检测更小的人脸。相比于opencv的人脸检测，召回率更高，误检率更低，且人脸框更准确，检测速度也同样很快。

![这里写图片描述][SouthEast 1]

[dlib官网][dlib] 有face\_detection的示例代码，包括C++和Python的，代码很精简，也是直接调用它给的接口。下面给出利用dlib进行人脸检测的python代码，用于检测单张图像、视频或摄像头中的人脸。

# coding: utf-8
    #face detection by dlib
    
    #   This face detector is made using the now classic Histogram of Oriented
    #   Gradients (HOG) feature combined with a linear classifier, an image
    #   pyramid, and sliding window detection scheme.  This type of object detector
    #   is fairly general and capable of detecting many types of semi-rigid objects
    #   in addition to human faces.  Therefore, if you are interested in making
    #   your own object detectors then read the train_object_detector.py example
    #   program.  
    import dlib
    import cv2
    import time
    
    def read_capture(detector):
    #    video = cv2.VideoCapture('xxx.mp4') #从视频中读取
        video = cv2.VideoCapture(0) #从摄像头读取
        if video.isOpened():
            success,frame = video.read()
            while success:
                im = frame.copy()
                #dlib的人脸检测器只能检测80x80和更大的人脸，如果需要检测比它小的人脸，需要对图像上采样，一次上采样图像尺寸放大一倍
                #rects = detector(img,1) #1次上采样
                rects = detector(im,0)
                for rect in rects: #rect.left(),rect.top(),rect.right(),rect.bottom()
                    cv2.rectangle(im,(rect.left(),rect.top()),(rect.right(),rect.bottom()),(0,0,255),2,8)
                cv2.imshow('capture face detection',im)
                if cv2.waitKey(1) >= 0 :
                    break
                success,frame = video.read()
            cv2.destroyAllWindows()
            video.release()
    
    if __name__ == '__main__' :
        detector = dlib.get_frontal_face_detector() #获取dlib正脸检测器
        start_time = time.time()                           
        #从单张图像中检测人脸
        im = cv2.imread('C:/Users/Administrator/Desktop/caffe/matlab/demo/1.jpg')
        #dlib的人脸检测器只能检测80x80和更大的人脸，如果需要检测比它小的人脸，需要对图像上采样，一次上采样图像尺寸放大一倍
        #rects = detector(img,1) #1次上采样
        #rects = detector(img,0) #0次上采样
        rects = detector(im,2)
        for rect in rects: #rect.left(),rect.top(),rect.right(),rect.bottom()
            cv2.rectangle(im,(rect.left(),rect.top()),(rect.right(),rect.bottom()),(0,0,255),2,8)
        end_time = time.time()
        print (end_time-start_time)*1000,'ms'
        cv2.imshow('dlib face detection',im)
        cv2.waitKey(0)
        cv2.destroyAllWindows()
    
        #read_capture(detector) #检测视频或摄像头中的人脸

--------------------

**5.libfacedetect人脸检测方法**

libfacedetect是深圳大学于仕祺老师提供的免费的人脸检测库，以二进制的方式免费发布可商用，可将这个库应用到系统中，无论是科研目的还是商业目的，无任何限制。该库调用非常简单，只有一个函数，纯C语言编译而成，不依赖任何第三方库。通过连续的优化，其人脸检测算法可以做到3.6毫秒处理一张VGA图像（最小48x48人脸），无论是检测速度、效果和人脸框位置的准确性都胜过opencv！！！  
libfacedetect提供了四套接口，分别为frontal、frontal\_surveillance、multiview、multiview\_reinforce，其中multiview\_reinforce效果最好，可以检测侧脸，速度比其它接口稍慢，但仍然比opencv的正脸检测要快，总而言之，非常好用，用起来非常简单！！！  
此外，libfacedetect和dlib一样提供了人脸特征点检测，且libfacedetect对侧脸检测仍有不错的效果。详情可以参考 [github][]

![这里写图片描述][SouthEast 2]

/*
    The MIT License (MIT)
    
    Copyright (c) 2015-2017 Shiqi Yu
    shiqi.yu@gmail.com
    
    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the "Software"), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:
    
    The above copyright notice and this permission notice shall be included in all
    copies or substantial portions of the Software.
    
    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
    SOFTWARE.
    */
    
    #include <stdio.h>
    #include <opencv2/opencv.hpp>
    #include "facedetect-dll.h"
    
    //#pragma comment(lib,"libfacedetect.lib")
    #pragma comment(lib,"libfacedetect-x64.lib")
    
    //define the buffer size. Do not change the size!
    #define DETECT_BUFFER_SIZE 0x20000
    using namespace cv;
    
    int main()
    {
        //load an image and convert it to gray (single-channel)
        Mat image = imread("C:\\Users\\Administrator\\Desktop\\faces\\39649a2f070828388e658cb3ba99a9014d08f1cc.jpg");
        Mat gray;
        cvtColor(image, gray, CV_BGR2GRAY);
    
    
        int * pResults = NULL;
        //pBuffer is used in the detection functions.
        //If you call functions in multiple threads, please create one buffer for each thread!
        unsigned char * pBuffer = (unsigned char *)malloc(DETECT_BUFFER_SIZE);
        if (!pBuffer)
        {
            fprintf(stderr, "Can not alloc buffer.\n");
            return -1;
        }
    
        int doLandmark = 1;
    
        ///////////////////////////////////////////
        // frontal face detection / 68 landmark detection
        // it's fast, but cannot detect side view faces
        //////////////////////////////////////////
        //!!! The input image must be a gray one (single-channel)
        //!!! DO NOT RELEASE pResults !!!
        pResults = facedetect_frontal(pBuffer, (unsigned char*)(gray.ptr(0)), gray.cols, gray.rows, (int)gray.step,
            1.2f, 2, 48, 0, doLandmark);
    
        printf("%d faces detected.\n", (pResults ? *pResults : 0));
        Mat result_frontal = image.clone();
        //print the detection results
        for (int i = 0; i < (pResults ? *pResults : 0); i++)
        {
            short * p = ((short*)(pResults + 1)) + 142 * i;
            int x = p[0];
            int y = p[1];
            int w = p[2];
            int h = p[3];
            int neighbors = p[4];
            int angle = p[5];
    
            printf("face_rect=[%d, %d, %d, %d], neighbors=%d, angle=%d\n", x, y, w, h, neighbors, angle);
            rectangle(result_frontal, Rect(x, y, w, h), Scalar(0, 255, 0), 2);
            if (doLandmark)
            {
                for (int j = 0; j < 68; j++)
                    circle(result_frontal, Point((int)p[6 + 2 * j], (int)p[6 + 2 * j + 1]), 1, Scalar(0, 255, 0));
            }
        }
        imshow("Results_frontal", result_frontal);
    
        /////////////////////////////////////////////
        //// frontal face detection designed for video surveillance / 68 landmark detection
        //// it can detect faces with bad illumination.
        ////////////////////////////////////////////
        ////!!! The input image must be a gray one (single-channel)
        ////!!! DO NOT RELEASE pResults !!!
        //pResults = facedetect_frontal_surveillance(pBuffer, (unsigned char*)(gray.ptr(0)), gray.cols, gray.rows, (int)gray.step,
        //  1.2f, 2, 48, 0, doLandmark);
        //printf("%d faces detected.\n", (pResults ? *pResults : 0));
        //Mat result_frontal_surveillance = image.clone();;
        ////print the detection results
        //for (int i = 0; i < (pResults ? *pResults : 0); i++)
        //{
        
        //  short * p = ((short*)(pResults + 1)) + 142 * i;
        //  int x = p[0];
        //  int y = p[1];
        //  int w = p[2];
        //  int h = p[3];
        //  int neighbors = p[4];
        //  int angle = p[5];
    
        //  printf("face_rect=[%d, %d, %d, %d], neighbors=%d, angle=%d\n", x, y, w, h, neighbors, angle);
        //  rectangle(result_frontal_surveillance, Rect(x, y, w, h), Scalar(0, 255, 0), 2);
        //  if (doLandmark)
        //  {
        
        //      for (int j = 0; j < 68; j++)
        //          circle(result_frontal_surveillance, Point((int)p[6 + 2 * j], (int)p[6 + 2 * j + 1]), 1, Scalar(0, 255, 0));
        //  }
        //}
        //imshow("Results_frontal_surveillance", result_frontal_surveillance);
    
    
        /////////////////////////////////////////////
        //// multiview face detection / 68 landmark detection
        //// it can detect side view faces, but slower than facedetect_frontal().
        ////////////////////////////////////////////
        ////!!! The input image must be a gray one (single-channel)
        ////!!! DO NOT RELEASE pResults !!!
        //pResults = facedetect_multiview(pBuffer, (unsigned char*)(gray.ptr(0)), gray.cols, gray.rows, (int)gray.step,
        //  1.2f, 2, 48, 0, doLandmark);
    
        //printf("%d faces detected.\n", (pResults ? *pResults : 0));
        //Mat result_multiview = image.clone();;
        ////print the detection results
        //for (int i = 0; i < (pResults ? *pResults : 0); i++)
        //{
        
        //  short * p = ((short*)(pResults + 1)) + 142 * i;
        //  int x = p[0];
        //  int y = p[1];
        //  int w = p[2];
        //  int h = p[3];
        //  int neighbors = p[4];
        //  int angle = p[5];
    
        //  printf("face_rect=[%d, %d, %d, %d], neighbors=%d, angle=%d\n", x, y, w, h, neighbors, angle);
        //  rectangle(result_multiview, Rect(x, y, w, h), Scalar(0, 255, 0), 2);
        //  if (doLandmark)
        //  {
        
        //      for (int j = 0; j < 68; j++)
        //          circle(result_multiview, Point((int)p[6 + 2 * j], (int)p[6 + 2 * j + 1]), 1, Scalar(0, 255, 0));
        //  }
        //}
        //imshow("Results_multiview", result_multiview);
    
    
        /////////////////////////////////////////////
        //// reinforced multiview face detection / 68 landmark detection
        //// it can detect side view faces, better but slower than facedetect_multiview().
        ////////////////////////////////////////////
        ////!!! The input image must be a gray one (single-channel)
        ////!!! DO NOT RELEASE pResults !!!
        //pResults = facedetect_multiview_reinforce(pBuffer, (unsigned char*)(gray.ptr(0)), gray.cols, gray.rows, (int)gray.step,
        //  1.2f, 3, 48, 0, doLandmark);
    
        //printf("%d faces detected.\n", (pResults ? *pResults : 0));
        //Mat result_multiview_reinforce = image.clone();;
        ////print the detection results
        //for (int i = 0; i < (pResults ? *pResults : 0); i++)
        //{
        
        //  short * p = ((short*)(pResults + 1)) + 142 * i;
        //  int x = p[0];
        //  int y = p[1];
        //  int w = p[2];
        //  int h = p[3];
        //  int neighbors = p[4];
        //  int angle = p[5];
    
        //  printf("face_rect=[%d, %d, %d, %d], neighbors=%d, angle=%d\n", x, y, w, h, neighbors, angle);
        //  rectangle(result_multiview_reinforce, Rect(x, y, w, h), Scalar(0, 255, 0), 2);
        //  if (doLandmark)
        //  {
        
        //      for (int j = 0; j < 68; j++)
        //          circle(result_multiview_reinforce, Point((int)p[6 + 2 * j], (int)p[6 + 2 * j + 1]), 1, Scalar(0, 255, 0));
        //  }
        //}
        //imshow("Results_multiview_reinforce", result_multiview_reinforce);
    
        waitKey();
    
        //release the buffer
        free(pBuffer);
    
        return 0;
    }

--------------------

**6.Seetaface人脸检测方法**

Seetaface由中科院计算所山世光研究员带领的人脸识别研究组研发，代码基于C++实现，不依赖第三方库，开源协议为BSD-2，可供学术界和工业界免费使用。SeetaFace人脸识别引擎包括了搭建一套全自动人脸识别系统所需的三个核心模块，即：人脸检测模块SeetaFace Detection、面部特征点定位模块SeetaFace Alignment以及人脸特征提取与比对模块SeetaFace Identification。它是一套包括所有技术模块的、完全开源的基准人脸识别系统。[开源Github地址][Github]  
其中，SeetaFace Detection采用了一种结合传统人造特征与多层感知机（MLP）的级联结构，在FDDB上达到了84.4%的召回率（100个误检时），并可在单个i7 CPU上实时处理VGA分辨率的图像。  
人脸检测模块SeetaFace Detection采用的是一种结合经典级联结构和多层神经网络的人脸检测方法实现，其所采用的漏斗型级联结构（Funnel-Structured Cascade，FuSt）专门针对多姿态人脸检测而设计，其中引入了由粗到精的设计理念，兼顾了速度和精度的平衡。  
如图1所示，FuSt级联结构在顶部由多个针对不同姿态的快速LAB级联分类器构成，紧接着是若干个基于SURF特征的多层感知机（MLP）级联结构，最后由一个统一的MLP级联结构（同样基于SURF特征）来处理所有姿态的候选窗口，整体上呈现出上宽下窄的漏斗形状。从上往下，各个层次上的分类器及其所采用的特征逐步变得复杂，从而可以保留人脸窗口并排除越来越难与人脸区分的非人脸候选窗口。  
![这里写图片描述][SouthEast 3]

本人在ubuntu下编译测试过，FaceDetection模块问题不大，FaceAlign和FaceIdentification有些小坑要踩，参照以下两个博客，测试效果还是很不错的。  
![这里写图片描述][SouthEast 4]  
![这里写图片描述][SouthEast 5]

--------------------

**小结**

对比上述六种人脸检测方法，其中opencv、dlib和libfacedetect、seetaface都提供了人脸检测的接口可以直接调用，检测效果上个人认为是dlib ≈ libfacedetect ≈ seetaface > opencv，检测速度上，libfacedetect > dlib ≈ seetaface ≈ opencv。另外，前两种是采用深度学习的方法，单个CNN做人脸检测，效果一般速度较慢，但实现简单，而级联CNN人脸检测方法检测效果和速度都与dlib接近，但训练起来可能麻烦点。对于一般的应用需求，除了单CNN的方法，另外五种基本都能达到要求，其中libfacedetect具有最好的实时性，seetaface优势在于有一整套的人脸识别系统。  
1.单个CNN人脸检测方法  
2.级联CNN人脸检测方法  
3.OpenCV人脸检测方法  
4.Dlib人脸检测方法  
5.libfacedetect人脸检测方法  
6.Seetaface人脸检测方法  
除了上述这些方法，还有很多很多优秀的算法，在国内face++在人脸方面非常成功，目前已经提供人脸检测、人脸比对、人脸搜索、人脸属性、人脸关键点等SDK，但好像是收费的，做的的确非常出色，百度也做了类似的工作，估计是个公司都做了这里就不多说了…之后会接着介绍人脸特征点检测（人脸对齐）和在此基础上的应用。

[SouthEast]: /images/20220616/fad40afd79d544a3915a69d4aaad5e4a.png
[Link 1]: http://www.opencv.org.cn/opencvdoc/2.3.2/html/modules/objdetect/doc/cascade_classification.html#cascadeclassifier-cascadeclassifier
[SouthEast 1]: /images/20220616/3db5962114b14eadb587831bc2bf8d1d.png
[dlib]: http://dlib.net/
[github]: https://github.com/ShiqiYu/libfacedetection
[SouthEast 2]: /images/20220616/2f8a10be68d1499984f07926c3627361.png
[Github]: https://github.com/seetaface/SeetaFaceEngine
[SouthEast 3]: /images/20220616/368fdeebb9fb4ee7b7252264c73044fc.png
[SouthEast 4]: /images/20220616/e030456d688d4525bffbf11169c811a5.png
[SouthEast 5]: /images/20220616/fd958062448640669e3c9f06571c9df5.png