使用Keras的面部表情识别

一时失言乱红尘 2024-04-01 17:41 134阅读 0赞

使用Keras的面部表情识别 项目实施…

介绍和概述

Keras是一个非常强大的开源Python库，它运行在TensorFlow、Theano等其他开源机器库之上，用于开发和评估深度学习模型并利用各种优化技术。

Keras完全支持递归神经网络和卷积神经网络 Keras在CPU和GPU上都能顺利运行 Keras神经网络是用Python编写的，主张简单和强大的调试能力。 Keras以其令人难以置信的表现力、灵活、最小的结构而闻名。 Keras具有一致、简单和可扩展的API Keras还以其高度的计算可扩展性而闻名。 对各种平台和后端的广泛支持 210个机器学习项目（含源代码），你今天就可以建立了 有源代码 中等.datadriveninvestor.com

在这个项目中，我们将使用Keras实现面部表情识别。我们的数据集（已经分为训练集和测试集）包括从Kaggle repo下载的不同面部表情的图像。

  1. Import Libraries
  2. import numpy as np
  3. import seaborn as sns
  4. import matplotlib.pyplot as plt
  5. import utils
  6. import os
  7. %matplotlib inline
  8. from tensorflow.keras.preprocessing.image import ImageDataGenerator
  9. from tensorflow.keras.layers import Dense, Input, Dropout,Flatten, Conv2D
  10. from tensorflow.keras.layers import BatchNormalization, Activation, MaxPooling2D
  11. from tensorflow.keras.models import Model, Sequential
  12. from tensorflow.keras.optimizers import Adam
  13. from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
  14. from tensorflow.keras.utils import plot_model
  15. from IPython.display import SVG, Image
  16. from livelossplot import PlotLossesTensorFlowKeras
  17. import tensorflow as tf
  18. print("Tensorflow version:", tf.__version__)

输出

  1. Tensorflow版本：2.1.0

绘制表达式图像

  1. utils.datasets.fer.plot_example_images（plt）.show()
  3. for expression in os.listdir("train/"):
  4.     print(str(len(os.listdir("train/" + expression)))+ " " + expression + " images")
  5. Output —
  7. 3171 surprise images
  8. 7215 happy images
  9. 4965 neutral images
  10. 3995 angry images
  11. 4830 sad images
  12. 436 disgust images
  13. 4097 fear images)
  15. 输出 -
  17. 3171张惊喜图片
  18. 7215张快乐的图像
  19. 4965幅中性图像
  20. 3995张愤怒的图像
  21. 4830个悲伤的图像
  22. 436 厌恶的图像
  23. 4097张恐惧图像

生成训练和验证批次

为了充分利用我们的训练实例，我们将通过一些随机变换来 “增强 “它们，这反过来有助于防止模型过拟合，并帮助模型更好地泛化。

keras.preprocessing.image.ImageDataGenerator类在keras允许—。

通过.flow(data, labels)或.flow_from_directory(directory)来实例化增强的图像批（及其标签）的生成器。

它可以与接受数据生成器作为输入的Keras模型方法、fit_generator、evaluation_generator和predict_generator一起使用。

配置随机变换和归一化操作，以便在训练期间对你的图像数据进行处理。

  1. img_size = 48
  2. batch_size = 64
  3. datagen_train = ImageDataGenerator(horizontal_flip=True)
  4. train_generator = datagen_train.flow_from_directory("train/",
  5.                   target_size = (img_size,img_size),
  6.                   color_mode='grayscale'。
  7.                   batch_size=batch_size。
  8.                   class_mode='categorical',shuffle= True)
  9. datagen_validation = ImageDataGenerator(horizontal_flip=True)
  10. validation_generator = datagen_validation.flow_from_directory("test/",
  11. target_size = (img_size,img_size),
  12. color_mode='grayscale'。
  13. batch_size=batch_size, class_mode='categorical',shuffle= True)

输出-

找到属于7个类别的28709张图片。 找到属于7个类别的7178张图片。 创建CNN模型 我们将使用convnet来完成这项任务。在你的模型中选择参数的数量是很重要的，即层的数量和每层的大小。

  1. model= Sequential()
  2. #1 Conv
  3. model.add(Conv2D(64,(3,3),padding='same',input_shape=(48,48,1)))
  4. model.add(BatchNormalization())
  5. model.add(Activation('relu'))
  6. model.add(MaxPooling2D(pool_size=(2,2)))
  7. model.add(Dropout(0.25))
  8. # 2 conv
  9. model.add(Conv2D(128,(5,5), padding = 'same'))
  10. model.add(BatchNormalization())
  11. model.add(Activation('relu'))
  12. model.add(MaxPooling2D(pool_size=(2,2)))
  13. model.add(Dropout(0.25))
  14. # 3 conv
  15. model.add(Conv2D(512,(3,3), padding = 'same'))
  16. model.add(BatchNormalization())
  17. model.add(Activation('relu'))
  18. model.add(MaxPooling2D(pool_size=(2,2)))
  19. model.add(Dropout(0.25))
  20. # 4 conv
  21. model.add(Conv2D(512,(3,3), padding = 'same'))
  22. model.add(BatchNormalization())
  23. model.add(Activation('relu'))
  24. model.add(MaxPooling2D(pool_size=(2,2)))
  25. model.add(Dropout(0.25))
  26. model.add(Flatten())
  27. model.add(Dense(256))
  28. model.add(BatchNormalization())
  29. model.add(Activation('relu'))
  30. model.add(Dropout(0.25))
  31. model.add(Dense(512))
  32. model.add(BatchNormalization())
  33. model.add(Activation('relu'))
  34. model.add(Dropout(0.25))
  35. model.add(Dense(7,activation='softmax'))
  36. opt=Adam(lr=0.0005)
  37. model.compile(optimizer=opt,loss='categorical_crossentropy',metrics=['accuracy'])
  38. model.summary()
  39. Output —
  41. Model: "sequential_3"
  43. Model: "sequential_3"
  44. _________________________________________________________________
  45. Layer (type)                 Output Shape              Param #   
  46. =================================================================
  47. conv2d_4 (Conv2D)            (None, 48, 48, 64)        640       
  48. _________________________________________________________________
  49. batch_normalization_6 (Batch (None, 48, 48, 64)        256       
  50. _________________________________________________________________
  51. activation_6 (Activation)    (None, 48, 48, 64)        0         
  52. _________________________________________________________________
  53. max_pooling2d_4 (MaxPooling2 (None, 24, 24, 64)        0         
  54. _________________________________________________________________
  55. dropout_6 (Dropout)          (None, 24, 24, 64)        0         
  56. _________________________________________________________________
  57. conv2d_5 (Conv2D)            (None, 24, 24, 128)       204928    
  58. _________________________________________________________________
  59. batch_normalization_7 (Batch (None, 24, 24, 128)       512       
  60. _________________________________________________________________
  61. activation_7 (Activation)    (None, 24, 24, 128)       0         
  62. _________________________________________________________________
  63. max_pooling2d_5 (MaxPooling2 (None, 12, 12, 128)       0         
  64. _________________________________________________________________
  65. dropout_7 (Dropout)          (None, 12, 12, 128)       0         
  66. _________________________________________________________________
  67. conv2d_6 (Conv2D)            (None, 12, 12, 512)       590336    
  68. _________________________________________________________________
  69. batch_normalization_8 (Batch (None, 12, 12, 512)       2048      
  70. _________________________________________________________________
  71. activation_8 (Activation)    (None, 12, 12, 512)       0         
  72. _________________________________________________________________
  73. max_pooling2d_6 (MaxPooling2 (None, 6, 6, 512)         0         
  74. _________________________________________________________________
  75. dropout_8 (Dropout)          (None, 6, 6, 512)         0         
  76. _________________________________________________________________
  77. conv2d_7 (Conv2D)            (None, 6, 6, 512)         2359808   
  78. _________________________________________________________________
  79. batch_normalization_9 (Batch (None, 6, 6, 512)         2048      
  80. _________________________________________________________________
  81. activation_9 (Activation)    (None, 6, 6, 512)         0         
  82. _________________________________________________________________
  83. max_pooling2d_7 (MaxPooling2 (None, 3, 3, 512)         0         
  84. _________________________________________________________________
  85. dropout_9 (Dropout)          (None, 3, 3, 512)         0         
  86. _________________________________________________________________
  87. flatten_1 (Flatten)          (None, 4608)              0         
  88. _________________________________________________________________
  89. dense_3 (Dense)              (None, 256)               1179904   
  90. _________________________________________________________________
  91. batch_normalization_10 (Batc (None, 256)               1024      
  92. _________________________________________________________________
  93. activation_10 (Activation)   (None, 256)               0         
  94. _________________________________________________________________
  95. dropout_10 (Dropout)         (None, 256)               0         
  96. _________________________________________________________________
  97. dense_4 (Dense)              (None, 512)               131584    
  98. _________________________________________________________________
  99. batch_normalization_11 (Batc (None, 512)               2048      
  100. _________________________________________________________________
  101. activation_11 (Activation)   (None, 512)               0         
  102. _________________________________________________________________
  103. dropout_11 (Dropout)         (None, 512)               0         
  104. _________________________________________________________________
  105. dense_5 (Dense)              (None, 7)                 3591      
  106. =================================================================
  107. Total params: 4,478,727
  108. Trainable params: 4,474,759
  109. Non-trainable params: 3,968
  110. _________________________________________________________________

Train and Evaluate Model

  1. epochs = 15
  2. steps_per_epoch = train_generator.n//train_generator.batch_size
  3. validation_steps= validation_generator.n//validation_generator.batch_size
  4. checkpoint= ModelCheckpoint("model_weights.h5",monitor='val_accuracy',
  5.                            save_weights_only= True,  
  6.                 mode='max',verbose=1)
  7. reduce_lr = ReduceLROnPlateau(monitor='val_loss',factor=0.1,patience=2,min_lr=0.00001,model='auto')
  8. callbacks=[PlotLossesTensorFlowKeras(),checkpoint, reduce_lr]
  9. history = model.fit(
  10.           x=train_generator,
  11.           steps_per_epoch=steps_per_epoch,
  12.           epochs =epochs,
  13.           validation_data=validation_generator,
  14.           validation_steps=validation_steps,
  15.           callbacks = callbacks
  16. )
  18. Log-loss (cost function):
  19. training   (min:    1.032, max:    1.790, cur:    1.032)
  20. validation (min:    1.041, max:    1.797, cur:    1.041)
  22. accuracy:
  23. training   (min:    0.315, max:    0.608, cur:    0.608)
  24. validation (min:    0.327, max:    0.612, cur:    0.612)
  26. Epoch 00015: saving model to model_weights.h5
  27. 448/448 [==============================] - 27s 61ms/step - loss: 1.0317 - accuracy: 0.6082 - val_loss: 1.0407 - val_accuracy: 0.6124

建立模型为JSON Str

  1. model_json=model.to_json()
  2. with open("model_json", "w") as json_file:
  3.     json_file.write(model_json)

构建一个用于预测面部表情的Flask应用程序

我们将使用open cv分类器来自动检测图像中的人脸，并画出边界框，用于构建应用程序并导入面部表情模型。

OpenCV提供了一个训练方法Cascade Classifier Training，可以使用cv::CascadeClassifier::load方法读取。由Paul Viola和Michael Jones提出的使用基于Haar特征的级联分类器的物体检测方法是一种基于机器学习的方法，从大量的正负图像中训练出一个级联函数。

  1. import cv2
  2. from model import FacialExpressionModel
  3. import numpy as np
  4. facec = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
  5. model = FacialExpressionModel("model.json", "model_weights.h5")
  6. font = cv2.FONT_HERSHEY_SIMPLEX
  7. class VideoCamera(object):
  8.     def __init__(self):
  9.         self.video = cv2.VideoCapture("path to video file")
  10. def __del__(self):
  11.         self.video.release()
  12. # returns camera frames along with bounding boxes and predictions
  13.     def get_frame(self):
  14.         _, fr = self.video.read()
  15.         gray_fr = cv2.cvtColor(fr, cv2.COLOR_BGR2GRAY)
  16.         faces = facec.detectMultiScale(gray_fr, 1.3, 5)
  17. for (x, y, w, h) in faces:
  18.             fc = gray_fr[y:y+h, x:x+w]
  19. roi = cv2.resize(fc, (48, 48))
  20.             pred = model.predict_emotion(roi[np.newaxis, :, :, 
  21.                    np.newaxis])
  22. cv2.putText(fr, pred, (x, y), font, 1, (255, 255, 0), 2)
  23.             cv2.rectangle(fr,(x,y),(x+w,y+h),(255,0,0),2)
  24. _, jpeg = cv2.imencode('.jpg', fr)
  25.         return jpeg.tobytes()

用面部表情列表作为model.py文件构建脚本

tf.keras.models.model_from_json解析我们上面实例化的JSON模型配置字符串并返回一个模型实例。面部表情识别是将面部图像上的表情分为各种类别的技术，如愤怒、恐惧、惊讶、悲伤、快乐等。

  1. from tensorflow.keras.models import model_from_json
  2. import numpy as np
  3. import tensorflow as tf
  4. config.tf.compat.v1.ConfigProto()
  5. config.gpu_options.per_process_gpu_memory_fraction =0.15
  6. session = tf.compat.v1.Session(config=config)
  7. class FacialExpressionModel(object):
  8. EMOTIONS_LIST= ["Happy","Neutral","Sad","Surprise","Angry","Disgust","Fear"]
  9. def __init__(self,model_json_file,model_weights_file):
  10.         with open(model_json_file, "r") as json_file:
  11.             loaded_model_json = json_file.read()
  12.             self.loaded_model = model_from_json(loaded_model_json)
  13. self.loaded_model.load_weights(model_weights_file)
  14.         self.loaded_model._make_predict_function()
  15. def predict_emotion(self,img):
  16.         self.preds = self.loaded_model.predict(img)
  17.         return FacialExpressionModel.EMOTIONS_LIST[np.argmax(self.preds)]

Build an HTML Template for the Flask App

  1. <html>
  2.   <head>
  3.      <title> Facial Expression Recognition </title>
  4.   </head>
  5.   <body>
  6.      <img id="bg" width=800px height = 640px src="{
  7.      {url_for('video_feed')}}">
  8.   </body>
  9. </html>

Run Model to Recognize Facial Expressions

  1. from flask import Flask, render_template, Response
  2. from camera import VideoCamera
  3. app = Flask(__name__)
  4. @app.route('/')
  5. def index():
  6.     return render_template('index.html')
  7. def gen(camera):
  8.     while True:
  9.         frame = camera.get_frame()
  10.         yield (b'--frame\r\n'
  11.                b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
  12. @app.route('/video_feed')
  13. def video_feed():
  14.     return Response(gen(VideoCamera()),
  15.                     mimetype='multipart/x-mixed-replace; boundary=frame')
  16. if __name__ == '__main__':
  17.     app.run(host='0.0.0.0', debug=True)
  18. Run the main.py and that’s it.
  20. You should be able to see the tagged facial expression for the video you used in your code.

本文由 mdnice 多平台发布

发表评论

表情：
评论列表 （有 0 条评论，134人围观）

还没有评论，来说两句吧...

相关阅读

    相关 keras图像识别

    终于抓住11月的尾部，赶在12月到来之前进行更新了，2018准备要跟我们说拜拜了，小伙伴们你们今年的目标实现了吗？，好了扯远了，今天趁着有时间更新一下博客，记录一下使用kera

    叁歲伎倆叁歲伎倆/ 0 赞/ 415 阅读