Keras框架神经网络算法训练MNIST分类准确率(实验) CNN-手写体识别

淩亂°似流年 2022-05-24 01:37 256阅读 0赞

MNIST数据集信息参考：[http://yann.lecun.com/exdb/mnist/index.html][http_yann.lecun.com_exdb_mnist_index.html]  
MNIST是手写数字0~10图片数据集，每一张图片包含28\*28个像素。  
**MNIST训练数据集**包含：（1.）60000张图片的像素信息，因为***神经网络的输入层表示特征的维度***，所以将图像表示成一个\[60000,28,28\]的pixel张量；（2.）60000张图片的标签信息，表示成一个\[60000,10\]的矩阵，因为图片的标签是介于0-9的数字，把标签分为10类进行one-hot编码(一个one-hot向量除了某一位数字是1以外，其余维度数字都是0)，比如标签0将表示为(\[1,0,0,0,0,0,0,0,0,0\])。  
**MNIST测试数据集**包含10000张图片的像素信息，同训练数据集一样，将这10000张图片的像素信息，表示成一个\[10000,28,28\]的张量，将10000张图片的标签表示成\[10000,10\]的矩阵。

此外，还需要把每张图片的像素信息转化成向量形式，即将\[28,28\]的矩阵按行拉平成\[1,784\]的向量，即把数据集转成 \[60000, 784\]才能放到网络中训练。 第一个维度数字用来索引图片，第二个维度数字用来索引每张图片中的像素点。 一般还需要把图片中的像素数组归一化为0-1之间。

### 1. 修改模型编译compile信息 ###

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.optimizers import SGD
    
    #载入数据
    (x_train,y_train),(x_test,y_test)=mnist.load_data()
    #60000,28,28
    print('x_shape:',x_train.shape)
    print('y_shape:',y_train.shape)
    #Output:
    #x_shape: (60000, 28, 28)
    #y_shape: (60000,)
    #重新定义数据格式
    #60000,784及归一化
    x_train=x_train.reshape(x_train.shape[0],-1)/255.0
    x_test=x_test.reshape(x_test.shape[0],-1)/255.0
    
    #转one-hot标签
    y_train=np_utils.to_categorical(y_train,num_classes=10)
    y_test=np_utils.to_categorical(y_test,num_classes=10)
    
    #创建模型,输入784个神经元,输出10个神经元
    model=Sequential([Dense(units=10,input_dim=784,bias='one',activation='softmax')])
    #定义优化器
    sgd=SGD(lr=0.2)
    #定义loss func
    model.compile(optimizer=sgd,loss='mse',metrics=['accuracy'])
    
    #训练模型
    model.fit(x_train,y_train,batch_size=32,epochs=10)
    
    #评估模型
    loss,accuracy=model.evaluate(x_test,y_test)
    
    print('loss:',loss)
    print('accuracy:',accuracy)

![70][]

#### 1.1. 修改目标函数 ####

**loss=’mse’ ——> loss=’categorical\_crossentropy’**

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.optimizers import SGD
    #载入数据
    (x_train,y_train),(x_test,y_test)=mnist.load_data()
    #重新定义数据格式
    #60000,784及归一化
    x_train=x_train.reshape(x_train.shape[0],-1)/255.0
    x_test=x_test.reshape(x_test.shape[0],-1)/255.0
    
    #转one-hot标签
    y_train=np_utils.to_categorical(y_train,num_classes=10)
    y_test=np_utils.to_categorical(y_test,num_classes=10)
    
    #创建模型,输入784个神经元,输出10个神经元
    model=Sequential([Dense(units=10,input_dim=784,bias='one',activation='softmax')])
    #定义优化器
    sgd=SGD(lr=0.2)
    #定义loss func
    #crossentropy比mse收敛快
    model.compile(optimizer=sgd,loss='categorical_crossentropy',metrics=['accuracy'])
    
    #训练模型
    model.fit(x_train,y_train,batch_size=32,epochs=10)
    
    #评估模型
    loss,accuracy=model.evaluate(x_test,y_test)
    
    print('loss:',loss)
    print('accuracy:',accuracy)

![70 1][]

#### 1.2. 修改优化器 ####

**optimizer=sgd ——> optimizer=adam**

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.optimizers import SGD,Adam   #导入不同优化器
    #载入数据
    (x_train,y_train),(x_test,y_test)=mnist.load_data()
    #重新定义数据格式
    #60000,784及归一化
    x_train=x_train.reshape(x_train.shape[0],-1)/255.0
    x_test=x_test.reshape(x_test.shape[0],-1)/255.0
    
    #转one-hot标签
    y_train=np_utils.to_categorical(y_train,num_classes=10)
    y_test=np_utils.to_categorical(y_test,num_classes=10)
    
    #创建模型,输入784个神经元,输出10个神经元
    model=Sequential([Dense(units=10,input_dim=784,bias='one',activation='softmax')])
    #定义优化器
    sgd=SGD(lr=0.2)
    adam=Adam(lr=0.001)
    #定义loss func
    #crossentropy比mse收敛快
    model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy'])
    
    #训练模型
    model.fit(x_train,y_train,batch_size=32,epochs=10)
    
    #评估模型
    loss,accuracy=model.evaluate(x_test,y_test)
    
    print('loss:',loss)
    print('accuracy:',accuracy)

![70 2][]

### 2. 修改模型网络层信息 ###

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.optimizers import SGD
    #载入数据
    (x_train,y_train),(x_test,y_test)=mnist.load_data()
    #重新定义数据格式
    #60000,784及归一化
    x_train=x_train.reshape(x_train.shape[0],-1)/255.0
    x_test=x_test.reshape(x_test.shape[0],-1)/255.0
    
    #转one-hot标签
    y_train=np_utils.to_categorical(y_train,num_classes=10)
    y_test=np_utils.to_categorical(y_test,num_classes=10)
    
    #创建模型,输入784个神经元,输出10个神经元
    model=Sequential([Dense(units=200,input_dim=784,bias='one',activation='tanh'),
                      Dense(units=100,bias='one',activation='tanh'),
                      Dense(units=10,bias='one',activation='softmax')])
    #定义优化器
    sgd=SGD(lr=0.2)
    #定义loss func
    model.compile(optimizer=sgd,loss='mse',metrics=['accuracy'])
    
    #训练模型
    model.fit(x_train,y_train,batch_size=32,epochs=10)
    
    #评估模型
    loss,accuracy=model.evaluate(x_test,y_test)
    
    print('test loss:',loss)
    print('test accuracy:',accuracy)
    
    print('train loss:',loss)
    print('train accuracy:',accuracy)

![这里写图片描述][SouthEast]

#### 2.1. 防止过拟合，选用Dropout策略 ####

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense,Dropout
    from keras.optimizers import SGD
    #载入数据
    (x_train,y_train),(x_test,y_test)=mnist.load_data()
    #重新定义数据格式
    #60000,784及归一化
    x_train=x_train.reshape(x_train.shape[0],-1)/255.0
    x_test=x_test.reshape(x_test.shape[0],-1)/255.0
    
    #转one-hot标签
    y_train=np_utils.to_categorical(y_train,num_classes=10)
    y_test=np_utils.to_categorical(y_test,num_classes=10)
    
    #创建模型,输入784个神经元,输出10个神经元(Dropout=0.4表示让前一层40%神经元不工作)
    model=Sequential([Dense(units=200,input_dim=784,bias='one',activation='tanh'),
                      Dropout(0.4),
                      Dense(units=100,bias='one',activation='tanh'),
                      Dropout(0.4),
                      Dense(units=10,bias='one',activation='softmax')])
    #定义优化器
    sgd=SGD(lr=0.2)
    #定义loss func
    model.compile(optimizer=sgd,loss='mse',metrics=['accuracy'])
    
    #训练模型
    model.fit(x_train,y_train,batch_size=32,epochs=10)
    
    #评估模型
    loss,accuracy=model.evaluate(x_test,y_test)
    
    print('test loss:',loss)
    print('test accuracy:',accuracy)
    
    print('train loss:',loss)
    print('train accuracy:',accuracy)

![这里写图片描述][SouthEast 1]

#### 2.2. 防止过拟合，选用正则化方法 ####

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.optimizers import SGD
    from keras.regularizers import l2
    #载入数据
    (x_train,y_train),(x_test,y_test)=mnist.load_data()
    #重新定义数据格式
    #60000,784及归一化
    x_train=x_train.reshape(x_train.shape[0],-1)/255.0
    x_test=x_test.reshape(x_test.shape[0],-1)/255.0
    
    #转one-hot标签
    y_train=np_utils.to_categorical(y_train,num_classes=10)
    y_test=np_utils.to_categorical(y_test,num_classes=10)
    
    #创建模型,输入784个神经元,输出10个神经元
    model=Sequential([Dense(units=200,input_dim=784,bias='one',activation='tanh',kernel_regularizer=l2(0.003)),
                      Dense(units=100,bias='one',activation='tanh',kernel_regularizer=l2(0.003)),
                      Dense(units=10,bias='one',activation='softmax',kernel_regularizer=l2(0.003))])
    #定义优化器
    sgd=SGD(lr=0.2)
    #定义loss func
    model.compile(optimizer=sgd,loss='mse',metrics=['accuracy'])
    
    #训练模型
    model.fit(x_train,y_train,batch_size=32,epochs=10)
    
    #评估模型
    loss,accuracy=model.evaluate(x_test,y_test)
    
    print('test loss:',loss)
    print('test accuracy:',accuracy)
    
    print('train loss:',loss)
    print('train accuracy:',accuracy)

![这里写图片描述][SouthEast 2]

### 3. 更优模型探索 ###

#### 3.1. RNN ####

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.layers.recurrent import SimpleRNN
    from keras.optimizers import Adam
    # 数据长度-一行有28个像素
    input_size = 28
    # 序列长度-一共有28行
    time_steps = 28
    # 隐藏层cell个数
    cell_size = 50 
    
    # 载入数据
    (x_train,y_train),(x_test,y_test) = mnist.load_data()
    # (60000,28,28)
    x_train = x_train/255.0
    x_test = x_test/255.0
    # 换one hot格式
    y_train = np_utils.to_categorical(y_train,num_classes=10)
    y_test = np_utils.to_categorical(y_test,num_classes=10)#one hot
    
    # 创建模型
    model = Sequential()
    
    # 循环神经网络
    model.add(SimpleRNN(
        units = cell_size, # 输出
        input_shape = (time_steps,input_size), #输入
    ))
    
    # 输出层
    model.add(Dense(10,activation='softmax'))
    
    # 定义优化器
    adam = Adam(lr=1e-4)
    
    # 定义优化器，loss function，训练过程中计算准确率
    model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy'])
    
    # 训练模型
    model.fit(x_train,y_train,batch_size=64,epochs=10)
    
    # 评估模型
    loss,accuracy = model.evaluate(x_test,y_test)
    
    print('test loss',loss)
    print('test accuracy',accuracy)

![这里写图片描述][SouthEast 3]

#### 3.2. CNN ####

import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense,Dropout,Convolution2D,MaxPooling2D,Flatten
    from keras.optimizers import Adam
    
    # 载入数据
    (x_train,y_train),(x_test,y_test) = mnist.load_data()
    
    # (60000,28,28)->(60000,28,28,1)
    x_train = x_train.reshape(-1,28,28,1)/255.0
    x_test = x_test.reshape(-1,28,28,1)/255.0
    # 换one hot格式
    y_train = np_utils.to_categorical(y_train,num_classes=10)
    y_test = np_utils.to_categorical(y_test,num_classes=10)
    
    # 定义顺序模型
    model = Sequential()
    
    # 第一个卷积层
    # input_shape 输入平面
    # filters 卷积核/滤波器个数
    # kernel_size 卷积窗口大小
    # strides 步长
    # padding padding方式 same/valid
    # activation 激活函数
    model.add(Convolution2D(
        input_shape = (28,28,1),
        filters = 32,
        kernel_size = 5,
        strides = 1,
        padding = 'same',
        activation = 'relu'
    ))
    # 第一个池化层
    model.add(MaxPooling2D(
        pool_size = 2,
        strides = 2,
        padding = 'same',
    ))
    
    # 第二个卷积层
    model.add(Convolution2D(64,5,strides=1,padding='same',activation = 'relu'))
    # 第二个池化层
    model.add(MaxPooling2D(2,2,'same'))
    # 把第二个池化层的输出扁平化为1维
    model.add(Flatten())
    
    # 第一个全连接层
    model.add(Dense(1024,activation = 'relu'))
    # Dropout
    model.add(Dropout(0.5))
    # 第二个全连接层
    model.add(Dense(10,activation='softmax'))
    
    # 定义优化器
    adam = Adam(lr=1e-4)
    # 定义优化器，loss function，训练过程中计算准确率
    model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['accuracy'])
    
    # 训练模型
    model.fit(x_train,y_train,batch_size=64,epochs=10)
    
    # 评估模型
    loss,accuracy = model.evaluate(x_test,y_test)
    
    print('test loss',loss)
    print('test accuracy',accuracy)

![这里写图片描述][SouthEast 4]

手写数字复杂网络层抽特征可视化工具[http://scs.ryerson.ca/~aharley/vis/conv/][http_scs.ryerson.ca_aharley_vis_conv]

[http_yann.lecun.com_exdb_mnist_index.html]: http://yann.lecun.com/exdb/mnist/index.html
[70]: /images/20220524/3313d4a80f66481ea9f7dc56e8e068cf.png
[70 1]: /images/20220524/313a6704c16941ffb87717486c995684.png
[70 2]: /images/20220524/0a421fad99154063ba4ae21631701caf.png
[SouthEast]: /images/20220524/cb4ce8444c384e06804f1258277cbe1a.png
[SouthEast 1]: /images/20220524/63fb44bfe6b94f8c862618a56a566776.png
[SouthEast 2]: /images/20220524/08dc09adf9664840a69fdf0ec293a2f0.png
[SouthEast 3]: /images/20220524/d35241aae76447f48983bed8de6ddc98.png
[SouthEast 4]: /images/20220524/f6a9af57cbf04836bce213924260e54a.png
[http_scs.ryerson.ca_aharley_vis_conv]: http://scs.ryerson.ca/~aharley/vis/conv/