单隐藏层神经网络对数据做2分类

骑猪看日落 2023-08-17 17:43 57阅读 0赞

例子代码位置：[https://github.com/lilihongjava/deep\_learning/tree/master/%E5%8D%95%E9%9A%90%E8%97%8F%E5%B1%822%E5%88%86%E7%B1%BB%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C][https_github.com_lilihongjava_deep_learning_tree_master_E5_8D_95_E9_9A_90_E8_97_8F_E5_B1_822_E5_88_86_E7_B1_BB_E7_A5_9E_E7_BB_8F_E7_BD_91_E7_BB_9C]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70][]

对上图的数据建立一个模型来做分类

X：一个numpy的矩阵，包含了这些数据点的数值，维度为: (2, 400)

Y：一个numpy的向量，对应着的是X的标签【0 | 1】（红色:0 ， 蓝色 :1），度为: (1, 400)

### 神经网络模型 ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70 1][]

n_x - 输入层节点的数量，图中为2
    n_h - 隐藏层节点的数量，图中为4
    n_y - 输出层节点的数量，图中为1
    W1 - 权重矩阵,维度为（n_h，n_x）---(4,2)
    b1 - 偏向量，维度为（n_h，1）---(4,1)
    W2 - 权重矩阵，维度为（n_y，n_h）---(1,4)
    b2 - 偏向量，维度为（n_y，1）---(1,1)

### 正向传播 ###

正向传播通过公式计算y，对于单个样本![x^\{(i)\}][x_i]公式：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70 2][]

隐藏层为什么用tanh，是因为tanh激活函数通常比隐藏层单元的sigmoid激活函数效果更好，因为其输出的平均值更接近于零。

而输出层用sigmoid，是因为此项目做的是二分类

向量化公式：

![Z^\{\[1\] \} = W^\{\[1\]\} X + B^\{\[1\]\}][Z_1_ _ _ W_1_ X _ B_1]

![A^\{\[1\] \} = \\tanh(Z^\{\[1\] \})][A_1_ _ _ _tanh_Z_1_]

![Z^\{\[2\] \} = W^\{\[2\]\} A^\{\[1\] \} + B^\{\[2\] \}][Z_2_ _ _ W_2_ A_1_ _ _ B_2_]

![\\hat\{Y\} = A^\{\[2\] \} = \\sigma(Z^\{ \[2\] \})][hat_Y_ _ A_2_ _ _ _sigma_Z_ _2_]

# 前向传播，python代码
    Z1 = np.dot(W1, X) + b1
    A1 = np.tanh(Z1)
    Z2 = np.dot(W2, A1) + b2
    A2 = sigmoid(Z2)

对于所有样本的成本函数J公式：

![J = - \\frac\{1\}\{m\} \\sum\\limits\_\{i = 0\}^\{m\} \\large\\left(\\small Y\\log\\left(A^\{\[2\] \}\\right) + (1-Y)\\log\\left(1- A^\{\[2\]\}\\right) \\large \\right) \\small][J _ - _frac_1_m_ _sum_limits_i _ 0_m_ _large_left_small Y_log_left_A_2_ _right_ _ _1-Y_log_left_1- A_2_right_ _large _right_ _small]

# 计算成本
    logprobs = np.multiply(np.log(A2), Y) + np.multiply((1 - Y), np.log(1 - A2))
    cost = - np.sum(logprobs) / m

### 反向传播 ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70 3][]

左边公式为单个样本![x^\{(i)\}][x_i]公式，右边为向量化后的公式，其中dZ1需要计算![g^\{\[1\]'\}(Z^\{\[1\]\})][g_1_Z_1]，g激活函数为tanh，求导为![1-(\\tanh (x))^\{2\}][1-_tanh _x_2]

所以![g^\{\[1\]'\}(Z^\{\[1\]\})][g_1_Z_1]为![1-a^2][1-a_2]

dZ2 = A2 - Y
        dW2 = (1 / m) * np.dot(dZ2, A1.T)
        db2 = (1 / m) * np.sum(dZ2, axis=1, keepdims=True)
        dZ1 = np.multiply(np.dot(W2.T, dZ2), 1 - np.power(A1, 2))
        dW1 = (1 / m) * np.dot(dZ1, X.T)
        db1 = (1 / m) * np.sum(dZ1, axis=1, keepdims=True)

### 更新参数 ###

使用反向传播的(dW1, db1, dW2, db2)来更新(W1, b1, W2, b2)

![\\theta = \\theta - \\alpha \\frac\{\\partial J \}\{ \\partial \\theta \}][theta _ _theta - _alpha _frac_partial J _ _partial _theta]

*  α：学习速率
 *  θ：参数

### 预测 ###

构建predict()来使用模型进行预测， 使用向前传播来预测结果。  
激活值大于0.5为1，小于为0

# forward_propagation正向传播公式
    A2, cache = forward_propagation(X, parameters)
    # np.round四舍五入，既大于0.5为1，其他为0
    predictions = np.round(A2)

### 构建神经网络步骤 ###

1. 定义神经网络结构（输入单元的数量，隐藏单元的数量等）。  
2. 初始化模型的参数，参数权重W是不能全部初始化为零，会导致![W\_1^\{\[1\]\}][W_1_1]始终等于![W\_2^\{\[1\]\}][W_2_1]，完全对称。这样隐藏层设置多个神经元就没有任何意义了。  
3. 循环：

*  实施前向传播
 *  计算损失
 *  实现向后传播
 *  更新参数（梯度下降）

[https_github.com_lilihongjava_deep_learning_tree_master_E5_8D_95_E9_9A_90_E8_97_8F_E5_B1_822_E5_88_86_E7_B1_BB_E7_A5_9E_E7_BB_8F_E7_BD_91_E7_BB_9C]: https://github.com/lilihongjava/deep_learning/tree/master/%E5%8D%95%E9%9A%90%E8%97%8F%E5%B1%822%E5%88%86%E7%B1%BB%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70]: /images/20230808/c7578feae2ba436ca9dcc7610f8a309b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70 1]: /images/20230808/e6f245f6d5f24b08b90ebe9274d27a28.png
[x_i]: https://private.codecogs.com/gif.latex?x%5E%7B%28i%29%7D
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70 2]: /images/20230808/1bbb690c1ac143e0b428d847b7d36512.png
[Z_1_ _ _ W_1_ X _ B_1]: https://private.codecogs.com/gif.latex?Z%5E%7B%5B1%5D%20%7D%20%3D%20W%5E%7B%5B1%5D%7D%20X%20&plus;%20B%5E%7B%5B1%5D%7D
[A_1_ _ _ _tanh_Z_1_]: https://private.codecogs.com/gif.latex?A%5E%7B%5B1%5D%20%7D%20%3D%20%5Ctanh%28Z%5E%7B%5B1%5D%20%7D%29
[Z_2_ _ _ W_2_ A_1_ _ _ B_2_]: https://private.codecogs.com/gif.latex?Z%5E%7B%5B2%5D%20%7D%20%3D%20W%5E%7B%5B2%5D%7D%20A%5E%7B%5B1%5D%20%7D%20&plus;%20B%5E%7B%5B2%5D%20%7D
[hat_Y_ _ A_2_ _ _ _sigma_Z_ _2_]: https://private.codecogs.com/gif.latex?%5Chat%7BY%7D%20%3D%20A%5E%7B%5B2%5D%20%7D%20%3D%20%5Csigma%28Z%5E%7B%20%5B2%5D%20%7D%29
[J _ - _frac_1_m_ _sum_limits_i _ 0_m_ _large_left_small Y_log_left_A_2_ _right_ _ _1-Y_log_left_1- A_2_right_ _large _right_ _small]: https://private.codecogs.com/gif.latex?J%20%3D%20-%20%5Cfrac%7B1%7D%7Bm%7D%20%5Csum%5Climits_%7Bi%20%3D%200%7D%5E%7Bm%7D%20%5Clarge%5Cleft%28%5Csmall%20Y%5Clog%5Cleft%28A%5E%7B%5B2%5D%20%7D%5Cright%29%20&plus;%20%281-Y%29%5Clog%5Cleft%281-%20A%5E%7B%5B2%5D%7D%5Cright%29%20%5Clarge%20%5Cright%29%20%5Csmall
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMzODczNDMx_size_16_color_FFFFFF_t_70 3]: /images/20230808/cc3c8b30359e45e9af49cf61c5621841.png
[g_1_Z_1]: https://private.codecogs.com/gif.latex?g%5E%7B%5B1%5D%27%7D%28Z%5E%7B%5B1%5D%7D%29
[1-_tanh _x_2]: https://private.codecogs.com/gif.latex?1-%28%5Ctanh%20%28x%29%29%5E%7B2%7D
[1-a_2]: https://private.codecogs.com/gif.latex?1-a%5E2
[theta _ _theta - _alpha _frac_partial J _ _partial _theta]: https://private.codecogs.com/gif.latex?%5Ctheta%20%3D%20%5Ctheta%20-%20%5Calpha%20%5Cfrac%7B%5Cpartial%20J%20%7D%7B%20%5Cpartial%20%5Ctheta%20%7D
[W_1_1]: https://private.codecogs.com/gif.latex?W_1%5E%7B%5B1%5D%7D
[W_2_1]: https://private.codecogs.com/gif.latex?W_2%5E%7B%5B1%5D%7D